12/17/2023 0 Comments Htaccess allow moz dotbotNo my urls do not have an ending trailing slash nor do links inside.Īnd since you asked I’ve noticed in apache logs that infected URLS mostly do end up with a trailing slash with some rare exceptions where it doesn’t or ends with. For example, do your URLs include a trailing slash? This can be improved if we know more precisely the format of your URLs and of the "spam" URLs you are trying to rid your system of. htaccess as requested Options +MultiViews To inspect search results I use keyword site:sitename and this gives different result lengths depending whether I include Currently I have no robots.txt to allow bots to crawl.Īm I missing something? Can someone point me to a right direction? One day there are 60+ other day it’s 100+ the other day its 80+ and so on… Sometimes they go down and sometimes they go up. txt file and submitted it to the tool.Īll this is the span of a few days and I understand that it could take a while to clear up, but what’s bothering me is that search results length are changing in number. I copied all unwanted search results and put them in a. Next I found out I can disavow links to my website by using this tool. I was hoping that if bots receive 410 status code they will stop indexing them and remove them, but like before my search results were not starting to clear up. Īfter these changes I can see in apache logs that crawler bots (e.g SemrushBot, Dotbot, Googlebot, Petalbot and etc.) or some unknown user agents (e.g The Knowledge AI, ANTIPIDERSIA) are requesting infected pages(that no longer exist) and receives mostly 410 status code or sometimes 301 and right after a duplicate request with 410 status code.įor example 66.33.212.13 - "GET / HTTP/1.1" 301 557 "-" "ANTIPIDERSIA"Ħ6.33.212.13 - "GET / HTTP/1.1" 200 43030 "-" "ANTIPIDERSIA"Ħ4.62.252.176 - "GET /cialis-long-term-effects/ HTTP/1.1" 301 675 "-" "The Knowledge AI"Ħ6.249.70.92 - "GET /viagra-discounts/ HTTP/1.1" 301 659 "-" "Mozilla/5.0 (Linux Android 6.0.1 Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/.90 Mobile Safari/537.36 (compatible Googlebot/2.1 +)"Ħ6.249.70.69 - "GET /viagra-discounts/ HTTP/1.1" 410 5502 "-" "Mozilla/5.0 (Linux Android 6.0.1 Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/.90 Mobile Safari/537.36 (compatible Googlebot/2.1 +)" htaccess file to redirect non-existing pages in my website to 404.php page with the code below. Next thing I tried was returning a 410 status code for every page that doesn’t exist by using. But sadly the search results stayed the same. In the tools google provides I tried using "URL Inspection tool" and requested indexing of the website, submitted a "sitemap.xml" and used "Removals tool" to remove cached content. The website is hosted on and I’ve rebuilt the droplet that it was using, added a firewall, redirected all http traffic to https and the only thing that is the same is IP Address with the domain name.Īfter cleaning the server and website I’ve begun to clean the search results by using Google search console. Since then we reworked the website(design and logic) and it’s a completely new website that is no longer using WORDPRESS or any other CMS, it’s just plain PHP, JS and CSS files with a few forms. My client’s website was hacked by a "pharma hack" which was a WORDPRESS website. All we need is an easy explanation of the problem, so here it is.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |