Quote from another site (non-public topic):Offhand I don't know how effective they are with Google bots, since Google hasn't been hammering my site anyway. They should work though.
The lines in question are not for phpBB syntax, but I added the phpBB equivalents to my robots.txt.Google ignores a lot of directives, e.g., nofollow, noindex & crawl-delay (in robots.txt). However, they do honor 'disallow' statements in robots.txt.
Recently I added these lines to robots.txt & this resulted in a fairly dramatic decrease in Google (and other crawler) traffic:
Code:
Disallow: *sid=Disallow: */index.php?p=Statistics: Posted by Gumboots — Fri May 02, 2025 6:25 pm