You can point Google at the news feeds to use as sitemap.We use SEO Sitemap too, but quickly learned not to let it just build an automatic list of all threads, all forums.
It's not necessarily a "huge" issue. The bot follows the link and finds canonical tag indicating the source page, this avoids the duplicate content issue and what URL the bot will use in index. The only issue is the bot is wasting your resources and it's resources when it's following links to duplicate content when it could be indexing new conetent re-indexing old content. .Hello, this is a huge issue.
You can fix these problems by editing tepmplate and hiding the links from Google, many of them are already hidden. For example in viewtopic_body.html around line 231 find:
Code:
<a {% if postrow.S_FIRST_UNREAD %}class="first-unread" {% endif %}href="{{ postrow.U_MINI_POST }}">{{ postrow.POST_SUBJECT }}</a>Code:
<!-- IF not S_IS_BOT --><a {% if postrow.S_FIRST_UNREAD %}class="first-unread" {% endif %}href="{{ postrow.U_MINI_POST }}">{{ postrow.POST_SUBJECT }}</a><!-- ELSE -->{{ postrow.POST_SUBJECT }}<!-- ENDIF -->Statistics: Posted by thecoalman — Fri Oct 04, 2024 11:42 pm