Anti LLM crawler mischief

Blocking LLM crawlers with HTTP 4xx error pages? Why not provide them with a custom error page that contains links that lead to one of the nonsense generators of your choice, with said links saying prominently not to follow them.

On the other hand, I have no idea if common LLM crawlers follow links found on error pages, or if they just ignore the entire HTML text of the error page. Energetic people are encouraged to find out.

0

If you have a fediverse account, you can quote this note from your own instance. Search https://mastodon.social/users/cks/statuses/115631048793631505 on your instance and quote it. (Note that quoting is not supported in Mastodon.)