The main script collecting the instance data for these posts should now (at least partially) respect robots.txt files. This script runs 4 times a day and currently identifies itself as "fediverse/2.195 (
@fediversecounter)", where the version number will change. There is also a secondary script (that doesn't include a version number) that runs every 15 minutes which doesn't query robots.txt as it only queries a subset of the instances that is selected by the main fediverse.
If you have a fediverse account, you can quote this note from your own instance. Search https://mastodon.social/users/fediversecounter/statuses/113991383690619666 on your instance and quote it. (Note that quoting is not supported in Mastodon.)