ok. what are the best studies of AI coding that actually *measure* it and don't just ask the devs or their managers how they *feel* about it?

not whether the dev thinks they're faster. but whether they clearly *measure* faster, by some reasonable methodology.

frankly, the best study i know of so far is the METR study. that's limited and provides all its own caveats in an extremely honest manner.

AI bros pooh pooh the METR study, but they conspicuously don't do it again in a way that would solve their objections.

instead, the AI bros just don't seem to measure shit.

but surely someone's done a study as good or better than the METR study, right?

0
0
0

If you have a fediverse account, you can quote this note from your own instance. Search https://circumstances.run/users/davidgerard/statuses/115885352921683222 on your instance and quote it. (Note that quoting is not supported in Mastodon.)