Jenniferplusplus @jenniferplusplus@hachyderm.io
Public

I wrote about AI again. On some level I don't now why I do this to myself

jenniferplusplus.com/what-is-a

Actually, I wrote most of it months ago, for work. But, it was well received, so I put it on my blog. This is the more generic version.

The short version is that AI is not magic. It's a real phenomenon with real behavior and tradeoffs. I'm deeply tired of *****ALL***** the tradeoffs being handwaved away. And so much imagination fills in for the actual behavior. So I tried to describe how it's built, because that informs how it works, which informs what it actually does. And to be clear, it does things. It's not useless. But that's not the same as being useful, or worthwhile.

Anyway, I already put ~4k words on this in the article, so I'll shut up and let it speak for itself.

What is a token

AI is meant to seem like magic. But there's no such thing as magic. It's all illusion. So, allow me to spoil that illusion for you.

jenniferplusplus.com · Jennifer++

hjhornbeck @hjhornbeck@freethought.online
Quiet public

@jenniferplusplus
Good article! I have a similar stance, as an academic computer scientist. I do have one minor quibble, though: transformers do not generate tokens. They generate a probability distribution of potential next tokens, and are dependent on a "second system" to collapse that distribution into a concrete token. I've demonstrated this here:

web.archive.org/web/2026011500

(Forgive the Wayback link, the blog is having technical issues.)

Aside: Let’s Bisect an LLM!

I previously took a lot of words to describe the guts of Markov chains and LLMs, and ended by pointing out that all LLMs can be split into two systems: one that takes in a list of tokens and output…

web.archive.org · Reprobate Spreadsheet

0
1
0

If you have a fediverse account, you can quote this note from your own instance. Search https://freethought.online/users/hjhornbeck/statuses/116138223749897389 on your instance and quote it. (Note that quoting is not supported in Mastodon.)