Jeff Atwood @codinghorror@infosec.exchange
Public

"They can build systems that optimize for agreement and politeness—and become interchangeable, low-trust utilities. Or they can build systems that are occasionally annoying, occasionally resistant, and sometimes say “no”—and become indispensable."

Conclusion: Alignment Is Not Agreement LLM companies face a choice they cannot avoid: They can build systems that optimize for agreement and politeness—and become interchangeable, low-trust utilities. Or they can build systems that are occasionally annoying, occasionally resistant, and sometimes say “no”—and become indispensable. Over-agreement feels like alignment. But alignment without friction is just flattery. And flattery has never been a durable business model.
0
0
0

If you have a fediverse account, you can quote this note from your own instance. Search https://infosec.exchange/users/codinghorror/statuses/115746014882797331 on your instance and quote it. (Note that quoting is not supported in Mastodon.)