Does have some kind of sshd connection throttling mechanism that is outside the PerSourcePenalties config? (I set mine to 'no')

Use blacklist is set to no.
pf is not running either.

Getting really strange intermittent timeouts where I just can't log in yet I can get to the banner with telnet on port 22.

Some PAM thing with FreeBSD perhaps?

Running tcpdump, the FreeBSD machine is constantly sending its own list of KexAlgorithms until the connection times out.

FreeBSD is based sshd: 9.9p2
Connecting client is Linux openssh 10.0p2

0

If you have a fediverse account, you can quote this note from your own instance. Search https://mastodon.bsd.cafe/users/pertho/statuses/114818896974501780 on your instance and quote it. (Note that quoting is not supported in Mastodon.)