The Colchis project to load JSON compactly but accessibly into memory in now uses compression when loading texts. This reduces peak memory usage until that text is finally unpacked in the end. Since this is done once tree structure information is already processed and transformed into succinct data, this decreases peak memory usage while loading JSON; it seems to be close to the size of the original document now (and then shrinks a bit again).

github.com/faassen/colchis

0

If you have a fediverse account, you can quote this note from your own instance. Search https://hachyderm.io/users/faassen/statuses/114613836343158899 on your instance and quote it. (Note that quoting is not supported in Mastodon.)