What's the fastest way to concatenate ~4000 equal-size files on CephFS (on hard drives but can do >800MB/s with parallelism) using Go? Resulting file will be ~33GB. Needs to finish in under 120 second. Asking for GoodSync (enforces a 120s timeout and retry on S3 multipart upload completion) and versitygw (currently concatenates parts serially during the completion request).

0

If you have a fediverse account, you can quote this note from your own instance. Search https://social.treehouse.systems/users/mxshift/statuses/115782930874326445 on your instance and quote it. (Note that quoting is not supported in Mastodon.)