Asciinema: upload failed: <urlopen error [Errno 32] Broken pipe>

Answering the question about the storage on asciinema.org:

The recordings are stored in a S3 bucket, there’s also local cache directory on disk which nginx uses to limit the number of requests to S3 to a minimum (to keep the cost low).

The recordings are not gzipped at rest - they’re saved on S3 as is. This is fine because a) asciicasts are small in general (compared to video files) and b) the local nginx cache limits the S3 requests as described above.

Then, the webserver is serving the recordings compressed (on the fly) to the browsers (when accept-encoding: gzip is present in request headers), so the server->client transfer is small and fast.

I believe I could raise the upload limit significantly without increasing the hosting costs much, however…

Another reason for the limit is:

The web player needs to fetch the whole recording file and parse it in order to play it. Even if it gets served gzipped to a browser, after parsing it sits in memory. I don’t have exact numbers, but let’s say 1MB recording file when parsed into player’s data structures uses 10MB of RAM. If we let users upload 50MB files (or 5MB gzipped which uncompresses to 50MB) then when loaded in the player it would use 500MB of RAM. Now, if you embed such recording on your site then every visitor will be affected by such memory usage.

So I put the low limit also to prevent situations like above.

One possible solution to this would be to stream the recording in chunks to the player, which is possible with the new v3 player (via Websocket or EventSource driver), this however disables pause and seek features, as these drivers perform the playback in a live broadcast fashion.

One more thing about the storage - see my comment here about BYOS (bring your own storage) idea: Workaround the upload limit? - #5 by sickill