Hey @bitplane, welcome!
That’s an interesting idea. Feel free to reach out to ffmpeg authors, although I doubt they’ll go for it 
However, one thing which could be done without too much hassle and too much work is to implement raw frame output in agg or in the new convert subcommand of the upcoming asciinema CLI 3.0. It could output a raw image for each frame, sequentially as one stream, which could be piped straight into ffmpeg. Once those frames hit ffmpeg you can perform anything (ffmpeg can) on it and encode it into any type of video format.