Container formats are much, much, much simpler and demand much less CPU than sample data formats. One could (and I have) write a perfectly useable ISO parser in emacs lisp, of all things. The reason that they tend to be written in C is because a container parser is not super useful outside of the context of handling the sample data; the overwhelming majority of usages of code that understands containers is to be able to pump out pointers to each individual sample. Doing this across process boundaries would be ... unpleasant.