Not just on failure. This is the same thing that is done in the audio
path and should prevent leaks in decoders that allocate a frame, but
then end up not writing into it.