The cleanup is only done now when
a picture is returned (assuming that it has to be done when its returned)
a error is returned (assuming that there will be no further progress on the frame)
the codec is not h264 (this is still needed due to some deadlocks in realvideo)

This fixes a decoding regression with 00017.MTS
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>