Avoids memleaks with audio when extended_data is nontrivial and the user
doesn't reset the frame.
Shouldn't have any effect for video for now, but might be useful in the
future.