For encoding, AVCodecContext.frame_size is the number of input samples to
send to the encoder and does not necessarily correspond directly to the
timestamps of the output packets.