Avoid to add frames to the vsrc_buffer in the case ist->pts <
start_time, as these frames are unused (and never released). In
particular this condition is verified with commands of the kind:
ffmpeg -i INPUT -ss TIME OUTPUT

Also allow a minor simplification.