The libopus encoder does the same thing and its better than
keeping track of when the empty flush frames appear.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>