Because if we drop both we have no timestamps at all for some files.
Fixes Ticket344
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>