It is used to store the difference between pointers, so ptrdiff_t is the
correct type.

This prevents potential overflows.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>