For ismv/isma, the time scale might not be the same as the sample
rate.
Signed-off-by: Martin Storsjö <martin@martin.st>