avfilter/vf_edgedetect: fix undefined behaviour

Fixes #8287

avfilter/vf_edgedetect: fix undefined behaviour
Fixes #8287
932913f0 · Paul B Mahol · 7b94b222 · 932913f0
Commit 932913f0 authored Oct 16, 2019 by Paul B Mahol
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

vf_edgedetect.c libavfilter/vf_edgedetect.c +1 -1

No files found.
--- a/libavfilter/vf_edgedetect.c
+++ b/libavfilter/vf_edgedetect.c
@@ -216,7 +216,7 @@ static int get_rounded_direction(int gx, int gy)

        if (gx < 0)
            gx = -gx, gy = -gy;
-        gy <<= 16;
+        gy *= (1 << 16);
        tanpi8gx  =  27146 * gx;
        tan3pi8gx = 158218 * gx;
        if (gy > -tan3pi8gx && gy < -tanpi8gx)  return DIRECTION_45UP;