avcodec/nvenc: set correct error code

In function process_output_surface(), the return value is 0 on the path
that av_mallocz() returns a NULL pointer. 0 indicates success, which
deviates from the fact. Return "AVERROR(ENOMEM)" instead of "0".
Signed-off-by: Pan Bian <bianpan2016@163.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>

libavcodec/nvenc.c

@@ -1763,8 +1763,10 @@ static int process_output_surface(AVCodecContext *avctx, AVPacket *pkt, NvencSur
     }
     slice_offsets = av_mallocz(slice_mode_data * sizeof(*slice_offsets));
 
-    if (!slice_offsets)
+    if (!slice_offsets) {
+        res = AVERROR(ENOMEM);
         goto error;
+    }
 
     lock_params.version = NV_ENC_LOCK_BITSTREAM_VER;