• Guo, Yejun's avatar
    avfilter/vf_dnn_processing: add a generic filter for image proccessing with dnn networks · 4d980a8c
    Guo, Yejun authored
    This filter accepts all the dnn networks which do image processing.
    Currently, frame with formats rgb24 and bgr24 are supported. Other
    formats such as gray and YUV will be supported next. The dnn network
    can accept data in float32 or uint8 format. And the dnn network can
    change frame size.
    
    The following is a python script to halve the value of the first
    channel of the pixel. It demos how to setup and execute dnn model
    with python+tensorflow. It also generates .pb file which will be
    used by ffmpeg.
    
    import tensorflow as tf
    import numpy as np
    import imageio
    in_img = imageio.imread('in.bmp')
    in_img = in_img.astype(np.float32)/255.0
    in_data = in_img[np.newaxis, :]
    filter_data = np.array([0.5, 0, 0, 0, 1., 0, 0, 0, 1.]).reshape(1,1,3,3).astype(np.float32)
    filter = tf.Variable(filter_data)
    x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
    y = tf.nn.conv2d(x, filter, strides=[1, 1, 1, 1], padding='VALID', name='dnn_out')
    sess=tf.Session()
    sess.run(tf.global_variables_initializer())
    output = sess.run(y, feed_dict={x: in_data})
    graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
    tf.train.write_graph(graph_def, '.', 'halve_first_channel.pb', as_text=False)
    output = output * 255.0
    output = output.astype(np.uint8)
    imageio.imsave("out.bmp", np.squeeze(output))
    
    To do the same thing with ffmpeg:
    - generate halve_first_channel.pb with the above script
    - generate halve_first_channel.model with tools/python/convert.py
    - try with following commands
      ./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.model:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=native -y out.native.png
      ./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.pb:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=tensorflow -y out.tf.png
    Signed-off-by: 's avatarGuo, Yejun <yejun.guo@intel.com>
    Signed-off-by: 's avatarPedro Arthur <bygrandao@gmail.com>
    4d980a8c
Name
Last commit
Last update
..
doxy Loading commit data...
examples Loading commit data...
.gitignore Loading commit data...
APIchanges Loading commit data...
Doxyfile Loading commit data...
Makefile Loading commit data...
authors.texi Loading commit data...
bitstream_filters.texi Loading commit data...
bootstrap.min.css Loading commit data...
build_system.txt Loading commit data...
codecs.texi Loading commit data...
decoders.texi Loading commit data...
default.css Loading commit data...
demuxers.texi Loading commit data...
developer.texi Loading commit data...
devices.texi Loading commit data...
doxy-wrapper.sh Loading commit data...
encoders.texi Loading commit data...
errno.txt Loading commit data...
faq.texi Loading commit data...
fate.texi Loading commit data...
fate_config.sh.template Loading commit data...
ffmpeg-bitstream-filters.texi Loading commit data...
ffmpeg-codecs.texi Loading commit data...
ffmpeg-devices.texi Loading commit data...
ffmpeg-filters.texi Loading commit data...
ffmpeg-formats.texi Loading commit data...
ffmpeg-protocols.texi Loading commit data...
ffmpeg-resampler.texi Loading commit data...
ffmpeg-scaler.texi Loading commit data...
ffmpeg-utils.texi Loading commit data...
ffmpeg.texi Loading commit data...
ffmpeg.txt Loading commit data...
ffplay.texi Loading commit data...
ffprobe.texi Loading commit data...
ffprobe.xsd Loading commit data...
fftools-common-opts.texi Loading commit data...
filter_design.txt Loading commit data...
filters.texi Loading commit data...
formats.texi Loading commit data...
general.texi Loading commit data...
git-howto.texi Loading commit data...
indevs.texi Loading commit data...
issue_tracker.txt Loading commit data...
lexicon Loading commit data...
libav-merge.txt Loading commit data...
libavcodec.texi Loading commit data...
libavdevice.texi Loading commit data...
libavfilter.texi Loading commit data...
libavformat.texi Loading commit data...
libavutil.texi Loading commit data...
libswresample.texi Loading commit data...
libswscale.texi Loading commit data...
mailing-list-faq.texi Loading commit data...
metadata.texi Loading commit data...
mips.txt Loading commit data...
multithreading.txt Loading commit data...
muxers.texi Loading commit data...
nut.texi Loading commit data...
optimization.txt Loading commit data...
outdevs.texi Loading commit data...
patchwork Loading commit data...
platform.texi Loading commit data...
print_options.c Loading commit data...
protocols.texi Loading commit data...
rate_distortion.txt Loading commit data...
resampler.texi Loading commit data...
scaler.texi Loading commit data...
snow.txt Loading commit data...
style.min.css Loading commit data...
swresample.txt Loading commit data...
swscale.txt Loading commit data...
t2h.init Loading commit data...
t2h.pm Loading commit data...
tablegen.txt Loading commit data...
texi2pod.pl Loading commit data...
texidep.pl Loading commit data...
undefined.txt Loading commit data...
utils.texi Loading commit data...
writing_filters.txt Loading commit data...