doc/encoders: partially rewrite and reformat libx264 docs

Format is based on the thread:
"[PATCH] doc/encoders: Add libopus encoder doc" (06-28-2013)
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/165368/

Also merge the two option sections (Mapping and Private options).

Patch partially edited by Stefano Sabatini.
Signed-off-by: Stefano Sabatini <stefasab@gmail.com>