1. 14 Feb, 2017 1 commit
    • Joel Cunningham's avatar
      HTTP: improve performance by reducing forward seeks · 8c8e5d52
      Joel Cunningham authored
      This commit optimizes HTTP performance by reducing forward seeks, instead
      favoring a read-ahead and discard on the current connection (referred to
      as a short seek) for seeks that are within a TCP window's worth of data.
      This improves performance because with TCP flow control, a window's worth
      of data will be in the local socket buffer already or in-flight from the
      sender once congestion control on the sender is fully utilizing the window.
      
      Note: this approach doesn't attempt to differentiate from a newly opened
      connection which may not be fully utilizing the window due to congestion
      control vs one that is. The receiver can't get at this information, so we
      assume worst case; that full window is in use (we did advertise it after all)
      and that data could be in-flight
      
      The previous behavior of closing the connection, then opening a new
      with a new HTTP range value results in a massive amounts of discarded
      and re-sent data when large TCP windows are used.  This has been observed
      on MacOS/iOS which starts with an initial window of 256KB and grows up to
      1MB depending on the bandwidth-product delay.
      
      When seeking within a window's worth of data and we close the connection,
      then open a new one within the same window's worth of data, we discard
      from the current offset till the end of the window.  Then on the new
      connection the server ends up re-sending the previous data from new
      offset till the end of old window.
      
      Example (assumes full window utilization):
      
      TCP window size: 64KB
      Position: 32KB
      Forward seek position: 40KB
      
            *                      (Next window)
      32KB |--------------| 96KB |---------------| 160KB
              *
        40KB |---------------| 104KB
      
      Re-sent amount: 96KB - 40KB = 56KB
      
      For a real world test example, I have MP4 file of ~25MB, which ffplay
      only reads ~16MB and performs 177 seeks. With current ffmpeg, this results
      in 177 HTTP GETs and ~73MB worth of TCP data communication.  With this
      patch, ffmpeg issues 4 HTTP GETs and 3 seeks for a total of ~22MB of TCP data
      communication.
      
      To support this feature, the short seek logic in avio_seek() has been
      extended to call a function to get the short seek threshold value.  This
      callback has been plumbed to the URLProtocol structure, which now has
      infrastructure in HTTP and TCP to get the underlying receiver window size
      via SO_RCVBUF.  If the underlying URL and protocol don't support returning
      a short seek threshold, the default s->short_seek_threshold is used
      
      This feature has been tested on Windows 7 and MacOS/iOS.  Windows support
      is slightly complicated by the fact that when TCP window auto-tuning is
      enabled, SO_RCVBUF doesn't report the real window size, but it does if
      SO_RCVBUF was manually set (disabling auto-tuning). So we can only use
      this optimization on Windows in the later case
      Signed-off-by: 's avatarJoel Cunningham <joel.cunningham@me.com>
      Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
      8c8e5d52
  2. 26 Jan, 2017 1 commit
    • Joel Cunningham's avatar
      tcp: set socket buffer sizes before listen/connect/accept · f3778108
      Joel Cunningham authored
      From e24d95c0e06a878d401ee34fd6742fcaddeeb95f Mon Sep 17 00:00:00 2001
      From: Joel Cunningham <joel.cunningham@me.com>
      Date: Mon, 9 Jan 2017 13:37:51 -0600
      Subject: [PATCH] tcp: set socket buffer sizes before listen/connect/accept
      
      Attempting to set SO_RCVBUF and SO_SNDBUF on TCP sockets after connection
      establishment is incorrect and some stacks ignore the set call on the socket at
      this point.  This has been observed on MacOS/iOS.  Windows 7 has some peculiar
      behavior where setting SO_RCVBUF after applies only if the buffer is increasing
      from the default while decreases are ignored.  This is possibly how the incorrect
      usage has gone unnoticed
      
      Unix Network Programming Vol. 1: The Sockets Networking API (3rd edition, seciton 7.5):
      
      "When setting the size of the TCP socket receive buffer, the ordering of the
      function calls is important.  This is because of TCP's window scale option,
      which is exchanged with the peer on SYN segments when the connection is
      established. For a client, this means the SO_RCVBUF socket option must be
      set before calling connect.  For a server, this means the socket option must
      be set for the listening socket before calling listen.  Setting this option
      for the connected socket will have no effect whatsoever on the possible window
      scale option because accept does not return with the connected socket until
      TCP's three-way handshake is complete.  This is why the option must be set on
      the listening socket. (The sizes of the socket buffers are always inherited from
      the listening socket by the newly created connected socket)"
      Signed-off-by: 's avatarJoel Cunningham <joel.cunningham@me.com>
      Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
      f3778108
  3. 06 Nov, 2016 1 commit
  4. 05 Nov, 2016 1 commit
  5. 24 Mar, 2016 1 commit
  6. 22 Feb, 2016 1 commit
    • Anton Khirnov's avatar
      lavf: reorganize URLProtocols · 2758cded
      Anton Khirnov authored
      Instead of a linked list constructed at av_register_all(), store them
      in a constant array of pointers.
      
      Since no registration is necessary now, this removes some global state
      from lavf. This will also allow the urlprotocol layer caller to limit
      the available protocols in a simple and flexible way in the following
      commits.
      2758cded
  7. 22 Jan, 2016 1 commit
    • Perette Barella's avatar
      libavformat/tcp.c : add send_buffer_size and recv_buffer_size options · 84110f4f
      Perette Barella authored
      adds two new options that may be set via the dictionary:
      
      - send_buffer_size
      - recv_buffer_size
      
      When present, setsockopt() is used with SO_SNDBUF and SO_RCVBUF to set
      socket buffer sizes.  I chose to make send and receive independent
      because buffering requirements are often asymmetric.
      
      Errors in setting the buffer size mean the socket will use its
      default, so they are ignored.
      
      There is no sanity checking on values, as the kernel/socket layers
      already impose reasonable limits if asked for something crazy.
      
      Rationale for enlarging receive buffers is to reduce susceptibility
      to intermittent network delays/congestion.  I added setting the send
      buffer for symmetry.
      Signed-off-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
      84110f4f
  8. 11 Sep, 2015 1 commit
  9. 31 Jul, 2015 2 commits
  10. 09 Apr, 2015 1 commit
  11. 28 Feb, 2015 1 commit
  12. 12 Nov, 2014 1 commit
  13. 10 Oct, 2014 1 commit
  14. 25 Aug, 2014 1 commit
  15. 25 Dec, 2013 2 commits
  16. 09 Sep, 2013 1 commit
  17. 25 Aug, 2013 1 commit
  18. 06 Aug, 2013 1 commit
  19. 05 Aug, 2013 1 commit
  20. 05 Jul, 2013 2 commits
  21. 04 Jun, 2013 1 commit
  22. 01 Jun, 2013 2 commits
  23. 12 Mar, 2013 1 commit
  24. 09 Oct, 2012 2 commits
  25. 25 Jul, 2012 1 commit
  26. 09 Jul, 2012 1 commit
  27. 27 Jun, 2012 2 commits
  28. 26 Jun, 2012 1 commit
  29. 20 Jun, 2012 1 commit
  30. 19 Jun, 2012 2 commits
  31. 17 Jun, 2012 1 commit
  32. 01 Jun, 2012 1 commit
  33. 31 May, 2012 1 commit