I'm hoping that this will address the remaining tsan fate-h264 issues:

WARNING: ThreadSanitizer: data race (pid=24478)
  Read of size 8 at 0x7dbc0001c828 by main thread (mutexes: write M3243):
    #0 ff_h264_ref_picture src/libavcodec/h264_picture.c:107 (ffmpeg+0x0000013b78d8)
[..]
  Previous write of size 1 at 0x7dbc0001c82e by thread T2 (mutexes: write M3245):
    #0 ff_h264_direct_ref_list_init src/libavcodec/h264_direct.c:137 (ffmpeg+0x000001382c93)

But I'm not sure because I haven't been able to reproduce locally.