With the current approach we only need to track using an unordered set as we can
still access the backing store pointer and length by the time we free the
backing store.

BUG=chromium:619491, chromium:611688
LOG=N
R=ulan@chromium.org

Review-Url: https://codereview.chromium.org/2107443002
Cr-Commit-Position: refs/heads/master@{#37318}