• Marja Hölttä's avatar
    [script streaming] Fix U+feff handling. · 4e453429
    Marja Hölttä authored
    U+feff is the UTF BOM but if it occurs inside the text, it's a "zero-width
    no-break space". However, the UTF-8 decoder in script streaming still thought
    it's a BOM and skipped it. The correct way to handle it would be to create a
    U+feff code point instead - the Scanner will then handle it as whitespace.
    
    This is a discrepancy between the Blink UTF-8 decoder and the V8 UTF-8 decoder,
    and caused the source positions be off by one. This bug went unnoticed, since
    normally off-by-one in this situation doesn't make the code to break.
    
    BUG=chromium:758508,chromium:758236
    
    Change-Id: Ib92a3ee65c402e21b77e42537db2a021cff55379
    Reviewed-on: https://chromium-review.googlesource.com/632096Reviewed-by: 's avatarAdam Klein <adamk@chromium.org>
    Commit-Queue: Marja Hölttä <marja@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#47583}
    4e453429
test-scanner-streams.cc 20 KB