1. 26 Jul, 2021 1 commit
    • Leszek Swirski's avatar
      Reland "[offthread] Template deserializer on Isolate" · 6f898234
      Leszek Swirski authored
      This is a reland of e24fa913
      It fixes the heap verification errors by going back to using MakeThin
      instead of manually creating a filler (that then makes the verifier
      think that this was array left-trimming).
      
      Original change's description:
      > [offthread] Template deserializer on Isolate
      >
      > Make the deserializer class templated on Isolate/LocalIsolate. This
      > allows the ObjectSerializer to be split into a main-thread and offthread
      > variant, with the latter taking a LocalIsolate.
      >
      > Eventually, we probably want to anyway split off the code-cache de/serializer
      > to a separate implementation (for various reasons), and this the only one that
      > wants off-thread finalization, and at this point the deserializer can revert
      > back to being un-templated, used only for bootstrapping. However, this is the
      > simplest way, for now, to enable off-thread deserialization.
      >
      > Bug: chromium:1075999
      > Change-Id: I49c0d2c5409f0aa58183673785296756c3714f22
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2562254
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#75834}
      
      Bug: chromium:1075999
      Change-Id: I1d81fad2550a2a9f04dd0f9d8e66422d28faf378
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3043960Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Reviewed-by: 's avatarOmer Katz <omerkatz@chromium.org>
      Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      Commit-Queue: Omer Katz <omerkatz@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#75918}
      6f898234
  2. 21 Jul, 2021 2 commits
    • Nico Hartmann's avatar
      Revert "[offthread] Template deserializer on Isolate" · c73d759b
      Nico Hartmann authored
      This reverts commit e24fa913.
      
      Reason for revert: https://ci.chromium.org/ui/p/v8/builders/ci/V8%20Mac64%20GC%20Stress/18917/overview
      
      Original change's description:
      > [offthread] Template deserializer on Isolate
      >
      > Make the deserializer class templated on Isolate/LocalIsolate. This
      > allows the ObjectSerializer to be split into a main-thread and offthread
      > variant, with the latter taking a LocalIsolate.
      >
      > Eventually, we probably want to anyway split off the code-cache de/serializer
      > to a separate implementation (for various reasons), and this the only one that
      > wants off-thread finalization, and at this point the deserializer can revert
      > back to being un-templated, used only for bootstrapping. However, this is the
      > simplest way, for now, to enable off-thread deserialization.
      >
      > Bug: chromium:1075999
      > Change-Id: I49c0d2c5409f0aa58183673785296756c3714f22
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2562254
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#75834}
      
      Bug: chromium:1075999
      Change-Id: Id699ebe0c17d3a61ec35b0f78417306175271647
      No-Presubmit: true
      No-Tree-Checks: true
      No-Try: true
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3041675Reviewed-by: 's avatarNico Hartmann <nicohartmann@chromium.org>
      Commit-Queue: Nico Hartmann <nicohartmann@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#75836}
      c73d759b
    • Leszek Swirski's avatar
      [offthread] Template deserializer on Isolate · e24fa913
      Leszek Swirski authored
      Make the deserializer class templated on Isolate/LocalIsolate. This
      allows the ObjectSerializer to be split into a main-thread and offthread
      variant, with the latter taking a LocalIsolate.
      
      Eventually, we probably want to anyway split off the code-cache de/serializer
      to a separate implementation (for various reasons), and this the only one that
      wants off-thread finalization, and at this point the deserializer can revert
      back to being un-templated, used only for bootstrapping. However, this is the
      simplest way, for now, to enable off-thread deserialization.
      
      Bug: chromium:1075999
      Change-Id: I49c0d2c5409f0aa58183673785296756c3714f22
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2562254Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Reviewed-by: 's avatarDominik Inführ <dinfuehr@chromium.org>
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#75834}
      e24fa913
  3. 10 May, 2021 1 commit
  4. 11 Mar, 2021 3 commits
    • Clemens Backes's avatar
      Reland "[no-wasm] Exclude src/wasm from compilation" · 3f9ff062
      Clemens Backes authored
      This is a reland of 80f5dfda. A condition
      in pipeline.cc was inverted, which lead to a CSA verifier error.
      
      Original change's description:
      > [no-wasm] Exclude src/wasm from compilation
      >
      > This is the biggest chunk, including
      > - all of src/wasm,
      > - torque file for wasm objects,
      > - torque file for wasm builtins,
      > - wasm builtins,
      > - wasm runtime functions,
      > - int64 lowering,
      > - simd scala lowering,
      > - WasmGraphBuilder (TF graph construction for wasm),
      > - wasm frame types,
      > - wasm interrupts,
      > - the JSWasmCall opcode,
      > - wasm backing store allocation.
      >
      > Those components are all recursively entangled, so I found no way to
      > split this change up further.
      >
      > Some includes that were recursively included by wasm headers needed to
      > be added explicitly now.
      >
      > backing-store-unittest.cc is renamed to wasm-backing-store-unittest.cc
      > because it only tests wasm backing stores. This file is excluded from
      > no-wasm builds then.
      >
      > R=jkummerow@chromium.org, jgruber@chromium.org, mlippautz@chromium.org, petermarshall@chromium.org
      >
      > Bug: v8:11238
      > Change-Id: I7558f2d12d2dd6c65128c4de7b79173668c80b2b
      > Cq-Include-Trybots: luci.v8.try:v8_linux64_no_wasm_compile_rel
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2742955
      > Commit-Queue: Clemens Backes <clemensb@chromium.org>
      > Reviewed-by: Peter Marshall <petermarshall@chromium.org>
      > Reviewed-by: Toon Verwaest <verwaest@chromium.org>
      > Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
      > Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#73344}
      
      TBR=jgruber@chromium.org
      
      Bug: v8:11238
      Change-Id: I20bd2847a59c68738b5a336cd42582b7b1499585
      Cq-Include-Trybots: luci.v8.try:v8_linux64_no_wasm_compile_rel
      Cq-Include-Trybots: luci.v8.try:v8_linux_verify_csa_rel_ng
      Cq-Include-Trybots: luci.v8.try:v8_linux64_verify_csa_rel_ng
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2752867Reviewed-by: 's avatarClemens Backes <clemensb@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Commit-Queue: Clemens Backes <clemensb@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#73348}
      3f9ff062
    • Clemens Backes's avatar
      Revert "[no-wasm] Exclude src/wasm from compilation" · 92bc3d38
      Clemens Backes authored
      This reverts commit 80f5dfda.
      
      Reason for revert: Fails CSA verification: https://ci.chromium.org/ui/p/v8/builders/ci/V8%20Linux%20-%20verify%20csa/21766/overview
      
      Original change's description:
      > [no-wasm] Exclude src/wasm from compilation
      >
      > This is the biggest chunk, including
      > - all of src/wasm,
      > - torque file for wasm objects,
      > - torque file for wasm builtins,
      > - wasm builtins,
      > - wasm runtime functions,
      > - int64 lowering,
      > - simd scala lowering,
      > - WasmGraphBuilder (TF graph construction for wasm),
      > - wasm frame types,
      > - wasm interrupts,
      > - the JSWasmCall opcode,
      > - wasm backing store allocation.
      >
      > Those components are all recursively entangled, so I found no way to
      > split this change up further.
      >
      > Some includes that were recursively included by wasm headers needed to
      > be added explicitly now.
      >
      > backing-store-unittest.cc is renamed to wasm-backing-store-unittest.cc
      > because it only tests wasm backing stores. This file is excluded from
      > no-wasm builds then.
      >
      > R=​jkummerow@chromium.org, jgruber@chromium.org, mlippautz@chromium.org, petermarshall@chromium.org
      >
      > Bug: v8:11238
      > Change-Id: I7558f2d12d2dd6c65128c4de7b79173668c80b2b
      > Cq-Include-Trybots: luci.v8.try:v8_linux64_no_wasm_compile_rel
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2742955
      > Commit-Queue: Clemens Backes <clemensb@chromium.org>
      > Reviewed-by: Peter Marshall <petermarshall@chromium.org>
      > Reviewed-by: Toon Verwaest <verwaest@chromium.org>
      > Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
      > Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#73344}
      
      Bug: v8:11238
      Change-Id: I93672002c1faa36bb0bb5b4a9cc2032ee2ccd814
      Cq-Include-Trybots: luci.v8.try:v8_linux64_no_wasm_compile_rel
      No-Presubmit: true
      No-Tree-Checks: true
      No-Try: true
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2752866
      Auto-Submit: Clemens Backes <clemensb@chromium.org>
      Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
      Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
      Cr-Commit-Position: refs/heads/master@{#73346}
      92bc3d38
    • Clemens Backes's avatar
      [no-wasm] Exclude src/wasm from compilation · 80f5dfda
      Clemens Backes authored
      This is the biggest chunk, including
      - all of src/wasm,
      - torque file for wasm objects,
      - torque file for wasm builtins,
      - wasm builtins,
      - wasm runtime functions,
      - int64 lowering,
      - simd scala lowering,
      - WasmGraphBuilder (TF graph construction for wasm),
      - wasm frame types,
      - wasm interrupts,
      - the JSWasmCall opcode,
      - wasm backing store allocation.
      
      Those components are all recursively entangled, so I found no way to
      split this change up further.
      
      Some includes that were recursively included by wasm headers needed to
      be added explicitly now.
      
      backing-store-unittest.cc is renamed to wasm-backing-store-unittest.cc
      because it only tests wasm backing stores. This file is excluded from
      no-wasm builds then.
      
      R=jkummerow@chromium.org, jgruber@chromium.org, mlippautz@chromium.org, petermarshall@chromium.org
      
      Bug: v8:11238
      Change-Id: I7558f2d12d2dd6c65128c4de7b79173668c80b2b
      Cq-Include-Trybots: luci.v8.try:v8_linux64_no_wasm_compile_rel
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2742955
      Commit-Queue: Clemens Backes <clemensb@chromium.org>
      Reviewed-by: 's avatarPeter Marshall <petermarshall@chromium.org>
      Reviewed-by: 's avatarToon Verwaest <verwaest@chromium.org>
      Reviewed-by: 's avatarMichael Lippautz <mlippautz@chromium.org>
      Reviewed-by: 's avatarJakob Kummerow <jkummerow@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#73344}
      80f5dfda
  5. 07 Oct, 2020 1 commit
    • Leszek Swirski's avatar
      Reland^4 "[serializer] Allocate during deserialization" · 3c508b38
      Leszek Swirski authored
      This relands commit 3f4e9bbe.
      which was a reland of c4a062a9
      which was a reland of 28a30c57
      which was a reland of 5d7a29c9
      
      The change had an issue that embedders implementing heap tracing (e.g.
      Unified Heap with Blink) could be passed an uninitialized pointer if
      marking happened during deserialization of an object containing such a
      pointer. Because of the 0xdeadbed0 uninitialized filler value, these
      embedders would then receive the value 0xdeadbed0deadbed0 as the
      'pointer', and crash on dereference.
      
      There is, however, special handling already for null pointers in heap
      tracing, also for dealing with not-yet initialized values. So, we can
      make the uninitialized Smi filler be 0x00000000, and that will make such
      embedded fields have a nullptr representation, making them follow the
      normal uninitialized value bailouts.
      
      In addition, it relands the following dependent changes, which are
      relanding unchanged and are followup performance improvements.
      Relanding them in the same change should allow for cleaner reverts
      should they be needed.
      
      This relands commit 76ad3ab5
      [identity-map] Change resize heuristic
      
      This relands commit 77cc96aa
      [identity-map] Cache the calculated Hash
      
      This relands commit bee5b996
      [serializer] Remove Deserializer::Initialize
      
      This relands commit c8f73f22
      [serializer] Cache instance type in PostProcessNewObject
      
      This relands commit 4e7c99ab
      [identity-map] Remove double-lookups in IdentityMap
      
      Original change's description:
      > Reland^3 "[serializer] Allocate during deserialization"
      >
      > This is a reland of c4a062a9
      > which was a reland of 28a30c57
      > which was a reland of 5d7a29c9
      >
      > Fixes TSAN errors from non-atomic writes in the deserializer. Now all
      > writes are (relaxed) atomic.
      >
      > Original change's description:
      > > Reland^2 "[serializer] Allocate during deserialization"
      > >
      > > This is a reland of 28a30c57
      > > which was a reland of 5d7a29c9
      > >
      > > The crashes were from calling RegisterDeserializerFinished on a null
      > > Isolate pointer, for a deserializer that was never initialised
      > > (specifically, ReadOnlyDeserializer when ROHeap is shared).
      > >
      > > Original change's description:
      > > > Reland "[serializer] Allocate during deserialization"
      > > >
      > > > This is a reland of 5d7a29c9
      > > >
      > > > This reland shuffles around the order of checks in Heap::AllocateRawWith
      > > > to not check the new space addresses until it's known that this is a new
      > > > space allocation. This fixes an UBSan failure during read-only space
      > > > deserialization, which happens before the new space is initialized.
      > > >
      > > > It also fixes some issues discovered by --stress-snapshot, around
      > > > serializing ThinStrings (which are now elided as part of serialization),
      > > > handle counts (I bumped the maximum handle count in that check), and
      > > > clearing map transitions (the map backpointer field needed a Smi
      > > > uninitialized value check).
      > > >
      > > > Original change's description:
      > > > > [serializer] Allocate during deserialization
      > > > >
      > > > > This patch removes the concept of reservations and a specialized
      > > > > deserializer allocator, and instead makes the deserializer allocate
      > > > > directly with the Heap's Allocate method.
      > > > >
      > > > > The major consequence of this is that the GC can now run during
      > > > > deserialization, which means that:
      > > > >
      > > > >   a) Deserialized objects are visible to the GC, and
      > > > >   b) Objects that the deserializer/deserialized objects point to can
      > > > >      move.
      > > > >
      > > > > Point a) is mostly not a problem due to previous work in making
      > > > > deserialized objects "GC valid", i.e. making sure that they have a valid
      > > > > size before any subsequent allocation/safepoint. We now additionally
      > > > > have to initialize the allocated space with a valid tagged value -- this
      > > > > is a magic Smi value to keep "uninitialized" checks simple.
      > > > >
      > > > > Point b) is solved by Handlifying the deserializer. This involves
      > > > > changing any vectors of objects into vectors of Handles, and any object
      > > > > keyed map into an IdentityMap (we can't use Handles as keys because
      > > > > the object's address is no longer a stable hash).
      > > > >
      > > > > Back-references can no longer be direct chunk offsets, so instead the
      > > > > deserializer stores a Handle to each deserialized object, and the
      > > > > backreference is an index into this handle array. This encoding could
      > > > > be optimized in the future with e.g. a second pass over the serialized
      > > > > array which emits a different bytecode for objects that are and aren't
      > > > > back-referenced.
      > > > >
      > > > > Additionally, the slot-walk over objects to initialize them can no
      > > > > longer use absolute slot offsets, as again an object may move and its
      > > > > slot address would become invalid. Now, slots are walked as relative
      > > > > offsets to a Handle to the object, or as absolute slots for the case of
      > > > > root pointers. A concept of "slot accessor" is introduced to share the
      > > > > code between these two modes, and writing the slot (including write
      > > > > barriers) is abstracted into this accessor.
      > > > >
      > > > > Finally, the Code body walk is modified to deserialize all objects
      > > > > referred to by RelocInfos before doing the RelocInfo walk itself. This
      > > > > is because RelocInfoIterator uses raw pointers, so we cannot allocate
      > > > > during a RelocInfo walk.
      > > > >
      > > > > As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      > > > > size rather than byte size -- the size is expected to be tagged-aligned
      > > > > anyway, so now we get an extra few bits in the size encoding.
      > > > >
      > > > > Bug: chromium:1075999
      > > > > Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      > > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      > > > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > > > Cr-Commit-Position: refs/heads/master@{#70229}
      
      Bug: chromium:1075999
      Change-Id: Ib514a4ef16bd02bfb60d046ecbf8fae1ead64a98
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2452689
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70366}
      3c508b38
  6. 05 Oct, 2020 2 commits
    • Adam Klein's avatar
      Revert "Reland^3 "[serializer] Allocate during deserialization"" · a10ec2be
      Adam Klein authored
      This reverts commit 3f4e9bbe, along
      with the following dependent changes (reverted to make this a clean revert):
      76ad3ab5 [identity-map] Change resize heuristic
      77cc96aa [identity-map] Cache the calculated Hash
      bee5b996 [serializer] Remove Deserializer::Initialize
      c8f73f22 [serializer] Cache instance type in PostProcessNewObject
      4e7c99ab [identity-map] Remove double-lookups in IdentityMap
      
      Reason for revert: major crash spike on Canary (https://crbug.com/1135027)
      
      Original change's description:
      > Reland^3 "[serializer] Allocate during deserialization"
      >
      > This is a reland of c4a062a9
      > which was a reland of 28a30c57
      > which was a reland of 5d7a29c9
      >
      > Fixes TSAN errors from non-atomic writes in the deserializer. Now all
      > writes are (relaxed) atomic.
      >
      > Original change's description:
      > > Reland^2 "[serializer] Allocate during deserialization"
      > >
      > > This is a reland of 28a30c57
      > > which was a reland of 5d7a29c9
      > >
      > > The crashes were from calling RegisterDeserializerFinished on a null
      > > Isolate pointer, for a deserializer that was never initialised
      > > (specifically, ReadOnlyDeserializer when ROHeap is shared).
      > >
      > > Original change's description:
      > > > Reland "[serializer] Allocate during deserialization"
      > > >
      > > > This is a reland of 5d7a29c9
      > > >
      > > > This reland shuffles around the order of checks in Heap::AllocateRawWith
      > > > to not check the new space addresses until it's known that this is a new
      > > > space allocation. This fixes an UBSan failure during read-only space
      > > > deserialization, which happens before the new space is initialized.
      > > >
      > > > It also fixes some issues discovered by --stress-snapshot, around
      > > > serializing ThinStrings (which are now elided as part of serialization),
      > > > handle counts (I bumped the maximum handle count in that check), and
      > > > clearing map transitions (the map backpointer field needed a Smi
      > > > uninitialized value check).
      > > >
      > > > Original change's description:
      > > > > [serializer] Allocate during deserialization
      > > > >
      > > > > This patch removes the concept of reservations and a specialized
      > > > > deserializer allocator, and instead makes the deserializer allocate
      > > > > directly with the Heap's Allocate method.
      > > > >
      > > > > The major consequence of this is that the GC can now run during
      > > > > deserialization, which means that:
      > > > >
      > > > >   a) Deserialized objects are visible to the GC, and
      > > > >   b) Objects that the deserializer/deserialized objects point to can
      > > > >      move.
      > > > >
      > > > > Point a) is mostly not a problem due to previous work in making
      > > > > deserialized objects "GC valid", i.e. making sure that they have a valid
      > > > > size before any subsequent allocation/safepoint. We now additionally
      > > > > have to initialize the allocated space with a valid tagged value -- this
      > > > > is a magic Smi value to keep "uninitialized" checks simple.
      > > > >
      > > > > Point b) is solved by Handlifying the deserializer. This involves
      > > > > changing any vectors of objects into vectors of Handles, and any object
      > > > > keyed map into an IdentityMap (we can't use Handles as keys because
      > > > > the object's address is no longer a stable hash).
      > > > >
      > > > > Back-references can no longer be direct chunk offsets, so instead the
      > > > > deserializer stores a Handle to each deserialized object, and the
      > > > > backreference is an index into this handle array. This encoding could
      > > > > be optimized in the future with e.g. a second pass over the serialized
      > > > > array which emits a different bytecode for objects that are and aren't
      > > > > back-referenced.
      > > > >
      > > > > Additionally, the slot-walk over objects to initialize them can no
      > > > > longer use absolute slot offsets, as again an object may move and its
      > > > > slot address would become invalid. Now, slots are walked as relative
      > > > > offsets to a Handle to the object, or as absolute slots for the case of
      > > > > root pointers. A concept of "slot accessor" is introduced to share the
      > > > > code between these two modes, and writing the slot (including write
      > > > > barriers) is abstracted into this accessor.
      > > > >
      > > > > Finally, the Code body walk is modified to deserialize all objects
      > > > > referred to by RelocInfos before doing the RelocInfo walk itself. This
      > > > > is because RelocInfoIterator uses raw pointers, so we cannot allocate
      > > > > during a RelocInfo walk.
      > > > >
      > > > > As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      > > > > size rather than byte size -- the size is expected to be tagged-aligned
      > > > > anyway, so now we get an extra few bits in the size encoding.
      > > > >
      > > > > Bug: chromium:1075999
      > > > > Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      > > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      > > > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > > > Cr-Commit-Position: refs/heads/master@{#70229}
      > > >
      > > > Bug: chromium:1075999
      > > > Change-Id: Ibc77cc48b3440b4a28b09746cfc47e50c340ce54
      > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2440828
      > > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > > Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      > > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > > Cr-Commit-Position: refs/heads/master@{#70267}
      > >
      > > Tbr: jgruber@chromium.org,ulan@chromium.org
      > > Bug: chromium:1075999
      > > Change-Id: Iaa8dc54895866ada0e34a7c9e8fff9ae1cb13f2d
      > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2444991
      > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > Cr-Commit-Position: refs/heads/master@{#70279}
      >
      > Tbr: jgruber@chromium.org,ulan@chromium.org
      > Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_rel_ng,v8_linux64_tsan_no_cm_rel_ng,v8_linux64_tsan_isolates_rel_ng
      > Bug: chromium:1075999
      > Change-Id: I0b9b11644aebc4cc8b07c62a0f765b24e4d73d89
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2445872
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      > Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#70288}
      
      TBR=ulan@chromium.org,jgruber@chromium.org,leszeks@chromium.org,dinfuehr@chromium.org
      
      Bug: chromium:1075999, chromium:1135027
      Change-Id: I5d0d9e49c0302d94ff7291834f5f18e7a0839eb7
      Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_rel_ng,v8_linux64_tsan_no_cm_rel_ng,v8_linux64_tsan_isolates_rel_ng
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2451030Reviewed-by: 's avatarAdam Klein <adamk@chromium.org>
      Commit-Queue: Adam Klein <adamk@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70328}
      a10ec2be
    • Leszek Swirski's avatar
      [serializer] Remove Deserializer::Initialize · bee5b996
      Leszek Swirski authored
      Remove the separate Initialize method from Deserializer, opting instead
      to pass around SnapshotData where appropriate and pass the isolate
      directly into the Deserializer's constructor.
      
      Change-Id: I0092fadd9c81f14b2ce75145fd81af37c3947c65
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2448466
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Commit-Queue: Dan Elphick <delphick@chromium.org>
      Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarDan Elphick <delphick@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70310}
      bee5b996
  7. 02 Oct, 2020 3 commits
    • Leszek Swirski's avatar
      Reland^3 "[serializer] Allocate during deserialization" · 3f4e9bbe
      Leszek Swirski authored
      This is a reland of c4a062a9
      which was a reland of 28a30c57
      which was a reland of 5d7a29c9
      
      Fixes TSAN errors from non-atomic writes in the deserializer. Now all
      writes are (relaxed) atomic.
      
      Original change's description:
      > Reland^2 "[serializer] Allocate during deserialization"
      >
      > This is a reland of 28a30c57
      > which was a reland of 5d7a29c9
      >
      > The crashes were from calling RegisterDeserializerFinished on a null
      > Isolate pointer, for a deserializer that was never initialised
      > (specifically, ReadOnlyDeserializer when ROHeap is shared).
      >
      > Original change's description:
      > > Reland "[serializer] Allocate during deserialization"
      > >
      > > This is a reland of 5d7a29c9
      > >
      > > This reland shuffles around the order of checks in Heap::AllocateRawWith
      > > to not check the new space addresses until it's known that this is a new
      > > space allocation. This fixes an UBSan failure during read-only space
      > > deserialization, which happens before the new space is initialized.
      > >
      > > It also fixes some issues discovered by --stress-snapshot, around
      > > serializing ThinStrings (which are now elided as part of serialization),
      > > handle counts (I bumped the maximum handle count in that check), and
      > > clearing map transitions (the map backpointer field needed a Smi
      > > uninitialized value check).
      > >
      > > Original change's description:
      > > > [serializer] Allocate during deserialization
      > > >
      > > > This patch removes the concept of reservations and a specialized
      > > > deserializer allocator, and instead makes the deserializer allocate
      > > > directly with the Heap's Allocate method.
      > > >
      > > > The major consequence of this is that the GC can now run during
      > > > deserialization, which means that:
      > > >
      > > >   a) Deserialized objects are visible to the GC, and
      > > >   b) Objects that the deserializer/deserialized objects point to can
      > > >      move.
      > > >
      > > > Point a) is mostly not a problem due to previous work in making
      > > > deserialized objects "GC valid", i.e. making sure that they have a valid
      > > > size before any subsequent allocation/safepoint. We now additionally
      > > > have to initialize the allocated space with a valid tagged value -- this
      > > > is a magic Smi value to keep "uninitialized" checks simple.
      > > >
      > > > Point b) is solved by Handlifying the deserializer. This involves
      > > > changing any vectors of objects into vectors of Handles, and any object
      > > > keyed map into an IdentityMap (we can't use Handles as keys because
      > > > the object's address is no longer a stable hash).
      > > >
      > > > Back-references can no longer be direct chunk offsets, so instead the
      > > > deserializer stores a Handle to each deserialized object, and the
      > > > backreference is an index into this handle array. This encoding could
      > > > be optimized in the future with e.g. a second pass over the serialized
      > > > array which emits a different bytecode for objects that are and aren't
      > > > back-referenced.
      > > >
      > > > Additionally, the slot-walk over objects to initialize them can no
      > > > longer use absolute slot offsets, as again an object may move and its
      > > > slot address would become invalid. Now, slots are walked as relative
      > > > offsets to a Handle to the object, or as absolute slots for the case of
      > > > root pointers. A concept of "slot accessor" is introduced to share the
      > > > code between these two modes, and writing the slot (including write
      > > > barriers) is abstracted into this accessor.
      > > >
      > > > Finally, the Code body walk is modified to deserialize all objects
      > > > referred to by RelocInfos before doing the RelocInfo walk itself. This
      > > > is because RelocInfoIterator uses raw pointers, so we cannot allocate
      > > > during a RelocInfo walk.
      > > >
      > > > As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      > > > size rather than byte size -- the size is expected to be tagged-aligned
      > > > anyway, so now we get an extra few bits in the size encoding.
      > > >
      > > > Bug: chromium:1075999
      > > > Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      > > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > > Cr-Commit-Position: refs/heads/master@{#70229}
      > >
      > > Bug: chromium:1075999
      > > Change-Id: Ibc77cc48b3440b4a28b09746cfc47e50c340ce54
      > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2440828
      > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > Cr-Commit-Position: refs/heads/master@{#70267}
      >
      > Tbr: jgruber@chromium.org,ulan@chromium.org
      > Bug: chromium:1075999
      > Change-Id: Iaa8dc54895866ada0e34a7c9e8fff9ae1cb13f2d
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2444991
      > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#70279}
      
      Tbr: jgruber@chromium.org,ulan@chromium.org
      Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_rel_ng,v8_linux64_tsan_no_cm_rel_ng,v8_linux64_tsan_isolates_rel_ng
      Bug: chromium:1075999
      Change-Id: I0b9b11644aebc4cc8b07c62a0f765b24e4d73d89
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2445872
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarDominik Inführ <dinfuehr@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70288}
      3f4e9bbe
    • Clemens Backes's avatar
      Revert "Reland^2 "[serializer] Allocate during deserialization"" · a81da102
      Clemens Backes authored
      This reverts commit c4a062a9.
      
      Reason for revert: TSan issues: https://ci.chromium.org/p/v8/builders/ci/V8%20Linux64%20TSAN/33504
      
      Original change's description:
      > Reland^2 "[serializer] Allocate during deserialization"
      >
      > This is a reland of 28a30c57
      > which was a reland of 5d7a29c9
      >
      > The crashes were from calling RegisterDeserializerFinished on a null
      > Isolate pointer, for a deserializer that was never initialised
      > (specifically, ReadOnlyDeserializer when ROHeap is shared).
      >
      > Original change's description:
      > > Reland "[serializer] Allocate during deserialization"
      > >
      > > This is a reland of 5d7a29c9
      > >
      > > This reland shuffles around the order of checks in Heap::AllocateRawWith
      > > to not check the new space addresses until it's known that this is a new
      > > space allocation. This fixes an UBSan failure during read-only space
      > > deserialization, which happens before the new space is initialized.
      > >
      > > It also fixes some issues discovered by --stress-snapshot, around
      > > serializing ThinStrings (which are now elided as part of serialization),
      > > handle counts (I bumped the maximum handle count in that check), and
      > > clearing map transitions (the map backpointer field needed a Smi
      > > uninitialized value check).
      > >
      > > Original change's description:
      > > > [serializer] Allocate during deserialization
      > > >
      > > > This patch removes the concept of reservations and a specialized
      > > > deserializer allocator, and instead makes the deserializer allocate
      > > > directly with the Heap's Allocate method.
      > > >
      > > > The major consequence of this is that the GC can now run during
      > > > deserialization, which means that:
      > > >
      > > >   a) Deserialized objects are visible to the GC, and
      > > >   b) Objects that the deserializer/deserialized objects point to can
      > > >      move.
      > > >
      > > > Point a) is mostly not a problem due to previous work in making
      > > > deserialized objects "GC valid", i.e. making sure that they have a valid
      > > > size before any subsequent allocation/safepoint. We now additionally
      > > > have to initialize the allocated space with a valid tagged value -- this
      > > > is a magic Smi value to keep "uninitialized" checks simple.
      > > >
      > > > Point b) is solved by Handlifying the deserializer. This involves
      > > > changing any vectors of objects into vectors of Handles, and any object
      > > > keyed map into an IdentityMap (we can't use Handles as keys because
      > > > the object's address is no longer a stable hash).
      > > >
      > > > Back-references can no longer be direct chunk offsets, so instead the
      > > > deserializer stores a Handle to each deserialized object, and the
      > > > backreference is an index into this handle array. This encoding could
      > > > be optimized in the future with e.g. a second pass over the serialized
      > > > array which emits a different bytecode for objects that are and aren't
      > > > back-referenced.
      > > >
      > > > Additionally, the slot-walk over objects to initialize them can no
      > > > longer use absolute slot offsets, as again an object may move and its
      > > > slot address would become invalid. Now, slots are walked as relative
      > > > offsets to a Handle to the object, or as absolute slots for the case of
      > > > root pointers. A concept of "slot accessor" is introduced to share the
      > > > code between these two modes, and writing the slot (including write
      > > > barriers) is abstracted into this accessor.
      > > >
      > > > Finally, the Code body walk is modified to deserialize all objects
      > > > referred to by RelocInfos before doing the RelocInfo walk itself. This
      > > > is because RelocInfoIterator uses raw pointers, so we cannot allocate
      > > > during a RelocInfo walk.
      > > >
      > > > As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      > > > size rather than byte size -- the size is expected to be tagged-aligned
      > > > anyway, so now we get an extra few bits in the size encoding.
      > > >
      > > > Bug: chromium:1075999
      > > > Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      > > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      > > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > > Cr-Commit-Position: refs/heads/master@{#70229}
      > >
      > > Bug: chromium:1075999
      > > Change-Id: Ibc77cc48b3440b4a28b09746cfc47e50c340ce54
      > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2440828
      > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > Cr-Commit-Position: refs/heads/master@{#70267}
      >
      > Tbr: jgruber@chromium.org,ulan@chromium.org
      > Bug: chromium:1075999
      > Change-Id: Iaa8dc54895866ada0e34a7c9e8fff9ae1cb13f2d
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2444991
      > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#70279}
      
      TBR=ulan@chromium.org,jgruber@chromium.org,leszeks@chromium.org
      
      Change-Id: Ib2f01db4cd9b55639d6a4af971bda865edb45e84
      No-Presubmit: true
      No-Tree-Checks: true
      No-Try: true
      Bug: chromium:1075999
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2445250Reviewed-by: 's avatarClemens Backes <clemensb@chromium.org>
      Commit-Queue: Clemens Backes <clemensb@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70280}
      a81da102
    • Leszek Swirski's avatar
      Reland^2 "[serializer] Allocate during deserialization" · c4a062a9
      Leszek Swirski authored
      This is a reland of 28a30c57
      which was a reland of 5d7a29c9
      
      The crashes were from calling RegisterDeserializerFinished on a null
      Isolate pointer, for a deserializer that was never initialised
      (specifically, ReadOnlyDeserializer when ROHeap is shared).
      
      Original change's description:
      > Reland "[serializer] Allocate during deserialization"
      >
      > This is a reland of 5d7a29c9
      >
      > This reland shuffles around the order of checks in Heap::AllocateRawWith
      > to not check the new space addresses until it's known that this is a new
      > space allocation. This fixes an UBSan failure during read-only space
      > deserialization, which happens before the new space is initialized.
      >
      > It also fixes some issues discovered by --stress-snapshot, around
      > serializing ThinStrings (which are now elided as part of serialization),
      > handle counts (I bumped the maximum handle count in that check), and
      > clearing map transitions (the map backpointer field needed a Smi
      > uninitialized value check).
      >
      > Original change's description:
      > > [serializer] Allocate during deserialization
      > >
      > > This patch removes the concept of reservations and a specialized
      > > deserializer allocator, and instead makes the deserializer allocate
      > > directly with the Heap's Allocate method.
      > >
      > > The major consequence of this is that the GC can now run during
      > > deserialization, which means that:
      > >
      > >   a) Deserialized objects are visible to the GC, and
      > >   b) Objects that the deserializer/deserialized objects point to can
      > >      move.
      > >
      > > Point a) is mostly not a problem due to previous work in making
      > > deserialized objects "GC valid", i.e. making sure that they have a valid
      > > size before any subsequent allocation/safepoint. We now additionally
      > > have to initialize the allocated space with a valid tagged value -- this
      > > is a magic Smi value to keep "uninitialized" checks simple.
      > >
      > > Point b) is solved by Handlifying the deserializer. This involves
      > > changing any vectors of objects into vectors of Handles, and any object
      > > keyed map into an IdentityMap (we can't use Handles as keys because
      > > the object's address is no longer a stable hash).
      > >
      > > Back-references can no longer be direct chunk offsets, so instead the
      > > deserializer stores a Handle to each deserialized object, and the
      > > backreference is an index into this handle array. This encoding could
      > > be optimized in the future with e.g. a second pass over the serialized
      > > array which emits a different bytecode for objects that are and aren't
      > > back-referenced.
      > >
      > > Additionally, the slot-walk over objects to initialize them can no
      > > longer use absolute slot offsets, as again an object may move and its
      > > slot address would become invalid. Now, slots are walked as relative
      > > offsets to a Handle to the object, or as absolute slots for the case of
      > > root pointers. A concept of "slot accessor" is introduced to share the
      > > code between these two modes, and writing the slot (including write
      > > barriers) is abstracted into this accessor.
      > >
      > > Finally, the Code body walk is modified to deserialize all objects
      > > referred to by RelocInfos before doing the RelocInfo walk itself. This
      > > is because RelocInfoIterator uses raw pointers, so we cannot allocate
      > > during a RelocInfo walk.
      > >
      > > As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      > > size rather than byte size -- the size is expected to be tagged-aligned
      > > anyway, so now we get an extra few bits in the size encoding.
      > >
      > > Bug: chromium:1075999
      > > Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > Cr-Commit-Position: refs/heads/master@{#70229}
      >
      > Bug: chromium:1075999
      > Change-Id: Ibc77cc48b3440b4a28b09746cfc47e50c340ce54
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2440828
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#70267}
      
      Tbr: jgruber@chromium.org,ulan@chromium.org
      Bug: chromium:1075999
      Change-Id: Iaa8dc54895866ada0e34a7c9e8fff9ae1cb13f2d
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2444991Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70279}
      c4a062a9
  8. 01 Oct, 2020 2 commits
    • Zhi An Ng's avatar
      Revert "Reland "[serializer] Allocate during deserialization"" · c7c0e790
      Zhi An Ng authored
      This reverts commit 28a30c57.
      
      Reason for revert: Broke Test262 https://ci.chromium.org/p/v8/builders/ci/V8%20Linux%20-%20shared/38638?
      
      Original change's description:
      > Reland "[serializer] Allocate during deserialization"
      >
      > This is a reland of 5d7a29c9
      >
      > This reland shuffles around the order of checks in Heap::AllocateRawWith
      > to not check the new space addresses until it's known that this is a new
      > space allocation. This fixes an UBSan failure during read-only space
      > deserialization, which happens before the new space is initialized.
      >
      > It also fixes some issues discovered by --stress-snapshot, around
      > serializing ThinStrings (which are now elided as part of serialization),
      > handle counts (I bumped the maximum handle count in that check), and
      > clearing map transitions (the map backpointer field needed a Smi
      > uninitialized value check).
      >
      > Original change's description:
      > > [serializer] Allocate during deserialization
      > >
      > > This patch removes the concept of reservations and a specialized
      > > deserializer allocator, and instead makes the deserializer allocate
      > > directly with the Heap's Allocate method.
      > >
      > > The major consequence of this is that the GC can now run during
      > > deserialization, which means that:
      > >
      > >   a) Deserialized objects are visible to the GC, and
      > >   b) Objects that the deserializer/deserialized objects point to can
      > >      move.
      > >
      > > Point a) is mostly not a problem due to previous work in making
      > > deserialized objects "GC valid", i.e. making sure that they have a valid
      > > size before any subsequent allocation/safepoint. We now additionally
      > > have to initialize the allocated space with a valid tagged value -- this
      > > is a magic Smi value to keep "uninitialized" checks simple.
      > >
      > > Point b) is solved by Handlifying the deserializer. This involves
      > > changing any vectors of objects into vectors of Handles, and any object
      > > keyed map into an IdentityMap (we can't use Handles as keys because
      > > the object's address is no longer a stable hash).
      > >
      > > Back-references can no longer be direct chunk offsets, so instead the
      > > deserializer stores a Handle to each deserialized object, and the
      > > backreference is an index into this handle array. This encoding could
      > > be optimized in the future with e.g. a second pass over the serialized
      > > array which emits a different bytecode for objects that are and aren't
      > > back-referenced.
      > >
      > > Additionally, the slot-walk over objects to initialize them can no
      > > longer use absolute slot offsets, as again an object may move and its
      > > slot address would become invalid. Now, slots are walked as relative
      > > offsets to a Handle to the object, or as absolute slots for the case of
      > > root pointers. A concept of "slot accessor" is introduced to share the
      > > code between these two modes, and writing the slot (including write
      > > barriers) is abstracted into this accessor.
      > >
      > > Finally, the Code body walk is modified to deserialize all objects
      > > referred to by RelocInfos before doing the RelocInfo walk itself. This
      > > is because RelocInfoIterator uses raw pointers, so we cannot allocate
      > > during a RelocInfo walk.
      > >
      > > As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      > > size rather than byte size -- the size is expected to be tagged-aligned
      > > anyway, so now we get an extra few bits in the size encoding.
      > >
      > > Bug: chromium:1075999
      > > Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      > > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > > Cr-Commit-Position: refs/heads/master@{#70229}
      >
      > Bug: chromium:1075999
      > Change-Id: Ibc77cc48b3440b4a28b09746cfc47e50c340ce54
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2440828
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#70267}
      
      TBR=ulan@chromium.org,jgruber@chromium.org,leszeks@chromium.org
      
      Change-Id: Ieed68332ef6a7ad36db061e3f48be0f28673d7a2
      No-Presubmit: true
      No-Tree-Checks: true
      No-Try: true
      Bug: chromium:1075999
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2441608Reviewed-by: 's avatarZhi An Ng <zhin@chromium.org>
      Commit-Queue: Zhi An Ng <zhin@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70268}
      c7c0e790
    • Leszek Swirski's avatar
      Reland "[serializer] Allocate during deserialization" · 28a30c57
      Leszek Swirski authored
      This is a reland of 5d7a29c9
      
      This reland shuffles around the order of checks in Heap::AllocateRawWith
      to not check the new space addresses until it's known that this is a new
      space allocation. This fixes an UBSan failure during read-only space
      deserialization, which happens before the new space is initialized.
      
      It also fixes some issues discovered by --stress-snapshot, around
      serializing ThinStrings (which are now elided as part of serialization),
      handle counts (I bumped the maximum handle count in that check), and
      clearing map transitions (the map backpointer field needed a Smi
      uninitialized value check).
      
      Original change's description:
      > [serializer] Allocate during deserialization
      >
      > This patch removes the concept of reservations and a specialized
      > deserializer allocator, and instead makes the deserializer allocate
      > directly with the Heap's Allocate method.
      >
      > The major consequence of this is that the GC can now run during
      > deserialization, which means that:
      >
      >   a) Deserialized objects are visible to the GC, and
      >   b) Objects that the deserializer/deserialized objects point to can
      >      move.
      >
      > Point a) is mostly not a problem due to previous work in making
      > deserialized objects "GC valid", i.e. making sure that they have a valid
      > size before any subsequent allocation/safepoint. We now additionally
      > have to initialize the allocated space with a valid tagged value -- this
      > is a magic Smi value to keep "uninitialized" checks simple.
      >
      > Point b) is solved by Handlifying the deserializer. This involves
      > changing any vectors of objects into vectors of Handles, and any object
      > keyed map into an IdentityMap (we can't use Handles as keys because
      > the object's address is no longer a stable hash).
      >
      > Back-references can no longer be direct chunk offsets, so instead the
      > deserializer stores a Handle to each deserialized object, and the
      > backreference is an index into this handle array. This encoding could
      > be optimized in the future with e.g. a second pass over the serialized
      > array which emits a different bytecode for objects that are and aren't
      > back-referenced.
      >
      > Additionally, the slot-walk over objects to initialize them can no
      > longer use absolute slot offsets, as again an object may move and its
      > slot address would become invalid. Now, slots are walked as relative
      > offsets to a Handle to the object, or as absolute slots for the case of
      > root pointers. A concept of "slot accessor" is introduced to share the
      > code between these two modes, and writing the slot (including write
      > barriers) is abstracted into this accessor.
      >
      > Finally, the Code body walk is modified to deserialize all objects
      > referred to by RelocInfos before doing the RelocInfo walk itself. This
      > is because RelocInfoIterator uses raw pointers, so we cannot allocate
      > during a RelocInfo walk.
      >
      > As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      > size rather than byte size -- the size is expected to be tagged-aligned
      > anyway, so now we get an extra few bits in the size encoding.
      >
      > Bug: chromium:1075999
      > Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#70229}
      
      Bug: chromium:1075999
      Change-Id: Ibc77cc48b3440b4a28b09746cfc47e50c340ce54
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2440828
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70267}
      28a30c57
  9. 30 Sep, 2020 2 commits
    • Leszek Swirski's avatar
      Revert "[serializer] Allocate during deserialization" · 74f3665c
      Leszek Swirski authored
      This reverts commit 5d7a29c9.
      
      Reason for revert: UBSan -- https://ci.chromium.org/p/v8/builders/ci/V8%20Linux64%20UBSan/13100
      
      Original change's description:
      > [serializer] Allocate during deserialization
      >
      > This patch removes the concept of reservations and a specialized
      > deserializer allocator, and instead makes the deserializer allocate
      > directly with the Heap's Allocate method.
      >
      > The major consequence of this is that the GC can now run during
      > deserialization, which means that:
      >
      >   a) Deserialized objects are visible to the GC, and
      >   b) Objects that the deserializer/deserialized objects point to can
      >      move.
      >
      > Point a) is mostly not a problem due to previous work in making
      > deserialized objects "GC valid", i.e. making sure that they have a valid
      > size before any subsequent allocation/safepoint. We now additionally
      > have to initialize the allocated space with a valid tagged value -- this
      > is a magic Smi value to keep "uninitialized" checks simple.
      >
      > Point b) is solved by Handlifying the deserializer. This involves
      > changing any vectors of objects into vectors of Handles, and any object
      > keyed map into an IdentityMap (we can't use Handles as keys because
      > the object's address is no longer a stable hash).
      >
      > Back-references can no longer be direct chunk offsets, so instead the
      > deserializer stores a Handle to each deserialized object, and the
      > backreference is an index into this handle array. This encoding could
      > be optimized in the future with e.g. a second pass over the serialized
      > array which emits a different bytecode for objects that are and aren't
      > back-referenced.
      >
      > Additionally, the slot-walk over objects to initialize them can no
      > longer use absolute slot offsets, as again an object may move and its
      > slot address would become invalid. Now, slots are walked as relative
      > offsets to a Handle to the object, or as absolute slots for the case of
      > root pointers. A concept of "slot accessor" is introduced to share the
      > code between these two modes, and writing the slot (including write
      > barriers) is abstracted into this accessor.
      >
      > Finally, the Code body walk is modified to deserialize all objects
      > referred to by RelocInfos before doing the RelocInfo walk itself. This
      > is because RelocInfoIterator uses raw pointers, so we cannot allocate
      > during a RelocInfo walk.
      >
      > As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      > size rather than byte size -- the size is expected to be tagged-aligned
      > anyway, so now we get an extra few bits in the size encoding.
      >
      > Bug: chromium:1075999
      > Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      > Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#70229}
      
      TBR=ulan@chromium.org,jgruber@chromium.org,leszeks@chromium.org
      
      Change-Id: I2bd792a24861e8f54897e51522769b50f8f814e2
      No-Presubmit: true
      No-Tree-Checks: true
      No-Try: true
      Bug: chromium:1075999
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2440827
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarLeszek Swirski <leszeks@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70231}
      74f3665c
    • Leszek Swirski's avatar
      [serializer] Allocate during deserialization · 5d7a29c9
      Leszek Swirski authored
      This patch removes the concept of reservations and a specialized
      deserializer allocator, and instead makes the deserializer allocate
      directly with the Heap's Allocate method.
      
      The major consequence of this is that the GC can now run during
      deserialization, which means that:
      
        a) Deserialized objects are visible to the GC, and
        b) Objects that the deserializer/deserialized objects point to can
           move.
      
      Point a) is mostly not a problem due to previous work in making
      deserialized objects "GC valid", i.e. making sure that they have a valid
      size before any subsequent allocation/safepoint. We now additionally
      have to initialize the allocated space with a valid tagged value -- this
      is a magic Smi value to keep "uninitialized" checks simple.
      
      Point b) is solved by Handlifying the deserializer. This involves
      changing any vectors of objects into vectors of Handles, and any object
      keyed map into an IdentityMap (we can't use Handles as keys because
      the object's address is no longer a stable hash).
      
      Back-references can no longer be direct chunk offsets, so instead the
      deserializer stores a Handle to each deserialized object, and the
      backreference is an index into this handle array. This encoding could
      be optimized in the future with e.g. a second pass over the serialized
      array which emits a different bytecode for objects that are and aren't
      back-referenced.
      
      Additionally, the slot-walk over objects to initialize them can no
      longer use absolute slot offsets, as again an object may move and its
      slot address would become invalid. Now, slots are walked as relative
      offsets to a Handle to the object, or as absolute slots for the case of
      root pointers. A concept of "slot accessor" is introduced to share the
      code between these two modes, and writing the slot (including write
      barriers) is abstracted into this accessor.
      
      Finally, the Code body walk is modified to deserialize all objects
      referred to by RelocInfos before doing the RelocInfo walk itself. This
      is because RelocInfoIterator uses raw pointers, so we cannot allocate
      during a RelocInfo walk.
      
      As a drive-by, the VariableRawData bytecode is tweaked to use tagged
      size rather than byte size -- the size is expected to be tagged-aligned
      anyway, so now we get an extra few bits in the size encoding.
      
      Bug: chromium:1075999
      Change-Id: I672c42f553f2669888cc5e35d692c1b8ece1845e
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2404451
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70229}
      5d7a29c9
  10. 22 Sep, 2020 1 commit
  11. 14 Aug, 2020 1 commit
    • Leszek Swirski's avatar
      [offthread] Change OffThreadIsolate to LocalIsolate · f1589bbe
      Leszek Swirski authored
      This patch introduces a new LocalIsolate and LocalFactory, which use
      LocalHeap and replace OffThreadIsolate and OffThreadFactory. This allows
      us to remove those classes, as well as the related OffThreadSpace,
      OffThreadLargeObjectSpace, OffThreadHeap, and OffThreadTransferHandle.
      OffThreadLogger becomes LocalLogger.
      
      LocalHeap behaves more like Heap than OffThreadHeap did, so this allows
      us to additionally remove the concept of "Finish" and "Publish" that the
      OffThreadIsolate had, and allows us to internalize strings directly with
      the newly-concurrent string table (where the implementation can now move
      to FactoryBase).
      
      This patch also removes the off-thread support from the deserializer
      entirely, as well as removing the LocalIsolateWrapper which allowed
      run-time distinction between Isolate and OffThreadIsolate. LocalHeap
      doesn't support the reservation model used by the deserializer, and we
      will likely move the deserializer to use LocalIsolate unconditionally
      once we figure out the details of how to do this.
      
      Bug: chromium:1011762
      
      Change-Id: I1a1a0a72952b19a8a4c167c11a863c153a1252fc
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2315990
      Commit-Queue: Andreas Haas <ahaas@chromium.org>
      Auto-Submit: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarAndreas Haas <ahaas@chromium.org>
      Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Reviewed-by: 's avatarDominik Inführ <dinfuehr@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#69397}
      f1589bbe
  12. 06 Aug, 2020 1 commit
    • Leszek Swirski's avatar
      [runtime] Move string table off-heap · 1546be9c
      Leszek Swirski authored
      Changes the isolate's string table into an off-heap structure. This
      allows the string table to be resized without allocating on the V8 heap,
      and potentially triggering a GC. This allows existing strings to be
      inserted into the string table without requiring allocation.
      
      This has two important benefits:
      
        1) It allows the deserializer to insert strings directly into the
           string table, rather than having to defer string insertion until
           deserialization completes.
      
        2) It simplifies the concurrent string table lookup to allow resizing
           the table inside the write lock, therefore eliminating the race
           where two concurrent lookups could both resize the table.
      
      The off-heap string table has the following properties:
      
        1) The general hashmap behaviour matches the HashTable, i.e. open
           addressing, power-of-two sized, quadratic probing. This could, of
           course, now be changed.
      
        2) The empty and deleted sentinels are changed to Smi 0 and 1,
           respectively, to make those comparisons a bit cheaper and not
           require roots access.
      
        3) When the HashTable is resized, the old elements array is kept
           alive in a linked list of previous arrays, so that concurrent
           lookups don't lose the data they're accessing. This linked list
           is cleared by the GC, as then we know that all threads are in
           a safepoint.
      
        4) The GC treats the hash table entries as weak roots, and only walks
           them for non-live reference clearing and for evacuation.
      
        5) Since there is no longer a FixedArray to serialize for the startup
           snapshot, there is now a custom serialization of the string table,
           and the string table root is considered unserializable during weak
           root iteration. As a bonus, the custom serialization is more
           efficient, as it skips non-string entries.
      
      As a drive-by, rename LookupStringExists_NoAllocate to
      TryStringToIndexOrLookupExisting, to make it clearer that it returns
      a non-string for the case when the string is an array index. As another
      drive-by, extract StringSet into a separate header.
      
      Bug: v8:10729
      Change-Id: I9c990fb2d74d1fe222920408670974a70e969bca
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2339104
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#69270}
      1546be9c
  13. 23 Jul, 2020 1 commit
  14. 22 Jul, 2020 2 commits
  15. 21 Jul, 2020 1 commit
  16. 27 May, 2020 1 commit
  17. 14 May, 2020 1 commit
    • Leszek Swirski's avatar
      [offthread] Add off thread deserialization · 595609fb
      Leszek Swirski authored
      Add a new OffThreadObjectDeserializer, which can deserialize a snapshot
      into an OffThreadIsolate.
      
      This involves templating the Deserializer base class on Isolate, and
      amending OffThreadHeap to be able to create Reservations same as the
      main-thread Heap can. Various off-thread incompatible methods are
      stubbed out as UNREACHABLE in OffThreadIsolate overloads.
      
      There is currently no API entry into the off-thread deserialization, but
      under --stress-background-compile it now runs the CodeDeserializer (i.e.
      code cache deserialization) in a background thread.
      
      Bug: chromium:1075999
      
      Change-Id: I2453f51ae31df4d4b6aa94b0804a9d6d3a03781e
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2172741
      Commit-Queue: Leszek Swirski <leszeks@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Reviewed-by: 's avatarIgor Sheludko <ishell@chromium.org>
      Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#67799}
      595609fb
  18. 11 May, 2020 1 commit
  19. 08 May, 2020 1 commit
  20. 04 May, 2020 1 commit
  21. 22 Apr, 2020 1 commit
    • Dan Elphick's avatar
      [snapshot] Rename partial snapshot to context snapshot · 71bd50e1
      Dan Elphick authored
      The partial snapshot/serializer/deserializer are renamed to context *,
      while the partial snapshot cache is renamed to startup object cache in
      line with the read-only object cache (as this better reflects where it
      lives and what it does).
      
      To avoid a gap in the file history due to renaming both the files and
      identifiers simulataneously, this leaves all the partial-*.* files in
      place. They will be renamed in a follow-up CL.
      
      Bug: v8:10416
      Change-Id: I5ef41cad751aaa24b35ee2b3c72bd0295832f2c6
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2144115
      Commit-Queue: Dan Elphick <delphick@chromium.org>
      Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
      Auto-Submit: Dan Elphick <delphick@chromium.org>
      Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#67306}
      71bd50e1
  22. 11 Feb, 2020 1 commit
  23. 06 Feb, 2020 1 commit
    • Steve Blackburn's avatar
      Serialization without heap assumptions. · 1d2c043a
      Steve Blackburn authored
      Currently back references to regular objects are encoded in terms of
      a relative address, index by chunk index and chunk offset.   This
      approach has the advantage of avoiding the need for a table of
      back-references at deserialization time, but makes strong assumptions
      about the way objects are organized in memory (for example, this will
      not work if the allocator uses a free list rather than a bump pointer).
      
      I did some quick measurements and found that the absolute number of
      objects and back-references is low, suggesting that simply indexing
      objects would work with little (or no) observable impact on peak
      memory use during deserialization.   Indexing only back referenced
      objects is not implemented in this simple CL, but could fairly easily
      be added.
      
      Given that the existing mechanism will remain in place, I have
      implemented the object index by simply making chunk size one, so
      every object lives on its own chunk (with offset zero).   This is
      the moral equivalent to indexing each object but is a more minimal
      change.  Directly encoding an object index will be more efficient,
      the trade off made here is just to keep the change absolutely minimal.
      
      If using an object index becomes the default, this can be optimized
      first by only using an index for each object that is actually back-
      referenced (about half of all objects in my measurements), and more
      aggressively, a technique like register allocation could be used at
      serialization time to limit the number of indices to the maximum
      number of outstanding back-references at any time (basically a live-
      range analysis of back-references).
      
      Bug: v8:9533
      Change-Id: I1b7ae87e954f67f6405c2bbdf3b4a4f385af8579
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2030742
      Commit-Queue: Steve Blackburn <steveblackburn@google.com>
      Reviewed-by: 's avatarUlan Degenbaev <ulan@chromium.org>
      Reviewed-by: 's avatarJakob Gruber <jgruber@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#66154}
      1d2c043a
  24. 04 Feb, 2020 1 commit
  25. 19 Dec, 2019 2 commits
  26. 17 Dec, 2019 1 commit
  27. 13 Dec, 2019 2 commits
  28. 15 Nov, 2019 1 commit
  29. 10 Sep, 2019 1 commit