• Mircea Trofin's avatar
    [wasm] Initialize parallel jobs with less memory. · b29bfffd
    Mircea Trofin authored
    Avoid constructing zones and large zone objects when initializing
    WasmCompilationUnit. The main reason we did that is so we can cache
    the CEntryStub node, which requires a code object, obtainable only
    on the main thread. We need that value, however, on background threads,
    which is also where we need the aforementioned large objects. We only
    need that for the WasmCompilationUnits being currently compiled, which
    is a number proportional to the number of background threads provided
    by the embedder. Specifically, one zone is needed only for the duration
    of the background compilation, while the second zone needs to survive 
    past that, so the compilation results may be committed to the GC heap
    as Code objects.
    
    The problem with these large objects is that the first allocation
    in a Zone is at minimum 8KB. We used to allocate 2 zones. For
    modules with 200K functions, that means 3.2GB of memory pre-allocated
    before any of it is actually needed.
    
    This change attaches a Handle to the CEntryStub on the WasmCompilationUnits,
    and delays zone creation to when needed. The change also adds a way to 
    cache CEntryStubs in a JSGraph from a given Code handle - limited to the
    scenario needed by wasm (and removable once we get wasm off the GC heap,
    which subsumes removing this dependency on CEntryStubs)
    
    An additional constraint for this change is that we want it to be easily 
    back-mergeable to address chromium:723899.
    
    For the wasm payload in question, collecting the max memory used by d8
    using /usr/bin/time --format='(%Xtext+%Ddata %Mmax)', we get the 
    following numbers (in KB):
    
    - unchanged: 3307480
    - patch 1: 1807140 (45% reduction)
    - patch 3: 1230320 (62% reduction from first)
    - patch 5/6: 519368 (84% reduction from first)
    
    Bug: chomium:732010, chromium:723899
    Change-Id: I45b96792daf8a9c8dc47d45fb52da75945a41401
    Reviewed-on: https://chromium-review.googlesource.com/530193
    Commit-Queue: Mircea Trofin <mtrofin@chromium.org>
    Reviewed-by: 's avatarAndreas Haas <ahaas@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#45880}
    b29bfffd
Name
Last commit
Last update
..
arm Loading commit data...
arm64 Loading commit data...
ia32 Loading commit data...
mips Loading commit data...
mips64 Loading commit data...
ppc Loading commit data...
s390 Loading commit data...
x64 Loading commit data...
x87 Loading commit data...
OWNERS Loading commit data...
STYLE Loading commit data...
access-builder.cc Loading commit data...
access-builder.h Loading commit data...
access-info.cc Loading commit data...
access-info.h Loading commit data...
all-nodes.cc Loading commit data...
all-nodes.h Loading commit data...
ast-graph-builder.cc Loading commit data...
ast-graph-builder.h Loading commit data...
ast-loop-assignment-analyzer.cc Loading commit data...
ast-loop-assignment-analyzer.h Loading commit data...
basic-block-instrumentor.cc Loading commit data...
basic-block-instrumentor.h Loading commit data...
branch-elimination.cc Loading commit data...
branch-elimination.h Loading commit data...
bytecode-analysis.cc Loading commit data...
bytecode-analysis.h Loading commit data...
bytecode-graph-builder.cc Loading commit data...
bytecode-graph-builder.h Loading commit data...
bytecode-liveness-map.cc Loading commit data...
bytecode-liveness-map.h Loading commit data...
c-linkage.cc Loading commit data...
checkpoint-elimination.cc Loading commit data...
checkpoint-elimination.h Loading commit data...
code-assembler.cc Loading commit data...
code-assembler.h Loading commit data...
code-generator-impl.h Loading commit data...
code-generator.cc Loading commit data...
code-generator.h Loading commit data...
common-node-cache.cc Loading commit data...
common-node-cache.h Loading commit data...
common-operator-reducer.cc Loading commit data...
common-operator-reducer.h Loading commit data...
common-operator.cc Loading commit data...
common-operator.h Loading commit data...
compiler-source-position-table.cc Loading commit data...
compiler-source-position-table.h Loading commit data...
control-builders.cc Loading commit data...
control-builders.h Loading commit data...
control-equivalence.cc Loading commit data...
control-equivalence.h Loading commit data...
control-flow-optimizer.cc Loading commit data...
control-flow-optimizer.h Loading commit data...
dead-code-elimination.cc Loading commit data...
dead-code-elimination.h Loading commit data...
diamond.h Loading commit data...
effect-control-linearizer.cc Loading commit data...
effect-control-linearizer.h Loading commit data...
escape-analysis-reducer.cc Loading commit data...
escape-analysis-reducer.h Loading commit data...
escape-analysis.cc Loading commit data...
escape-analysis.h Loading commit data...
frame-elider.cc Loading commit data...
frame-elider.h Loading commit data...
frame-states.cc Loading commit data...
frame-states.h Loading commit data...
frame.cc Loading commit data...
frame.h Loading commit data...
gap-resolver.cc Loading commit data...
gap-resolver.h Loading commit data...
graph-assembler.cc Loading commit data...
graph-assembler.h Loading commit data...
graph-reducer.cc Loading commit data...
graph-reducer.h Loading commit data...
graph-trimmer.cc Loading commit data...
graph-trimmer.h Loading commit data...
graph-visualizer.cc Loading commit data...
graph-visualizer.h Loading commit data...
graph.cc Loading commit data...
graph.h Loading commit data...
instruction-codes.h Loading commit data...
instruction-scheduler.cc Loading commit data...
instruction-scheduler.h Loading commit data...
instruction-selector-impl.h Loading commit data...
instruction-selector.cc Loading commit data...
instruction-selector.h Loading commit data...
instruction.cc Loading commit data...
instruction.h Loading commit data...
int64-lowering.cc Loading commit data...
int64-lowering.h Loading commit data...
js-builtin-reducer.cc Loading commit data...
js-builtin-reducer.h Loading commit data...
js-call-reducer.cc Loading commit data...
js-call-reducer.h Loading commit data...
js-context-specialization.cc Loading commit data...
js-context-specialization.h Loading commit data...
js-create-lowering.cc Loading commit data...
js-create-lowering.h Loading commit data...
js-frame-specialization.cc Loading commit data...
js-frame-specialization.h Loading commit data...
js-generic-lowering.cc Loading commit data...
js-generic-lowering.h Loading commit data...
js-graph.cc Loading commit data...
js-graph.h Loading commit data...
js-inlining-heuristic.cc Loading commit data...
js-inlining-heuristic.h Loading commit data...
js-inlining.cc Loading commit data...
js-inlining.h Loading commit data...
js-intrinsic-lowering.cc Loading commit data...
js-intrinsic-lowering.h Loading commit data...
js-native-context-specialization.cc Loading commit data...
js-native-context-specialization.h Loading commit data...
js-operator.cc Loading commit data...
js-operator.h Loading commit data...
js-type-hint-lowering.cc Loading commit data...
js-type-hint-lowering.h Loading commit data...
js-typed-lowering.cc Loading commit data...
js-typed-lowering.h Loading commit data...
jump-threading.cc Loading commit data...
jump-threading.h Loading commit data...
linkage.cc Loading commit data...
linkage.h Loading commit data...
live-range-separator.cc Loading commit data...
live-range-separator.h Loading commit data...
liveness-analyzer.cc Loading commit data...
liveness-analyzer.h Loading commit data...
load-elimination.cc Loading commit data...
load-elimination.h Loading commit data...
loop-analysis.cc Loading commit data...
loop-analysis.h Loading commit data...
loop-peeling.cc Loading commit data...
loop-peeling.h Loading commit data...
loop-variable-optimizer.cc Loading commit data...
loop-variable-optimizer.h Loading commit data...
machine-graph-verifier.cc Loading commit data...
machine-graph-verifier.h Loading commit data...
machine-operator-reducer.cc Loading commit data...
machine-operator-reducer.h Loading commit data...
machine-operator.cc Loading commit data...
machine-operator.h Loading commit data...
memory-optimizer.cc Loading commit data...
memory-optimizer.h Loading commit data...
move-optimizer.cc Loading commit data...
move-optimizer.h Loading commit data...
node-aux-data.h Loading commit data...
node-cache.cc Loading commit data...
node-cache.h Loading commit data...
node-marker.cc Loading commit data...
node-marker.h Loading commit data...
node-matchers.cc Loading commit data...
node-matchers.h Loading commit data...
node-properties.cc Loading commit data...
node-properties.h Loading commit data...
node.cc Loading commit data...
node.h Loading commit data...
opcodes.cc Loading commit data...
opcodes.h Loading commit data...
operation-typer.cc Loading commit data...
operation-typer.h Loading commit data...
operator-properties.cc Loading commit data...
operator-properties.h Loading commit data...
operator.cc Loading commit data...
operator.h Loading commit data...
osr.cc Loading commit data...
osr.h Loading commit data...
pipeline-statistics.cc Loading commit data...
pipeline-statistics.h Loading commit data...
pipeline.cc Loading commit data...
pipeline.h Loading commit data...
raw-machine-assembler.cc Loading commit data...
raw-machine-assembler.h Loading commit data...
redundancy-elimination.cc Loading commit data...
redundancy-elimination.h Loading commit data...
register-allocator-verifier.cc Loading commit data...
register-allocator-verifier.h Loading commit data...
register-allocator.cc Loading commit data...
register-allocator.h Loading commit data...
representation-change.cc Loading commit data...
representation-change.h Loading commit data...
schedule.cc Loading commit data...
schedule.h Loading commit data...
scheduler.cc Loading commit data...
scheduler.h Loading commit data...
select-lowering.cc Loading commit data...
select-lowering.h Loading commit data...
simd-scalar-lowering.cc Loading commit data...
simd-scalar-lowering.h Loading commit data...
simplified-lowering.cc Loading commit data...
simplified-lowering.h Loading commit data...
simplified-operator-reducer.cc Loading commit data...
simplified-operator-reducer.h Loading commit data...
simplified-operator.cc Loading commit data...
simplified-operator.h Loading commit data...
state-values-utils.cc Loading commit data...
state-values-utils.h Loading commit data...
store-store-elimination.cc Loading commit data...
store-store-elimination.h Loading commit data...
tail-call-optimization.cc Loading commit data...
tail-call-optimization.h Loading commit data...
type-cache.cc Loading commit data...
type-cache.h Loading commit data...
typed-optimization.cc Loading commit data...
typed-optimization.h Loading commit data...
typer.cc Loading commit data...
typer.h Loading commit data...
types.cc Loading commit data...
types.h Loading commit data...
unwinding-info-writer.h Loading commit data...
value-numbering-reducer.cc Loading commit data...
value-numbering-reducer.h Loading commit data...
verifier.cc Loading commit data...
verifier.h Loading commit data...
wasm-compiler.cc Loading commit data...
wasm-compiler.h Loading commit data...
wasm-linkage.cc Loading commit data...
zone-stats.cc Loading commit data...
zone-stats.h Loading commit data...