Commit 79aee245 authored by jgruber's avatar jgruber Committed by Commit Bot

[builtins] Implement lazy deserialization for TFJ builtins

This adds support for lazy deserialization of JS-linkage (TFJ) builtins,
still gated behind the --lazy-deserialization flag. If enabled, we
proceed as follows:

During isolate initialization, only eager builtins are deserialized. All
references to lazy builtins are replaced by the DeserializeLazy builtin.
In particular, this happens in the builtin table (Builtins::builtins_)
and in SharedFunctionInfo objects.

When calling into a not-yet deserialized function (i.e. the JSFunction's
code object is the DeserializeLazy builtin), the DeserializeLazy builtin
takes over.  It checks the builtin table to see if the target builtin
(determined by looking at the builtin id stored on the
SharedFunctionInfo) has already been deserialized. If so, it simply
copies the builtin code object to the JSFunction and SharedFunctionInfo.
Otherwise, we enter Runtime::kDeserializeLazy to deserialize the
builtin.

With --lazy-deserialization, isolate deserialization is 11% faster
(1.5ms vs.  1.7ms), and code_space->Size() is 33% lower (984K vs.
1475K).

Moving relocation infos & handler tables out of the partial snapshot
cache would additionally let us save up to 30K per isolate. Adding code
stubs to that list increases further potential savings to 262K.

Bug: v8:6624
Change-Id: I0ac7d05d165d2466998269bd431ac076a311cbeb
Reviewed-on: https://chromium-review.googlesource.com/649166
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: 's avatarYang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47818}
parent bec24736
......@@ -813,6 +813,10 @@ ExternalReference ExternalReference::isolate_address(Isolate* isolate) {
return ExternalReference(isolate);
}
ExternalReference ExternalReference::builtins_address(Isolate* isolate) {
return ExternalReference(isolate->builtins()->builtins_table_address());
}
ExternalReference ExternalReference::interpreter_dispatch_table_address(
Isolate* isolate) {
return ExternalReference(isolate->interpreter()->dispatch_table_address());
......
......@@ -830,6 +830,9 @@ class ExternalReference BASE_EMBEDDED {
// Isolate as an external reference.
static ExternalReference isolate_address(Isolate* isolate);
// The builtins table as an external reference, used by lazy deserialization.
static ExternalReference builtins_address(Isolate* isolate);
// One-of-a-kind references. These references are not part of a general
// pattern. This means that they have to be added to the
// ExternalReferenceTable in serialize.cc manually.
......
......@@ -31,33 +31,50 @@ TF_BUILTIN(DeserializeLazy, CodeStubAssembler) {
LoadObjectField(function, JSFunction::kSharedFunctionInfoOffset);
CSA_ASSERT(this, IsSharedFunctionInfo(shared));
Node* shared_code = LoadObjectField(shared, SharedFunctionInfo::kCodeOffset);
CSA_ASSERT(this, IsCodeMap(LoadMap(shared_code)));
Node* shared_code_builtin_id = LoadObjectField(
shared_code, Code::kBuiltinIndexOffset, MachineType::Int32());
// The shared function info stores the target builtin id that needs to be
// deserialized (originally set by Factory::NewSharedFunctionInfo).
Node* target_builtin_id =
LoadObjectField(shared, SharedFunctionInfo::kFunctionDataOffset);
CSA_ASSERT(this, TaggedIsSmi(target_builtin_id));
CSA_ASSERT(this, SmiNotEqual(target_builtin_id,
SmiConstant(Builtins::kDeserializeLazy)));
// The builtin may already have been deserialized. If that is the case, it is
// stored in the builtins table, and we can copy to correct code object to
// both the shared function info and function without calling into runtime.
//
// Otherwise, we need to call into runtime to deserialize.
Label copy_from_builtins_table(this), deserialize_in_runtime(this), out(this);
Node* code_object_in_table;
// Check what's in the builtins table.
{
Node* builtins_table =
ExternalConstant(ExternalReference::builtins_address(isolate()));
Node* builtin_ptr =
IntPtrAdd(UncheckedCast<IntPtrT>(builtins_table),
SmiUntag(SmiShl(target_builtin_id, kPointerSizeLog2)));
// TODO(6624): Once lazy deserialization has been implemented, add a path
// here that checks whether the appropriate builtin has already been
// deserialized into the builtin table. If so, copy it into the function &
// shared function info here instead of going through runtime.
code_object_in_table = Load(MachineType::AnyTagged(), builtin_ptr);
CSA_ASSERT(this, IsCodeMap(LoadMap(code_object_in_table)));
// It could be that the shared function info already has a deserialized copy
// of the builtin. In that case, simply copy the code over to the function and
// tail-call into it. Otherwise, we need to call into runtime to deserialize.
Node* maybe_deserialized_builtin_id = LoadObjectField(
code_object_in_table, Code::kBuiltinIndexOffset, MachineType::Int32());
Label copy_from_shared(this), deserialize_in_runtime(this), out(this);
Branch(Word32Equal(shared_code_builtin_id,
Int32Constant(Builtins::kDeserializeLazy)),
&deserialize_in_runtime, &copy_from_shared);
Branch(Word32Equal(maybe_deserialized_builtin_id,
Int32Constant(Builtins::kDeserializeLazy)),
&deserialize_in_runtime, &copy_from_builtins_table);
}
BIND(&copy_from_shared);
BIND(&copy_from_builtins_table);
{
CSA_ASSERT(this, Int32GreaterThanOrEqual(shared_code_builtin_id,
Int32Constant(0)));
CSA_ASSERT(this, Int32LessThan(shared_code_builtin_id,
Int32Constant(Builtins::builtin_count)));
StoreObjectField(function, JSFunction::kCodeOffset, shared_code);
CSA_ASSERT(this, SmiGreaterThanOrEqual(target_builtin_id, SmiConstant(0)));
CSA_ASSERT(this, SmiLessThan(target_builtin_id,
SmiConstant(Builtins::builtin_count)));
StoreObjectField(shared, SharedFunctionInfo::kCodeOffset,
code_object_in_table);
StoreObjectField(function, JSFunction::kCodeOffset, code_object_in_table);
Goto(&out);
}
......@@ -72,17 +89,16 @@ TF_BUILTIN(DeserializeLazy, CodeStubAssembler) {
Node* function_code_builtin_id = LoadObjectField(
function_code, Code::kBuiltinIndexOffset, MachineType::Int32());
Node* function_builtin_id =
LoadObjectField(shared, SharedFunctionInfo::kFunctionDataOffset);
CSA_ASSERT(this, TaggedIsSmi(function_builtin_id));
CSA_ASSERT(this, Word32Equal(function_code_builtin_id,
SmiToWord32(function_builtin_id)));
SmiToWord32(target_builtin_id)));
#endif
Goto(&out);
}
// Finally, tail-call into the deserialized code, passing along any existing
// arguments on the stack unmodified.
BIND(&out);
TailCallStub(CodeFactory::Call(isolate()), context, function, argc);
}
......
......@@ -86,6 +86,10 @@ class Builtins {
V8_EXPORT_PRIVATE Handle<Code> builtin_handle(Name name);
// Used by lazy deserialization to determine whether a given builtin has been
// deserialized. See the DeserializeLazy builtin.
Object** builtins_table_address() { return &builtins_[0]; }
V8_EXPORT_PRIVATE static Callable CallableFor(Isolate* isolate, Name name);
static int GetStackParameterCount(Name name);
......
......@@ -89,6 +89,7 @@ void ExternalReferenceTable::AddReferences(Isolate* isolate) {
Add(ExternalReference::address_of_one_half().address(),
"LDoubleConstant::one_half");
Add(ExternalReference::isolate_address(isolate).address(), "isolate");
Add(ExternalReference::builtins_address(isolate).address(), "builtins");
Add(ExternalReference::interpreter_dispatch_table_address(isolate).address(),
"Interpreter::dispatch_table_address");
Add(ExternalReference::bytecode_size_table_address(isolate).address(),
......
......@@ -17,6 +17,7 @@
#include "src/messages.h"
#include "src/parsing/parse-info.h"
#include "src/parsing/parsing.h"
#include "src/snapshot/snapshot.h"
#include "src/wasm/wasm-module.h"
namespace v8 {
......@@ -510,24 +511,33 @@ RUNTIME_FUNCTION(Runtime_DeserializeLazy) {
DCHECK(FLAG_lazy_deserialization);
// TODO(6624): Lazy-deserialize instead of simply loading from the builtins
// list. In fact, this should never be called if the builtin has already
// been deserialized.
Handle<SharedFunctionInfo> shared(function->shared(), isolate);
int builtin_id = shared->lazy_deserialization_builtin_id();
#ifdef DEBUG
Builtins::Name builtin_name = static_cast<Builtins::Name>(builtin_id);
// At this point, the builtins table should definitely have DeserializeLazy
// set at the position of the target builtin. Also, we should never lazily
// deserialize DeserializeLazy.
DCHECK_NE(Builtins::kDeserializeLazy, builtin_name);
DCHECK_EQ(Builtins::kDeserializeLazy,
isolate->builtins()->builtin(builtin_name)->builtin_index());
// The DeserializeLazy builtin tail-calls the deserialized builtin. This only
// works with JS-linkage.
DCHECK(Builtins::IsLazy(builtin_id));
DCHECK_EQ(Builtins::TFJ, Builtins::KindOf(builtin_id));
#endif // DEBUG
if (FLAG_trace_lazy_deserialization) {
PrintF("Lazy-deserializing %s\n", Builtins::name(builtin_id));
}
Code* code =
isolate->builtins()->builtin(static_cast<Builtins::Name>(builtin_id));
Code* code = Snapshot::DeserializeBuiltin(isolate, builtin_id);
DCHECK_EQ(builtin_id, code->builtin_index());
DCHECK_EQ(code, isolate->builtins()->builtin(builtin_name));
shared->set_code(code);
function->set_code(code);
......
......@@ -32,7 +32,8 @@ class DeserializingBuiltinScope {
DISALLOW_COPY_AND_ASSIGN(DeserializingBuiltinScope)
};
BuiltinDeserializer::BuiltinDeserializer(const BuiltinSnapshotData* data)
BuiltinDeserializer::BuiltinDeserializer(Isolate* isolate,
const BuiltinSnapshotData* data)
: Deserializer(data, false) {
// We may have to relax this at some point to pack reloc infos and handler
// tables into the builtin blob (instead of the partial snapshot cache).
......@@ -42,19 +43,23 @@ BuiltinDeserializer::BuiltinDeserializer(const BuiltinSnapshotData* data)
DCHECK_EQ(Builtins::builtin_count, builtin_offsets_.length());
DCHECK(std::is_sorted(builtin_offsets_.begin(), builtin_offsets_.end()));
builtin_sizes_ = ExtractBuiltinSizes();
DCHECK_EQ(Builtins::builtin_count, builtin_sizes_.size());
Initialize(isolate);
}
void BuiltinDeserializer::DeserializeEagerBuiltins() {
DCHECK(!AllowHeapAllocation::IsAllowed());
DCHECK_EQ(0, source()->position());
// TODO(jgruber): Replace lazy builtins with DeserializeLazy.
Builtins* builtins = isolate()->builtins();
for (int i = 0; i < Builtins::builtin_count; i++) {
builtins->set_builtin(i, DeserializeBuiltin(i));
if (FLAG_lazy_deserialization && Builtins::IsLazy(i)) {
// Do nothing. These builtins have been replaced by DeserializeLazy in
// InitializeBuiltinsTable.
DCHECK_EQ(builtins->builtin(Builtins::kDeserializeLazy),
builtins->builtin(static_cast<Builtins::Name>(i)));
} else {
builtins->set_builtin(i, DeserializeBuiltin(i));
}
}
#ifdef DEBUG
......@@ -114,23 +119,28 @@ uint32_t BuiltinDeserializer::ExtractBuiltinSize(int builtin_id) {
return result;
}
std::vector<uint32_t> BuiltinDeserializer::ExtractBuiltinSizes() {
std::vector<uint32_t> result;
result.reserve(Builtins::builtin_count);
for (int i = 0; i < Builtins::builtin_count; i++) {
result.push_back(ExtractBuiltinSize(i));
}
return result;
}
Heap::Reservation BuiltinDeserializer::CreateReservationsForEagerBuiltins() {
DCHECK(ReservesOnlyCodeSpace());
Heap::Reservation result;
// DeserializeLazy is always the first reservation (to simplify logic in
// InitializeBuiltinsTable).
{
DCHECK(!Builtins::IsLazy(Builtins::kDeserializeLazy));
uint32_t builtin_size = ExtractBuiltinSize(Builtins::kDeserializeLazy);
DCHECK_LE(builtin_size, MemoryAllocator::PageAreaSize(CODE_SPACE));
result.push_back({builtin_size, nullptr, nullptr});
}
for (int i = 0; i < Builtins::builtin_count; i++) {
// TODO(jgruber): Skip lazy builtins.
if (i == Builtins::kDeserializeLazy) continue;
// Skip lazy builtins. These will be replaced by the DeserializeLazy code
// object in InitializeBuiltinsTable and thus require no reserved space.
if (FLAG_lazy_deserialization && Builtins::IsLazy(i)) continue;
const uint32_t builtin_size = builtin_sizes_[i];
uint32_t builtin_size = ExtractBuiltinSize(i);
DCHECK_LE(builtin_size, MemoryAllocator::PageAreaSize(CODE_SPACE));
result.push_back({builtin_size, nullptr, nullptr});
}
......@@ -138,40 +148,83 @@ Heap::Reservation BuiltinDeserializer::CreateReservationsForEagerBuiltins() {
return result;
}
void BuiltinDeserializer::InitializeBuiltinFromReservation(
const Heap::Chunk& chunk, int builtin_id) {
DCHECK_EQ(ExtractBuiltinSize(builtin_id), chunk.size);
DCHECK_EQ(chunk.size, chunk.end - chunk.start);
SkipList::Update(chunk.start, chunk.size);
isolate()->builtins()->set_builtin(builtin_id,
HeapObject::FromAddress(chunk.start));
}
void BuiltinDeserializer::InitializeBuiltinsTable(
const Heap::Reservation& reservation) {
DCHECK(!AllowHeapAllocation::IsAllowed());
// Other builtins can be replaced by DeserializeLazy so it may not be lazy.
DCHECK(!Builtins::IsLazy(Builtins::kDeserializeLazy));
Builtins* builtins = isolate()->builtins();
int reservation_index = 0;
for (int i = 0; i < Builtins::builtin_count; i++) {
// TODO(jgruber): Replace lazy builtins with DeserializeLazy.
Address start = reservation[reservation_index].start;
DCHECK_EQ(builtin_sizes_[i], reservation[reservation_index].size);
DCHECK_EQ(builtin_sizes_[i], reservation[reservation_index].end - start);
// Other builtins can be replaced by DeserializeLazy so it may not be lazy.
// It always occupies the first reservation slot.
{
DCHECK(!Builtins::IsLazy(Builtins::kDeserializeLazy));
InitializeBuiltinFromReservation(reservation[reservation_index],
Builtins::kDeserializeLazy);
reservation_index++;
}
builtins->set_builtin(i, HeapObject::FromAddress(start));
Code* deserialize_lazy = builtins->builtin(Builtins::kDeserializeLazy);
reservation_index++;
for (int i = 0; i < Builtins::builtin_count; i++) {
if (i == Builtins::kDeserializeLazy) continue;
if (FLAG_lazy_deserialization && Builtins::IsLazy(i)) {
builtins->set_builtin(i, deserialize_lazy);
} else {
InitializeBuiltinFromReservation(reservation[reservation_index], i);
reservation_index++;
}
}
DCHECK_EQ(reservation.size(), reservation_index);
}
void BuiltinDeserializer::ReserveAndInitializeBuiltinsTableForBuiltin(
int builtin_id) {
DCHECK(AllowHeapAllocation::IsAllowed());
DCHECK(isolate()->builtins()->is_initialized());
DCHECK(Builtins::IsBuiltinId(builtin_id));
DCHECK_NE(Builtins::kDeserializeLazy, builtin_id);
DCHECK_EQ(Builtins::kDeserializeLazy,
isolate()
->builtins()
->builtin(static_cast<Builtins::Name>(builtin_id))
->builtin_index());
const uint32_t builtin_size = ExtractBuiltinSize(builtin_id);
DCHECK_LE(builtin_size, MemoryAllocator::PageAreaSize(CODE_SPACE));
Handle<HeapObject> o =
isolate()->factory()->NewFillerObject(builtin_size, false, CODE_SPACE);
// Note: After this point and until deserialization finishes, heap allocation
// is disallowed. We currently can't safely assert this since we'd need to
// pass the DisallowHeapAllocation scope out of this function.
// Write the allocated filler object into the builtins table. It will be
// returned by our custom Allocate method below once needed.
isolate()->builtins()->set_builtin(builtin_id, *o);
}
Address BuiltinDeserializer::Allocate(int space_index, int size) {
DCHECK_EQ(CODE_SPACE, space_index);
DCHECK_EQ(ExtractBuiltinSize(current_builtin_id_), size);
Object* obj = isolate()->builtins()->builtin(
static_cast<Builtins::Name>(current_builtin_id_));
DCHECK(Internals::HasHeapObjectTag(obj));
Address address = HeapObject::cast(obj)->address();
SkipList::Update(address, size);
return address;
return HeapObject::cast(obj)->address();
}
} // namespace internal
......
......@@ -16,16 +16,16 @@ class BuiltinSnapshotData;
// Deserializes the builtins blob.
class BuiltinDeserializer final : public Deserializer {
public:
explicit BuiltinDeserializer(const BuiltinSnapshotData* data);
// Expose Deserializer::Initialize.
using Deserializer::Initialize;
BuiltinDeserializer(Isolate* isolate, const BuiltinSnapshotData* data);
// Builtins deserialization is tightly integrated with deserialization of the
// startup blob. In particular, we need to ensure that no GC can occur
// between startup- and builtins deserialization, as all builtins have been
// pre-allocated and their pointers may not be invalidated.
void DeserializeEagerBuiltins();
// Deserializes the single given builtin. Assumes that reservations have
// already been allocated.
Code* DeserializeBuiltin(int builtin_id);
// These methods are used to pre-allocate builtin objects prior to
......@@ -35,10 +35,18 @@ class BuiltinDeserializer final : public Deserializer {
Heap::Reservation CreateReservationsForEagerBuiltins();
void InitializeBuiltinsTable(const Heap::Reservation& reservation);
// Creates reservations and initializes the builtins table in preparation for
// lazily deserializing a single builtin.
void ReserveAndInitializeBuiltinsTableForBuiltin(int builtin_id);
private:
// Extracts the size builtin Code objects (baked into the snapshot).
uint32_t ExtractBuiltinSize(int builtin_id);
std::vector<uint32_t> ExtractBuiltinSizes();
// Used after memory allocation prior to isolate initialization, to register
// the newly created object in code space and add it to the builtins table.
void InitializeBuiltinFromReservation(const Heap::Chunk& chunk,
int builtin_id);
// Allocation works differently here than in other deserializers. Instead of
// a statically-known memory area determined at serialization-time, our
......@@ -62,10 +70,6 @@ class BuiltinDeserializer final : public Deserializer {
static const int kNoBuiltinId = -1;
int current_builtin_id_ = kNoBuiltinId;
// The sizes of each builtin Code object in its deserialized state. This list
// is used to determine required space prior to deserialization.
std::vector<uint32_t> builtin_sizes_;
// The offsets of each builtin within the serialized data. Equivalent to
// BuiltinSerializer::builtin_offsets_ but on the deserialization side.
Vector<const uint32_t> builtin_offsets_;
......
......@@ -9,6 +9,7 @@
#include "src/api.h"
#include "src/base/platform/platform.h"
#include "src/objects-inl.h"
#include "src/snapshot/builtin-deserializer.h"
#include "src/snapshot/builtin-serializer.h"
#include "src/snapshot/partial-deserializer.h"
#include "src/snapshot/snapshot-source-sink.h"
......@@ -85,6 +86,34 @@ MaybeHandle<Context> Snapshot::NewContextFromSnapshot(
return result;
}
// static
Code* Snapshot::DeserializeBuiltin(Isolate* isolate, int builtin_id) {
base::ElapsedTimer timer;
if (FLAG_profile_deserialization) timer.Start();
const v8::StartupData* blob = isolate->snapshot_blob();
Vector<const byte> builtin_data = Snapshot::ExtractBuiltinData(blob);
BuiltinSnapshotData builtin_snapshot_data(builtin_data);
BuiltinDeserializer builtin_deserializer(isolate, &builtin_snapshot_data);
builtin_deserializer.ReserveAndInitializeBuiltinsTableForBuiltin(builtin_id);
DisallowHeapAllocation no_gc;
Code* code = builtin_deserializer.DeserializeBuiltin(builtin_id);
DCHECK_EQ(code, isolate->builtins()->builtin(
static_cast<Builtins::Name>(builtin_id)));
if (FLAG_profile_deserialization) {
double ms = timer.Elapsed().InMillisecondsF();
int bytes = code->Size();
PrintF("[Deserializing builtin %s (%d bytes) took %0.3f ms]\n",
Builtins::name(builtin_id), bytes, ms);
}
return code;
}
void ProfileDeserialization(
const SnapshotData* startup_snapshot, const SnapshotData* builtin_snapshot,
const std::vector<SnapshotData*>& context_snapshots) {
......
......@@ -86,22 +86,33 @@ class BuiltinSnapshotData final : public SnapshotData {
class Snapshot : public AllStatic {
public:
// ---------------- Deserialization ----------------
// Initialize the Isolate from the internal snapshot. Returns false if no
// snapshot could be found.
static bool Initialize(Isolate* isolate);
// Create a new context using the internal partial snapshot.
static MaybeHandle<Context> NewContextFromSnapshot(
Isolate* isolate, Handle<JSGlobalProxy> global_proxy,
size_t context_index,
v8::DeserializeEmbedderFieldsCallback embedder_fields_deserializer);
static bool HasContextSnapshot(Isolate* isolate, size_t index);
// Deserializes a single given builtin code object. Intended to be called at
// runtime after the isolate (and the builtins table) has been fully
// initialized.
static Code* DeserializeBuiltin(Isolate* isolate, int builtin_id);
// ---------------- Helper methods ----------------
static bool HasContextSnapshot(Isolate* isolate, size_t index);
static bool EmbedsScript(Isolate* isolate);
// To be implemented by the snapshot source.
static const v8::StartupData* DefaultSnapshotBlob();
// ---------------- Serialization ----------------
static v8::StartupData CreateSnapshotBlob(
const SnapshotData* startup_snapshot,
const BuiltinSnapshotData* builtin_snapshot,
......
......@@ -15,8 +15,7 @@ namespace internal {
void StartupDeserializer::DeserializeInto(Isolate* isolate) {
Initialize(isolate);
BuiltinDeserializer builtin_deserializer(builtin_data_);
builtin_deserializer.Initialize(isolate);
BuiltinDeserializer builtin_deserializer(isolate, builtin_data_);
if (!Deserializer::ReserveSpace(this, &builtin_deserializer)) {
V8::FatalProcessOutOfMemory("StartupDeserializer");
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment