[heap] Scavenger completes sweeping if possible

Replacing EnsureSweepingCompleted() with DrainSweepingWorklistForSpace()
caused sweeping to be completed later. This resulted in longer scavenger
pauses, since the scavenger had to stop & resume sweeping more often.
To avoid this, the scavenger now completes sweeping when all pages are
swept.

Bug: chromium:1106295, v8:10315
Change-Id: Ia46efdfb1f2492e9228f0349936be32a467f502b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2301935Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68900}

src/heap/paged-spaces.cc

@@ -909,11 +909,6 @@ bool PagedSpace::RawRefillLabMain(int size_in_bytes, AllocationOrigin origin) {
   MarkCompactCollector* collector = heap()->mark_compact_collector();
   // Sweeping is still in progress.
   if (collector->sweeping_in_progress()) {
-    if (FLAG_concurrent_sweeping && !is_compaction_space() &&
-        !collector->sweeper()->AreSweeperTasksRunning()) {
-      collector->DrainSweepingWorklistForSpace(identity());
-    }
-
     // First try to refill the free-list, concurrent sweeper threads
     // may have freed some objects in the meantime.
     RefillFreeList();

src/heap/scavenger.cc

@@ -267,6 +267,17 @@ void ScavengerCollector::CollectGarbage() {
 
   {
     Sweeper* sweeper = heap_->mark_compact_collector()->sweeper();
+
+    // Try to finish sweeping here, such that the following code doesn't need to
+    // pause & resume sweeping.
+    if (sweeper->sweeping_in_progress() && FLAG_concurrent_sweeping &&
+        !sweeper->AreSweeperTasksRunning()) {
+      // At this point we know that all concurrent sweeping tasks have run
+      // out-of-work and quit: all pages are swept. The main thread still needs
+      // to complete sweeping though.
+      heap_->mark_compact_collector()->EnsureSweepingCompleted();
+    }
+
     // Pause the concurrent sweeper.
     Sweeper::PauseOrCompleteScope pause_scope(sweeper);
     // Filter out pages from the sweeper that need to be processed for old to