When we used the tasks API, we distributed the compilation deadlines
such that different tasks would finish (and publish) at slightly
different times. When moving to the jobs API, this feature was lost, and
all tasks now only publish once all compilation is done. This creates
major contention and synchronization overhead after compilation, which
can take roughly as long as the actual concurrent compilation.

This CL reintroduces a mechanism for ensuring regular publishing. We
choose a limit which is high enough to ensure that we don't publish too
often (because that comes with overhead), but low enough to ensure that
not too many units are published at the end, when all compilation is
done. At that point, there will still be some contention.
Also, choose a slightly different limit per task to ensure proper
distribution of publishing over time (different tasks are likely to
publish at different times).

This removes the last remaining regressions when compiling big modules
via the jobs API. In local measurements, it makes us even faster than
with the tasks API.

R=ahaas@chromium.org

Bug: chromium:1101340, chromium:1113234
Change-Id: I504f32606b8ad31a951449709cf407c471fa9b25
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2375196Reviewed-by: Andreas Haas <ahaas@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69566}