This is the second CL in a line of two (see crrev.com/c/2835237) to
bring write-protection to the WebAssembly code space. The previous CL
changed the page permissions from W^X (only either writable or
executable can be active, but never both) to write-protection (due to
concurrent execution in the main thread). However, write-protection
still did not work, because in several places the code space is
modified without properly switching it to writable beforehand.

This CL fixes --wasm-write-protect-code-memory such that it can now be
enabled again (with potentially high overhead due to frequent page
protection switches). For that, it adds the missing switching to
writable by adding {NativeModuleModificationScope} objects (similar to
the already existing {CodeSpaceWriteScope} objects for Apple M1
hardware).

This CL also fixes a race condition between checking for the current
writable permission and actually setting the permission, by protecting
the counter of currently active writers with the same lock as the
{WasmCodeAllocator} itself. (Before multi-threaded compilation, this
was not necessary.)

Finally, this CL also changes the {Mutex} protecting the
{WasmCodeAllocator} to a {RecursiveMutex} because it can be requested
multiple times in the call hierarchy of the same thread, which would
cause a deadlock otherwise. Since {TryLock()} of a {RecursiveMutex}
never fails, this also removes the (now failing) DCHECKs.

R=clemensb@chromium.org
CC=​​jkummerow@chromium.org

Bug: v8:11663
Change-Id: I4db27ad0a9348021b0b663dbe88b3432a4d8d6b5
Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_rel_ng
Cq-Include-Trybots: luci.v8.try:v8_linux64_tsan_isolates_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2835238
Commit-Queue: Daniel Lehmann <dlehmann@google.com>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#74163}