• Mark Mentovai's avatar
    mac-arm64: Provide an arm64-native ninja-mac for builds in ¼ of the time · 4005bb74
    Mark Mentovai authored
    ninja-mac is expanded to become a universal (“fat”) executable. The
    existing x86_64 slice was taken from the x86_64 ninja-mac as it
    previously existed in this location, and a new arm64 slice was added.
    
    This is an aid to native development on arm64. In macOS, subprocesses
    inherit their parent’s “binpref,” meaning that children of a translated
    process will, by default, also be translated, if there’s a choice
    between native and translated. Where ninja-mac is x86_64-only and runs
    translated on arm64, all tools it invokes will also run translated, even
    if native versions are available. This has an impact on any part of the
    toolchain for which universal versions are available: in Chrome’s case,
    tools such as ld64 are affected. In Crashpad’s build, which uses the
    compiler provided by Xcode, it even affects the compiler, leading to
    significant differences in build time on arm64.
    
    mark@arm-and-hammer zsh% file $(which ninja-mac)
    …/ninja-mac: Mach-O 64-bit executable x86_64
    mark@arm-and-hammer zsh% time ninja -C out/release
    ninja: Entering directory `out/release'
    [1/1] Regenerating ninja files
    [511/511] LINK ./crashpad_minidump_test
    ninja -C out/release  1384.03s user 111.05s system 393% cpu 6:19.53 total
    
    mark@arm-and-hammer zsh% file $(which ninja-mac)
    …/ninja-mac: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
    …/ninja-mac (for architecture x86_64):	Mach-O 64-bit executable x86_64
    …/ninja-mac (for architecture arm64):	Mach-O 64-bit executable arm64
    mark@arm-and-hammer zsh% time ninja -C out/release
    ninja: Entering directory `out/release'
    [1/1] Regenerating ninja files
    [511/511] LINK ./crashpad_minidump_test
    ninja -C out/release  606.80s user 34.71s system 701% cpu 1:31.41 total
    
    This differs from the previous attempt, e64cd3ae
    (https://chromium-review.googlesource.com/c/2288417) in that the arm64
    version was built with Xcode 12.2b1 (12B5018i). The linker in this
    version of Xcode produces linker-signed output for arm64 by default. As
    of macOS 11.0db6 (20A5364e), all arm64 code must be signed. More
    information at
    https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-universal-apps-beta-release-notes#Code-Signing.
    In the previous attempt, an older Xcode was used and the output was not
    signed at all. Because it did not run on systems upgraded to 11.0db6,
    this version of ninja-mac was backed out in a070dd18
    (https://chromium-review.googlesource.com/c/2393775).
    
    To create this ninja-mac:
    
    % git clone https://github.com/ninja-build/ninja.git
    % cd ninja
    % git checkout v1.8.2
    % mkdir ../build
    % cd ../build
    % LDFLAGS=-Wl,-s ../ninja/configure.py --bootstrap
    % lipo -create -output ninja-mac $(which ninja-mac) ninja
    
    An explanation of the LDFLAGS: e64cd3ae used “strip” on the output,
    matching the stripped x86_64 slice, but “strip” is not currently
    compatible with linker-signed executables. This is documented at
    https://developer.apple.com/documentation/xcode-release-notes/xcode-12_2-beta-release-notes#Apple-Clang-Compiler.
    I was unable to use the recommended workaround in this case. Rather than
    leave the output unstripped, I asked ld64 to strip symbols using its -s
    option, which it believes is obsolete, but in fact provides the intended
    behavior.
    
    The x86_64 slice is unsigned, as it was previously. The new arm64 slice
    is linker-signed with an ad-hoc signature. The x86_64 slice was built
    with the macOS 10.12 SDK and runs on Mac OS X 10.6 or later. The arm64
    slice was built with and has a minimum runtime requirement of macOS
    11.0. In practice, the difference doesn’t matter, as the 11.0 SDK is
    required to build for arm64, and no macOS version earlier than 11.0 will
    run on arm64.
    
    Bug: chromium:1103315
    Change-Id: I9f201d0d6f476e187acc0008681c3fd2c1a72aa0
    Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2427051Reviewed-by: 's avatarNico Weber <thakis@chromium.org>
    Commit-Queue: Mark Mentovai <mark@chromium.org>
    4005bb74
Name
Last commit
Last update
bootstrap Loading commit data...
fetch_configs Loading commit data...
git-templates Loading commit data...
infra Loading commit data...
man Loading commit data...
python-bin Loading commit data...
recipes Loading commit data...
testing_support Loading commit data...
tests Loading commit data...
third_party Loading commit data...
win_toolchain Loading commit data...
zsh-goodies Loading commit data...
.cipd_impl.ps1 Loading commit data...
.gitattributes Loading commit data...
.gitignore Loading commit data...
.style.yapf Loading commit data...
.vpython Loading commit data...
.vpython3 Loading commit data...
CROS_OWNERS Loading commit data...
GOMA_OWNERS Loading commit data...
LICENSE Loading commit data...
LUCI_OWNERS Loading commit data...
OWNERS Loading commit data...
PRESUBMIT.py Loading commit data...
README.gclient.md Loading commit data...
README.git-cl.md Loading commit data...
README.md Loading commit data...
WATCHLISTS Loading commit data...
auth.py Loading commit data...
autoninja Loading commit data...
autoninja.bat Loading commit data...
autoninja.py Loading commit data...
bb Loading commit data...
bb.bat Loading commit data...
bootstrap_python3 Loading commit data...
breakpad.py Loading commit data...
cbuildbot Loading commit data...
chrome_set_ver Loading commit data...
cipd Loading commit data...
cipd.bat Loading commit data...
cipd_bin_setup.bat Loading commit data...
cipd_bin_setup.sh Loading commit data...
cipd_client_version Loading commit data...
cipd_client_version.digests Loading commit data...
cipd_manifest.txt Loading commit data...
cipd_manifest.versions Loading commit data...
cit Loading commit data...
cit.bat Loading commit data...
cit.py Loading commit data...
clang-format Loading commit data...
clang-format.bat Loading commit data...
clang_format.py Loading commit data...
clang_format_merge_driver Loading commit data...
clang_format_merge_driver.bat Loading commit data...
clang_format_merge_driver.py Loading commit data...
codereview.settings Loading commit data...
compile_single_file Loading commit data...
compile_single_file.bat Loading commit data...
compile_single_file.py Loading commit data...
cpplint.bat Loading commit data...
cpplint.py Loading commit data...
cpplint_chromium.py Loading commit data...
cros Loading commit data...
cros_sdk Loading commit data...
crosjobs Loading commit data...
detect_host_arch.py Loading commit data...
dirmd Loading commit data...
dirmd.bat Loading commit data...
download_from_google_storage Loading commit data...
download_from_google_storage.bat Loading commit data...
download_from_google_storage.py Loading commit data...
ensure_bootstrap Loading commit data...
fetch Loading commit data...
fetch.bat Loading commit data...
fetch.py Loading commit data...
fix_encoding.py Loading commit data...
gclient Loading commit data...
gclient-new-workdir.py Loading commit data...
gclient.bat Loading commit data...
gclient.py Loading commit data...
gclient_completion.sh Loading commit data...
gclient_eval.py Loading commit data...
gclient_paths.py Loading commit data...
gclient_scm.py Loading commit data...
gclient_utils.py Loading commit data...
gerrit_client.py Loading commit data...
gerrit_util.py Loading commit data...
git-cache Loading commit data...
git-cl Loading commit data...
git-crrev-parse Loading commit data...
git-drover Loading commit data...
git-find-releases Loading commit data...
git-footers Loading commit data...
git-freeze Loading commit data...
git-gs Loading commit data...
git-hyper-blame Loading commit data...
git-map Loading commit data...
git-map-branches Loading commit data...
git-mark-merge-base Loading commit data...
git-nav-downstream Loading commit data...
git-nav-upstream Loading commit data...
git-new-branch Loading commit data...
git-number Loading commit data...
git-rebase-update Loading commit data...
git-rename-branch Loading commit data...
git-reparent-branch Loading commit data...
git-retry Loading commit data...
git-runhooks Loading commit data...
git-squash-branch Loading commit data...
git-thaw Loading commit data...
git-upstream-diff Loading commit data...
git_cache.py Loading commit data...
git_cl.py Loading commit data...
git_cl_completion.sh Loading commit data...
git_common.py Loading commit data...
git_dates.py Loading commit data...
git_drover.py Loading commit data...
git_find_releases.py Loading commit data...
git_footers.py Loading commit data...
git_freezer.py Loading commit data...
git_hyper_blame.py Loading commit data...
git_map.py Loading commit data...
git_map_branches.py Loading commit data...
git_mark_merge_base.py Loading commit data...
git_nav_downstream.py Loading commit data...
git_new_branch.py Loading commit data...
git_number.py Loading commit data...
git_rebase_update.py Loading commit data...
git_rename_branch.py Loading commit data...
git_reparent_branch.py Loading commit data...
git_retry.py Loading commit data...
git_squash_branch.py Loading commit data...
git_upstream_diff.py Loading commit data...
gn Loading commit data...
gn.bat Loading commit data...
gn.py Loading commit data...
goma_auth Loading commit data...
goma_auth.bat Loading commit data...
goma_ctl Loading commit data...
goma_ctl.bat Loading commit data...
gsutil.py Loading commit data...
gsutil.py.bat Loading commit data...
gsutil.vpython Loading commit data...
led Loading commit data...
led.bat Loading commit data...
lockfile.py Loading commit data...
luci-auth Loading commit data...
luci-auth.bat Loading commit data...
lucicfg Loading commit data...
lucicfg.bat Loading commit data...
mac_toolchain Loading commit data...
metrics.README.md Loading commit data...
metrics.py Loading commit data...
metrics_utils.py Loading commit data...
my_activity.py Loading commit data...
ninja Loading commit data...
ninja-linux32 Loading commit data...
ninja-linux64 Loading commit data...
ninja-mac Loading commit data...
ninja.exe Loading commit data...
ninjalog.README.md Loading commit data...
ninjalog_uploader.py Loading commit data...
ninjalog_uploader_wrapper.py Loading commit data...
owners.py Loading commit data...
owners_finder.py Loading commit data...
post_build_ninja_summary.py Loading commit data...
presubmit_canned_checks.py Loading commit data...
presubmit_support.py Loading commit data...
profile.xml Loading commit data...
prpc Loading commit data...
prpc.bat Loading commit data...
pylint Loading commit data...
pylint-1.5 Loading commit data...
pylint-1.6 Loading commit data...
pylint-1.7 Loading commit data...
pylint-1.8 Loading commit data...
pylint-1.9 Loading commit data...
pylint.bat Loading commit data...
pylint_main.py Loading commit data...
pylintrc Loading commit data...
python_runner.sh Loading commit data...
rdb Loading commit data...
rdb.bat Loading commit data...
rdb_wrapper.py Loading commit data...
repo Loading commit data...
roll-dep Loading commit data...
roll-dep.bat Loading commit data...
roll_dep.py Loading commit data...
scm.py Loading commit data...
setup_color.py Loading commit data...
split_cl.py Loading commit data...
subcommand.py Loading commit data...
subprocess2.py Loading commit data...
update_depot_tools Loading commit data...
update_depot_tools.bat Loading commit data...
update_depot_tools_toggle.py Loading commit data...
upload_metrics.py Loading commit data...
upload_to_google_storage.py Loading commit data...
vpython Loading commit data...
vpython.bat Loading commit data...
vpython3 Loading commit data...
vpython3.bat Loading commit data...
watchlists.py Loading commit data...
weekly Loading commit data...
win32imports.py Loading commit data...
wtf Loading commit data...
yapf Loading commit data...
yapf.bat Loading commit data...