• ssanfilippo's avatar
    [Swarming] work around slow calls in archive.py · 1c1b70c9
    ssanfilippo authored
    Apparently, the tarfile Python module spends a lot of time in
    grp.getgrid for retrieving a piece information (the name of the
    primary group) which we don't need anyway. There is no
    proper way to disable these slow calls, but there's a workaround
    which relies on the way in which grp (and pwd) is used.
    
    In fact, pwd and grp are imported in this fashion:
    
        try:
            import grp, pwd
        except ImportError:
            grp = pwd = None
    
    and then used with the following pattern [2]:
    
          if grp:
              try:
                  tarinfo.gname = grp.getgrgid(tarinfo.gid)[0]
              except KeyError:
                  pass
    
    By setting grp and pwd to None, thus skipping the calls, I was
    able to achieve a 35x speedup on my workstation.
    
    The user and group names are set to test262 when building the tar.
    
    The downside to this approach is that we are relying on an
    implementation detail, which is not in the public API.
    However, the blamelist shows that the relevant bits of the module
    have not been updated since 2003 [3], so we might as well assume
    that the workaround will keep working, on cPython 2.x at least.
    
    ---
    
    [1] https://hg.python.org/cpython/file/2.7/Lib/tarfile.py#l56
    [2] https://hg.python.org/cpython/file/2.7/Lib/tarfile.py#l1933
    [3] https://hg.python.org/cpython/rev/f9a5ed092660
    
    BUG=chromium:535160
    LOG=N
    
    Review URL: https://codereview.chromium.org/1727773002
    
    Cr-Commit-Position: refs/heads/master@{#34245}
    1c1b70c9
Name
Last commit
Last update
benchmarks Loading commit data...
build Loading commit data...
docs Loading commit data...
include Loading commit data...
infra Loading commit data...
samples Loading commit data...
src Loading commit data...
test Loading commit data...
testing Loading commit data...
third_party/binutils Loading commit data...
tools Loading commit data...
.clang-format Loading commit data...
.gitignore Loading commit data...
.ycm_extra_conf.py Loading commit data...
AUTHORS Loading commit data...
BUILD.gn Loading commit data...
ChangeLog Loading commit data...
DEPS Loading commit data...
LICENSE Loading commit data...
LICENSE.strongtalk Loading commit data...
LICENSE.v8 Loading commit data...
LICENSE.valgrind Loading commit data...
Makefile Loading commit data...
Makefile.android Loading commit data...
Makefile.nacl Loading commit data...
OWNERS Loading commit data...
PRESUBMIT.py Loading commit data...
README.md Loading commit data...
WATCHLISTS Loading commit data...
codereview.settings Loading commit data...
snapshot_toolchain.gni Loading commit data...