Skip to content

C
Chromium_Depot_tools
Switch branch/tag
  1. 25 Apr, 2019 1 commit
    • Daniel Bratell's avatar
      Speed up some really slow OWNERS operations for large/huge patches · b2b66999
      Daniel Bratell authored 
      The current owners systems scales badly on the number of files in a
patch, on the number of OWNERS files and on the number of "set noparent"
rules. This patch addresses all three of those.

If you have a patch with many (thousands) of files, you run into a lot
of repeated work because the parent directories are visited for every
file. By using a cache or remembering what directories have been
visited, a lot of work can be avoided.

If you have many OWNERS files or "set noparent" rules in scope of a
patch, owners and stop_looking lookups became slow.  By splitting the
rules by longest glob-free directory path, you get much smaller lists
to process, which makes the code much faster.

(Test case with "all the files" went from >15 minutes to 30 seconds).

This saves about 2.5 second for git cl upload with a patch of
350 files spread over the tree.

The addition of many noparent rules in the Chromium tree might have
been the major reason this is needed, but there are also more files to
change, allowing larger patches.

Bug: 920591
Change-Id: If21178746ca9b88c2c07d265bd583b556d1734b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1404172Reviewed-by: 's avatarDirk Pranke <dpranke@chromium.org>
Auto-Submit: Daniel Bratell <bratell@opera.com>
Commit-Queue: Daniel Bratell <bratell@opera.com>
      b2b66999
  3. 24 Apr, 2019 5 commits
  5. 23 Apr, 2019 3 commits
  7. 22 Apr, 2019 1 commit
  9. 18 Apr, 2019 3 commits
    • btolsch's avatar
      Add openscreen gclient config copy-pasta · 8dfc0bce
      btolsch authored 
      This change should hopefully enable the openscreen repo to use the
run_presubmit recipe.

Bug: openscreen:43
Change-Id: I672cddfeddd09ccc25ca851c0b613293191f17f9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1574189
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: 's avatarAndrii Shyshkalov <tandrii@chromium.org>
      8dfc0bce
    • Elly Fong-Jones's avatar
      depot_tools: bail out if $PATH contains tildes · 7b294392
      Elly Fong-Jones authored 
      Bash and its descendants allow having $PATH entries containing literal tildes,
like:
  ...:~/depot_tools:...
If the user has their shell configured this way, their shell will be able to
invoke `fetch` and other depot_tools commands, but depot_tools will not in turn
be able to invoke subcommands via popen because popen does not have this
bash-specific behavior when not in shell mode.

This change has gclient detect this configuration problem and error out with a
descriptive error message. Not doing so leads to a puzzling condition where
gclient complains about being unable to find a program that (from the user's
perspective) is in their $PATH and invokable interactively.

It also has 'fetch' suppress its own failure stack trace if it runs gclient and
gclient fails; the 'fetch' stack trace pushes the messages from gclient (which
are actually diagnostic) off the screen.

Bug: chromium:952865
Change-Id: Ibba4d2fccee405aa68392ce141493f1de21ec018
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1573204
Commit-Queue: Elly Fong-Jones <ellyjones@chromium.org>
Reviewed-by: 's avatarAaron Gable <agable@chromium.org>
      7b294392
    • Aaron Gable's avatar
      Restore py2/py3 compatibility with basestring · ac9b0f37
      Aaron Gable authored 
      A previous CL replaced "basestring" with "str", because basestring
does not exist in Python 3. However, this broke Python 2's ability
to interoperate with unicode strings. This CL introduces a workaround
(defining basestring to be equivalent to string, if it doesn't exist
already), and restores the references to basestring. This workaround
can be fixed when we're 100% on Python 3.

It also undoes some unnecessary and harder-to-read formatting changes.

Bug: 942522
Change-Id: I4a31ee46dc048134c2e4832b6c44ea00ce341899
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1572441
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: 's avatarDirk Pranke <dpranke@chromium.org>
      ac9b0f37
  11. 17 Apr, 2019 2 commits
  13. 16 Apr, 2019 1 commit
    • Vadim Shtayura's avatar
      [lucicfg] Bump lucicfg version v1.6.0 -> v1.7.0. · 7e7523be
      Vadim Shtayura authored 
      Changes:
2019-04-16 vadimsh@chromium.org [lucicfg] Allow to use relative paths in load(...) and exec(...).
2019-04-16 vadimsh@chromium.org [lucicfg] Remove config_set meta option, finish declare_config_set.
2019-04-13 vadimsh@chromium.org [lucicfg] Parallel validation of multiple config sets.
2019-04-12 vadimsh@chromium.org [lucicfg] Add ctx.declare_config_set(...) API.
2019-04-12 vadimsh@chromium.org [lucicfg] Convert Output from a map alias to a struct.
2019-04-12 vadimsh@chromium.org [lucicfg] Use ctx.output instead of ctx.config_set in generators.
2019-04-11 vadimsh@chromium.org [lucicfg] Start refactoring to support multiple output config sets.
2019-04-11 vadimsh@chromium.org [starlark] Fix tests after recent go.starlark.net roll.
2019-04-10 vadimsh@chromium.org [lucicfg] Finish the doc.
2019-04-09 vadimsh@chromium.org [lucicfg] Add a doc for "migrating from existing configs".
2019-04-09 vadimsh@chromium.org [lucicfg] Add examples of realistic lucicfg configs.
2019-04-05 vadimsh@chromium.org [lucicfg] Write an introduction doc.
2019-04-05 vadimsh@chromium.org [lucicfg] Write "Defining cron schedules" doc.
2019-04-05 seanabraham@chromium.org Improve emit_defaults documentation.
2019-04-04 nodir@google.com [buildbucket] Move config protos

R=tandrii@chromium.org
BUG=945048

Change-Id: I250c13fee50fbc963d44fccef1cba3510e834905
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1570412
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Auto-Submit: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: 's avatarAndrii Shyshkalov <tandrii@chromium.org>
      7e7523be
  15. 15 Apr, 2019 1 commit
  17. 12 Apr, 2019 2 commits
  19. 11 Apr, 2019 4 commits
  21. 10 Apr, 2019 5 commits
  23. 09 Apr, 2019 1 commit
  25. 08 Apr, 2019 1 commit
    • Raul Tambre's avatar
      Fix UTF-8 output in gclient_utils.CheckCallAndFilter · 1fb04630
      Raul Tambre authored 
      Hooks for Electron output UTF-8 characters.
Example error for when "✔" is output:
  File "/home/markus/depot_tools/metrics.py", line 266, in print_notice_and_exit
    yield
  File "/home/markus/depot_tools/gclient.py", line 3112, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/markus/depot_tools/gclient.py", line 3098, in main
    return dispatcher.execute(OptionParser(), argv)
  File "/home/markus/depot_tools/subcommand.py", line 252, in execute
    return command(parser, args[1:])
  File "/home/markus/depot_tools/gclient.py", line 2677, in CMDsync
    ret = client.RunOnDeps('update', args)
  File "/home/markus/depot_tools/gclient.py", line 1746, in RunOnDeps
    self.RunHooksRecursively(self._options, pm)
  File "/home/markus/depot_tools/gclient.py", line 1052, in RunHooksRecursively
    hook.run()
  File "/home/markus/depot_tools/gclient.py", line 245, in run
    cmd, cwd=self.effective_cwd, always=self._verbose)
  File "/home/markus/depot_tools/gclient_utils.py", line 344, in CheckCallAndFilterAndHeader
    return CheckCallAndFilter(args, **kwargs)
  File "/home/markus/depot_tools/gclient_utils.py", line 576, in CheckCallAndFilter
    stdout.write(in_byte.decode())
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe2 in position 0: unexpected end of data

This issue was introduced in CL:1524583.

Bug: 942522
Change-Id: I3c4355b925b34398c800d142f942531a829e0297
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1541334
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: 's avatarDirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
      1fb04630
  27. 05 Apr, 2019 4 commits
  29. 04 Apr, 2019 2 commits
  31. 03 Apr, 2019 3 commits
  33. 02 Apr, 2019 1 commit
    • Vadim Shtayura's avatar
      [lucicfg] Bump lucicfg version v1.5.12 -> v1.6.0. · 52f1494c
      Vadim Shtayura authored 
      Changes:
2019-04-02 vadimsh@chromium.org [lucicfg] Rename 'recipe = ...' field into 'executable = ...'.
2019-04-02 vadimsh@chromium.org [lucicfg] Implement semantic diff of CQ configs.
2019-04-02 vadimsh@chromium.org [lucicfg] Accept non-standard text protos as input of semantic-diff.
2019-04-02 vadimsh@chromium.org [lucicfg] Remove header IDs from normalized Milo configs.
2019-04-02 vadimsh@chromium.org [lucicfg] Implement semantic diff of Milo configs.
2019-04-02 vadimsh@chromium.org [lucicfg] Implement semantic diff of Scheduler configs.

TBR=tandrii@chromium.org

Change-Id: I1ace2edb3a2b686d069dbfb3d753ecc2540f01fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1549945Reviewed-by: 's avatarVadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
      52f1494c