Bug:1286319
Change-Id: I7b205e48605f3d889a4780043fdfc54e900aff01
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3692861Reviewed-by: Leina Sun <sunxiaodi@google.com>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Auto-Submit: Ryan Heise <heiserya@google.com>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>

AFAIK, flash is long gone from Chromium.

Bug: 1064651
Change-Id: I25b98ac2b44e52fa3e3cfdbc9d7716ee736a4b6a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3687701
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Scott Violet <sky@chromium.org>
Auto-Submit: Victor Vianna <victorvianna@google.com>

This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8811985614603312065

recipe_engine:
https://chromium.googlesource.com/infra/luci/recipes-py.git/+/51b3aed56d1ba5b0e8ac65f94c20b15513a5f87c
  51b3aed (jkusuma@google.com)
      Update go.chromium.org/luci protos related to resultdb.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I60997a4ef19b45a3ce381ea885ba0932bf78f6d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3694138
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

Changes:
1. Windows: exit /b %errorlevel% should be used instead of goto :EOF to get valid exit codes during cmd /c <script>.bat invocation.
2. Windows: delayed var expansion is required in update_depot_tools.bat exit code generation.
3. Posix: update_depot_tools returns exit code from update_git_repo function in case of a failure.

A rule of thumb on Windows: goto :EOF should not be used if %errorlevel% must be returned for all possible invocations.

Test case for update_depot_tools changes:
1. Make a change to depot_tools sources that will conflict with next depot_tools update
2. Run update_depot_tools either directly or via gclient
3. Expect a git error is triggered
4. Inspect %errorlevel% or $? depending on platform
5. Expected 1, but the actual result is 0.

Test case for changes in other .bat files:
1. Make a change to depot_tools sources that will conflict with next depot_tools update
2. Run cmd /c gclient
3. Expect a git error is triggered
4. Inspect %errorlevel%
5. Expected 1, but the actual result is 0.

Change-Id: I64459982bcd9cc3db1319a9b39224b7a7af8c5aa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3688632Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8812059854989420705

recipe_engine:
https://chromium.googlesource.com/infra/luci/recipes-py.git/+/92f7100ca350a89b67174e89eaef52fb04c5e8ef
  92f7100 (iannucci@chromium.org)
      [proto] Add ability to inject python files when scanning protos.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I11df85101f6262d6e3529db3598b2a294d170216
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3691023
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

A run of "git cl presubmit --all" shows that PyLint 1.5 is being run
from nine locations, however the deprecation warnings don't say where.
This makes it difficult to file bugs or fix the remaining instances.
This changes the message to list the path to the presubmit that is
running PyLint 1.5.

This also adds instructions on how to change to version 2.7.

Before:

  pylint-1.5 is deprecated, please switch to 2.7 before 2022-07-11

After (with word wrapping):

  pylint-1.5 is being run on ...src\tools\find_runtime_symbols
  and is deprecated, please switch to 2.7 before 2022-07-11 (add
  version='2.7' to RunPylint call)

Change-Id: Iece2cb904f5d26ad66e3ab78f7ce7aef1878bfd1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3688839Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

When a repository doesn't work with Gerrit, it doesn't make sense to
create GerritAccessor and pass it to presubmit checks.
gerrit.host is stored into git config when codereview.settings file contains GERRIT_HOST item.

Change-Id: I9740950caf85a1da9a6e4ae12e3612625cf0cec5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3687235Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>

Tested manually since we have no unit tests that cover this bit.

R=gavinmak@google.com

Bug: 1105747
Change-Id: I113e27236ed450490e59df9a6f698925ff556bc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3685350Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>

When PRESUBMIT is run with a custom --upstream, dry run cl format should
also use this value, otherwise format warnings would be inconsistent.

Change-Id: I1370f614618ecc03706e54cd4b151a207d9b3bae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3688631
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>

Bug: b/234078033
Change-Id: I54589891b54986859db539e45c3a208f97081bc6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3689029Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>

Currently, the ninja binaries in depot_tools are committed to the repo directly.
This CL installs CIPD ninja packages to replace them.
Switching to CIPD ninja will be done in the next CL.

Also, see the performance comparison and CIPD packages in the comments below.
https://crbug.com/931218#c84
https://crbug.com/931218#c86

Bug: 931218
Change-Id: I44d5e9ea1447588b32a10db20cfba088dd748f48
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674981
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>

Bug:1330629
Change-Id: I522255528e36d4806eada70c22afee1277a6778d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3668429Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>

This CL makes fetch default to sso:// within the cog env. It also adds a helper within gclient_scm to check if the env is cog.

Bug: chrome-operations:170
Change-Id: I8c35348653406549fadbd08df3d9991bb24ca776
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3670721Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Remove unnecessary os.listdir call and fix infinite loop on Windows if
no file was found in a tree.

Change-Id: I82a8763e807bbc0ce6fcae6b35a370ffe3b34943
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3687234Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Aleksey Khoroshilov <akhoroshilov@brave.com>

This reverts commit 013e47be.

Reason for revert: This will be relanded as a warning, although the
bugs which the original change exposed should be fixed soon.

Original change's description:
> Revert "Fix not-run-test detection."
>
> This reverts commit 41691abe.
>
> Reason for revert: I thought https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683378 would solve the problem but turns out this is the issue where it now reports it as a critical error.
>
> Original change's description:
> > Fix not-run-test detection.
> >
> > In crrev.com/c/2986400 the skip_shebang_check flag was added to address
> > the problem of some tests being run neither on Python 2 or Python 3. At
> > the same time reporting was added to warn if tests were run on neither
> > Python version.
> >
> > Unfortunately the reporting code was incorrect. A PresubmitPromptWarning
> > object was created but was not added to the list of results, so it had
> > no effect. Also the skipped tests name was added where a list was
> > expected which meant that each character of the test name showed up on
> > its own line.
> >
> > This change fixes the reporting. It also changes the format of the
> > report and makes it an error - a failure to run tests at all deserves
> > that severity level.
> >
> > A test run with the fixed check showed that no errors have crept in -
> > all tests were being run with one or the other. However the fix did
> > find a bug in an in-progress change I was working on.
> >
> > Bug: 1223478
> > Change-Id: Ibb44b5e60e7a7a5de08302f19ee4035cdfac5212
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674199
> > Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
> > Commit-Queue: Fabrice de Gans <fdegans@chromium.org>
> > Reviewed-by: Fabrice de Gans <fdegans@chromium.org>
>
> Bug: 1223478
> Change-Id: If4395e90784c3f91692f398b4e5230a51139f78a
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683105
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Reviewed-by: Bruce Dawson <brucedawson@chromium.org>

Bug: 1223478, 1330859
Change-Id: I6a0a78e70db77c9c5fda5bab27960cd4e71279e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3685349Reviewed-by: Fabrice de Gans <fdegans@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

This reverts commit c03ebf0e.

Reason for revert: the original change was not the cause of the test
failures. The actual cause was crrev.com/c/3683105 which was reverted.

Original change's description:
> Revert "Add Python 3 support to GetPythonUnitTests"
>
> This reverts commit dfc71bbe.
>
> Reason for revert: Breaks all presubmits that do not have skip_shebang_check=True set.
>
> Original change's description:
> > Add Python 3 support to GetPythonUnitTests
> >
> > GetPythonUnitTests (and by extension RunPythonUnitTests) always ran
> > scripts using Python 2. This change adds a python3=False parameter to
> > GetPythonUnitTests to give the option of switching to Python 3, while
> > leaving the default as Python 2. The child scripts are run under one or
> > the other, based on this parameter.
> >
> > This change is needed in support of crrev.com/c/3679801.
> >
> > Bug: 1313804
> > Change-Id: Ic59287352d4941707adaf7981ed7af4201b7d526
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3680099
> > Reviewed-by: Jesse McKenna <jessemckenna@google.com>
> > Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
>
> Bug: 1313804
> Change-Id: I1bd7096b6cfdfed6205e6dc0b5d4498b5b821b97
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683378
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Christoffer Jansson <jansson@chromium.org>
> Owners-Override: Ben Pastene <bpastene@chromium.org>
> Reviewed-by: Tomas Gunnarsson <tommi@chromium.org>

Bug: 1313804
Change-Id: I15ed90cc35d88ece71eb681acc0acc06cb5c3bcc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3685229
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Jesse McKenna <jessemckenna@google.com>

This is a reland of commit 6817010e

Additional Changes:
This reland fixes https://crbug.com/1330995 by stopping gclient from accidentally updating the scheme when one is not present.

Original change's description:
> Update fetch protocol using --protocol-override flag
>
> This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
>
> Bug: chrome-operations:170
> Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Bug: 1330995
Change-Id: I1447a5e884e41d671d8556c35193f1635f2f6936
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3684112Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Currently cipd packages listed in src/DEPS are being created and
installed in symlink mode (see bug for more info). This CL adds `$OverrideInstallMode copy` to the ensure files for cipd packages installed during gclient runhooks. `$OverrideInstallMode` was
implemented here, https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/3124202,
so that every cipd package can be installed directly to the installation
target, instead of being symlinked to an executable inside the cache/builder/.cipd dir.

The added `$OverrideInstallMode copy` can be removed once all cipd
packages themselves or the default mode is updated to be created in
copy mode.

Bug: 1329641
Change-Id: I68dc718e20f193c384d005c9b6b87603753943d4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3673698Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>

This reverts commit 6817010e.

Reason for revert: Breaks gclient sync for flutter

Original change's description:
> Update fetch protocol using --protocol-override flag
>
> This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.
>
> Bug: chrome-operations:170
> Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Bug: chrome-operations:170
Change-Id: I76a49030f48fa266eeea336307e7153b22ded4e9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3684110
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>

Should be landed once the new cipd is rolled out everywhere (including
bots)

Bug: 1329641
Change-Id: I32edaa167dd26dbf891435215eeb7c833aba091f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3680100Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>

This reverts commit 41691abe.

Reason for revert: I thought https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683378 would solve the problem but turns out this is the issue where it now reports it as a critical error.

Original change's description:
> Fix not-run-test detection.
>
> In crrev.com/c/2986400 the skip_shebang_check flag was added to address
> the problem of some tests being run neither on Python 2 or Python 3. At
> the same time reporting was added to warn if tests were run on neither
> Python version.
>
> Unfortunately the reporting code was incorrect. A PresubmitPromptWarning
> object was created but was not added to the list of results, so it had
> no effect. Also the skipped tests name was added where a list was
> expected which meant that each character of the test name showed up on
> its own line.
>
> This change fixes the reporting. It also changes the format of the
> report and makes it an error - a failure to run tests at all deserves
> that severity level.
>
> A test run with the fixed check showed that no errors have crept in -
> all tests were being run with one or the other. However the fix did
> find a bug in an in-progress change I was working on.
>
> Bug: 1223478
> Change-Id: Ibb44b5e60e7a7a5de08302f19ee4035cdfac5212
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674199
> Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
> Commit-Queue: Fabrice de Gans <fdegans@chromium.org>
> Reviewed-by: Fabrice de Gans <fdegans@chromium.org>

Bug: 1223478
Change-Id: If4395e90784c3f91692f398b4e5230a51139f78a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683105
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>

This CL updates gclient sync to use the protocol of the URL specified in the solutions for cloning all the child dependencies of it.

Bug: chrome-operations:170
Change-Id: I33588059788b677fbae8c3b434100af5c7979a67
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3631600Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

This reverts commit dfc71bbe.

Reason for revert: Breaks all presubmits that do not have skip_shebang_check=True set.

Original change's description:
> Add Python 3 support to GetPythonUnitTests
>
> GetPythonUnitTests (and by extension RunPythonUnitTests) always ran
> scripts using Python 2. This change adds a python3=False parameter to
> GetPythonUnitTests to give the option of switching to Python 3, while
> leaving the default as Python 2. The child scripts are run under one or
> the other, based on this parameter.
>
> This change is needed in support of crrev.com/c/3679801.
>
> Bug: 1313804
> Change-Id: Ic59287352d4941707adaf7981ed7af4201b7d526
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3680099
> Reviewed-by: Jesse McKenna <jessemckenna@google.com>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

Bug: 1313804
Change-Id: I1bd7096b6cfdfed6205e6dc0b5d4498b5b821b97
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3683378
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Christoffer Jansson <jansson@chromium.org>
Owners-Override: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Tomas Gunnarsson <tommi@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8812608469227296113

recipe_engine:
https://chromium.googlesource.com/infra/luci/recipes-py.git/+/5ba6e0d0cd472c2e7783bc95424b44bb46ce2749
  5ba6e0d (chromium-autoroll@skia-public.iam.gserviceaccount.com)
      Roll CAS Client from 27d906266697 to 7bf7ff13b591

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I7ee0f0de3825a12d2c478bedfb372d6cadf38a4a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3681981
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8812613803095882881

recipe_engine:
https://chromium.googlesource.com/infra/luci/recipes-py.git/+/d14718e984d2b7b539f8f9c8bec3215807c0a681
  d14718e (alexschulze@chromium.org)
      [file] Add fail_silently to copy and copytree

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I10eff17643f710878340d5da77d4ebf6e6813ac2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3679516
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

GetPythonUnitTests (and by extension RunPythonUnitTests) always ran
scripts using Python 2. This change adds a python3=False parameter to
GetPythonUnitTests to give the option of switching to Python 3, while
leaving the default as Python 2. The child scripts are run under one or
the other, based on this parameter.

This change is needed in support of crrev.com/c/3679801.

Bug: 1313804
Change-Id: Ic59287352d4941707adaf7981ed7af4201b7d526
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3680099Reviewed-by: Jesse McKenna <jessemckenna@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

In crrev.com/c/2986400 the skip_shebang_check flag was added to address
the problem of some tests being run neither on Python 2 or Python 3. At
the same time reporting was added to warn if tests were run on neither
Python version.

Unfortunately the reporting code was incorrect. A PresubmitPromptWarning
object was created but was not added to the list of results, so it had
no effect. Also the skipped tests name was added where a list was
expected which meant that each character of the test name showed up on
its own line.

This change fixes the reporting. It also changes the format of the
report and makes it an error - a failure to run tests at all deserves
that severity level.

A test run with the fixed check showed that no errors have crept in -
all tests were being run with one or the other. However the fix did
find a bug in an in-progress change I was working on.

Bug: 1223478
Change-Id: Ibb44b5e60e7a7a5de08302f19ee4035cdfac5212
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674199
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Fabrice de Gans <fdegans@chromium.org>
Reviewed-by: Fabrice de Gans <fdegans@chromium.org>

Some presubmit failures result in uncaught exceptions which historically
have halted presubmit runs. This behavior makes it more difficult to
find all failures, measure presubmit times, or detect other behavior
over time. This is particularly problematic when running "git cl
presubmit --all".

As an example, presubmit --all was broken and would not complete from
when crrev.com/3642633 landed until crrev.com/c/3657600. The minimal
repro was this presubmit command:

    git cl presubmit --files ui\accessibility\ax_mode.h

This command, or presubmit --all, triggered this error, which halts the
presubmits:

Evaluation of CheckChangeOnCommit failed: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'
Traceback (most recent call last):
  File "presubmit_support.py", line 2038, in <module>
    sys.exit(main())
  File "presubmit_support.py", line 2015, in main
    return DoPresubmitChecks(
  File "presubmit_support.py", line 1738, in DoPresubmitChecks
    results += executer.ExecPresubmitScript(presubmit_script, filename)
  File "presubmit_support.py", line 1602, in ExecPresubmitScript
    self._run_check_function(function_name, context, sink,
  File "presubmit_support.py", line 1640, in _run_check_function
    six.reraise(e_type, e_value, e_tb)
  File "C:\Users\brucedawson\.vpython-root\e726d2\lib\site-packages\six.py", line 686, in reraise
    raise value
  File "presubmit_support.py", line 1630, in _run_check_function
    result = eval(function_name + '(*__args)', context)
  File "<string>", line 1, in <module>
  File "PRESUBMIT.py", line 314, in CheckChangeOnCommit
    errs.extend(CheckModesMatch(input_api, output_api))
  File "PRESUBMIT.py", line 283, in CheckModesMatch
    ax_modes_in_js = GetAccessibilityModesFromFile(
  File "PRESUBMIT.py", line 245, in GetAccessibilityModesFromFile
    for line in open(fullpath).readlines():
FileNotFoundError: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'

With this change the exception is handled and presubmits continue after
printing this message:

Evaluation of CheckChangeOnCommit failed: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js', Traceback (most recent call last):
  File "presubmit_support.py", line 1630, in _run_check_function
    result = eval(function_name + '(*__args)', context)
  File "<string>", line 1, in <module>
  File "PRESUBMIT.py", line 314, in CheckChangeOnCommit
    errs.extend(CheckModesMatch(input_api, output_api))
  File "PRESUBMIT.py", line 283, in CheckModesMatch
    ax_modes_in_js = GetAccessibilityModesFromFile(
  File "PRESUBMIT.py", line 245, in GetAccessibilityModesFromFile
    for line in open(fullpath).readlines():
FileNotFoundError: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'

That is, the crucial information regarding the failure is printed, but
subsequent presubmits are not skipped.

Bug: 1309977
Change-Id: I7cfeda85c830e6c8e567c0df3c50f27af1dbe835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3653978Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8812990573718281121

recipe_engine:
https://chromium.googlesource.com/infra/luci/recipes-py.git/+/e52acbc0987bdb31369aba51894b98286a418582
  e52acbc (andrewlamb@chromium.org)
      [recipes-py] roll recipe proto.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I1a16c436667dfcb2a44c760e2d59ad30cf8d8a7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3673905
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

Bug:1328653
Change-Id: I812eae65b6ea4f5bcd01d315405e90e0b3f1e732
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3673717
Auto-Submit: Ryan Heise <heiserya@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

R=gavinmak@google.com

Change-Id: I7d9ca9530640ffa4b8e60c247f809b0ff477d88e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3673716
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>

A typical cpplint error report during a presubmit looks like this:

    .../linux_ui.cc:64:  Almost always, snprintf is better than strcpy  [runtime/printf] [4]
    ...
    Changelist failed cpplint.py check.

The problem is that the details of the error go to stderr when
CheckChangeLintsClean is run, whereas the "Changelist failed cpplint.py
check." message is printed at the end of the run. If there a lot of
errors, warnings, or messages then the association between them can be
completely lost.

This change adds the tag (cpplint) to the error messages, and adds a
comment to the "failed cpplint.py check" message to suggest looking for
the tag.

Bug: 1309977
Change-Id: I0d073b57f215e28495cbc3e009cab6ac2f23b65e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3632947
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>

This is to fix
https://ci.chromium.org/ui/p/chromium/builders/ci/Win%20x64%20Builder%20(py2%20less)/1151/overview

Bug: 1327534
Change-Id: I3dcead03d0f4e213e1c4e610d8b90fe95ea843d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3670999
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>

Bug: b/233146956
Change-Id: I15ff4a79e4edf05f26bbb1cd3f7abfc99f209fc8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3668430Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Some of the expensive checks when running presubmit --all, such as
CheckStableMojomChanges (~300 s) and CheckAddedDepsHaveTargetApprovals
(~200 s) only look at diffs and are therefore guaranteed to be NOPs when
running presubmit --all or --files=. Passing along the no_diffs state
lets these expensive checks be skipped, thus allowing for faster
iteration times.

Initial testing suggests that (with some supporting changes in the
Chromium repo) this reduces "presubmit --all" times by about ten
minutes, or a bit more than 10%, and additional improvements may be
possible.

Special handling for the no-diffs case also offers a simple way to avoid
presubmit failures that happen whenever all files are flagged as being
changed.

Finally, and perhaps most importantly for having a presubmit --all bot,
when --no_diffs is passed we can treat errors like "Issue wasn't
uploaded" and "Add a description to the CL" as messages, thus making it
possible to have zero presubmit errors when run on origin/main.

Bug: 1320937, 1322936
Change-Id: I0d09dd4aae8fdaa48c8b2f89337441cf96dcff72
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3628368
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>

This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8813160854717135793

recipe_engine:
https://chromium.googlesource.com/infra/luci/recipes-py.git/+/d7417d09a5a31edda68c3d086752704f39e56323
  d7417d0 (chromium-autoroll@skia-public.iam.gserviceaccount.com)
      Roll CAS Client from c73a8d4e4fa5 to 7d578d09794d

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I0fe76bd2c53d93f19e37959baaa024b5ec19279f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3669677
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

Presubmit message types were printed in an inconsistent order, generally
determined by which type was encountered first. This unpredictability
can be confusing when comparing runs. This change enforces a consistent
order, in increasing order of severity. ERRORS always go last so that
they will be the most visible when running presubmits locally.

This also adds a "There were Python %d presubmit errors." message to
the end to make it easier to tell when there were presubmit errors.
This is particularly useful when manually running presubmits.

Bug: 1309977
Change-Id: Ib2b73c7625789bad5b21ae12abf238b746cd11e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3668724Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

In https://crbug.com/1327371, we started encountering hanging gsutil
calls. These were triggered under very specific circumstances, but
were essentially due to a multi-processing bug in python on MacOS:
https://bugs.python.org/issue33725

When running gsutil on mac, it explicitly suggests disabling
multi-processing for this reason:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/catapult/third_party/gsutil/gslib/command.py;drc=3a12d6ccdec28da8bda09d9ff826aae1f9504e59;l=1331

So this CL simply puts that suggestion into practice. There are two
options for doing so: adding a line to an active Boto file, or adding
a "-o" arg to every gsutil invocation. Since chromium has multiple
copies of gsutil and multiple different locations where `gsutil` is
invoked, the latter option is considered intractable.

Instead, we choose the Boto file option by adding a context manager
to the gsutil recipe module that will insert a custom Boto file into
the environment that only disables multi-threading. Due to the fact
that the BOTO_PATH env var takes multiple paths, we can safely
incorporate both our custom Boto along with LUCI's per-build Boto file
in the same gsutil invocation. And when the context manager exits, the
environment should revert back to its original state.

Chromium incorporates this method in crrev.com/c/3662023. See it take
effect during the "gclient runhooks" step in this led build:
https://ci.chromium.org/swarming/task/5b0c5174e70cb010?server=chromium-swarm.appspot.com

Bug: 1327371
Change-Id: I75cfdf16d0ec023bf81ea57991dde996694a46c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3661949Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ben Pastene <bpastene@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8813260264718091857

recipe_engine:
https://chromium.googlesource.com/infra/luci/recipes-py.git/+/edc3cd1e9d177fd0ae245fbb9d25e141191add5f
  edc3cd1 (nicholasbishop@google.com)
      walkthrough.md: update test specification style

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ifa6fd468598e783fcbd1da3c4d1165d77823f126
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3664180
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

It's possible that remote closes connection, which would result in a
stacktrace being shown in the stderr. Silence this exception, as we do
with other exceptions.

R=gavinmak@google.com

Bug: 1328518
Change-Id: Ifae2aca64fe5d572c0b71008f6dacbb63ba77e06
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3664178Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>