Commits · e3703bb8b8af7978ecf1bdfbfe8c3e6c910be8bc · Linshizhi / Chromium_Depot_tools

11 Oct, 2019 3 commits

Andrii Shyshkalov authored Oct 11, 2019

Notably:
 * doesn't support luci migration host
 * supports swarming.parent_run_id.

R=tikuta

Bug: 1004140
Change-Id: Iffc5b418766c60a99b7b7ec5a5772703d0cc23a5
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1855369
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>

e3703bb8

python3 improvements · 8e57b4bc

Marc-Antoine Ruel authored Oct 11, 2019

Ran:
  vi $(git grep --name-only iteritems | grep -v third_party)
  vi $(git grep --name-only itervalues | grep -v third_party)
  vi $(git grep --name-only 'print ' | grep -v third_party)

and edited the files quickly with adhoc macros. Then ran in recipes/:
  ./recipes.py test train

There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.

Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>

8e57b4bc

depot_tools: Add httplib2 to .vpython · e1410883

Edward Lemur authored Oct 11, 2019

Check that things won't break before further changes are made.

Bug: 1002153
Change-Id: I41866f26334bf9ec2732bc0f25007234a95130e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854749
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

e1410883

10 Oct, 2019 4 commits

Roll recipe dependencies (trivial). · b2b51418

recipe-roller authored Oct 10, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/a1609e19fa4bc5914cd681e2819539cf922113bd buildbucket: support swarming.parent_run_id when triggering builds. (tandrii@google.com)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I2986b9ca1f9dcd76225cffcff7e39b15ab991fe5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1853748Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

b2b51418

git-cl: add --fixed / -x option to upload · d8b04ca8

Dan Beam authored Oct 10, 2019

This option take the same format as -b / --bug but
spits out "Fixed: #,#" instead of "Bug: #,#".

This CL also looks for fix-, fixed-, and fixes-
branch prefixes and uses those as cue to add "Fixed:"
to the CL description when editing. This is a slight
behavioral change for fix- specifically, as it used
to produce "Bug: " instead.

Bug: monorail:4470
Change-Id: Ib24a1ff33ca3674e53fc5437f459ea5708988290
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1846785
Commit-Queue: Dan Beam <dbeam@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Dan Beam <dbeam@chromium.org>

d8b04ca8

gsutil: Bump asn1crypto version to 1.0.1 · 6a1d7786

Edward Lemur authored Oct 10, 2019

Bug: 1012612
Change-Id: I909514ff89111bc829a2f6ddd91026348fc6cb0c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854010
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

6a1d7786

Roll recipe dependencies (trivial). · 34819029

recipe-roller authored Oct 10, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/47a94f44d8b345593510faf052494627c69c5c44 [buildbucket] Let child builds use builder priority (olivernewman@google.com)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ie01695cb155c905893c5d64bc1b6663d26f73d67
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1850018Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

34819029

09 Oct, 2019 5 commits

depot_tools: Remove depot-tools-auth · b4a587df

Edward Lemur authored Oct 09, 2019

Users must use luci-auth now.

Bug: 1001756
Change-Id: I04cab6bdbfbd958f386a4cab761dfe4d34073afc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1849810
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>

b4a587df

git-common: Explicitly specify utf-8 when encoding. · 71681bf1

Edward Lemur authored Oct 09, 2019

Bug: 1012773
Change-Id: I95006ce78b4473c8fb435162089a787b57c0f835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1850926
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

71681bf1

git cl: record additional tag for builds triggered via retry-failed. · aeee6a83

Andrii Shyshkalov authored Oct 09, 2019

Tag is the same as used by Gerrit plugin for the same functionality:
  retry_failed:1

R=ehmaldonado

Change-Id: I14ef435b676088c86ae7cc0536c6a68fa58bd89f
Bug: 445502
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1848258
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

aeee6a83

Reland "gclient_utils: buffer output as bytestrings in Annotated" · cb1eb489

Edward Lemur authored Oct 09, 2019

This is a reland of 5d284fdf

Always convert the input to bytes and write to sys.stdout (in Python2)
and sys.stdout.buffer (in Python 3).

Original change's description:
> gclient_utils: buffer output as bytestrings in Annotated
>
> In Python 3 byestrings and normal strings can't be concatenated.
> To fix this we buffer as bytestrings in the Annotated wrapper.
> We can't decode to a string because the output might come byte-by-byte, so it doesn't work with Unicode characters like ✔.
>
> Also had to update gclient_test.py, where double-wrapping stdout with Annotated caused made output not work and include_zero=True working caused other unintended side-effects.
>
> Example error from "fetch chromium":
> Traceback (most recent call last):
>   File "C:\Google\depot_tools\gclient_scm.py", line 1045, in _Clone
>     self._Run(clone_cmd, options, cwd=self._root_dir, retry=True,
>   File "C:\Google\depot_tools\gclient_scm.py", line 1370, in _Run
>     gclient_utils.CheckCallAndFilter(cmd, env=env, **kwargs)
>   File "C:\Google\depot_tools\gclient_utils.py", line 583, in CheckCallAndFilter
>     show_header_if_necessary(needs_header, attempt)
>   File "C:\Google\depot_tools\gclient_utils.py", line 533, in show_header_if_necessary
>     stdout_write(header.encode())
>   File "C:\Google\depot_tools\gclient_utils.py", line 391, in write
>     obj[0] += out
> TypeError: can only concatenate str (not "bytes") to str
>
> Bug: 984182
> Change-Id: If7037d30e9faf524f2405258281f6e6cd0bcdcae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1778745
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Auto-Submit: Raul Tambre <raul@tambre.ee>

Bug: 984182
Change-Id: Ifafb5e16a517c4c38dd4c9d5c6d4c3f994838bc9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1845504Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

cb1eb489

git-cl: Use buildbucket v2 to fetch tryjob results. · baaf6bec

Edward Lemur authored Oct 09, 2019

Bug: 976104
Change-Id: Icf761f1cd093f7600ad43b71af474e52780f1997
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1842335Reviewed-by: Anthony Polito <apolito@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

baaf6bec

08 Oct, 2019 11 commits

git-hyper-blame: Make tests run on Python 3. · be83c310

Edward Lemur authored Oct 08, 2019

Bug: 1009809
Change-Id: If3099898a75837d3cee35bf9920680665dd4d322
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1838257
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

be83c310

Roll recipe dependencies (trivial). · cef11fda

recipe-roller authored Oct 08, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/1cc8317a701750c47ac3fe5183af8e105f9c36bc swarming: always mock bot and task IDs in test mode. (tandrii@google.com)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I47dc290d63ef5c35c2adae5cd2d0480281eaadaa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1848260Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

cef11fda

Roll recipe dependencies (trivial). · b7a7f1c0

recipe-roller authored Oct 08, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/3d26d5f05680aa707e527b8c291d298cd3d35748 swarming: support overriding default bot_id and task_id in test_api. (tandrii@google.com)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I3a1609364e8a9abb3a3a3d7f73ce5126ee8338f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1847962Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

b7a7f1c0

Revert "git-cl: Set legacy properties." · c38e0005

Edward Lemur authored Oct 08, 2019

This reverts commit f0faf48d.

Reason for revert:
Legacy properties are no longer used by Skia.

Original change's description:
> git-cl: Set legacy properties.
> 
> Will be removed back again on Oct 9, 2019.
> 
> Bug: 1004447
> Change-Id: I7003fac9ae8b184e2a7073b516535e8fd599f4f5
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1825446
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

TBR=tandrii@google.com,ehmaldonado@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 1004447
Change-Id: I4e1fb31b6179bf10c3d9554cd313f7040aaf1b1f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1847959Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>

c38e0005

Roll recipe dependencies (trivial). · 15676c0e

recipe-roller authored Oct 08, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/f84207a8e5eb80f11bb414e98b4177471d69419e Make tandrii@ top level recipe engine OWNER. (tandrii@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I94d1a8cf32e7c29e5f94c588ecb272f82213eb93
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1847957Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

15676c0e

git-cl: Run using vpython · 1f3bafba

Edward Lemur authored Oct 08, 2019

Bug: 1002153
Change-Id: I02ffa574f072575c4b6662a7f32678428ed3f85d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1845498Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

1f3bafba

git-number: Make tests run on Python 3. · 352808fa

Edward Lemur authored Oct 08, 2019

Bug: 1009809
Change-Id: I044f10a7a04f4837cba6e17d8f3053471cd7a643
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1841875
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

352808fa

recipe_module git: remove non-LUCI mode. · 1458d572

Andrii Shyshkalov authored Oct 08, 2019

R=maruel

Bug: 1012148
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: build
Change-Id: I386b540ef1d68d2f20031111ad5b693eb35c57a4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1846736
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

1458d572

git cl upload --retry-failed: search back up to 5 patchsets. · 1ad5811a

Andrii Shyshkalov authored Oct 08, 2019

Make it work the same way as "git cl try --retry-failed".

R=ehmaldonado

Change-Id: Id3a709dc40b6b6d068acbba0c610da254ced8ff3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1845478
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>

1ad5811a

git_cl: fix test. · e7ae514a

Andrii Shyshkalov authored Oct 08, 2019

R=ehmaldonado

Change-Id: Ia5c5d5fffdace2ce18fb0b2e5535370b9619dde4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1846771
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

e7ae514a

git-drover: Fix tests to run on Python 3. · 6f661161

Edward Lemur authored Oct 08, 2019

Bug: 1009809
Change-Id: I5f453c7e019376e502716ab2e9891a97663d1f28
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1838254
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>

6f661161

07 Oct, 2019 9 commits

depot_tools: Run git_cache_test and git_footers_test on windows. · ab51754c

Edward Lesmes authored Oct 07, 2019

Bug: Some
Change-Id: I74eef44bfe3d940c773e548e6d1035dee4cd45ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1842638
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

ab51754c

Revert "gclient_utils: buffer output as bytestrings in Annotated" · eeafa0ea

Bob Haarman authored Oct 07, 2019

This reverts commit 5d284fdf.

Reason for revert: breaks buildbots, crbug.com/1011982

Original change's description:
> gclient_utils: buffer output as bytestrings in Annotated
> 
> In Python 3 byestrings and normal strings can't be concatenated.
> To fix this we buffer as bytestrings in the Annotated wrapper.
> We can't decode to a string because the output might come byte-by-byte, so it doesn't work with Unicode characters like ✔.
> 
> Also had to update gclient_test.py, where double-wrapping stdout with Annotated caused made output not work and include_zero=True working caused other unintended side-effects.
> 
> Example error from "fetch chromium":
> Traceback (most recent call last):
>   File "C:\Google\depot_tools\gclient_scm.py", line 1045, in _Clone
>     self._Run(clone_cmd, options, cwd=self._root_dir, retry=True,
>   File "C:\Google\depot_tools\gclient_scm.py", line 1370, in _Run
>     gclient_utils.CheckCallAndFilter(cmd, env=env, **kwargs)
>   File "C:\Google\depot_tools\gclient_utils.py", line 583, in CheckCallAndFilter
>     show_header_if_necessary(needs_header, attempt)
>   File "C:\Google\depot_tools\gclient_utils.py", line 533, in show_header_if_necessary
>     stdout_write(header.encode())
>   File "C:\Google\depot_tools\gclient_utils.py", line 391, in write
>     obj[0] += out
> TypeError: can only concatenate str (not "bytes") to str
> 
> Bug: 984182
> Change-Id: If7037d30e9faf524f2405258281f6e6cd0bcdcae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1778745
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Auto-Submit: Raul Tambre <raul@tambre.ee>

TBR=dpranke@chromium.org,ehmaldonado@chromium.org,raul@tambre.ee

Change-Id: I5ea8d3249c58a3487996649a264bb5be059fe884
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 984182
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1845500Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

eeafa0ea

depot_tools: Remove buildbucket.py · 3306bbe4

Edward Lemur authored Oct 07, 2019

Change-Id: I6b66880361a5fcbff4652b40adbe96da85ecabac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1845819
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

3306bbe4

Roll recipe dependencies (trivial). · c9598be3

recipe-roller authored Oct 07, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/60cb7d1afaec1ce649a9d233e9575dc752881b5f scheduler: stop supporting using `properties`. (tandrii@google.com)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I068e32f1df035e83701cbc25872175fb765a9086
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1845820Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

c9598be3

gclient_utils: buffer output as bytestrings in Annotated · 5d284fdf

Raul Tambre authored Oct 07, 2019

In Python 3 byestrings and normal strings can't be concatenated.
To fix this we buffer as bytestrings in the Annotated wrapper.
We can't decode to a string because the output might come byte-by-byte, so it doesn't work with Unicode characters like ✔.

Also had to update gclient_test.py, where double-wrapping stdout with Annotated caused made output not work and include_zero=True working caused other unintended side-effects.

Example error from "fetch chromium":
Traceback (most recent call last):
File "C:\Google\depot_tools\gclient_scm.py", line 1045, in _Clone
self._Run(clone_cmd, options, cwd=self._root_dir, retry=True,
File "C:\Google\depot_tools\gclient_scm.py", line 1370, in _Run
gclient_utils.CheckCallAndFilter(cmd, env=env, **kwargs)
File "C:\Google\depot_tools\gclient_utils.py", line 583, in CheckCallAndFilter
show_header_if_necessary(needs_header, attempt)
File "C:\Google\depot_tools\gclient_utils.py", line 533, in show_header_if_necessary
stdout_write(header.encode())
File "C:\Google\depot_tools\gclient_utils.py", line 391, in write
obj[0] += out
TypeError: can only concatenate str (not "bytes") to str

Bug: 984182
Change-Id: If7037d30e9faf524f2405258281f6e6cd0bcdcae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1778745
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>

5d284fdf

download_from_google_storage: Decode string passed to re.search() · 5189047f

Raphael Kubo da Costa authored Oct 07, 2019

The code in _downloader_worker_thread() does not work with Python 3:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rakuco/src/depot_tools/download_from_google_storage.py", line 343, in _downloader_worker_thread
    elif re.search(r'executable:\s*1', out):
  File "/usr/lib/python3.7/re.py", line 183, in search
    return _compile(pattern, flags).search(string)
TypeError: cannot use a string pattern on a bytes-like object

Bug: 984182, 1007872
Change-Id: I09df8169d802b010596ac4f34501b4bda805e6f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1844833
Auto-Submit: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

5189047f

download_from_google_storage: Do not call decode() on a string twice · e225ef6a

Raphael Kubo da Costa authored Oct 07, 2019

This throws an exception when running with Python 3:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rakuco/src/depot_tools/download_from_google_storage.py", line 259, in _downloader_worker_thread
    file_url = '%s/%s' % (base_url, input_sha1_sum.decode())
AttributeError: 'str' object has no attribute 'decode'

We are already calling bytes.decode() in enumerate_input(), whose output
is passed to _downloader_worker_thread().

Bug: 984182, 1007872
Change-Id: I55c4835192caf69a8dee9e89ae1bb5531f4bacae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1844832
Auto-Submit: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

e225ef6a

gclient_utils: Stop using execfile() · 107c97c8

Raphael Kubo da Costa authored Oct 07, 2019

execfile() does not exist in Python 3; replace it with open(), read()
and exec().

Bug: 984182
Change-Id: I2266ff2bd63bffc458bd04500c8c0d6c360f1b4a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1844831
Auto-Submit: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

107c97c8

Roll recipe dependencies (trivial). · 4102985e

recipe-roller authored Oct 07, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/ac1ee2b1e0f55d7bdd89ab7746bc00d679c93728 Followup to https://chromium-review.googlesource.com/c/1843211. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ie4c20535c7526a5c7825cb077480ae0120176ba6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1844635Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

4102985e

05 Oct, 2019 2 commits

Roll recipe dependencies (trivial). · c9256e16

recipe-roller authored Oct 05, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/b3a48f21daa4a8f74a59d9d868c760aecfba360d [futures] Expose metadata for Futures. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I8b5d4c94b8c1958bc0980e9f175ce84b4f2816fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1843271Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

c9256e16

[git_cache] Add option to break locks on populate. · 09315982

Robert Iannucci authored Oct 05, 2019

R=ehmaldonado@chromium.org, tandrii@chromium.org

Change-Id: Icd7b0afc98e6d9365f74e8a6eb232700c6362d17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1842832
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

09315982

04 Oct, 2019 6 commits

Roll recipe dependencies (trivial). · d696f201

recipe-roller authored Oct 04, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/8dd8812fab57cb92382a2d20a0be10164b51cdf5 [vpython] Add pywin32 wheel on 32bit windows too. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ief5cef362e4353dfdb3ed7987a3ba440cd7685c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1842336Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

d696f201

Roll recipe dependencies (trivial). · 37c463fe

recipe-roller authored Oct 04, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/82c1aa10460d1e0f498026a8f5cff566a269b1c9 [third_party/logdog] Only asyncify the file after the handshake. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I7812d0c02f69256ddfd4b742ac0dc23e05d94f91
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1841189Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

37c463fe

Roll recipe dependencies (trivial). · 8a859625

recipe-roller authored Oct 04, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/6277a86cea12e7c3e24b78f32271f0a40b035726 [luciexe] Allow luciexe recipes to always set summary_markdown. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I08be31f5d56d81e2a45233474a0a3ea673f2ed11
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1841182Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

8a859625

Roll recipe dependencies (trivial). · aa4d8a75

recipe-roller authored Oct 04, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/e051b6e61887a79ca02fa2f912163032754d7948 [context] Fix context module to use greenlet-specific storage. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ie2ea83d559c18f7c59f82c37bf39186c14107e13
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1839548Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

aa4d8a75

Roll recipe dependencies (trivial). · 94c33f91

recipe-roller authored Oct 04, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/cb5e29908c24ae791cbc0c1fc17e9098bc2157cc Revert "recipe module buildbucket: migrate to protos for inputs." (tandrii@google.com)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ibbdaebb39481e6d5c817f074a88bdfb2fb91164e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1839252Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

94c33f91

Roll recipe dependencies (trivial). · d2b74ffc

recipe-roller authored Oct 04, 2019

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/aa84e2a4f50fe7ed19690fa61b0d1f8294964b14 Account for step resource costs before opening file handles for them. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Iec475ff0be5440f789dc1ca1f105e1ed8c2b2a77
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1838708Reviewed-by: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>

d2b74ffc