Currently gclient might include extra commits when applying patches.
For example, in this case we checkout |patch| and rebase it on top of |base|,
thus including an |extra commit| that we shouldn't.

o master
|
. o patch
|/
o extra commit
|
o base (what gclient synced src at)

This change uses the merge-base between |patch| and |master| to cherry-pick only
the changes belonging to the patch.

Bug: 850812
Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
Reviewed-on: https://chromium-review.googlesource.com/1137052
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>

The new version supports '$ParanoidMode CheckPresence' in ensure files.

R=iannucci@chromium.org
BUG=794764

Change-Id: I8add0ebbaa324fedf99959a0a5d1301172ef3704
Reviewed-on: https://chromium-review.googlesource.com/1138732
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

I want to use this in the Codesearch builder to replace the hack in
crrev.com/c/1126693.

Bug: 860397
Change-Id: I174a684cbff0f993b5c657bc2e32105cb49d253a
Reviewed-on: https://chromium-review.googlesource.com/1132266
Commit-Queue: Joey Scarr <jsca@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

The update_scripts step doesn't set '--reset'
Additionally, if there was no fetch config, don't fail trying to unset it.

Tbr: agable@chromium.org
No-Try: True
Bug: 862547
Change-Id: I90886ca7d1dd20ae59b378a5998de47dc67c60f9
Reviewed-on: https://chromium-review.googlesource.com/1137693
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

Forgot to remove it in https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1135645

No-Presubmit: True
Tbr: agable@chromium.org
Bug: 862547
Change-Id: I926959cd91a89d8e77ae000543b00b667b5eb25f
Reviewed-on: https://chromium-review.googlesource.com/1137362
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

If a ref is incorrectly set, it will affect later runs.
For example, [1] caused a bad refspec to be added to the fetch config,
(+refs/branch-heads/refs/branch-heads/6.7:refs/branch-heads/refs/branch-heads/6.7)
which affects all subsequent runs in the same bot: [2], [3].

This makes bot_update clear the fetch config before each run, so it is more
robust against these kind of failures.

[1] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8942033142534034848%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout
[2] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8941307871443322944%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout
[3] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8941333453153279680%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout

Bug: 862547
Change-Id: I2f849c604656e81ebd7377465d287226b8bdea1a
Reviewed-on: https://chromium-review.googlesource.com/1135645
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

This reverts commit d328b47e.

Reason for revert: suspect this caused https://bugs.chromium.org/p/chromium/issues/detail?id=863211

Original change's description:
> gclient: Add support for the branch:revision format.
> 
> Bug: 850812, 853032
> Change-Id: I597acbde2b3c229813b7eba8fcba52d5877130b2
> Reviewed-on: https://chromium-review.googlesource.com/1119235
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

TBR=agable@chromium.org,tandrii@chromium.org,ehmaldonado@chromium.org

Change-Id: I1e4c00b83a2840cc5a87621469da866dad58a20c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812, 853032
Reviewed-on: https://chromium-review.googlesource.com/1135893Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>

Bug: 850812, 853032
Change-Id: I597acbde2b3c229813b7eba8fcba52d5877130b2
Reviewed-on: https://chromium-review.googlesource.com/1119235
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>

Check that everything works with git nav-downstream, which we're not afraid of breaking

Bug: 832386
Change-Id: I7c7561c749bc72cc12804c4ebce8f827ed8be02b
Reviewed-on: https://chromium-review.googlesource.com/1134518Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

The code to load .netrc file copies it to a temporary file
named $TEMP/XXXXXXXXXgerrit_util/netrc before loading it
to workaround crbug.com/664664.

However, using a netrc file is deprecated (in favor of a
gitcookies file instead) so most of the time, the file is
empty. Instead of creating an empty file, load data from
/dev/null (named os.devnull).

This avoid leaving an empty file lying around (since the
temporary file is not deleted), and avoid race-conditions
on Windows when multiple processes/threads try to create
the same file.

Bug: none
Change-Id: I873b72402038ca3f0d32387353dc89b731aaa927
Reviewed-on: https://chromium-review.googlesource.com/1132998
Commit-Queue: Sylvain Defresne <sdefresne@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>

Change-Id: I64c3d143186be938042c12e2455bdb034d3bcba8
Reviewed-on: https://chromium-review.googlesource.com/1079639
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>

When a git mirror is configured, it fetches only refs/heads/*,
and possibly branch-heads and tags, but not refs/changes.

When gclient attempts to sync refs/changes/* from a mirror,
it fails, since the mirror has no such objects.

See for example [1], where the DEPS entry for builtools was
modified to sync to a CL, but all the bots failed to sync to it.

This change modifies gclient to always fetch refs/changes/*
directly from the repo, even when a mirror is configured.

[1] https://chromium-review.googlesource.com/c/chromium/src/+/1119098/6

Bug: 858894

Change-Id: I71bb313e4325a81b2985db1d00c70a8844dc7c22
Reviewed-on: https://chromium-review.googlesource.com/1119525
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>

R=machenbach@chromium.org, smut@google.com

Change-Id: Ibdad06fa32294c19347eb4f52322b050f45b6a01
Reviewed-on: https://chromium-review.googlesource.com/1127895Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: smut <smut@google.com>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>

Skip adding origin if the branch name already has 'origin/'.

Bug: 859703
Change-Id: I0141c04189d00187c202a7885a7c6bb74571e7ea
Reviewed-on: https://chromium-review.googlesource.com/1128390Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

When syncing remote branches, checkout origin/<branch> instead of
<branch>, since there might be a local branch with that name.

Bug: 859703
Change-Id: I9a49b940f53f75a97e5458fdcf578c3b52b26de2
Reviewed-on: https://chromium-review.googlesource.com/1128271Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Obviates need to call api.step.active_result by callers.

Bug: 711249
Change-Id: Ibd3278df01fc8f9d7cc73eacc72d9fc57578808d
Reviewed-on: https://chromium-review.googlesource.com/1127372Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

Today, whenever LUCI scheduler sets gitiles_commit, a revision property
is also always set, thus this CL will have no effect on today's production.

The goal of CL is to pave the way to avoid this duplication. For starters,
one could start real writing tests with

    + api.buildbucket.ci_build(..., revision='sha1')

without the need to duplicate the same revision as a property. All
recipes which WAI with such input are thus ready to work w/o 'revision'
property. After that, we'll be able to stop the duplication in LUCI scheduler.

R=hinoka

Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: I1773b1c3d2a98a9dbd1cfdd9ac2b4fb99745ee99
Reviewed-on: https://chromium-review.googlesource.com/1123713
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>

Change-Id: Ie95e10784725176c94a584fa6dadef2318142053
Reviewed-on: https://chromium-review.googlesource.com/1123650Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

gclient setdep used to assume that the version of a CIPD package was always
of the form 'version:<something>'.
However, the following are all valid:
  * SHA1 hashes that specify a single instance
  * tags (tagname:tagvalue)
  * References (e.g. 'latest') that are dynamically resolved by the CIPD server.

This change adds support for all of them:
  * GetCIPD simply returns whatever the 'version' entry is, without assuming
    it begins with 'version:'.
  * SetCIPD sets 'version' ta whatever it is passed, without prepending 'version:'
    at the beginning.

Bug: 858978
Change-Id: I53669c5df7fb51cde42e0af271139b5719a47622
Reviewed-on: https://chromium-review.googlesource.com/1120943Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/782ed0377a0a23f393a3a873194954825972b961 buildbucket recipe_module: accept typical git_repo URLs, too. (tandrii@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ic13c85b02d8271d22223ecd9c88e306d46445523
Reviewed-on: https://chromium-review.googlesource.com/1123626
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Reviewed-by: Recipe Roller <recipe-roller@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/af0fe7159155bbbf93ae62322b78291ea2131a60 Fix nits in buildbucket recipe module. (tandrii@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ib16763954eb8b264bce93cdf18eb4e77303f4e98
Reviewed-on: https://chromium-review.googlesource.com/1123623
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Reviewed-by: Recipe Roller <recipe-roller@chromium.org>

Adds '.ts' to list of extensions to be handled by clang-format.

Change-Id: Ia00e59ba3ab617ebf248cfcf6df07dd98cdc14ce
Reviewed-on: https://chromium-review.googlesource.com/1120325Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Deepanjan Roy <dproy@chromium.org>

It is not possible to use cwd argument, since none of the methods called by this
module define cwd explicitly and low-level modules like api.step do not support
it anymore, resulting in a TypeError exception thrown due to unexpected keyword
argument, e.g. see https://ci.chromium.org/swarming/task/3e74a52905a71110. Given
that any users using cwd argument would be broken anyway, it should be safe to
make this change.

Planned usage: https://crrev.com/c/1122874

R=tandrii@chromium.org, iannucci@chromium.org

Bug: 793687
Change-Id: Iadf6bf91d8d41dbc6751a848bb69d7922a206ade
Reviewed-on: https://chromium-review.googlesource.com/1122873Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/57600791c9ce0381bffff5b8d5ffef235c661a09 Remove bogus test file. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ie3173cf9044b45b1570b5283ff52d04c61d83d95
Reviewed-on: https://chromium-review.googlesource.com/1122699
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Reviewed-by: Recipe Roller <recipe-roller@chromium.org>

Follow up on https://crrev.com/c/1113036/2/my_activity.py#752

R=sergiyb@chromium.org

Change-Id: I56e4936f2f2bc94a734aeb510594ee8384d6e2a9
Reviewed-on: https://chromium-review.googlesource.com/1121816Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

R=sergiyb

Change-Id: I020580302db491e4f355000daf901223b99d91e3
Reviewed-on: https://chromium-review.googlesource.com/1113036Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

R=sergiyb

Change-Id: I97b117576f73ceb98435eda3158e22c31a5e851e
Reviewed-on: https://chromium-review.googlesource.com/1113035Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

R=phosek@chromium.org, tandrii@chromium.org, vadimsh@chromium.org

Bug: 790650
Change-Id: Ia878004d6983dbbad882ec683da2e0db5e727c66
Reviewed-on: https://chromium-review.googlesource.com/1116073
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

This is neccessary to ensure that generated ninja commands invoke python from
PATH instead of the one from VPython's VirtualEnv. On LUCI builders that replace
system python with VPython using vpython:native-python-wrapper swarming tag, a
new VPython VirtualEnv bubble will be created for each script and contain that
script's individual dependencies.

Based on https://cs.chromium.org/chromium/infra/recipes-py/recipes.py?l=88&rcl=66d9d8c66f10ca37dffa4b6d374c3d802a061b01.

R=iannucci@chromium.org

Bug: 793687
Change-Id: I633d2e0dfeda0097072daba102fb013656aaafa2
Reviewed-on: https://chromium-review.googlesource.com/1120817
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

This CL makes a couple changes:
  * The goal here is to be able to specify git cache entirely from the
    environment variable $GIT_CACHE_PATH and not require special treatment from
    bot_update. Eventually this will be specified at the swarming task level
    instead of in the recipe (i.e. "cached git" will eventually be an
    implementation detail of git on the bots and completely transparent to
    all other software).
  * Removal of the general --cache-dir option from gclient. This option was
    error-prone; it doesn't actually make sense to configure this on
    a per-invocation basis. The sole exception was `gclient config`, which
    now allows this option to be set directly.
  * Consolidation of GitWrapper.cache_dir and GitWrapper._GetMirror; previously
    these two things could disagree, leading to weird intermediate states. Now
    they're the same value.

R=agable@chromium.org, ehmaldonado@chromium.org, tandrii@chromium.org

Bug: 823434
Change-Id: I14adc7619b5fc10768ce32be2651c6215ba94aff
Reviewed-on: https://chromium-review.googlesource.com/1105473Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>

Let me make them exported via InputAPI to be used by our repo.

Change-Id: I1a0c7f49ab5a3c00b2617062444d67bbb9600837
Reviewed-on: https://chromium-review.googlesource.com/1117958
Commit-Queue: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/66d9d8c66f10ca37dffa4b6d374c3d802a061b01 Update the user guide to refer to the correct script. (gbeaty@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I8766783fafc1c67d046587fcbbcc644bc57c6d1f
Reviewed-on: https://chromium-review.googlesource.com/1119405
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Reviewed-by: Recipe Roller <recipe-roller@chromium.org>

R=iannucci@chromium.org, tandrii@chromium.org
BUG=801423

Change-Id: I28331e8b6210cbd972cc687fa71a234e75da4b95
Reviewed-on: https://chromium-review.googlesource.com/1119374Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

crrev.com/c/1097623 broke the official bots because the cipd variable
escaping present in the normal DEPS file wasn't present in the flattened
DEPS file.

Change-Id: Iaae8b95a9d8c5ce0ed80071169e731a24e98b4c4
Reviewed-on: https://chromium-review.googlesource.com/1112935
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/5c0c63e50c92e7a1a5fd81b3283f03b096abe667 Add stats and safemode to archive module. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ia8af13b0f9785237d425810b8a884755cef8a255
Reviewed-on: https://chromium-review.googlesource.com/1118863Reviewed-by: Recipe Roller <recipe-roller@chromium.org>
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>

It causes flakiness, since the package that is responsible for
printing the dependency is not always the same.

Bug: None
Change-Id: I887d740d6a903cb0d5e6c080b76feab78a54effd
Reviewed-on: https://chromium-review.googlesource.com/1116191Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/82e233fa4e6fed0a5dd99501e0d52d3dce25c994 [cipd] Make the argument to ensure a class (phosek@google.com)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9d98ce4b9ab792409ccade93df1fbcfc6056679e
Reviewed-on: https://chromium-review.googlesource.com/1117329Reviewed-by: Recipe Roller <recipe-roller@chromium.org>
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>

Bug: 855791
Change-Id: I964df1a781da4335e9b77192faf7913d3546de13
Reviewed-on: https://chromium-review.googlesource.com/1112817Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/b1546529ec629fa8ddf804916c94b6e06c44d693 Add simple `archive` module for tar + zip. (iannucci@chromium.org)


TBR=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I4a8a4f1269b9d791b87ffb720796b1d33e7cd5ae
Reviewed-on: https://chromium-review.googlesource.com/1115784
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Reviewed-by: Recipe Roller <recipe-roller@chromium.org>

Report the dependencies that were not synced, due to their, or their parent's
condition evaluating to False.

Bug: 853010
Change-Id: I375703a1b91e3c3e31e444b0df1c95ecae17b6ba
Reviewed-on: https://chromium-review.googlesource.com/1111113Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>