Skip to content

C
Chromium_Depot_tools
Commit c3d1208d authored by Michael Achenbach's avatar Michael Achenbach Committed by Commit Bot
Browse files
Options

Revert "bot_update recipe: Upload source manifest" 

This reverts commit 037f9b80.

Reason for revert: The structural change to manifest breaks the recipe
tester:
https://cs.chromium.org/chromium/infra/recipes/recipes/recipe_roll_tryjob.py?q=recipe_roll_tryjob.py&sq=package:chromium&dr&l=149
https://crbug.com/776299

Original change's description:
> bot_update recipe: Upload source manifest
> 
> Bug: 772529
> Change-Id: I3f833740ddfbc71be42b140c99dcd4bbfeed54fe
> Reviewed-on: https://chromium-review.googlesource.com/706071
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

TBR=iannucci@chromium.org,hinoka@chromium.org,agable@chromium.org

Change-Id: Iba9653d73b7f117ca2f94619ae5da900cd0c8c98
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 772529,776299
Reviewed-on: https://chromium-review.googlesource.com/728059Reviewed-by: 's avatarMichael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
parent 037f9b80
Hide whitespace changes
Inline Side-by-side
Showing with 22 additions and 363 deletions
recipes/README.recipes.md
View file @ c3d1208d
...@@ -38,7 +38,7 @@ ...@@ -38,7 +38,7 @@
### *recipe_modules* / [bot\_update](/recipes/recipe_modules/bot_update) ### *recipe_modules* / [bot\_update](/recipes/recipe_modules/bot_update)
[DEPS](/recipes/recipe_modules/bot_update/__init__.py#1): [depot\_tools](#recipe_modules-depot_tools), [gclient](#recipe_modules-gclient), [gerrit](#recipe_modules-gerrit), [rietveld](#recipe_modules-rietveld), [tryserver](#recipe_modules-tryserver), [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/source\_manifest][recipe_engine/recipe_modules/source_manifest], [recipe\_engine/step][recipe_engine/recipe_modules/step] [DEPS](/recipes/recipe_modules/bot_update/__init__.py#1): [depot\_tools](#recipe_modules-depot_tools), [gclient](#recipe_modules-gclient), [gerrit](#recipe_modules-gerrit), [rietveld](#recipe_modules-rietveld), [tryserver](#recipe_modules-tryserver), [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
Recipe module to ensure a checkout is consistent on a bot. Recipe module to ensure a checkout is consistent on a bot.
...@@ -50,12 +50,12 @@ Wrapper for easy calling of bot_update. ...@@ -50,12 +50,12 @@ Wrapper for easy calling of bot_update.
&mdash; **def [apply\_gerrit\_ref](/recipes/recipe_modules/bot_update/api.py#49)(self, root, gerrit_no_reset=False, gerrit_no_rebase_patch_ref=False, gerrit_repo=None, gerrit_ref=None, step_name='apply_gerrit', \*\*kwargs):** &mdash; **def [apply\_gerrit\_ref](/recipes/recipe_modules/bot_update/api.py#49)(self, root, gerrit_no_reset=False, gerrit_no_rebase_patch_ref=False, gerrit_repo=None, gerrit_ref=None, step_name='apply_gerrit', \*\*kwargs):**
&mdash; **def [deapply\_patch](/recipes/recipe_modules/bot_update/api.py#445)(self, bot_update_step):** &mdash; **def [deapply\_patch](/recipes/recipe_modules/bot_update/api.py#434)(self, bot_update_step):**
Deapplies a patch, taking care of DEPS and solution revisions properly. Deapplies a patch, taking care of DEPS and solution revisions properly.
&mdash; **def [ensure\_checkout](/recipes/recipe_modules/bot_update/api.py#71)(self, gclient_config=None, suffix=None, patch=True, update_presentation=True, patch_root=None, no_shallow=False, with_branch_heads=False, with_tags=False, refs=None, patch_oauth2=False, oauth2_json=False, use_site_config_creds=True, clobber=False, root_solution_revision=None, rietveld=None, issue=None, patchset=None, gerrit_no_reset=False, gerrit_no_rebase_patch_ref=False, disable_syntax_validation=False, manifest_name=None, \*\*kwargs):** &mdash; **def [ensure\_checkout](/recipes/recipe_modules/bot_update/api.py#71)(self, gclient_config=None, suffix=None, patch=True, update_presentation=True, patch_root=None, no_shallow=False, with_branch_heads=False, with_tags=False, refs=None, patch_oauth2=False, oauth2_json=False, use_site_config_creds=True, clobber=False, root_solution_revision=None, rietveld=None, issue=None, patchset=None, gerrit_no_reset=False, gerrit_no_rebase_patch_ref=False, disable_syntax_validation=False, \*\*kwargs):**
Args: Args:
use_site_config_creds: If the oauth2 credentials are in the buildbot use_site_config_creds: If the oauth2 credentials are in the buildbot
...@@ -71,10 +71,8 @@ Args: ...@@ -71,10 +71,8 @@ Args:
disable_syntax_validation: (legacy) Disables syntax validation for DEPS. disable_syntax_validation: (legacy) Disables syntax validation for DEPS.
Needed as migration paths for recipes dealing with older revisions, Needed as migration paths for recipes dealing with older revisions,
such as bisect. such as bisect.
manifest_name: The name of the manifest to upload to LogDog. This must
be unique for the whole build.
&mdash; **def [get\_project\_revision\_properties](/recipes/recipe_modules/bot_update/api.py#422)(self, project_name, gclient_config=None):** &mdash; **def [get\_project\_revision\_properties](/recipes/recipe_modules/bot_update/api.py#411)(self, project_name, gclient_config=None):**
Returns all property names used for storing the checked-out revision of Returns all property names used for storing the checked-out revision of
a given project. a given project.
...@@ -841,7 +839,6 @@ Move things around in a loop! ...@@ -841,7 +839,6 @@ Move things around in a loop!
[recipe_engine/recipe_modules/properties]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-properties [recipe_engine/recipe_modules/properties]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-properties
[recipe_engine/recipe_modules/python]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-python [recipe_engine/recipe_modules/python]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-python
[recipe_engine/recipe_modules/raw_io]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-raw_io [recipe_engine/recipe_modules/raw_io]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-raw_io
[recipe_engine/recipe_modules/source_manifest]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-source_manifest
[recipe_engine/recipe_modules/step]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-step [recipe_engine/recipe_modules/step]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-step
[recipe_engine/recipe_modules/url]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-url [recipe_engine/recipe_modules/url]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/README.recipes.md#recipe_modules-url
[recipe_engine/wkt/RecipeApi]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/recipe_engine/recipe_api.py#992 [recipe_engine/wkt/RecipeApi]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/c07de907b2223efcacbf45d7b793fc0d8a964f31/recipe_engine/recipe_api.py#992
recipes/recipe_modules/bot_update/__init__.py
View file @ c3d1208d
...@@ -9,7 +9,6 @@ DEPS = [ ...@@ -9,7 +9,6 @@ DEPS = [
'recipe_engine/properties', 'recipe_engine/properties',
'recipe_engine/python', 'recipe_engine/python',
'recipe_engine/raw_io', 'recipe_engine/raw_io',
'recipe_engine/source_manifest',
'recipe_engine/step', 'recipe_engine/step',
'rietveld', 'rietveld',
'tryserver', 'tryserver',
......
recipes/recipe_modules/bot_update/api.py
View file @ c3d1208d
...@@ -77,8 +77,7 @@ class BotUpdateApi(recipe_api.RecipeApi): ...@@ -77,8 +77,7 @@ class BotUpdateApi(recipe_api.RecipeApi):
root_solution_revision=None, rietveld=None, issue=None, root_solution_revision=None, rietveld=None, issue=None,
patchset=None, gerrit_no_reset=False, patchset=None, gerrit_no_reset=False,
gerrit_no_rebase_patch_ref=False, gerrit_no_rebase_patch_ref=False,
disable_syntax_validation=False, manifest_name=None, disable_syntax_validation=False, **kwargs):
**kwargs):
""" """
Args: Args:
use_site_config_creds: If the oauth2 credentials are in the buildbot use_site_config_creds: If the oauth2 credentials are in the buildbot
...@@ -94,8 +93,6 @@ class BotUpdateApi(recipe_api.RecipeApi): ...@@ -94,8 +93,6 @@ class BotUpdateApi(recipe_api.RecipeApi):
disable_syntax_validation: (legacy) Disables syntax validation for DEPS. disable_syntax_validation: (legacy) Disables syntax validation for DEPS.
Needed as migration paths for recipes dealing with older revisions, Needed as migration paths for recipes dealing with older revisions,
such as bisect. such as bisect.
manifest_name: The name of the manifest to upload to LogDog. This must
be unique for the whole build.
""" """
refs = refs or [] refs = refs or []
# We can re-use the gclient spec from the gclient module, since all the # We can re-use the gclient spec from the gclient module, since all the
...@@ -264,6 +261,7 @@ class BotUpdateApi(recipe_api.RecipeApi): ...@@ -264,6 +261,7 @@ class BotUpdateApi(recipe_api.RecipeApi):
root, first_sln, reverse_rev_map, self._fail_patch, root, first_sln, reverse_rev_map, self._fail_patch,
fixed_revisions=fixed_revisions) fixed_revisions=fixed_revisions)
# Add suffixes to the step name, if specified.
name = 'bot_update' name = 'bot_update'
if not patch: if not patch:
name += ' (without patch)' name += ' (without patch)'
...@@ -298,15 +296,6 @@ class BotUpdateApi(recipe_api.RecipeApi): ...@@ -298,15 +296,6 @@ class BotUpdateApi(recipe_api.RecipeApi):
step_text = result['step_text'] step_text = result['step_text']
step_result.presentation.step_text = step_text step_result.presentation.step_text = step_text
# Export the step results as a Source Manifest to LogDog.
if manifest_name:
if not patch:
# The param "patched" is purely cosmetic to mean "if false, this
# bot_update run exists purely to unpatch an existing patch".
manifest_name += '_unpatched'
self.m.source_manifest.set_json_manifest(
manifest_name, result.get('manifest', {}))
# Set the "checkout" path for the main solution. # Set the "checkout" path for the main solution.
# This is used by the Chromium module to figure out where to look for # This is used by the Chromium module to figure out where to look for
# the checkout. # the checkout.
......
recipes/recipe_modules/bot_update/examples/full.expected/with_manifest_name.json deleted 100644 → 0
View file @ 037f9b80
[
{
"cmd": [
"python",
"-u",
"RECIPE_MODULE[depot_tools::bot_update]/resources/bot_update.py",
"--spec-path",
"cache_dir = '[GIT_CACHE]'\nsolutions = [{'deps_file': '.DEPS.git', 'managed': True, 'name': 'src', 'url': 'https://chromium.googlesource.com/chromium/src.git'}]",
"--patch_root",
"src",
"--revision_mapping_file",
"{\"got_angle_revision\": \"src/third_party/angle\", \"got_cr_revision\": \"src\", \"got_revision\": \"src\", \"got_v8_revision\": \"src/v8\"}",
"--git-cache-dir",
"[GIT_CACHE]",
"--cleanup-dir",
"[CLEANUP]/bot_update",
"--output_json",
"/path/to/tmp/json",
"--revision",
"src@HEAD",
"--disable-syntax-validation"
],
"env_prefixes": {
"PATH": [
"RECIPE_PACKAGE_REPO[depot_tools]"
]
},
"infra_step": true,
"name": "bot_update",
"~followup_annotations": [
"@@@STEP_TEXT@Some step text@@@",
"@@@STEP_LOG_LINE@json.output@{@@@",
"@@@STEP_LOG_LINE@json.output@ \"did_run\": true, @@@",
"@@@STEP_LOG_LINE@json.output@ \"fixed_revisions\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"src\": \"HEAD\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"manifest\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"src\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"src/third_party/angle\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src/third_party/angle.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"fac9503c46405f77757b9a728eb85b8d7bc6080c\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"src/v8\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src/v8.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"801ada225ddc271c132c3a35f03975671d43e399\"@@@",
"@@@STEP_LOG_LINE@json.output@ }@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"patch_failure\": false, @@@",
"@@@STEP_LOG_LINE@json.output@ \"patch_root\": \"src\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"properties\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"got_angle_revision\": \"fac9503c46405f77757b9a728eb85b8d7bc6080c\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_angle_revision_cp\": \"refs/heads/master@{#297276}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_cr_revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_cr_revision_cp\": \"refs/heads/master@{#170242}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_revision_cp\": \"refs/heads/master@{#170242}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_v8_revision\": \"801ada225ddc271c132c3a35f03975671d43e399\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_v8_revision_cp\": \"refs/heads/master@{#43426}\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"root\": \"src\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"step_text\": \"Some step text\"@@@",
"@@@STEP_LOG_LINE@json.output@}@@@",
"@@@STEP_LOG_END@json.output@@@",
"@@@SET_BUILD_PROPERTY@got_angle_revision@\"fac9503c46405f77757b9a728eb85b8d7bc6080c\"@@@",
"@@@SET_BUILD_PROPERTY@got_angle_revision_cp@\"refs/heads/master@{#297276}\"@@@",
"@@@SET_BUILD_PROPERTY@got_cr_revision@\"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\"@@@",
"@@@SET_BUILD_PROPERTY@got_cr_revision_cp@\"refs/heads/master@{#170242}\"@@@",
"@@@SET_BUILD_PROPERTY@got_revision@\"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\"@@@",
"@@@SET_BUILD_PROPERTY@got_revision_cp@\"refs/heads/master@{#170242}\"@@@",
"@@@SET_BUILD_PROPERTY@got_v8_revision@\"801ada225ddc271c132c3a35f03975671d43e399\"@@@",
"@@@SET_BUILD_PROPERTY@got_v8_revision_cp@\"refs/heads/master@{#43426}\"@@@"
]
},
{
"cmd": [
"python",
"-u",
"RECIPE_MODULE[depot_tools::bot_update]/resources/bot_update.py",
"--spec-path",
"cache_dir = '[GIT_CACHE]'\nsolutions = [{'deps_file': '.DEPS.git', 'managed': True, 'name': 'src', 'url': 'https://chromium.googlesource.com/chromium/src.git'}]",
"--patch_root",
"src",
"--revision_mapping_file",
"{\"got_angle_revision\": \"src/third_party/angle\", \"got_cr_revision\": \"src\", \"got_revision\": \"src\", \"got_v8_revision\": \"src/v8\"}",
"--git-cache-dir",
"[GIT_CACHE]",
"--cleanup-dir",
"[CLEANUP]/bot_update",
"--output_json",
"/path/to/tmp/json",
"--revision",
"src@f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9"
],
"env_prefixes": {
"PATH": [
"RECIPE_PACKAGE_REPO[depot_tools]"
]
},
"infra_step": true,
"name": "bot_update (without patch)",
"~followup_annotations": [
"@@@STEP_TEXT@Some step text@@@",
"@@@STEP_LOG_LINE@json.output@{@@@",
"@@@STEP_LOG_LINE@json.output@ \"did_run\": true, @@@",
"@@@STEP_LOG_LINE@json.output@ \"fixed_revisions\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"src\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"manifest\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"src\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"src/third_party/angle\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src/third_party/angle.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"fac9503c46405f77757b9a728eb85b8d7bc6080c\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"src/v8\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src/v8.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"801ada225ddc271c132c3a35f03975671d43e399\"@@@",
"@@@STEP_LOG_LINE@json.output@ }@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"patch_failure\": false, @@@",
"@@@STEP_LOG_LINE@json.output@ \"patch_root\": \"src\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"properties\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"got_angle_revision\": \"fac9503c46405f77757b9a728eb85b8d7bc6080c\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_angle_revision_cp\": \"refs/heads/master@{#297276}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_cr_revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_cr_revision_cp\": \"refs/heads/master@{#170242}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_revision_cp\": \"refs/heads/master@{#170242}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_v8_revision\": \"801ada225ddc271c132c3a35f03975671d43e399\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_v8_revision_cp\": \"refs/heads/master@{#43426}\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"root\": \"src\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"step_text\": \"Some step text\"@@@",
"@@@STEP_LOG_LINE@json.output@}@@@",
"@@@STEP_LOG_END@json.output@@@"
]
},
{
"name": "$result",
"recipe_result": null,
"status_code": 0
}
]
\ No newline at end of file
recipes/recipe_modules/bot_update/examples/full.expected/with_manifest_name_no_patch.json deleted 100644 → 0
View file @ 037f9b80
[
{
"cmd": [
"python",
"-u",
"RECIPE_MODULE[depot_tools::bot_update]/resources/bot_update.py",
"--spec-path",
"cache_dir = '[GIT_CACHE]'\nsolutions = [{'deps_file': '.DEPS.git', 'managed': True, 'name': 'src', 'url': 'https://chromium.googlesource.com/chromium/src.git'}]",
"--patch_root",
"src",
"--revision_mapping_file",
"{\"got_angle_revision\": \"src/third_party/angle\", \"got_cr_revision\": \"src\", \"got_revision\": \"src\", \"got_v8_revision\": \"src/v8\"}",
"--git-cache-dir",
"[GIT_CACHE]",
"--cleanup-dir",
"[CLEANUP]/bot_update",
"--output_json",
"/path/to/tmp/json",
"--revision",
"src@HEAD",
"--disable-syntax-validation"
],
"env_prefixes": {
"PATH": [
"RECIPE_PACKAGE_REPO[depot_tools]"
]
},
"infra_step": true,
"name": "bot_update (without patch)",
"~followup_annotations": [
"@@@STEP_TEXT@Some step text@@@",
"@@@STEP_LOG_LINE@json.output@{@@@",
"@@@STEP_LOG_LINE@json.output@ \"did_run\": true, @@@",
"@@@STEP_LOG_LINE@json.output@ \"fixed_revisions\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"src\": \"HEAD\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"manifest\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"src\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"src/third_party/angle\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src/third_party/angle.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"fac9503c46405f77757b9a728eb85b8d7bc6080c\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"src/v8\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"repository\": \"https://fake.org/src/v8.git\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"revision\": \"801ada225ddc271c132c3a35f03975671d43e399\"@@@",
"@@@STEP_LOG_LINE@json.output@ }@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"patch_failure\": false, @@@",
"@@@STEP_LOG_LINE@json.output@ \"patch_root\": \"src\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"properties\": {@@@",
"@@@STEP_LOG_LINE@json.output@ \"got_angle_revision\": \"fac9503c46405f77757b9a728eb85b8d7bc6080c\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_angle_revision_cp\": \"refs/heads/master@{#297276}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_cr_revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_cr_revision_cp\": \"refs/heads/master@{#170242}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_revision\": \"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_revision_cp\": \"refs/heads/master@{#170242}\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_v8_revision\": \"801ada225ddc271c132c3a35f03975671d43e399\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"got_v8_revision_cp\": \"refs/heads/master@{#43426}\"@@@",
"@@@STEP_LOG_LINE@json.output@ }, @@@",
"@@@STEP_LOG_LINE@json.output@ \"root\": \"src\", @@@",
"@@@STEP_LOG_LINE@json.output@ \"step_text\": \"Some step text\"@@@",
"@@@STEP_LOG_LINE@json.output@}@@@",
"@@@STEP_LOG_END@json.output@@@",
"@@@SET_BUILD_PROPERTY@got_angle_revision@\"fac9503c46405f77757b9a728eb85b8d7bc6080c\"@@@",
"@@@SET_BUILD_PROPERTY@got_angle_revision_cp@\"refs/heads/master@{#297276}\"@@@",
"@@@SET_BUILD_PROPERTY@got_cr_revision@\"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\"@@@",
"@@@SET_BUILD_PROPERTY@got_cr_revision_cp@\"refs/heads/master@{#170242}\"@@@",
"@@@SET_BUILD_PROPERTY@got_revision@\"f27fede2220bcd326aee3e86ddfd4ebd0fe58cb9\"@@@",
"@@@SET_BUILD_PROPERTY@got_revision_cp@\"refs/heads/master@{#170242}\"@@@",
"@@@SET_BUILD_PROPERTY@got_v8_revision@\"801ada225ddc271c132c3a35f03975671d43e399\"@@@",
"@@@SET_BUILD_PROPERTY@got_v8_revision_cp@\"refs/heads/master@{#43426}\"@@@"
]
},
{
"name": "$result",
"recipe_result": null,
"status_code": 0
}
]
\ No newline at end of file
recipes/recipe_modules/bot_update/examples/full.py
View file @ c3d1208d
...@@ -49,7 +49,6 @@ def RunSteps(api): ...@@ -49,7 +49,6 @@ def RunSteps(api):
gerrit_no_reset = True if api.properties.get('gerrit_no_reset') else False gerrit_no_reset = True if api.properties.get('gerrit_no_reset') else False
gerrit_no_rebase_patch_ref = bool( gerrit_no_rebase_patch_ref = bool(
api.properties.get('gerrit_no_rebase_patch_ref')) api.properties.get('gerrit_no_rebase_patch_ref'))
manifest_name = api.properties.get('manifest_name')
if api.properties.get('test_apply_gerrit_ref'): if api.properties.get('test_apply_gerrit_ref'):
prop2arg = { prop2arg = {
...@@ -80,8 +79,7 @@ def RunSteps(api): ...@@ -80,8 +79,7 @@ def RunSteps(api):
suffix=suffix, suffix=suffix,
gerrit_no_reset=gerrit_no_reset, gerrit_no_reset=gerrit_no_reset,
gerrit_no_rebase_patch_ref=gerrit_no_rebase_patch_ref, gerrit_no_rebase_patch_ref=gerrit_no_rebase_patch_ref,
disable_syntax_validation=True, disable_syntax_validation=True)
manifest_name=manifest_name)
if patch: if patch:
api.bot_update.deapply_patch(bot_update_step) api.bot_update.deapply_patch(bot_update_step)
...@@ -91,13 +89,6 @@ def GenTests(api): ...@@ -91,13 +89,6 @@ def GenTests(api):
patch=False, patch=False,
revision='abc' revision='abc'
) )
yield api.test('with_manifest_name_no_patch') + api.properties(
manifest_name='checkout',
patch=False
)
yield api.test('with_manifest_name') + api.properties(
manifest_name='checkout'
)
yield api.test('buildbot') + api.properties( yield api.test('buildbot') + api.properties(
path_config='buildbot', path_config='buildbot',
patch=False, patch=False,
......
recipes/recipe_modules/bot_update/resources/bot_update.py
View file @ c3d1208d
...@@ -49,6 +49,9 @@ COMMIT_FOOTER_ENTRY_RE = re.compile(r'([^:]+):\s*(.*)') ...@@ -49,6 +49,9 @@ COMMIT_FOOTER_ENTRY_RE = re.compile(r'([^:]+):\s*(.*)')
COMMIT_POSITION_FOOTER_KEY = 'Cr-Commit-Position' COMMIT_POSITION_FOOTER_KEY = 'Cr-Commit-Position'
COMMIT_ORIGINAL_POSITION_FOOTER_KEY = 'Cr-Original-Commit-Position' COMMIT_ORIGINAL_POSITION_FOOTER_KEY = 'Cr-Original-Commit-Position'
# Regular expression to parse gclient's revinfo entries.
REVINFO_RE = re.compile(r'^([^:]+):\s+([^@]+)@(.+)$')
# Copied from scripts/recipes/chromium.py. # Copied from scripts/recipes/chromium.py.
GOT_REVISION_MAPPINGS = { GOT_REVISION_MAPPINGS = {
...@@ -361,75 +364,18 @@ def gclient_revinfo(): ...@@ -361,75 +364,18 @@ def gclient_revinfo():
return call_gclient('revinfo', '-a') or '' return call_gclient('revinfo', '-a') or ''
def normalize_git_url(url): def create_manifest():
"""Normalize a git url to be consistent. manifest = {}
output = gclient_revinfo()
This recognizes urls to the googlesoruce.com domain. It ensures that for line in output.strip().splitlines():
the url: match = REVINFO_RE.match(line.strip())
* Do not end in .git if match:
* Do not contain /a/ in their path. manifest[match.group(1)] = {
""" 'repository': match.group(2),
try: 'revision': match.group(3),
p = urlparse.urlparse(url)
except Exception:
# Not a url, just return it back.
return url
if not p.netloc.endswith('.googlesource.com'):
# Not a googlesource.com URL, can't normalize this, just return as is.
return url
path = p.path
if path.startswith('/a'):
path = path[len('/a'):]
if path.endswith('.git'):
path = path[:-len('.git')]
return 'https://%s%s' % (p.netloc, path)
# TODO(hinoka): Include patch revision.
def create_manifest(gclient_output, patch_root, gerrit_ref):
"""Return the JSONPB equivilent of the source manifest proto.
The source manifest proto is defined here:
https://chromium.googlesource.com/infra/luci/recipes-py/+/master/recipe_engine/source_manifest.proto
This is based off of:
* The gclient_output (from calling gclient.py --output-json) which contains
the directory -> repo:revision mapping.
* Gerrit Patch info which contains info about patched revisions.
We normalize the URLs such that if they are googlesource.com urls, they:
"""
manifest = {
'version': 0, # Currently the only valid version is 0.
}
dirs = {}
if patch_root:
patch_root = patch_root.strip('/') # Normalize directory names.
for directory, info in gclient_output.get('solutions', {}).iteritems():
directory = directory.strip('/') # Normalize the directory name.
# There are two places to the the revision from, we do it in this order:
# 1. In the "revision" field
# 2. At the end of the URL, after @
repo = ''
revision = info.get('revision', '')
# The format of the url is "https://repo.url/blah.git@abcdefabcdef" or
# just "https://repo.url/blah.git"
url_split = info.get('url', '').split('@')
if not revision and len(url_split) == 2:
revision = url_split[1]
if url_split:
repo = normalize_git_url(url_split[0])
if repo:
dirs[directory] = {
'git_checkout': {
'repo_url': repo,
'revision': revision,
}
} }
if patch_root == directory: else:
dirs[directory]['git_checkout']['patch_fetch_ref'] = gerrit_ref print "WARNING: Couldn't match revinfo line:\n%s" % line
manifest['directories'] = dirs
return manifest return manifest
...@@ -1158,8 +1104,7 @@ def checkout(options, git_slns, specs, revisions, step_text, shallow): ...@@ -1158,8 +1104,7 @@ def checkout(options, git_slns, specs, revisions, step_text, shallow):
step_text=step_text, step_text=step_text,
fixed_revisions=revisions, fixed_revisions=revisions,
properties=got_revisions, properties=got_revisions,
manifest=create_manifest( manifest=create_manifest())
gclient_output, options.patch_root, options.gerrit_ref))
def print_debug_info(): def print_debug_info():
......
tests/bot_update_coverage_test.py
View file @ c3d1208d
...@@ -245,38 +245,6 @@ class BotUpdateUnittests(unittest.TestCase): ...@@ -245,38 +245,6 @@ class BotUpdateUnittests(unittest.TestCase):
setattr(os, 'remove', old_os_remove) setattr(os, 'remove', old_os_remove)
self.assertTrue(os.path.join(path, lockfile) in removed) self.assertTrue(os.path.join(path, lockfile) in removed)
def testGenerateManifestsBasic(self):
gclient_output = {
'solutions': {
'breakpad/': {
'revision': None,
'scm': None,
'url': ('https://chromium.googlesource.com/breakpad/breakpad.git' +
'@5f638d532312685548d5033618c8a36f73302d0a')
},
"src/": {
'revision': 'f671d3baeb64d9dba628ad582e867cf1aebc0207',
'scm': None,
'url': 'https://chromium.googlesource.com/a/chromium/src.git'
},
}
}
out = bot_update.create_manifest(gclient_output, None, None)
self.assertEquals(len(out['directories']), 2)
print out
self.assertEquals(
out['directories']['src']['git_checkout']['revision'],
'f671d3baeb64d9dba628ad582e867cf1aebc0207')
self.assertEquals(
out['directories']['src']['git_checkout']['repo_url'],
'https://chromium.googlesource.com/chromium/src')
self.assertEquals(
out['directories']['breakpad']['git_checkout']['revision'],
'5f638d532312685548d5033618c8a36f73302d0a')
self.assertEquals(
out['directories']['breakpad']['git_checkout']['repo_url'],
'https://chromium.googlesource.com/breakpad/breakpad')
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment