Copy presubmit logic up into the presubmit recipe module.

Bug: 925774
Change-Id: I27e8670578095db71384d77cdb3d606bfdcf1077
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1684760
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>

recipes/README.recipes.md

@@ -36,6 +36,8 @@
   * [infra_paths:examples/full](#recipes-infra_paths_examples_full)
   * [osx_sdk:examples/full](#recipes-osx_sdk_examples_full)
   * [presubmit:examples/full](#recipes-presubmit_examples_full)
+  * [presubmit:tests/execute](#recipes-presubmit_tests_execute)
+  * [presubmit:tests/prepare](#recipes-presubmit_tests_prepare)
   * [tryserver:examples/full](#recipes-tryserver_examples_full)
   * [tryserver:tests/gerrit_change_fetch_ref](#recipes-tryserver_tests_gerrit_change_fetch_ref)
   * [windows_sdk:examples/full](#recipes-windows_sdk_examples_full)
@@ -739,15 +741,39 @@ Raises:
     StepFailure or InfraFailure.
 ### *recipe_modules* / [presubmit](/recipes/recipe_modules/presubmit)
 
-[DEPS](/recipes/recipe_modules/presubmit/__init__.py#1): [depot\_tools](#recipe_modules-depot_tools), [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/step][recipe_engine/recipe_modules/step]
+[DEPS](/recipes/recipe_modules/presubmit/__init__.py#11): [bot\_update](#recipe_modules-bot_update), [depot\_tools](#recipe_modules-depot_tools), [gclient](#recipe_modules-gclient), [git](#recipe_modules-git), [tryserver](#recipe_modules-tryserver), [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/cq][recipe_engine/recipe_modules/cq], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/step][recipe_engine/recipe_modules/step]
 
-#### **class [PresubmitApi](/recipes/recipe_modules/presubmit/api.py#7)([RecipeApi][recipe_engine/wkt/RecipeApi]):**
+#### **class [PresubmitApi](/recipes/recipe_modules/presubmit/api.py#11)([RecipeApi][recipe_engine/wkt/RecipeApi]):**
 
-— **def [\_\_call\_\_](/recipes/recipe_modules/presubmit/api.py#12)(self, \*args, \*\*kwargs):**
+— **def [\_\_call\_\_](/recipes/recipe_modules/presubmit/api.py#27)(self, \*args, \*\*kwargs):**
 
 Return a presubmit step.
 
-&emsp; **@property**<br>&mdash; **def [presubmit\_support\_path](/recipes/recipe_modules/presubmit/api.py#8)(self):**
+&mdash; **def [execute](/recipes/recipe_modules/presubmit/api.py#74)(self, bot_update_step):**
+
+Runs presubmit and sets summary markdown if applicable.
+
+Args:
+  bot_update_step: the StepResult from a previously executed bot_update step.
+Returns:
+  a RawResult object, suitable for being returned from RunSteps.
+
+&mdash; **def [prepare](/recipes/recipe_modules/presubmit/api.py#39)(self):**
+
+Set up a presubmit run.
+
+This includes:
+
+  - setting up the checkout w/ bot_update
+  - locally committing the applied patch
+  - running hooks, if requested
+
+This expects the gclient configuration to already have been set.
+
+Returns:
+  the StepResult from the bot_update step.
+
+&emsp; **@property**<br>&mdash; **def [presubmit\_support\_path](/recipes/recipe_modules/presubmit/api.py#23)(self):**
 ### *recipe_modules* / [tryserver](/recipes/recipe_modules/tryserver)
 
 [DEPS](/recipes/recipe_modules/tryserver/__init__.py#5): [gerrit](#recipe_modules-gerrit), [git](#recipe_modules-git), [git\_cl](#recipe_modules-git_cl), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
@@ -987,6 +1013,16 @@ Move things around in a loop!
 [DEPS](/recipes/recipe_modules/presubmit/examples/full.py#5): [presubmit](#recipe_modules-presubmit), [recipe\_engine/json][recipe_engine/recipe_modules/json]
 
 &mdash; **def [RunSteps](/recipes/recipe_modules/presubmit/examples/full.py#11)(api):**
+### *recipes* / [presubmit:tests/execute](/recipes/recipe_modules/presubmit/tests/execute.py)
+
+[DEPS](/recipes/recipe_modules/presubmit/tests/execute.py#10): [gclient](#recipe_modules-gclient), [presubmit](#recipe_modules-presubmit), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/cq][recipe_engine/recipe_modules/cq], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/runtime][recipe_engine/recipe_modules/runtime]
+
+&mdash; **def [RunSteps](/recipes/recipe_modules/presubmit/tests/execute.py#29)(api, patch_project, patch_repository_url):**
+### *recipes* / [presubmit:tests/prepare](/recipes/recipe_modules/presubmit/tests/prepare.py)
+
+[DEPS](/recipes/recipe_modules/presubmit/tests/prepare.py#9): [gclient](#recipe_modules-gclient), [presubmit](#recipe_modules-presubmit), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/runtime][recipe_engine/recipe_modules/runtime]
+
+&mdash; **def [RunSteps](/recipes/recipe_modules/presubmit/tests/prepare.py#26)(api, patch_project, patch_repository_url):**
 ### *recipes* / [tryserver:examples/full](/recipes/recipe_modules/tryserver/examples/full.py)
 
 [DEPS](/recipes/recipe_modules/tryserver/examples/full.py#5): [gerrit](#recipe_modules-gerrit), [tryserver](#recipe_modules-tryserver), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
@@ -1007,6 +1043,7 @@ Move things around in a loop!
 [recipe_engine/recipe_modules/cipd]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/9fc4304dd3a91f553108a76aa90c70c27891601a/README.recipes.md#recipe_modules-cipd
 [recipe_engine/recipe_modules/commit_position]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/9fc4304dd3a91f553108a76aa90c70c27891601a/README.recipes.md#recipe_modules-commit_position
 [recipe_engine/recipe_modules/context]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/9fc4304dd3a91f553108a76aa90c70c27891601a/README.recipes.md#recipe_modules-context
+[recipe_engine/recipe_modules/cq]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/9fc4304dd3a91f553108a76aa90c70c27891601a/README.recipes.md#recipe_modules-cq
 [recipe_engine/recipe_modules/file]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/9fc4304dd3a91f553108a76aa90c70c27891601a/README.recipes.md#recipe_modules-file
 [recipe_engine/recipe_modules/json]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/9fc4304dd3a91f553108a76aa90c70c27891601a/README.recipes.md#recipe_modules-json
 [recipe_engine/recipe_modules/path]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/9fc4304dd3a91f553108a76aa90c70c27891601a/README.recipes.md#recipe_modules-path

recipes/recipe_modules/presubmit/__init__.py

+# Copyright 2019 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+from recipe_engine.config import ConfigGroup, Single
+from recipe_engine.recipe_api import Property
+
+from PB.recipe_modules.depot_tools.presubmit import properties
+
+
 DEPS = [
+  'bot_update',
   'depot_tools',
-  'recipe_engine/json',
+  'gclient',
+  'git',
   'recipe_engine/context',
+  'recipe_engine/cq',
+  'recipe_engine/json',
   'recipe_engine/path',
+  'recipe_engine/properties',
   'recipe_engine/python',
   'recipe_engine/step',
+  'tryserver',
 ]
+
+
+PROPERTIES = properties.InputProperties

recipes/recipe_modules/presubmit/api.py

@@ -4,7 +4,22 @@
 
 from recipe_engine import recipe_api
 
+from PB.recipe_engine import result as result_pb2
+from PB.go.chromium.org.luci.buildbucket.proto import common as common_pb2
+
+
 class PresubmitApi(recipe_api.RecipeApi):
+
+  def __init__(self, properties, **kwargs):
+    super(PresubmitApi, self).__init__(**kwargs)
+
+    self._runhooks = properties.runhooks
+    # 8 minutes seems like a reasonable upper bound on presubmit timings.
+    # According to event mon data we have, it seems like anything longer than
+    # this is a bug, and should just instant fail.
+    self._timeout_s = properties.timeout_s
+    self._vpython_spec_path = properties.vpython_spec_path
+
   @property
   def presubmit_support_path(self):
     return self.repo_resource('presubmit_support.py')
@@ -20,3 +35,215 @@ class PresubmitApi(recipe_api.RecipeApi):
       step_data = self.m.python(
           name, self.presubmit_support_path, presubmit_args, **kwargs)
       return step_data.json.output
+
+  def prepare(self):
+    """Set up a presubmit run.
+
+    This includes:
+
+      - setting up the checkout w/ bot_update
+      - locally committing the applied patch
+      - running hooks, if requested
+
+    This expects the gclient configuration to already have been set.
+
+    Returns:
+      the StepResult from the bot_update step.
+    """
+    # Expect callers to have already set up their gclient configuration.
+
+    bot_update_step = self.m.bot_update.ensure_checkout()
+    relative_root = self.m.gclient.get_gerrit_patch_root().rstrip('/')
+
+    abs_root = self.m.context.cwd.join(relative_root)
+    with self.m.context(cwd=abs_root):
+      # TODO(unowned): Consider either:
+      #  - extracting user name & email address from the issue, or
+      #  - using a dedicated and clearly nonexistent name/email address
+      self.m.git('-c', 'user.email=commit-bot@chromium.org',
+              '-c', 'user.name=The Commit Bot',
+              'commit', '-a', '-m', 'Committed patch',
+              name='commit-git-patch', infra_step=False)
+
+    if self._runhooks:
+      with self.m.context(cwd=self.m.path['checkout']):
+        self.m.gclient.runhooks()
+
+    return bot_update_step
+
+  def execute(self, bot_update_step):
+    """Runs presubmit and sets summary markdown if applicable.
+
+    Args:
+      bot_update_step: the StepResult from a previously executed bot_update step.
+    Returns:
+      a RawResult object, suitable for being returned from RunSteps.
+    """
+    relative_root = self.m.gclient.get_gerrit_patch_root().rstrip('/')
+    abs_root = self.m.context.cwd.join(relative_root)
+    got_revision_properties = self.m.bot_update.get_project_revision_properties(
+        # Replace path.sep with '/', since most recipes are written assuming '/'
+        # as the delimiter. This breaks on windows otherwise.
+        relative_root.replace(self.m.path.sep, '/'), self.m.gclient.c)
+    upstream = bot_update_step.json.output['properties'].get(
+        got_revision_properties[0])
+
+    presubmit_args = [
+      '--issue', self.m.tryserver.gerrit_change.change,
+      '--patchset', self.m.tryserver.gerrit_change.patchset,
+      '--gerrit_url', 'https://%s' % self.m.tryserver.gerrit_change.host,
+      '--gerrit_fetch',
+    ]
+    if self.m.cq.state == self.m.cq.DRY:
+      presubmit_args.append('--dry_run')
+
+    presubmit_args.extend([
+      '--root', abs_root,
+      '--commit',
+      '--verbose', '--verbose',
+      '--skip_canned', 'CheckTreeIsOpen',
+      '--skip_canned', 'CheckBuildbotPendingBuilds',
+      '--upstream', upstream,  # '' if not in bot_update mode.
+    ])
+
+    venv = None
+    # TODO(iannucci): verify that presubmit_support.py correctly finds and
+    # uses .vpython files, then remove this configuration.
+    if self._vpython_spec_path:
+      venv = abs_root.join(self._vpython_spec_path)
+
+    raw_result = result_pb2.RawResult()
+    step_json = self(
+        *presubmit_args,
+        venv=venv, timeout=self._timeout_s,
+        # ok_ret='any' causes all exceptions to be ignored in this step
+        ok_ret='any')
+    # Set recipe result values
+    if step_json:
+      raw_result.summary_markdown = _createSummaryMarkdown(step_json)
+
+    retcode = self.m.step.active_result.retcode
+    if retcode == 0:
+      raw_result.status = common_pb2.SUCCESS
+      return raw_result
+
+    self.m.step.active_result.presentation.status = 'FAILURE'
+    if self.m.step.active_result.exc_result.had_timeout:
+      # TODO(iannucci): Shouldn't we also mark failure on timeouts?
+      raw_result.summary_markdown += (
+          '\n\nTimeout occurred during presubmit step.')
+    if retcode == 1:
+      raw_result.status = common_pb2.FAILURE
+      self.m.tryserver.set_test_failure_tryjob_result()
+    else:
+      raw_result.status = common_pb2.INFRA_FAILURE
+      self.m.tryserver.set_invalid_test_results_tryjob_result()
+    # Handle unexpected errors not caught by json output
+    if raw_result.summary_markdown == '':
+      raw_result.status = common_pb2.INFRA_FAILURE
+      raw_result.summary_markdown = (
+        'Something unexpected occurred'
+        ' while running presubmit checks.'
+        ' Please [file a bug](https://bugs.chromium.org'
+        '/p/chromium/issues/entry?components='
+        'Infra%3EClient%3EChrome&status=Untriaged)'
+      )
+    return raw_result
+
+
+def _limitSize(message_list, char_limit=450):
+  """Returns a list of strings within a certain character length.
+
+  Args:
+     * message_list (List[str]) - The message to truncate as a list
+       of lines (without line endings).
+  """
+  hint = ('**The complete output can be'
+          ' found at the bottom of the presubmit stdout.**')
+  char_count = 0
+  for index, message in enumerate(message_list):
+    char_count += len(message)
+    if char_count > char_limit:
+      total_errors = len(message_list)
+      oversized_msg = ('**Error size > %d chars, '
+      'there are %d more error(s) (%d total)**') % (
+        char_limit, total_errors - index, total_errors
+      )
+      if index == 0:
+        # Show at minimum part of the first error message
+        first_message = message_list[index].replace('\n\n', '\n')
+        return ['\n\n'.join(
+          _limitSize(first_message.splitlines())
+          )
+        ]
+      return message_list[:index] + [oversized_msg, hint]
+  return message_list
+
+
+def _createSummaryMarkdown(step_json):
+  """Returns a string with data on errors, warnings, and notifications.
+
+  Extracts the number of errors, warnings and notifications
+  from the dictionary(step_json).
+
+  Then it lists all the errors line by line.
+
+  Args:
+      * step_json = {
+        'errors': [
+          {
+            'message': string,
+            'long_text': string,
+            'items: [string],
+            'fatal': boolean
+          }
+        ],
+        'notifications': [
+          {
+            'message': string,
+            'long_text': string,
+            'items: [string],
+            'fatal': boolean
+          }
+        ],
+        'warnings': [
+          {
+            'message': string,
+            'long_text': string,
+            'items: [string],
+            'fatal': boolean
+          }
+        ]
+      }
+  """
+  errors = step_json['errors']
+  warning_count = len(step_json['warnings'])
+  notif_count = len(step_json['notifications'])
+  description = (
+    'There are %d error(s), %d warning(s),'
+    ' and %d notifications(s). Here are the errors:') % (
+      len(errors), warning_count, notif_count
+  )
+  error_messages = []
+
+  for error in errors:
+    # markdown represents new lines with 2 spaces
+    # replacing the \n with \n\n because \n gets replaced with an empty space.
+    # This way it will work on both markdown and plain text.
+    error_messages.append(
+      '**ERROR**\n\n%s\n\n%s' % (
+      error['message'].replace('\n', '\n\n'),
+      error['long_text'].replace('\n', '\n\n'))
+    )
+
+  error_messages = _limitSize(error_messages)
+  # Description is not counted in the total message size.
+  # It is inserted afterward to ensure it is the first message seen.
+  error_messages.insert(0, description)
+  if warning_count or notif_count:
+    error_messages.append(
+      ('To see notifications and warnings,'
+      ' look at the stdout of the presubmit step.')
+    )
+  return '\n\n'.join(error_messages)
+

recipes/recipe_modules/presubmit/properties.proto

+// Copyright 2019 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+syntax = "proto3";
+package recipe_modules.depot_tools.presubmit;
+
+message InputProperties {
+  // Whether gclient hooks should be run when setting up the checkout.
+  bool runhooks = 1;
+  // Timeout for presubmit execution, in seconds.
+  int32 timeout_s = 2;
+  // The path to the vpython spec to use, relative to the repository root.
+  string vpython_spec_path = 3;
+}

recipes/recipe_modules/presubmit/test_api.py

+# Copyright 2019 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+from recipe_engine import recipe_test_api
+
+from PB.recipe_modules.depot_tools.presubmit import properties
+
+class PresubmitTestApi(recipe_test_api.RecipeTestApi):
+
+  def __call__(self, runhooks=False, timeout_s=480, vpython_spec_path=None):
+    return self.m.properties(
+        **{
+            '$depot_tools/presubmit': properties.InputProperties(
+                runhooks=runhooks,
+                timeout_s=timeout_s,
+                vpython_spec_path=vpython_spec_path,
+            ),
+        }
+    )

recipes/recipe_modules/presubmit/tests/execute.py

+# Copyright 2019 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+import textwrap
+
+from recipe_engine import post_process
+from recipe_engine import recipe_api
+
+DEPS = [
+  'gclient',
+  'presubmit',
+  'recipe_engine/buildbucket',
+  'recipe_engine/context',
+  'recipe_engine/cq',
+  'recipe_engine/json',
+  'recipe_engine/path',
+  'recipe_engine/properties',
+  'recipe_engine/runtime',
+]
+
+
+PROPERTIES = {
+  'patch_project': recipe_api.Property(None),
+  'patch_repository_url': recipe_api.Property(None),
+}
+
+
+def RunSteps(api, patch_project, patch_repository_url):
+  api.gclient.set_config('infra')
+  with api.context(cwd=api.path['cache'].join('builder')):
+    bot_update_step = api.presubmit.prepare()
+    return api.presubmit.execute(bot_update_step)
+
+
+def GenTests(api):
+  yield (
+      api.test('success') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.step_data(
+          'presubmit',
+          api.json.output({'errors': [], 'notifications': [], 'warnings': []}),
+      ) +
+      api.post_process(post_process.StatusSuccess) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  yield (
+      api.test('cq_dry_run') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.cq(dry_run=True) +
+      api.post_process(post_process.StatusSuccess) +
+      api.post_process(post_process.StepCommandContains, 'presubmit', ['--dry_run']) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  yield (
+      api.test('root_vpython') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.presubmit(vpython_spec_path='.vpython') +
+      api.post_process(post_process.StatusSuccess) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  yield (
+      api.test('timeout') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.presubmit(timeout_s=600) +
+      api.step_data(
+          'presubmit',
+          api.json.output({'errors': [], 'notifications': [], 'warnings': []}),
+          times_out_after=1200,
+      ) +
+      api.post_process(post_process.StatusFailure) +
+      api.post_process(
+          post_process.ResultReason,
+         (u'There are 0 error(s), 0 warning(s), and 0 notifications(s).'
+          ' Here are the errors:'
+          '\n\nTimeout occurred during presubmit step.')) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  yield (
+      api.test('failure') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.step_data('presubmit', api.json.output(
+          {
+            'errors': [
+              {
+                'message': 'Missing LGTM',
+                'long_text': 'Here are some suggested OWNERS: fake@',
+                'items': [],
+                'fatal': True
+              },
+              {
+                'message': 'Syntax error in fake.py',
+                'long_text': 'Expected "," after item in list',
+                'items': [],
+                'fatal': True
+              }
+            ],
+            'notifications': [
+              {
+                'message': 'If there is a bug associated please add it.',
+                'long_text': '',
+                'items': [],
+                'fatal': False
+              }
+            ],
+            'warnings': [
+              {
+                'message': 'Line 100 has more than 80 characters',
+                'long_text': '',
+                'items': [],
+                'fatal': False
+              }
+            ]
+          }, retcode=1)
+      ) +
+      api.post_process(post_process.StatusFailure) +
+      api.post_process(post_process.ResultReason, textwrap.dedent(u'''
+          There are 2 error(s), 1 warning(s), and 1 notifications(s). Here are the errors:
+
+          **ERROR**
+
+          Missing LGTM
+
+          Here are some suggested OWNERS: fake@
+
+          **ERROR**
+
+          Syntax error in fake.py
+
+          Expected "," after item in list
+
+          To see notifications and warnings, look at the stdout of the presubmit step.
+        ''').strip()
+      ) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  long_message = (u'Here are some suggested OWNERS:' +
+    u'\nreallyLongFakeAccountNameEmail@chromium.org' * 10)
+  yield (
+      api.test('failure-long-message') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.step_data('presubmit', api.json.output(
+          {
+            'errors': [
+              {
+                'message': 'Missing LGTM',
+                'long_text': long_message,
+                'items': [],
+                'fatal': True
+              }
+            ],
+            'notifications': [],
+            'warnings': []
+          }, retcode=1)
+      ) +
+      api.post_process(post_process.StatusFailure) +
+      api.post_process(post_process.ResultReason, textwrap.dedent('''
+          There are 1 error(s), 0 warning(s), and 0 notifications(s). Here are the errors:
+
+          **ERROR**
+
+          Missing LGTM
+
+          Here are some suggested OWNERS:
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          reallyLongFakeAccountNameEmail@chromium.org
+
+          **Error size > 450 chars, there are 1 more error(s) (13 total)**
+
+          **The complete output can be found at the bottom of the presubmit stdout.**
+        ''').strip()
+      ) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  yield (
+      api.test('infra-failure') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.step_data('presubmit', api.json.output(
+          {
+            'errors': [
+              {
+                'message': 'Infra Failure',
+                'long_text': '',
+                'items': [],
+                'fatal': True
+              }
+            ],
+            'notifications': [],
+            'warnings': []
+          }, retcode=2)
+      ) +
+      api.post_process(post_process.StatusFailure) +
+      api.post_process(post_process.ResultReason, textwrap.dedent(u'''
+          There are 1 error(s), 0 warning(s), and 0 notifications(s). Here are the errors:
+
+          **ERROR**
+
+          Infra Failure
+
+        ''').lstrip()
+      ) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  bug_msg = (
+    'Something unexpected occurred'
+    ' while running presubmit checks.'
+    ' Please [file a bug](https://bugs.chromium.org'
+    '/p/chromium/issues/entry?components='
+    'Infra%3EClient%3EChrome&status=Untriaged)'
+  )
+  yield (
+      api.test('failure-no-json') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.step_data('presubmit', api.json.output(None, retcode=1)) +
+      api.post_process(post_process.StatusFailure) +
+      api.post_process(post_process.ResultReason, bug_msg) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  yield (
+      api.test('infra-failure-no-json') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.step_data('presubmit', api.json.output(None, retcode=2)) +
+      api.post_process(post_process.StatusFailure) +
+      api.post_process(post_process.ResultReason, bug_msg) +
+      api.post_process(post_process.DropExpectation)
+  )
+

recipes/recipe_modules/presubmit/tests/prepare.py

+# Copyright 2019 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+from recipe_engine import post_process
+from recipe_engine import recipe_api
+
+
+DEPS = [
+  'gclient',
+  'presubmit',
+  'recipe_engine/buildbucket',
+  'recipe_engine/context',
+  'recipe_engine/path',
+  'recipe_engine/properties',
+  'recipe_engine/runtime',
+]
+
+
+PROPERTIES = {
+  'patch_project': recipe_api.Property(None),
+  'patch_repository_url': recipe_api.Property(None),
+}
+
+
+def RunSteps(api, patch_project, patch_repository_url):
+  api.gclient.set_config('infra')
+  with api.context(cwd=api.path['cache'].join('builder')):
+    bot_update_step = api.presubmit.prepare()
+
+
+def GenTests(api):
+  yield (
+      api.test('basic') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.post_process(post_process.StatusSuccess) +
+      api.post_process(post_process.DropExpectation)
+  )
+
+  yield (
+      api.test('runhooks') +
+      api.runtime(is_experimental=False, is_luci=True) +
+      api.buildbucket.try_build(project='infra') +
+      api.presubmit(runhooks=True) +
+      api.post_process(post_process.MustRun, 'gclient runhooks') +
+      api.post_process(post_process.StatusSuccess) +
+      api.post_process(post_process.DropExpectation)
+  )