[Swarming] Let test runner exit gracefully after test failures.

The flake detection is done on the infra-side according to
the contents of the json test results. We don't want the
runner to fail after flakes.

This was controlled on the infra side by accepting any exit
codes so far. After the swarming switch, this is more
difficult, because the runner is wrapped by the swarming
collect script. There, failing exit codes can mean many
things, including network failures. Therefore, we now
force exit code 0 with test failures if those failures
are reported in the formal test results json.

The infrastructure will take care of reporting the flakes
and failures accordingly.

BUG=chromium:535160
LOG=n

Review URL: https://codereview.chromium.org/1416373005

Cr-Commit-Position: refs/heads/master@{#31740}

tools/run-tests.py

@@ -788,6 +788,11 @@ def Execute(arch, mode, args, options, suites):
   if num_tests == 0:
     print("Warning: no tests were run!")
 
+  if exit_code == 1 and options.json_test_results:
+    print("Force exit code 0 after failures. Json test results file generated "
+          "with failure information.")
+    exit_code = 0
+
   return exit_code
 
 

tools/testrunner/local/execution.py

@@ -216,8 +216,10 @@ class Runner(object):
     self.indicator.Starting()
     self._RunInternal(jobs)
     self.indicator.Done()
-    if self.failed or self.remaining:
+    if self.failed:
       return 1
+    elif self.remaining:
+      return 2
     return 0
 
   def _RunInternal(self, jobs):