Commit a974dd7e authored by Paolo Severini's avatar Paolo Severini Committed by Commit Bot

[Test] Make CSuite benchmark runner work better on Windows

The csuite.py script does not work correctly on Windows. It runs
correctly in baseline mode, but there are two problems when running in
compare mode:

1. In compare mode the output of benchmark.py is piped to the
   compare-baseline.py script, but Windows only execute python files if
   python.exe is the default program to open '.py' files, and this is
   not the case, by default, when python is installed as part of the
   depot_tools.

   Fix: explicitly add the 'python' command before compare-baseline.py.

2. By default CSuite prints the results to stdout using escapes codes
   that add color highlights. But this does not work on Windows when
   compare-baseline.py is launched with a pipe:

   python test/benchmarks/csuite/benchmark.py <...> |
       python test/benchmarks/csuite/compare-baseline.py <baseline_results>

   Fix: Do not use a pipe. Write the benchmark numbers for the
   compare-run into a separate file, and pass the path to this file to
   compare-baseline.py

Change-Id: Ic22d5bd4b47901f0ba0f35bc2496441346d21c6a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2656855Reviewed-by: 's avatarMichael Stanton <mvstanton@chromium.org>
Commit-Queue: Paolo Severini <paolosev@microsoft.com>
Cr-Commit-Position: refs/heads/master@{#72807}
parent 30cc618c
......@@ -144,14 +144,23 @@ if __name__ == '__main__':
if mode == "baseline":
cmdline = "%s > %s" % (cmdline_base, output_file)
else:
cmdline = "%s | %s %s" \
% (cmdline_base, compare_baseline_py_path, output_file)
output_file_compare = output_file + "_compare"
cmdline = "%s > %s" % (cmdline_base, output_file_compare)
if opts.verbose:
print("Spawning subprocess: %s." % cmdline)
return_code = subprocess.call(cmdline, shell=True, cwd=suite_path)
if return_code < 0:
print("Error return code: %d." % return_code)
if mode == "baseline":
print("Wrote %s." % output_file)
print("Run %s again with compare mode to see results." % suite)
else:
print("Wrote %s." % output_file_compare)
cmdline = "python %s %s -f %s" % (compare_baseline_py_path, output_file, output_file_compare)
if opts.verbose:
print("Spawning subprocess: %s." % cmdline)
return_code = subprocess.call(cmdline, shell=True, cwd=suite_path)
if return_code < 0:
print("Error return code: %d." % return_code)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment