Runtime Stats Komparator

Usage

1. Build chrome.

Telemetry benchmark

1. Run v8.browsing benchmarks:

   $CHROMIUM_DIR/tools/perf/run_benchmark run v8.browsing_desktop \
             --browser=exact --browser-executable=$CHROMIUM_DIR/out/release/chrome \
             --story-filter='.*2020 ' \
             --also-run-disabled-tests
           

2. Install jq.
3. Convert the telemetry JSON files to callstats JSON file:

             $V8_DIR/tools/callstats-from-telemetry.sh $CHROMIUM_DIR/tools/perf/artifacts/run_XXXX
           

4. Load the generated out.json

Merged CSV from results.html

1. Open a results.html page for RCS-enabled benchmarks
2. Select "Export merged CSV" in the toolbar
3. Load the downloading .csv file normally in callstats.html

Aggregated raw txt output

1. Install scipy, e.g. sudo aptitude install python-scipy
2. Check out a known working version of webpagereply:

   git -C $CHROME_DIR/third_party/webpagereplay checkout 7dbd94752d1cde5536ffc623a9e10a51721eff1d

3. Run callstats.py with a web-page-replay archive:

   $V8_DIR/tools/callstats.py run \
           --replay-bin=$CHROME_SRC/third_party/webpagereplay/replay.py \
           --replay-wpr=$INPUT_DIR/top25.wpr \
           --js-flags="" \
           --with-chrome=$CHROME_SRC/out/Release/chrome \
           --sites-file=$INPUT_DIR/top25.json

4. Move results file to a subdirectory: mkdir $VERSION_DIR; mv *.txt $VERSION_DIR
5. Repeat from step 1 with a different configuration (e.g. --js-flags="--nolazy").
6. Create the final results file: ./callstats.py json $VERSION_DIR1 $VERSION_DIR2 > result.json
7. Use results.json on this site.