- 16 Mar, 2016 1 commit
-
-
machenbach authored
The feature was removed from the bots a while ago. It was superseeded by the flaky-test detection which reruns tests. Remaining tests still marked as flaky most certainly pass since a while. Referencing all the bugs whose expectations lines get removed by this. BUG=v8:3838,v8:3525,v8:3125 LOG=n Review URL: https://codereview.chromium.org/1802983002 Cr-Commit-Position: refs/heads/master@{#34804}
-
- 15 Mar, 2016 1 commit
-
-
jyan authored
R=danno@chromium.org,jkummerow@chromium.org,jochen@chromium.org,joransiu@ca.ibm.com,michael_dawson@ca.ibm.com,mbrandy@us.ibm.com BUG= Review URL: https://codereview.chromium.org/1805533002 Cr-Commit-Position: refs/heads/master@{#34795}
-
- 14 Mar, 2016 1 commit
-
-
mstarzinger authored
This also runs the webkit test suite against Ignition. By now most of the tests pass, the failing ones have been blacklisted. R=machenbach@chromium.org TEST=webkit Review URL: https://codereview.chromium.org/1799603003 Cr-Commit-Position: refs/heads/master@{#34749}
-
- 11 Mar, 2016 1 commit
-
-
mstarzinger authored
This also runs the message test suite against Ignition. By now most of the source positions (and exception messages) are accurate, the failing ones have been blacklisted. R=machenbach@chromium.org Review URL: https://codereview.chromium.org/1783773003 Cr-Commit-Position: refs/heads/master@{#34708}
-
- 10 Mar, 2016 1 commit
-
-
yangguo authored
Changes include: - better test coverage for builds with snapshot - write snapshot blobs to buffer instead of test serialization files - renamed tests R=machenbach@chromium.org Review URL: https://codereview.chromium.org/1777213002 Cr-Commit-Position: refs/heads/master@{#34657}
-
- 09 Mar, 2016 1 commit
-
-
machenbach authored
With this, the test runner automatically merges sancov files after testing. There's no need to do this by some external infrastructure. In a future CL, we could even merge during testing to lift harddisk pressure. BUG=chromium:568949 LOG=n NOTRY=true Review URL: https://codereview.chromium.org/1776123002 Cr-Commit-Position: refs/heads/master@{#34606}
-
- 08 Mar, 2016 1 commit
-
-
machenbach authored
This adds sanitizer-coverage compilation, test-runner features and post processing. Sanitizer coverage is expected to be used together with asan. During test runner execution, the produced sancov files are disambiguated and match the pattern: <executable name>.test.<test id>.sancov. Two additional scripts are added for merging raw sancov files and for generating json data containing all instrumented lines + all covered lines from merged sancov files. Both scripts use multiprocessing for speed. The json data will later be uploaded to google storage for further use, e.g. to show coverage data in rietveld. Sancov documentation: http://clang.llvm.org/docs/SanitizerCoverage.html BUG=chromium:568949 LOG=n NOTRY=true TEST=python -m unittest sancov_formatter_test TEST=python -m unittest sancov_merger_test Review URL: https://codereview.chromium.org/1737263003 Cr-Commit-Position: refs/heads/master@{#34578}
-
- 26 Jan, 2016 1 commit
-
-
jochen authored
BUG=chromium:577261 R=machenbach@chromium.org,jarin@chromium.org LOG=n Review URL: https://codereview.chromium.org/1604203002 Cr-Commit-Position: refs/heads/master@{#33508}
-
- 14 Jan, 2016 1 commit
-
-
machenbach authored
NOTRY=true Review URL: https://codereview.chromium.org/1588073002 Cr-Commit-Position: refs/heads/master@{#33303}
-
- 05 Jan, 2016 1 commit
-
-
rmcilroy authored
BUG=v8:4280 LOG=N Review URL: https://codereview.chromium.org/1559013004 Cr-Commit-Position: refs/heads/master@{#33114}
-
- 04 Jan, 2016 1 commit
-
-
machenbach authored
Add control for the ignition test suites. This will be followed by a change on the infra side that targets the ignition set instead of the hardcoded mjsunit. After that, cctest can be enabled in a separate CL. This CL already includes the logic for running cctest for ignition on swarming. BUG=v8:4280 LOG=N Review URL: https://codereview.chromium.org/1559943002 Cr-Commit-Position: refs/heads/master@{#33089}
-
- 14 Dec, 2015 1 commit
-
-
machenbach authored
BUG=chromium:568949 LOG=n NOTRY=true Review URL: https://codereview.chromium.org/1520163003 Cr-Commit-Position: refs/heads/master@{#32840}
-
- 27 Nov, 2015 1 commit
-
-
machenbach authored
This will allow moving the test outcome check beyond the multi-process boundary in a follow up. It'll allow wrapping more complex test jobs like predicable mode on the multi- process side, which will make the code easier to maintain. BUG= Review URL: https://codereview.chromium.org/1469833002 Cr-Commit-Position: refs/heads/master@{#32373}
-
- 23 Nov, 2015 1 commit
-
-
machenbach authored
The feature sometimes prevents subsequent swarming bots to delete the work directory on windows. The data file is not kept on swarming bots anyways, therefore this switches off the feature completely. BUG=chromium:535160 LOG=n Review URL: https://codereview.chromium.org/1468933002 Cr-Commit-Position: refs/heads/master@{#32168}
-
- 06 Nov, 2015 1 commit
-
-
rmcilroy authored
Adds a blacklist of tests which are currently unsupported or broken in Ignition to the mjsunit and test262 test status. Also removes --ignition-script-filter flag, and adds a --ignition_fallback_on_eval_and_catch flag which fallsback to fullcodegen for functions which call eval or contain a catch block. BUG=v8:4280 LOG=N Review URL: https://codereview.chromium.org/1420963009 Cr-Commit-Position: refs/heads/master@{#31864}
-
- 03 Nov, 2015 1 commit
-
-
machenbach authored
The flake detection is done on the infra-side according to the contents of the json test results. We don't want the runner to fail after flakes. This was controlled on the infra side by accepting any exit codes so far. After the swarming switch, this is more difficult, because the runner is wrapped by the swarming collect script. There, failing exit codes can mean many things, including network failures. Therefore, we now force exit code 0 with test failures if those failures are reported in the formal test results json. The infrastructure will take care of reporting the flakes and failures accordingly. BUG=chromium:535160 LOG=n Review URL: https://codereview.chromium.org/1416373005 Cr-Commit-Position: refs/heads/master@{#31740}
-
- 30 Oct, 2015 1 commit
-
-
machenbach authored
This is currently hardcoded in the infra-side test runner setup, but will be removed there, so that swarming triggers don't need a custom environment. BUG=chromium:535160 LOG=n Review URL: https://codereview.chromium.org/1411733008 Cr-Commit-Position: refs/heads/master@{#31682}
-
- 20 Oct, 2015 1 commit
-
-
bmeurer authored
Review URL: https://codereview.chromium.org/1402353006 Cr-Commit-Position: refs/heads/master@{#31409}
-
- 19 Oct, 2015 2 commits
-
-
machenbach authored
Fix after https://codereview.chromium.org/1413023002/ BUG=chromium:535160 LOG=n NOTRY=true TBR=jkummerow@chromium.org Review URL: https://codereview.chromium.org/1411973003 Cr-Commit-Position: refs/heads/master@{#31376}
-
machenbach authored
The test suite is ran in 60% of the bots anyway and the step is very short. For swarming, it's better to run this together in one step as each step triggers a different bot. BUG=chromium:535160 LOG=n NOTRY=true Review URL: https://codereview.chromium.org/1413023002 Cr-Commit-Position: refs/heads/master@{#31360}
-
- 16 Oct, 2015 1 commit
-
-
machenbach authored
NOTRY=true Review URL: https://codereview.chromium.org/1413563002 Cr-Commit-Position: refs/heads/master@{#31328}
-
- 15 Oct, 2015 1 commit
-
-
machenbach authored
BUG=chromium:535160 LOG=n Review URL: https://codereview.chromium.org/1397593004 Cr-Commit-Position: refs/heads/master@{#31308}
-
- 12 Oct, 2015 1 commit
-
-
machenbach authored
BUG=chromium:535160 LOG=n NOTRY=true Review URL: https://codereview.chromium.org/1395343002 Cr-Commit-Position: refs/heads/master@{#31207}
-
- 06 Oct, 2015 1 commit
-
-
machenbach authored
This adds the unittests to the "default" test set. Now that the "default" and the DEFAULT_TESTS (i.e. runner with no arguments) are the same, removed DEFAULT_TESTS and use TEST_MAP["default"] instead. On the bots, where unittests and default were run in separation before, the explicit unittests step should now be skipped. This is necessary for swarming, as the unittests step is too small to justify its own swarming job. BUG=chromium:535160 LOG=n Review URL: https://codereview.chromium.org/1374733006 Cr-Commit-Position: refs/heads/master@{#31122}
-
- 02 Oct, 2015 1 commit
-
-
machenbach authored
BUG=chromium:535160,v8:4459 LOG=n NOTRY=true Review URL: https://codereview.chromium.org/1384683002 Cr-Commit-Position: refs/heads/master@{#31073}
-
- 29 Sep, 2015 2 commits
-
-
dusan.m.milosavljevic authored
TEST= BUG= Review URL: https://codereview.chromium.org/1334793004 Cr-Commit-Position: refs/heads/master@{#31011}
-
cbruni authored
[run-tests.py] - adding more detailed information about the flags - show more detailed error message on failing Popen commands BUG= Review URL: https://codereview.chromium.org/1369343002 Cr-Commit-Position: refs/heads/master@{#30997}
-
- 17 Sep, 2015 1 commit
-
-
ishell authored
Review URL: https://codereview.chromium.org/1352803002 Cr-Commit-Position: refs/heads/master@{#30799}
-
- 11 Aug, 2015 1 commit
-
-
adamk authored
The previous approach to this (in 1b1de2d2) did not work well with bots that intentionally run no tests. But I think this warning message is still useful for developers. Review URL: https://codereview.chromium.org/1281313004 Cr-Commit-Position: refs/heads/master@{#30116}
-
- 10 Aug, 2015 3 commits
-
-
machenbach authored
Revert of Make run-tests.py warn when it's not testing anything (patchset #1 id:1 of https://codereview.chromium.org/1283513003/ ) Reason for revert: Some bots that don't run tests (by design) don't return the test results json anymore which makes the infrastructure side fail now. Original issue's description: > Make run-tests.py warn when it's not testing anything > > I've often been confused by an "all passed" run of the test script > when it turned out that either all the tests I cared about were skipped > or, more likely, I mistyped the name(s) of the tests I wanted to run. > This patch aims to fix that (and gives a useful diagnostic for the > "all matched tests were skipped" case). > > R=machenbach@chromium.org > NOTRY=true > > Committed: https://crrev.com/1b1de2d221a0ed23e529e91bf9fa3dc287acb2ca > Cr-Commit-Position: refs/heads/master@{#30095} TBR=jkummerow@chromium.org,adamk@chromium.org NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review URL: https://codereview.chromium.org/1283593005 Cr-Commit-Position: refs/heads/master@{#30097}
-
adamk authored
I've often been confused by an "all passed" run of the test script when it turned out that either all the tests I cared about were skipped or, more likely, I mistyped the name(s) of the tests I wanted to run. This patch aims to fix that (and gives a useful diagnostic for the "all matched tests were skipped" case). R=machenbach@chromium.org NOTRY=true Review URL: https://codereview.chromium.org/1283513003 Cr-Commit-Position: refs/heads/master@{#30095}
-
adamk authored
R=machenbach@chromium.org NOTRY=true Review URL: https://codereview.chromium.org/1279613005 Cr-Commit-Position: refs/heads/master@{#30094}
-
- 07 Aug, 2015 1 commit
-
-
machenbach authored
BUG=chromium:511215 LOG=n Review URL: https://codereview.chromium.org/1276853002 Cr-Commit-Position: refs/heads/master@{#30057}
-
- 06 Aug, 2015 1 commit
-
-
machenbach authored
Use the same path separator for test cases on the cmd-line on all platforms as in testcase.GetLabel, which is used to report failures. BUG=chromium:511215 LOG=n NOTRY=true Review URL: https://codereview.chromium.org/1281453003 Cr-Commit-Position: refs/heads/master@{#30045}
-
- 29 Jul, 2015 2 commits
-
-
machenbach authored
BUG=chromium:511215 NOTREECHECKS=true NOTRY=true LOG=n TBR=jkummerow@chromium.org Review URL: https://codereview.chromium.org/1262113002 Cr-Commit-Position: refs/heads/master@{#29900}
-
machenbach authored
This allows variants to be named on test failures (follow up) and then to be used in the test runner for a repro. This also speeds up variant iteration for test262 and fixes a bug with variants for benchmarks. BUG=chromium:511215 NOTREECHECKS=true LOG=n Review URL: https://codereview.chromium.org/1245623005 Cr-Commit-Position: refs/heads/master@{#29899}
-
- 13 Jul, 2015 1 commit
-
-
machenbach authored
BUG=chromium:508921 LOG=n Review URL: https://codereview.chromium.org/1234443003 Cr-Commit-Position: refs/heads/master@{#29605}
-
- 23 Jun, 2015 1 commit
-
-
jochen authored
BUG=none R=machenbach@chromium.org LOG=n Review URL: https://codereview.chromium.org/1204643003 Cr-Commit-Position: refs/heads/master@{#29221}
-
- 11 Jun, 2015 1 commit
-
-
machenbach authored
NOTRY=true Review URL: https://codereview.chromium.org/1180473003 Cr-Commit-Position: refs/heads/master@{#28936}
-
- 10 Jun, 2015 1 commit
-
-
machenbach authored
Reason for revert: All build problems had other reasons. Original issue's description: > Revert of [test] Refactoring - Let runner handle test IDs. (patchset #1 id:1 of https://codereview.chromium.org/1168303007/) > > Reason for revert: > [Sheriff] Revert until the tree is in a better state. > > Original issue's description: > > [test] Refactoring - Let runner handle test IDs. > > > > This prepares for properly rerunning tests. Currently when > > tests are rerun, the same test object is reused. This > > will be changed in a follow up. > > > > Committed: https://crrev.com/f41a81b8a513fc360c500c066b74f223bc9c0223 > > Cr-Commit-Position: refs/heads/master@{#28864} > > TBR=jkummerow@chromium.org,tandrii@chromium.org > NOPRESUBMIT=true > NOTREECHECKS=true > NOTRY=true > > Committed: https://crrev.com/f83444a5f24381ec9ccc28b1e18e1e9370415bb2 > Cr-Commit-Position: refs/heads/master@{#28891} TBR=jkummerow@chromium.org,tandrii@chromium.org NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review URL: https://codereview.chromium.org/1179483002 Cr-Commit-Position: refs/heads/master@{#28903}
-