1. 26 Jul, 2022 1 commit
  2. 18 Jul, 2022 1 commit
    • Liviu Rau's avatar
      [test] Refactor testrunner (4) · b3477fdd
      Liviu Rau authored
       - Removed duplication and unnecessary indirection from all suites testcfgs.
       - Introduced a more comprehensive context to cover both command context and other platform specific concerns.
       - Propagated above context to TestLoader to allow for test counting command execution on all platforms.
       - Wrapped original pool with another class to give it a new interface and allow injecting different implementations in the future.
       - Consolidated progress indicators under a single processor in the pipeline.
       - Consolidated result retention requirements calculation outside of pipeline chain.
       - Refactored LoaderProc and got it under tests.
       - Added some more tests for the standard runner.
       - Extracted BuildConfig class.
      
      
      Bug: v8:12785
      Change-Id: I87be040e91f792a983662bb5a10d55b36a14ea7f
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3701595Reviewed-by: 's avatarMichael Achenbach <machenbach@chromium.org>
      Commit-Queue: Liviu Rau <liviurau@chromium.org>
      Cr-Commit-Position: refs/heads/main@{#81770}
      b3477fdd
  3. 14 Jun, 2022 1 commit
    • Liviu Rau's avatar
      [test] Refactor testrunner (3) · 9733dacc
      Liviu Rau authored
      - removed some 'feature envy' instances:
        - created a AugmentedOptions class to take care of non trivial option look-ups
        - moved some calculation closer the the actual data
      - promoted parameter that was passed around a lot (options) to filed in the offending class
      - similar object creation looks a bit more similar
      - CommandContext provides a wrapper that ensures resource de-allocation
      - separate tests from standard and num_fuzz runners
      - added a couple of more tests to improve coverage
      
      This is still in flux. But further changes risk creating a disconnect between the original implementation and further refactorings.
      
      Bug: v8:12785
      Change-Id: I0ec2e9c6a81604a5cd1d4a80982dd3329c1b48db
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3686411
      Commit-Queue: Liviu Rau <liviurau@chromium.org>
      Reviewed-by: 's avatarMichael Achenbach <machenbach@chromium.org>
      Cr-Commit-Position: refs/heads/main@{#81148}
      9733dacc
  4. 02 Jun, 2022 1 commit
  5. 01 Jun, 2022 1 commit
  6. 25 Mar, 2022 1 commit
  7. 17 Mar, 2022 2 commits
  8. 04 Feb, 2022 1 commit
    • Michael Achenbach's avatar
      [infra] Make various scripts compatible with Python3 · efd28c14
      Michael Achenbach authored
      This fixes all Python3 problems in scripts and tests running via
      v8_presubmit.py. It includes:
      - Test runner
      - Release tools
      - Perf runner
      - Torque formatter
      - V8's main presubmit
      
      On bots, v8_presubmit is run with vpython, hence we also add
      the required dependencies. After the Python3 migration, most
      of the transitional code in this CL can be removed again.
      
      Bug: chromium:1293709,chromium:1292016
      Change-Id: Ic25e5965948b212c047e9d5194d2a4b6db1fa91b
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3432213Reviewed-by: 's avatarLiviu Rau <liviurau@chromium.org>
      Commit-Queue: Michael Achenbach <machenbach@chromium.org>
      Cr-Commit-Position: refs/heads/main@{#78936}
      efd28c14
  9. 27 Jan, 2022 1 commit
  10. 15 Dec, 2021 1 commit
    • Samuel Groß's avatar
      V8 Sandbox rebranding · 277fdd1d
      Samuel Groß authored
      This CL renames a number of things related to the V8 sandbox.
      Mainly, what used to be under V8_HEAP_SANDBOX is now under
      V8_SANDBOXED_EXTERNAL_POINTERS, while the previous V8 VirtualMemoryCage
      is now simply the V8 Sandbox:
      
      V8_VIRTUAL_MEMORY_CAGE => V8_SANDBOX
      V8_HEAP_SANDBOX => V8_SANDBOXED_EXTERNAL_POINTERS
      V8_CAGED_POINTERS => V8_SANDBOXED_POINTERS
      V8VirtualMemoryCage => Sandbox
      CagedPointer => SandboxedPointer
      fake cage => partially reserved sandbox
      src/security => src/sandbox
      
      This naming scheme should simplify things: the sandbox is now the large
      region of virtual address space inside which V8 mainly operates and
      which should be considered untrusted. Mechanisms like sandboxed pointers
      are then used to attempt to prevent escapes from the sandbox (i.e.
      corruption of memory outside of it). Furthermore, the new naming scheme
      avoids the confusion with the various other "cages" in V8, in
      particular, the VirtualMemoryCage class, by dropping that name entirely.
      
      Future sandbox features are developed under their own V8_SANDBOX_X flag,
      and will, once final, be merged into V8_SANDBOX. Current future features
      are sandboxed external pointers (using the external pointer table), and
      sandboxed pointers (pointers guaranteed to point into the sandbox, e.g.
      because they are encoded as offsets). This CL then also introduces a new
      build flag, v8_enable_sandbox_future, which enables all future features.
      
      Bug: v8:10391
      Change-Id: I5174ea8f5ab40fb96a04af10853da735ad775c96
      Cq-Include-Trybots: luci.v8.try:v8_linux64_heap_sandbox_dbg_ng,v8_linux_arm64_sim_heap_sandbox_dbg_ng
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3322981Reviewed-by: 's avatarHannes Payer <hpayer@chromium.org>
      Reviewed-by: 's avatarIgor Sheludko <ishell@chromium.org>
      Reviewed-by: 's avatarMichael Achenbach <machenbach@chromium.org>
      Reviewed-by: 's avatarToon Verwaest <verwaest@chromium.org>
      Commit-Queue: Samuel Groß <saelo@chromium.org>
      Cr-Commit-Position: refs/heads/main@{#78384}
      277fdd1d
  11. 25 Aug, 2021 1 commit
  12. 27 Jul, 2021 1 commit
  13. 21 Jul, 2021 1 commit
  14. 20 Jul, 2021 2 commits
  15. 26 May, 2021 1 commit
    • Michael Achenbach's avatar
      [test] Add verbose output of flaky tests · f370d607
      Michael Achenbach authored
      The verbose output shown on bots didn't print the first failing result
      of a flaky test before. Now the result line shows all results and
      the details in the end show the output of the first failure.
      
      Previously it was confusing as it seemed that the json results and
      the test runner output differed.
      
      We now print PASS in all caps like the other statuses. A test for
      this case already existed and the output is now updated.
      
      Bug: v8:8434
      Change-Id: I473ec392e0028bf64b3da53d4b37446ffcd17277
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2919670
      Commit-Queue: Liviu Rau <liviurau@chromium.org>
      Auto-Submit: Michael Achenbach <machenbach@chromium.org>
      Reviewed-by: 's avatarLiviu Rau <liviurau@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#74792}
      f370d607
  16. 21 May, 2021 1 commit
    • Michael Achenbach's avatar
      [test] Run heavy tests sequentially · ee56a986
      Michael Achenbach authored
      This adds a new status file indicator "HEAVY" to mark tests with high
      resource demands. There will be other tests running in parallel,
      but only a limited number of other heavy tests. The limit is
      controlled with a new parameter --max-heavy-tests and defaults to 1.
      
      The change also marks a variety of tests as heavy that recently had
      flaky timeouts. Heavy also implies slow, hence heavy tests are
      executed at the beginning with a higher timeout like other slow tests.
      
      The implementation is encapsulated in the test-processor chain. A
      new processor buffers heavy tests in a queue and adds buffered tests
      only if other heavy tests have ended their computation.
      
      Bug: v8:5861
      Change-Id: I89648ad0030271a3a5af588ecc9c43285b728d6d
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2905767
      Commit-Queue: Michael Achenbach <machenbach@chromium.org>
      Reviewed-by: 's avatarLiviu Rau <liviurau@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#74712}
      ee56a986
  17. 12 Apr, 2021 1 commit
  18. 22 Feb, 2021 1 commit
    • Clemens Backes's avatar
      [no-wasm] Skip wasm tests if wasm is disabled · 18136654
      Clemens Backes authored
      This CL introduces a test runner flag to detect if webassembly has been
      disabled. Since all tests that require wasm are alrady skipped in
      lite mode, we introduce a has_webassembly flag for the test runner which
      checks for v8_enable_webassembly=true and v8_enable_lite_mode=false.
      As a drive-by, we also do not set the V8_ENABLE_WEBASSEMBLY
      preprocessor flag if lite mode is enabled.
      
      The status files are updated by splitting wasm tests from the
      "lite_mode" section and checking for "not has_webassembly" instead.
      
      Note that the v8_enable_webassembly=false configuration is not tested
      on any bot currently, but I will make sure that all tests keep passing
      on further changes in this configuration.
      
      R=machenbach@chromium.org
      
      Bug: v8:11238
      Cq-Include-Trybots: luci.v8.try:v8_linux64_no_wasm_compile_rel
      Change-Id: I1841eb1f1633cb47e0c079f4a4a4d769ca3a9cbb
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2710425Reviewed-by: 's avatarMichael Achenbach <machenbach@chromium.org>
      Commit-Queue: Clemens Backes <clemensb@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#72898}
      18136654
  19. 08 Oct, 2020 1 commit
  20. 07 Oct, 2020 1 commit
    • Michael Achenbach's avatar
      [test] Overhaul mode processing in test runner · 608b732d
      Michael Achenbach authored
      This simplifies mode processing as follows:
      - Passing the --mode parameter is deprecated.
      - The build output is now only searched in the --outdir parameter
      that was passed (previously some combinations of mode and outdir
      were possible).
      - The mode is deduced from the build artifacts based on the gn
      arguments "is_debug" and "dcheck_always_on".
      - Timeouts and status file entries in release mode with dchecks are
      treated like in debug mode.
      
      This change was prepared on the infrastructure side by deprecating
      the --mode flag and passing --outdir=out/build:
      https://crrev.com/c/2426643
      
      Bug: chromium:1132088, v8:10893
      Change-Id: I0f34ebc003b220f07df4ecdbf69ea6c06ac1f66a
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2450016Reviewed-by: 's avatarJakob Kummerow <jkummerow@chromium.org>
      Commit-Queue: Michael Achenbach <machenbach@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#70363}
      608b732d
  21. 16 Sep, 2020 1 commit
    • Michael Achenbach's avatar
      [test] Print less in verbose mode · d8d6110b
      Michael Achenbach authored
      I/O is quite expensive on the bots. This cuts down a bit of it by
      printing slightly fewer characters per test in verbose mode.
      
      This leads to an overall speed improvement of ~20% for large test
      suites, e.g. Test262 output-collection time goes from ~2m30 to ~2m.
      
      The averages to a 5-10% overall speed improvement for slow tryjobs.
      
      Bug: v8:10916
      Change-Id: I56dcb072af8eb32a1e09e17a05db5782c6d79315
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2414038
      Auto-Submit: Michael Achenbach <machenbach@chromium.org>
      Commit-Queue: Maya Lekova <mslekova@chromium.org>
      Reviewed-by: 's avatarMaya Lekova <mslekova@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#69950}
      d8d6110b
  22. 05 May, 2020 1 commit
  23. 19 Feb, 2020 1 commit
  24. 22 Oct, 2019 2 commits
  25. 16 Oct, 2019 1 commit
    • Simon Zünd's avatar
      Reland "Unconditionally enable snapshot builds and remove 'v8_use_snapshot'" · 69efc4c3
      Simon Zünd authored
      This is a reland of 1c56974f
      
      This is a plain reland of the original CL. The original CL was speculatively
      reverted, but ended up not being the cause for bot failures.
      
      Original change's description:
      > Unconditionally enable snapshot builds and remove 'v8_use_snapshot'
      >
      > This CL removes 'v8_use_snapshot' and the usages of the implied
      > V8_USE_SNAPSHOT define. One test runner unittest was updated to use the
      > "asan" variant instead of the now obsolete "no_snap" variant.
      >
      > Related chromium CL: https://crrev.com/c/1796325.
      >
      > Bug: v8:8531
      > Change-Id: I5da7c9f8e9110fe7bc0f4e4f821bcb7f7d98f927
      > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1784282
      > Commit-Queue: Simon Zünd <szuend@chromium.org>
      > Reviewed-by: Tamer Tas <tmrts@chromium.org>
      > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
      > Reviewed-by: Nico Weber <thakis@chromium.org>
      > Reviewed-by: Jakob Gruber <jgruber@chromium.org>
      > Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
      > Cr-Commit-Position: refs/heads/master@{#64290}
      
      TBR=thakis@chromium.org,machenbach@chromium.org,mstarzinger@chromium.org,jgruber@chromium.org,tmrts@chromium.org,szuend@chromium.org
      
      Bug: v8:8531
      Change-Id: Id75a802279238138f7aefec62e0b6425a5acc08d
      Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1864649Reviewed-by: 's avatarSimon Zünd <szuend@chromium.org>
      Reviewed-by: 's avatarTamer Tas <tmrts@chromium.org>
      Commit-Queue: Simon Zünd <szuend@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#64305}
      69efc4c3
  26. 15 Oct, 2019 2 commits
  27. 03 Apr, 2019 1 commit
  28. 19 Feb, 2019 1 commit
  29. 12 Feb, 2019 3 commits
    • Tamer Tas's avatar
      [testrunner] enable the progress indicator · baeb4e32
      Tamer Tas authored
      Using test generators meant that we had to remove the progress indicator since
      the total number of tests weren't known before-hand.
      
      This CL implements a progress indicator using test number estimations.
      
      cctest and unittests progress indicator is accurate, however estimating
      means the progress will terminate over 100% in big test suites and sometimes
      under 100%.
      
      R=machenbach@chromium.org
      CC=​sergiyb@chromium.org,yangguo@chromium.org
      
      Bug: v8:8769
      Change-Id: I40ca5b40f9b1223376d33707f0945900ea98cea3
      Reviewed-on: https://chromium-review.googlesource.com/c/1460471
      Commit-Queue: Tamer Tas <tmrts@chromium.org>
      Reviewed-by: 's avatarMichael Achenbach <machenbach@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#59538}
      baeb4e32
    • Tamer Tas's avatar
      [fuzzer] create a test-suite for the fuzzer · e2207c67
      Tamer Tas authored
      Testrunner test-suite doesn't have automated tests for the Fuzzer. A significant
      change for the testrunner may break the fuzzers. We don't have a way to catch
      even the syntax errors for the fuzzer before landing CLs.
      
      This CL creates a rudimentary test-suite that runs the fuzzer without any tests
      to see if APIs still work as intended.
      
      R=machenbach@chromium.org
      CC=yangguo@chromium.org,sergiyb@chromium.org
      
      No-Try: true
      Bug: v8:8763
      Change-Id: Ib4519ca093de2af5279964edd164cee04aae77a8
      Reviewed-on: https://chromium-review.googlesource.com/c/1466301Reviewed-by: 's avatarMichael Achenbach <machenbach@chromium.org>
      Commit-Queue: Tamer Tas <tmrts@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#59535}
      e2207c67
    • Tamer Tas's avatar
      [testrunner] enable variant sharding · ad5b6d7d
      Tamer Tas authored
      Testrunner runs variants of a test sequentially without taking sharding into
      account. A slow test with slow variants slows down the whole test run no matter
      the sharding configuration.
      
      This CL implements a test hashing algorithm and variant sharding for test
      variants.
      
      R=machenbach@chromium.org
      CC=yangguo@chromium.org,sergiyb@chromium.org
      
      Bug: v8:8174
      Change-Id: I15f8c547fa2f361fb6c53bf8d5df055d3df38d3e
      Reviewed-on: https://chromium-review.googlesource.com/c/1458016
      Commit-Queue: Tamer Tas <tmrts@chromium.org>
      Reviewed-by: 's avatarMichael Achenbach <machenbach@chromium.org>
      Cr-Commit-Position: refs/heads/master@{#59530}
      ad5b6d7d
  30. 07 Feb, 2019 1 commit
  31. 01 Feb, 2019 3 commits
  32. 21 Jan, 2019 1 commit