Commit 457588ef by Jamie Madill Committed by Commit Bot

Add more documentation on testing infrastructure.

Adds more info on the amount of testing ANGLE runs and our various configs. Adds a few other notes and cross-links. Bug: angleproject:1944 Change-Id: I76e46783bd6a3f84d3e170bd30ea2e6d57911143 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2105715 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: 's avatarYuly Novikov <ynovikov@chromium.org> Reviewed-by: 's avatarGeoff Lang <geofflang@chromium.org>
parent f1eb10ef
...@@ -81,4 +81,5 @@ View the [Dev setup instructions](doc/DevSetup.md). ...@@ -81,4 +81,5 @@ View the [Dev setup instructions](doc/DevSetup.md).
* Watch a [short presentation](https://youtu.be/QrIKdjmpmaA) on the Vulkan back-end. * Watch a [short presentation](https://youtu.be/QrIKdjmpmaA) on the Vulkan back-end.
* Track the [dEQP test conformance](doc/dEQP-Charts.md) * Track the [dEQP test conformance](doc/dEQP-Charts.md)
* Read design docs on the [Vulkan back-end](src/libANGLE/renderer/vulkan/README.md) * Read design docs on the [Vulkan back-end](src/libANGLE/renderer/vulkan/README.md)
* Read about ANGLE's [testing infrastructure](infra/README.md)
* If you use ANGLE in your own project, we'd love to hear about it! * If you use ANGLE in your own project, we'd love to hear about it!
...@@ -64,6 +64,8 @@ ...@@ -64,6 +64,8 @@
* We do not currently have the capability to run individual bots or tests in a run. * We do not currently have the capability to run individual bots or tests in a run.
* Tests can also be run locally, ANGLE's main testing methods are: * Tests can also be run locally, ANGLE's main testing methods are:
* `angle_unittests`, `angle_end2end_tests` and `angle_white_box_tests` targets. * `angle_unittests`, `angle_end2end_tests` and `angle_white_box_tests` targets.
* The OpenGL CTS also known as [drawElements Quality Program (dEQP)](dEQP.md).
If you're working on a new feature, there may be some extensive tests for it already written.
* The [Top-of-Tree WebGL Conformance tests][WebGL-CTS]. * The [Top-of-Tree WebGL Conformance tests][WebGL-CTS].
* If you are a Chromium developer, see * If you are a Chromium developer, see
[Building ANGLE for Chromium Development][build-ANGLE-for-chromium] for instructions on [Building ANGLE for Chromium Development][build-ANGLE-for-chromium] for instructions on
...@@ -84,8 +86,6 @@ ...@@ -84,8 +86,6 @@
ANGLE patches that have been committed. There may be hardware configurations that are not ANGLE patches that have been committed. There may be hardware configurations that are not
tested by the ANGLE trybots, if you notice breakage on this waterfall after landing a patch, tested by the ANGLE trybots, if you notice breakage on this waterfall after landing a patch,
please notify a project member. please notify a project member.
* ANGLE also includes the [drawElements Quality Program (dEQP)](dEQP.md) for additional testing.
If you're working on a new feature, there may be some extensive tests for it already written.
[WebGL-CTS]: https://www.khronos.org/registry/webgl/sdk/tests/webgl-conformance-tests.html [WebGL-CTS]: https://www.khronos.org/registry/webgl/sdk/tests/webgl-conformance-tests.html
[build-ANGLE-for-Chromium]: BuildingAngleForChromiumDevelopment.md [build-ANGLE-for-Chromium]: BuildingAngleForChromiumDevelopment.md
......
...@@ -13,7 +13,12 @@ As an ANGLE Sheriff. Your job is to: ...@@ -13,7 +13,12 @@ As an ANGLE Sheriff. Your job is to:
If you're not an ANGLE team member, you can contact us on the public ANGLE project If you're not an ANGLE team member, you can contact us on the public ANGLE project
[Google group](https://groups.google.com/forum/#!forum/angleproject). [Google group](https://groups.google.com/forum/#!forum/angleproject).
## Task 1: The Try Waterfall **Note**: It's highly recommend that all wranglers install the [Chromium Flake Linker][Flaker]
extension for inspecting bot builds. It'll save you a lot of time.
[Flaker]: https://chrome.google.com/webstore/detail/flake-linker/boamnmbgmfnobomddmenbaicodgglkhc
## Task 1: Montitor ANGLE CI and Try Testers
Your first job is to keep the Your first job is to keep the
[ANGLE Try Waterfall](https://ci.chromium.org/p/chromium/g/angle.try/builders) healthy. Some [ANGLE Try Waterfall](https://ci.chromium.org/p/chromium/g/angle.try/builders) healthy. Some
......
# ANGLE Standalone Testing Infrastructure # ANGLE Testing Infrastructure
ANGLE runs hundreds of thousands of tests on every change before it lands in
the tree. We scale our pre-commit and post-commit testing to many machines
using [Chromium Swarming][Swarming]. Our testing setup heavily leverages
existing work in Chromium. We also run compile-only
[Standalone Testing][Standalone] that does not depend on a Chromium checkout.
Also see the documentation on [ANGLE Wrangling][Wrangling] for more info.
## Pre-Commit Testing
See the pre-commit try waterfall here:
[`https://ci.chromium.org/p/chromium/g/angle.try/builders`](https://ci.chromium.org/p/chromium/g/angle.try/builders)
We currently run pre-commit tests on:
* Windows 32-bit AMD and Windows 64-bit Intel and NVIDIA GPUs
* Linux 64-bit NVIDIA and Intel GPUs
* Mac NVIDIA, Intel and AMD GPUs
* Pixel 2 and Nexus 5X
* Fuchsia testing in a VM
Looking at an example build shows how tests are split up between machines. See for example:
[`https://ci.chromium.org/p/chromium/builders/try/mac-angle-rel/5905`](https://ci.chromium.org/p/chromium/builders/try/mac-angle-rel/5905)
This build ran 82 test steps across 3 GPU families. In some cases (e.g.
webgl2_conformance_tests) the test is split up between multiple machines to
run faster (in this case 20 different machines at once). This build took 27
minutes to complete multiple hours of real automated testing.
For more details on running and working with our test sets see the docs in [Contributing Code][Contrib].
[Swarming]: https://chromium-swarm.appspot.com/
[Standalone]: #ANGLE-Standalone-Testing
[Contrib]: ../doc/ContributingCode.md#Testing
[Wrangling]: ANGLEWrangling.md
## ANGLE Standalone Testing
In addition to the ANGLE try bots using Chrome, and the GPU.FYI bots, ANGLE In addition to the ANGLE try bots using Chrome, and the GPU.FYI bots, ANGLE
has standalone testing on the Chrome infrastructure. Currently these tests are has standalone testing on the Chrome infrastructure. Currently these tests are
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment