Commit 3b0c9918 by Manh Nguyen Committed by Commit Bot

Update CaptureAndReplay.md and Python script with links

Links to the Python script in CaptureAndReplay.md Links to CaptureAndReplay.md's testing section in the Python script Bug: angleproject:4817 Change-Id: I584f8508661950e4907dae1adb8f0fe300387198 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2309226 Commit-Queue: Manh Nguyen <nguyenmh@google.com> Reviewed-by: 's avatarCody Northrop <cnorthrop@google.com> Reviewed-by: 's avatarJamie Madill <jmadill@chromium.org>
parent 4ccf4d0a
...@@ -170,10 +170,20 @@ as the GLES driver for your application. ...@@ -170,10 +170,20 @@ as the GLES driver for your application.
## Testing ## Testing
### Regression Testing Architecture ### Regression Testing Architecture
The python script uses the job queue pattern. There are n processes running asynchronously and independently, one on each CPU core. Whenever a process (except for the main process) finishes a job and becomes available, it grabs the next job from a shared job queue and runs that job on its CPU core. They keep doing this until there is no more job in the shared job queue. The main process, after filling up the shared job queue and spawning the (n-1) subprocesses that run on the remaining (n-1) CPU cores, waits until all the subprocesses are finished. It then reports the results and cleans up. The [python script][link_to_python_script] uses the job queue pattern. We spawn n-1 independent
worker processes, where n is the value returned by multiprocessing.cpu_count(). Whenever a worker
process finishes a job and becomes available, it grabs the next job from a shared job queue and
runs that job on its CPU core. When there are no more jobs in the queue, the worker processes
terminate and the main process reports results.
![Point-in-time snapshot of the job queue](https://screenshot.googleplex.com/w2QY5ui5FMg.png) ![Point-in-time snapshot of the job queue](img/RegressionTestingArchitecture.png)
### Job unit ### Job unit
A job unit is a test batch. Each test has to go through 3 stages: capture run, replay build, and replay run. The test batch batches the replay build stage of multiple tests together, and the replay run stage of multiple tests together. A job unit is a test batch. Each test has to go through 3 stages: capture run, replay build, and
![A test batch as a job unit](https://screenshot.googleplex.com/A4w8ogsnfLw.png) replay run. The test batch batches the replay build stage of multiple tests together, and the
replay run stage of multiple tests together.
![A test batch as a job unit](img/JobUnit.png)
[link_to_python_script]:https://chromium.googlesource.com/angle/angle/+/refs/heads/master/src/tests/capture_replay_tests.py
...@@ -230,6 +230,8 @@ def SetCWDToAngleFolder(): ...@@ -230,6 +230,8 @@ def SetCWDToAngleFolder():
return cwd return cwd
# RunTest gets run on each spawned subprocess.
# See https://chromium.googlesource.com/angle/angle/+/refs/heads/master/doc/CaptureAndReplay.md#testing for architecture
def RunTest(job_queue, gn_path, autoninja_path, capture_build_dir, replay_build_dir, test_exec, def RunTest(job_queue, gn_path, autoninja_path, capture_build_dir, replay_build_dir, test_exec,
replay_exec, trace_dir, result_list): replay_exec, trace_dir, result_list):
trace_folder_path = os.path.join(REPLAY_SAMPLE_FOLDER, trace_dir) trace_folder_path = os.path.join(REPLAY_SAMPLE_FOLDER, trace_dir)
...@@ -243,12 +245,14 @@ def RunTest(job_queue, gn_path, autoninja_path, capture_build_dir, replay_build_ ...@@ -243,12 +245,14 @@ def RunTest(job_queue, gn_path, autoninja_path, capture_build_dir, replay_build_
test = job_queue.get() test = job_queue.get()
print("Running " + test.full_test_name) print("Running " + test.full_test_name)
sys.stdout.flush() sys.stdout.flush()
# stage 1 from the diagram linked above: capture run
returncode, output = test.Run(test_exec_path, trace_folder_path) returncode, output = test.Run(test_exec_path, trace_folder_path)
if returncode != 0 or not CanRunReplay(trace_folder_path): if returncode != 0 or not CanRunReplay(trace_folder_path):
result_list.append((test.full_test_name, "Skipped", result_list.append((test.full_test_name, "Skipped",
"Skipping replay since capture didn't produce appropriate files or has crashed. " \ "Skipping replay since capture didn't produce appropriate files or has crashed. " \
+ "Error message: " + output)) + "Error message: " + output))
continue continue
# stage 2 from the diagram linked above: replay build
returncode, output = test.BuildReplay(gn_path, autoninja_path, replay_build_dir, returncode, output = test.BuildReplay(gn_path, autoninja_path, replay_build_dir,
trace_dir, replay_exec) trace_dir, replay_exec)
if returncode != 0: if returncode != 0:
...@@ -256,6 +260,7 @@ def RunTest(job_queue, gn_path, autoninja_path, capture_build_dir, replay_build_ ...@@ -256,6 +260,7 @@ def RunTest(job_queue, gn_path, autoninja_path, capture_build_dir, replay_build_
(test.full_test_name, "Skipped", (test.full_test_name, "Skipped",
"Skipping replay since failing to build replay. Error message: " + output)) "Skipping replay since failing to build replay. Error message: " + output))
continue continue
# stage 2 from the diagram linked above: replay run
returncode, output = test.RunReplay(replay_exec_path) returncode, output = test.RunReplay(replay_exec_path)
if returncode != 0: if returncode != 0:
result_list.append((test.full_test_name, "Failed", output)) result_list.append((test.full_test_name, "Failed", output))
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment