As previously discussed, let's flip the switch ^^.
This exposes the problem that it will now be run
for everyone, even if one did not read the help
about the recommended repetition count.
This is not good. So i think we can do the smart thing:
```
$ ./compare.py benchmarks gbench/Inputs/test3_run{0,1}.json
Comparing gbench/Inputs/test3_run0.json to gbench/Inputs/test3_run1.json
Benchmark Time CPU Time Old Time New CPU Old CPU New
--------------------------------------------------------------------------------------------------------
BM_One -0.1000 +0.1000 10 9 100 110
BM_Two +0.1111 -0.0111 9 10 90 89
BM_Two +0.2500 +0.1125 8 10 80 89
BM_Two_pvalue 0.2207 0.6831 U Test, Repetitions: 2. WARNING: Results unreliable! 9+ repetitions recommended.
BM_Two_stat +0.0000 +0.0000 8 8 80 80
```
(old screenshot)

Or, in the good case (noise omitted):
```
s$ ./compare.py benchmarks /tmp/run{0,1}.json
Comparing /tmp/run0.json to /tmp/run1.json
Benchmark Time CPU Time Old Time New CPU Old CPU New
---------------------------------------------------------------------------------------------------------------------------------
<99 more rows like this>
./_T012014.RW2/threads:8/real_time +0.0160 +0.0596 46 47 10 10
./_T012014.RW2/threads:8/real_time_pvalue 0.0000 0.0000 U Test, Repetitions: 100
./_T012014.RW2/threads:8/real_time_mean +0.0094 +0.0609 46 47 10 10
./_T012014.RW2/threads:8/real_time_median +0.0104 +0.0613 46 46 10 10
./_T012014.RW2/threads:8/real_time_stddev -0.1160 -0.1807 1 1 0 0
```
(old screenshot)

| Name |
Last commit
|
Last update |
|---|---|---|
| cmake | Loading commit data... | |
| docs | Loading commit data... | |
| include/benchmark | Loading commit data... | |
| src | Loading commit data... | |
| test | Loading commit data... | |
| tools | Loading commit data... | |
| .clang-format | Loading commit data... | |
| .gitignore | Loading commit data... | |
| .travis-libcxx-setup.sh | Loading commit data... | |
| .travis.yml | Loading commit data... | |
| .ycm_extra_conf.py | Loading commit data... | |
| AUTHORS | Loading commit data... | |
| BUILD.bazel | Loading commit data... | |
| CMakeLists.txt | Loading commit data... | |
| CONTRIBUTING.md | Loading commit data... | |
| CONTRIBUTORS | Loading commit data... | |
| LICENSE | Loading commit data... | |
| README.md | Loading commit data... | |
| WORKSPACE | Loading commit data... | |
| appveyor.yml | Loading commit data... | |
| mingw.py | Loading commit data... | |
| releasing.md | Loading commit data... |