Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
B
benchmark
Project
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Chen Yisong
benchmark
Commits
daf276ff
Commit
daf276ff
authored
Dec 01, 2019
by
Martin Blanchard
Committed by
Roman Lebedev
Dec 01, 2019
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Document environment variables options usage (#894)
See
https://github.com/google/benchmark/issues/881
parent
49aa79b6
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
44 additions
and
8 deletions
+44
-8
README.md
README.md
+44
-8
No files found.
README.md
View file @
daf276ff
# Benchmark
# Benchmark
[

](https://travis-ci.org/google/benchmark)
[

](https://travis-ci.org/google/benchmark)
[

](https://ci.appveyor.com/project/google/benchmark/branch/master)
[

](https://ci.appveyor.com/project/google/benchmark/branch/master)
[

](https://coveralls.io/r/google/benchmark)
[

](https://coveralls.io/r/google/benchmark)
[

](https://slackin-iqtfqnpzxd.now.sh/)
[

](https://slackin-iqtfqnpzxd.now.sh/)
A library to benchmark code snippets, similar to unit tests. Example:
A library to benchmark code snippets, similar to unit tests. Example:
```
c++
```
c++
...
@@ -149,7 +149,9 @@ this branch. However, this branch provides no stability guarantees and reserves
...
@@ -149,7 +149,9 @@ this branch. However, this branch provides no stability guarantees and reserves
the right to change and break the API at any time.
the right to change and break the API at any time.
## Usage
## Usage
### Basic usage
### Basic usage
Define a function that executes the code to measure, register it as a benchmark
Define a function that executes the code to measure, register it as a benchmark
function using the
`BENCHMARK`
macro, and ensure an appropriate
`main`
function
function using the
`BENCHMARK`
macro, and ensure an appropriate
`main`
function
is available:
is available:
...
@@ -239,15 +241,19 @@ too (`-lkstat`).
...
@@ -239,15 +241,19 @@ too (`-lkstat`).
## User Guide
## User Guide
### Command Line
### Command Line
[
Output Formats
](
#output-formats
)
[
Output Formats
](
#output-formats
)
[
Output Files
](
#output-files
)
[
Output Files
](
#output-files
)
[
Running Benchmarks
](
#running-benchmarks
)
[
Running a Subset of Benchmarks
](
#running-a-subset-of-benchmarks
)
[
Running a Subset of Benchmarks
](
#running-a-subset-of-benchmarks
)
[
Result Comparison
](
#result-comparison
)
[
Result Comparison
](
#result-comparison
)
### Library
### Library
[
Runtime and Reporting Considerations
](
#runtime-and-reporting-considerations
)
[
Runtime and Reporting Considerations
](
#runtime-and-reporting-considerations
)
[
Passing Arguments
](
#passing-arguments
)
[
Passing Arguments
](
#passing-arguments
)
...
@@ -282,17 +288,20 @@ too (`-lkstat`).
...
@@ -282,17 +288,20 @@ too (`-lkstat`).
[
Disabling CPU Frequency Scaling
](
#disabling-cpu-frequency-scaling
)
[
Disabling CPU Frequency Scaling
](
#disabling-cpu-frequency-scaling
)
<a
name=
"output-formats"
/>
<a
name=
"output-formats"
/>
### Output Formats
### Output Formats
The library supports multiple output formats. Use the
The library supports multiple output formats. Use the
`--benchmark_format=<console|json|csv>`
flag to set the format type.
`console`
`--benchmark_format=<console|json|csv>`
(or set the
is the default format.
`BENCHMARK_FORMAT=<console|json|csv>`
environment variable) flag to set
the format type.
`console`
is the default format.
The Console format is intended to be a human readable format. By default
The Console format is intended to be a human readable format. By default
the format generates color output. Context is output on stderr and the
the format generates color output. Context is output on stderr and the
tabular data on stdout. Example tabular output looks like:
tabular data on stdout. Example tabular output looks like:
```
```
Benchmark Time(ns) CPU(ns) Iterations
Benchmark Time(ns) CPU(ns) Iterations
----------------------------------------------------------------------
----------------------------------------------------------------------
...
@@ -306,6 +315,7 @@ The `context` attribute contains information about the run in general, including
...
@@ -306,6 +315,7 @@ The `context` attribute contains information about the run in general, including
information about the CPU and the date.
information about the CPU and the date.
The
`benchmarks`
attribute contains a list of every benchmark run. Example json
The
`benchmarks`
attribute contains a list of every benchmark run. Example json
output looks like:
output looks like:
```
json
```
json
{
{
"context"
:
{
"context"
:
{
...
@@ -346,6 +356,7 @@ output looks like:
...
@@ -346,6 +356,7 @@ output looks like:
The CSV format outputs comma-separated values. The
`context`
is output on stderr
The CSV format outputs comma-separated values. The
`context`
is output on stderr
and the CSV itself on stdout. Example CSV output looks like:
and the CSV itself on stdout. Example CSV output looks like:
```
```
name,iterations,real_time,cpu_time,bytes_per_second,items_per_second,label
name,iterations,real_time,cpu_time,bytes_per_second,items_per_second,label
"BM_SetInsert/1024/1",65465,17890.7,8407.45,475768,118942,
"BM_SetInsert/1024/1",65465,17890.7,8407.45,475768,118942,
...
@@ -357,16 +368,31 @@ name,iterations,real_time,cpu_time,bytes_per_second,items_per_second,label
...
@@ -357,16 +368,31 @@ name,iterations,real_time,cpu_time,bytes_per_second,items_per_second,label
### Output Files
### Output Files
Write benchmark results to a file with the
`--benchmark_out=<filename>`
option.
Write benchmark results to a file with the
`--benchmark_out=<filename>`
option
Specify the output format with
`--benchmark_out_format={json|console|csv}`
. Note that Specifying
(or set
`BENCHMARK_OUT`
). Specify the output format with
`--benchmark_out_format={json|console|csv}`
(or set
`BENCHMARK_OUT_FORMAT={json|console|csv}`
). Note that specifying
`--benchmark_out`
does not suppress the console output.
`--benchmark_out`
does not suppress the console output.
<a
name=
"running-benchmarks"
/>
### Running Benchmarks
Benchmarks are executed by running the produced binaries. Benchmarks binaries,
by default, accept options that may be specified either through their command
line interface or by setting environment variables before execution. For every
`--option_flag=<value>`
CLI swich, a corresponding environment variable
`OPTION_FLAG=<value>`
exist and is used as default if set (CLI switches always
prevails). A complete list of CLI options is available running benchmarks
with the
`--help`
switch.
<a
name=
"running-a-subset-of-benchmarks"
/>
<a
name=
"running-a-subset-of-benchmarks"
/>
### Running a Subset of Benchmarks
### Running a Subset of Benchmarks
The
`--benchmark_filter=<regex>`
option can be used to only run the benchmarks
The
`--benchmark_filter=<regex>`
option (or
`BENCHMARK_FILTER=<regex>`
which match the specified
`<regex>`
. For example:
environment variable) can be used to only run the benchmarks that match
the specified
`<regex>`
. For example:
```
bash
```
bash
$
./run_benchmarks.x
--benchmark_filter
=
BM_memcpy/32
$
./run_benchmarks.x
--benchmark_filter
=
BM_memcpy/32
...
@@ -384,7 +410,8 @@ BM_memcpy/32k 1834 ns 1837 ns 357143
...
@@ -384,7 +410,8 @@ BM_memcpy/32k 1834 ns 1837 ns 357143
### Result comparison
### Result comparison
It is possible to compare the benchmarking results. See
[
Additional Tooling Documentation
](
docs/tools.md
)
It is possible to compare the benchmarking results.
See
[
Additional Tooling Documentation
](
docs/tools.md
)
<a
name=
"runtime-and-reporting-considerations"
/>
<a
name=
"runtime-and-reporting-considerations"
/>
...
@@ -450,6 +477,7 @@ range multiplier is changed to multiples of two.
...
@@ -450,6 +477,7 @@ range multiplier is changed to multiples of two.
```
c++
```
c++
BENCHMARK
(
BM_memcpy
)
->
RangeMultiplier
(
2
)
->
Range
(
8
,
8
<<
10
);
BENCHMARK
(
BM_memcpy
)
->
RangeMultiplier
(
2
)
->
Range
(
8
,
8
<<
10
);
```
```
Now arguments generated are
[
8, 16, 32, 64, 128, 256, 512, 1024, 2k, 4k, 8k
]
.
Now arguments generated are
[
8, 16, 32, 64, 128, 256, 512, 1024, 2k, 4k, 8k
]
.
The preceding code shows a method of defining a sparse range. The following
The preceding code shows a method of defining a sparse range. The following
...
@@ -466,6 +494,7 @@ static void BM_DenseRange(benchmark::State& state) {
...
@@ -466,6 +494,7 @@ static void BM_DenseRange(benchmark::State& state) {
}
}
BENCHMARK
(
BM_DenseRange
)
->
DenseRange
(
0
,
1024
,
128
);
BENCHMARK
(
BM_DenseRange
)
->
DenseRange
(
0
,
1024
,
128
);
```
```
Now arguments generated are
[
0, 128, 256, 384, 512, 640, 768, 896, 1024
]
.
Now arguments generated are
[
0, 128, 256, 384, 512, 640, 768, 896, 1024
]
.
You might have a benchmark that depends on two or more inputs. For example, the
You might have a benchmark that depends on two or more inputs. For example, the
...
@@ -535,6 +564,7 @@ void BM_takes_args(benchmark::State& state, ExtraArgs&&... extra_args) {
...
@@ -535,6 +564,7 @@ void BM_takes_args(benchmark::State& state, ExtraArgs&&... extra_args) {
// the specified values to `extra_args`.
// the specified values to `extra_args`.
BENCHMARK_CAPTURE
(
BM_takes_args
,
int_string_test
,
42
,
std
::
string
(
"abc"
));
BENCHMARK_CAPTURE
(
BM_takes_args
,
int_string_test
,
42
,
std
::
string
(
"abc"
));
```
```
Note that elements of
`...args`
may refer to global variables. Users should
Note that elements of
`...args`
may refer to global variables. Users should
avoid modifying global state inside of a benchmark.
avoid modifying global state inside of a benchmark.
...
@@ -659,6 +689,7 @@ Also you can create templated fixture by using the following macros:
...
@@ -659,6 +689,7 @@ Also you can create templated fixture by using the following macros:
*
`BENCHMARK_TEMPLATE_DEFINE_F(ClassName, Method, ...)`
*
`BENCHMARK_TEMPLATE_DEFINE_F(ClassName, Method, ...)`
For example:
For example:
```
c++
```
c++
template
<
typename
T
>
template
<
typename
T
>
class
MyFixture
:
public
benchmark
::
Fixture
{};
class
MyFixture
:
public
benchmark
::
Fixture
{};
...
@@ -813,6 +844,7 @@ BM_CalculatePiRange/256k 2434095 ns 2434186 ns 288 3.1416
...
@@ -813,6 +844,7 @@ BM_CalculatePiRange/256k 2434095 ns 2434186 ns 288 3.1416
BM_CalculatePiRange/1024k 9721140 ns 9721413 ns 71 3.14159
BM_CalculatePiRange/1024k 9721140 ns 9721413 ns 71 3.14159
BM_CalculatePi/threads:8 2255 ns 9943 ns 70936
BM_CalculatePi/threads:8 2255 ns 9943 ns 70936
```
```
Note above the additional header printed when the benchmark changes from
Note above the additional header printed when the benchmark changes from
``BM_UserCounter``
to
``BM_Factorial``
. This is because
``BM_Factorial``
does
``BM_UserCounter``
to
``BM_Factorial``
. This is because
``BM_Factorial``
does
not have the same counter set as
``BM_UserCounter``
.
not have the same counter set as
``BM_UserCounter``
.
...
@@ -1222,11 +1254,15 @@ the benchmark loop should be preferred.
...
@@ -1222,11 +1254,15 @@ the benchmark loop should be preferred.
<a
name=
"disabling-cpu-frequency-scaling"
/>
<a
name=
"disabling-cpu-frequency-scaling"
/>
### Disabling CPU Frequency Scaling
### Disabling CPU Frequency Scaling
If you see this error:
If you see this error:
```
```
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
```
```
you might want to disable the CPU frequency scaling while running the benchmark:
you might want to disable the CPU frequency scaling while running the benchmark:
```
bash
```
bash
sudo
cpupower frequency-set
--governor
performance
sudo
cpupower frequency-set
--governor
performance
./mybench
./mybench
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment