From the team that brought you

Using Herbgrind

Herbgrind analyzes binaries to find inaccurate floating point expressions. The binaries can come from anywhere-C source, Fortran source, even unknown origins. This tutorial runs Herbgrind on the benchmark programs that Herbgrind ships with.

Compiling the benchmark program

Herbgrind ships test binaries in its bench/ directory. You can build them with:

make -C bench all

Let's analyze the diff-roots-simple.out binary that you just compiled. Run Herbgrind on that binary with:

herbgrind-path/ bench/diff-roots-simple.out

This should produce output that looks like this:

==16725== Herbgrind, a valgrind tool for Herbie
==16725== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==16725== Command: bench/diff-roots-simple.out
Writing report out to bench/

The printed value, 1.578592e-07, is printed by the diff-roots-simple.out binary. Herbgrind writes its results to the named file, bench/ This file contains one record for each operation; the only operation found in diff-roots-simple.c is:

(FPCore ()
  :type binary64
  (- (sqrt (+ 1.000000 10000000000000.000000)) (sqrt 10000000000000.000000)))
subtraction in main at diff-roots-simple.c:12 (address 400A00)
43.129555 bits average error
43.129555 bits max error
Aggregated over 1 instances

The first line gives the expression inaccurately evaluated, and the second line gives its location. That line in diff-roots-simple.c is actually:

y = sqrt(x + 1) - sqrt(x);

Since this line of code is run only once, Herbgrind doesn't know that x is intended to be a variable, and instead inlines its value. In a slightly more complicated example, diff-roots.c, this code is executed in a loop with different values for x, so Herbgrind will report it as a variable.

The next three lines of the output give the error incurred by the inaccurate computation: 43.1 bits of error over 1 instance of computing that expression.

Turning Herbgrind on and off

While running on diff-roots-simple.out, Herbgrind found inaccurate computations not only in diff-roots-simple.out but also in several GNU library calls. Herbgrind has a feature to avoid tracking floating point operations in libraries and other code not within your control by adding instrumentation to your source code.

Simply surround the numerically-interesting parts of your computation in the HERBGRIND_BEGIN() and HERBGRIND_END() macros:

// initialization code ...
// numerical code ...
// cleanup code ...

The diff-roots-simple.c example does this on lines 11 and 13. You can then run Herbgrind with the --start-off flag, which tells Herbgrind not to begin analyzing floating point operations until it sees a HERBGRIND_BEGIN() region:

./herbgrind --start-off bench/diff-roots-simple.out

The report file now contains only the inaccurate expression described before, and no library computations.

The HERBGRIND_BEGIN()/HERBGRIND_END() regions can be sprinkled anywhere in your source code; it's common to use them to start Herbgrind only after initializing your program and before cleaning up and outputting results. Herbgrind can be turned on and off multiple times.