One line, one honest number
Mastering Single mode in NBenchmark - handling async, complex state, and interpreting BenchmarkResult.
In the previous post, we saw that Benchmark.Run is the fastest way to get an honest number. For many tasks - like verifying a specific optimization or checking the cost of a new utility method - this "Single mode" is all you need.
But real-world code rarely exists in a vacuum. You have async methods, state that needs to be prepared before the timer starts, and results that need to be analyzed beyond a simple printout.
The overload family: Action vs Func
NBenchmark provides several overloads for Benchmark.Run to match the shape of your code. At first glance, you'll see overloads for both Action (void) and Func<T> (returning a value).
While you can benchmark a void method, you should prefer returning a value whenever possible.
When you return a value, the NBenchmark engine consumes it through a "sink." This is a critical detail: by ensuring the return value is used, NBenchmark prevents the .NET JIT compiler from performing dead-code elimination. If the compiler can prove that a calculation's result is never read, it may simply delete the entire operation.
If you provide an Action, the analyzer NB0010 will warn you if it detects a "throwaway" lambda - a method that does work but doesn't produce an observable side effect or return a value. The fix is simple: change your benchmark to return the result of the calculation.
Handling Async code
Modern .NET is async. If you are measuring a method that returns a Task or ValueTask, use Benchmark.RunAsync.
using NBenchmark;
var result = await Benchmark.RunAsync(async () =>
{
await Task.Delay(1); // Imagine a real async operation here
});
result.Print();
RunAsync captures the full awaited duration, including the overhead of the state machine, giving you the actual wall-clock time the caller experiences.
The Prepare/Body pattern
Some benchmarks require a significant amount of setup. If you need to load a 100MB dataset into memory before you can measure a search algorithm, you don't want that loading time to be part of your measurement.
NBenchmark solves this with the prepare and body split.
var result = Benchmark.Run(
prepare: () =>
{
// This runs once per worker process, before warmup
return LoadLargeDataset();
},
body: (data) =>
{
// This is the timed portion
return data.Search("target");
}
);
The prepare delegate runs once inside the worker process. The returned value is then passed into the body for every sample. This ensures your data is "hot" and ready in the worker's memory, but the cost of building that data is completely excluded from the timing.
Per-sample setup and teardown
While prepare happens once, sometimes you need a clean slate for every single sample. For example, if your benchmark modifies a collection, you need to reset that collection before the next run.
You can pass setup and teardown delegates to Benchmark.Run. These run around every sample, but - crucially - they are executed outside the timed window.
var result = Benchmark.Run(
body: () => { /* ... */ },
setup: () => {
// Reset state here
},
teardown: () => {
// Clean up here
}
);
Understanding the BenchmarkResult
When Benchmark.Run completes, it returns a BenchmarkResult. While .Print() is great for a quick look, the result object contains the full statistical picture:
- Median: The middle value of the trimmed sample set. This is your primary metric because it is robust against the remaining noise.
- Mean: The arithmetic average. This is used to build the confidence interval.
- Error: The $\pm$ margin of error. A tight error (e.g., $\pm 0.5$ ns) means the engine has pinned the mean with high precision.
- P95 / P99: The 95th and 99th percentiles. These tell you about the "tail" of your performance - the worst-case scenarios.
- Alloc/op: The average number of bytes allocated per operation.
Diving into the raw data
If you need to perform your own analysis, you can access the samples directly via result.RawSamples.
var samples = result.RawSamples;
Console.WriteLine($"Total samples captured: {samples.Length}");
By default, NBenchmark caps the number of raw samples stored in memory to prevent massive runs from crashing your process. You can tune this using MeasurementOptions.MaxRawSamples.
For a deeper look at how to configure these runs - including tuning sample counts and warmup durations - see the next post.
Go deeper: We've looked at one-off measurements. But what happens when you need to compare two different implementations side-by-side?