When the built-ins are not enough

For most engineers, a Markdown table in a PR is the end of the road. But as your performance needs grow, you'll hit a wall where the built-in tools aren't enough.

Maybe you need to stream your benchmark results into a corporate Grafana dashboard. Maybe you're working with specialized hardware that produces a very specific type of timing outlier that the IQR fence doesn't catch. Or maybe you want to correlate your benchmark samples with BCL instrumentation to see exactly which internal method is stealing your cycles.

NBenchmark is designed as a platform. Through a set of extension points, you can customize how results are reported, how outliers are detected, and how samples are observed in real-time.

Custom Reporters

If the standard formats don't fit your needs, you can implement your own IReporter. A custom reporter is responsible for taking the BenchmarkResult and turning it into your desired output.

The key to building a great reporter is the BenchmarkTable.Build utility. You don't need to re-derive the complex logic for ratios and significance tests; BenchmarkTable does that for you. Your reporter simply decides how to render that table.

To make your reporter discoverable via the --reporter CLI flag, you register it with the ReporterRegistry using a [ModuleInitializer].

public class MyCorporateReporter : IReporter
{
    public void Report(BenchmarkResult result, ReportDetailLevel detail)
    {
        var table = BenchmarkTable.Build(result);
        // Send table to corporate API...
    }
}

public static class ReporterExtensions
{
    [ModuleInitializer]
    public static void Initialize()
    {
        ReporterRegistry.Register("corp", new MyCorporateReporter());
    }
}

Specialized Outlier Detection

The default IQR (Interquartile Range) fence is a great general-purpose tool, but it isn't perfect. If you're dealing with bimodal distributions or hardware-specific jitters, you can implement a custom IOutlierDetector.

By using WithOutlierDetector, you can define a rule that examines the entire sample set and decides exactly which timings are "noise" and which are "signal." This is essential for tail-latency analysis, where a single unexpected spike might be a critical bug rather than an OS interrupt.

Real-time Observability with Observers

Sometimes you need to know what's happening during the run, not just after it. IMeasurementObserver allows you to hook into the engine's lifecycle.

Observers provide events for:

• OnPhase: When the engine moves from warmup to sampling.
• OnSample: Every time a new K-batch is completed.
• OnDetector: When the outlier detector trims a sample.
• OnResult: When the final statistics are calculated.

There is one strict rule for observers: they must be invisible. They should return immediately, never allocate on the hot path, and never throw. If an observer adds 10ns of overhead to a 50ns operation, it has just invalidated the benchmark.

BCL Instrumentation and OpenTelemetry

The most powerful way to observe a benchmark is to use the BCL's own instrumentation. NBenchmark provides a set of nbenchmark.* instruments and a span hierarchy that allow you to track execution without adding manual timing code to your body.

For those who need enterprise-grade observability, the NBenchmark.Exporters.OpenTelemetry package allows you to stream every single sample as an OTLP signal.

By using WithOpenTelemetry, your benchmarks stop being static reports and start being live telemetry. You can see your samples move in real-time in Jaeger or Honeycomb, allowing you to correlate a timing spike with a specific internal event or resource contention.

---

Go deeper: We've covered the extensions, the hardware, and the runtimes. To close the series, we're going to open the engine itself and look at the math and the protocols that make an honest number possible.

The engine, opened up →