Fail the build when it gets slower

A benchmark that nobody enforces is just trivia.

It's great to have a table in a pull request showing that a refactor is 10% faster, but that's a manual check. The moment a developer forgets to look at the report, or a reviewer assumes the numbers are "close enough," performance begins to drift. To stop the slow bleed of performance degradation, you have to move from observation to enforcement.

You need to turn your numbers into consequences.

Gate one: The CLI regression limit

For dedicated benchmark projects, the simplest way to enforce performance is the --max-regression-percent flag.

This flag allows you to define a hard limit on how much slower a benchmark can be compared to its baseline. If any median result exceeds this percentage, the process exits with a non-zero code, failing your CI pipeline.

# Fail the build if any benchmark is more than 5% slower than the baseline
dotnet benchmark --max-regression-percent 5

The beauty of this approach is that it doesn't block the evidence. Even when the build fails, NBenchmark still flushes the reporters. Your GitHub Action or Azure DevOps pipeline will fail, but the Markdown table will still be there, showing exactly which implementation crossed the line and by how much.

Gate two: Test-framework integration

While CLI flags are great for standalone projects, most engineers want their performance gates to live where their other tests live: in xUnit, NUnit, or MSTest.

NBenchmark provides integration packages (e.g., NBenchmark.Integration.xUnit) that allow you to write performance tests as first-class citizens. Instead of a [Fact], you use [PerformanceFact].

using NBenchmark.Integration.xUnit;

public class PaymentProcessingTests
{
    [PerformanceFact]
    public void ProcessPayment_ShouldBeFast()
    {
        // The entire method body is the benchmark
        var processor = new PaymentProcessor();
        processor.Process(new PaymentRequest(100));
    }
}

Absolute vs. Relative thresholds

Once you've integrated benchmarks into your tests, you have two ways to define "failure."

Absolute thresholds set a hard limit on the cost of an operation:

• MaxMeanNs: Fail if the mean exceeds this value.
• MaxP95Ns: Fail if the 95th percentile exceeds this value.
• MaxAllocatedBytes: Fail if the allocation cost is too high.

Absolute thresholds are useful for strict SLAs (e.g., "this API must respond in under 50ms"), but they are brittle. They fail when you move from a powerful build server to a slower runner.

Relative thresholds are the professional choice. By specifying a ReferenceMethod and a MaxSlowdownRatio, you compare the current implementation against a known-good reference measured in the same run.

[PerformanceFact(
    ReferenceMethod = nameof(LegacyProcessPayment), 
    MaxSlowdownRatio = 1.2)] // Fail if 20% slower than legacy
public void NewProcessPayment_ShouldNotRegress() 
{
    // ...
}

Because both the reference and the target are measured on the same machine in matching worker processes, the absolute speed of the hardware doesn't matter. A fast laptop and a slow CI runner will produce the same ratio. You no longer have to check in "baseline files" that go stale the moment you update your compiler.

Precision in the gate

By default, [Performance] tests run with a single launch to keep the test suite fast. While this is fine for catching 2x regressions, it's not enough for high-precision gates.

If you need to enforce a tight 5% limit, you should increase the LaunchCount. Raising the launch count provides a paired-interval ratio, which ensures that a "flaky" run on a noisy CI server doesn't trigger a false positive.

For those who need to perform assertions manually within a test, PerformanceAssert.Run (NUnit/MSTest) or BenchmarkAssert.Validate (xUnit) allows you to trigger a measurement and validate the result against your constraints in a single line.

Choosing your gate

Which enforcement strategy should you use?

Scenario	Recommended Gate	Why
Standalone perf project	--max-regression-percent	Simple, fast, and reports to CLI
Unit test suite	[PerformanceFact]	Integrated into existing test runners
Strict SLA / Latency Cap	MaxP95Ns	Ensures absolute limits are met
Refactor / Comparison	ReferenceMethod	Hardware-agnostic and stable

By turning your benchmarks into gates, you stop guessing if your code is fast enough. You define the limit, and the build ensures you never cross it.

---

Go deeper: You've set up the gates, but occasionally, the gates will fail and the numbers will look impossible. When that happens, you need a field guide to diagnose the symptoms.

When the numbers look wrong →