When you run a benchmark suite, NBenchmark produces a table. At first glance, it looks like a standard set of performance metrics, but the columns are designed to answer very specific questions about the stability, honesty, and significance of your code.

Understanding these columns is the difference between seeing a "faster number" and knowing you have a "better implementation."

The primary metrics: Median, Mean, and Error

The first few columns provide the central tendency of your samples.

Median is your primary metric. It represents the middle value of your sample set. We prioritize the median because it is robust; a single massive spike caused by an OS context switch won't pull the median away from the typical execution time.

Mean is the arithmetic average. While less robust than the median, the mean is mathematically necessary to build the confidence interval.

Error is the ± margin of the confidence interval around the mean. It is usually displayed as a time value and a percentage (e.g., ±1.2ns (0.4%)).

A wide error margin does not necessarily mean you need more samples. The NBenchmark adaptive loop already handles sample counts to reach a target precision. A stubbornly wide interval usually points to external interference - such as a background process stealing your CPU cycles - rather than a lack of data.

The two sample sets: Trimmed vs Raw

This is the most important distinction in the NBenchmark engine. To give you an honest number, the engine does not treat all samples equally.

The trimmed set

The engine applies a statistical fence to identify and remove extreme outliers (spikes). The resulting trimmed set is used to calculate:

Mean
Standard Deviation (StdDev)
Coefficient of Variation (CV)
The Confidence Interval (Error)

By calculating these on the trimmed set, we ensure that the "typical" performance isn't skewed by a random thermal throttle event.

The raw set

The raw pre-trim set contains every single sample collected, including the spikes. This set is used for the tail metrics:

P95 and P99 (the 95th and 99th percentiles)
Min and Max
The histogram

We use the raw set here because the "tail" is exactly where the spikes live. If you are building a low-latency system, you care deeply about the P99. You want to know the worst-case scenario, and that includes the samples the fence would have removed. By default, NBenchmark uses TailMetricsBasis.Raw to ensure your tail metrics are honest.

Comparison and Cost: Ratio, Sig, and Magnitude

When you define a baseline in a suite, NBenchmark adds three columns that turn raw numbers into a comparison story.

Ratio tells you how much slower or faster a method is compared to the baseline. A ratio of 1.50 means the method is 50% slower than the baseline.

Sig (Significance) answers: "Is this difference real, or is it just noise?" It uses a non-parametric test to determine if the two distributions are statistically distinct. A checkmark (✓) means the difference is significant; a cross (✗) means it is not.

Magnitude classifies the effect size as Negligible, Small, Medium, or Large.

Crucially, Ratio, Sig, and Magnitude only mean something when read together. A result that is "Significant" but "Negligible" is a difference that is mathematically real but practically useless.

Alloc/op shows the average bytes allocated per operation. This is the cost the Garbage Collector sees. If your median time is low but your allocations are high, you are borrowing time from the future; eventually, the GC will trigger a pause that lands in your P99.

The Advanced detail block

If you run with ReportDetail.Advanced, NBenchmark appends a detailed diagnostic block below the table. This block exposes the "why" behind the numbers:

Quartiles and Fences: The exact boundaries used to identify outliers.
Shape Statistics: Skewness and Kurtosis, which tell you if your distribution is symmetrical or leaning heavily toward the tail.
Samples Trimmed: Exactly how many samples were discarded as noise.
Auto-tuned Line: The history of how the engine decided when to stop sampling.

Example: Simple vs Advanced

Consider a comparison between two JSON serializers:

var suite = new BenchmarkSuite()
    .Add("System.Text.Json", () => JsonSerializer.Serialize(data))
    .Add("FastJson", () => FastJson.Serialize(data))
    .WithBaseline("System.Text.Json");

await suite.RunAsync();

Simple Output:

Method	Median	Mean	Error	Ratio	Sig	Mag	Alloc/op
System.Text.Json	450ns	452ns	±2ns (0.4%)	1.00	-	-	128B
FastJson	310ns	315ns	±3ns (0.9%)	0.69	✓	Large	64B

Advanced Detail for FastJson:

Samples: 100 (Trimmed: 97, Outliers: 3)
Fence: [280ns, 340ns]
P95: 335ns | P99: 410ns
Skewness: 1.2 (Right-skewed)
AutoTune: Target 0.5% reached at sample 82.

In this example, FastJson is significantly faster (Ratio 0.69) with a large magnitude of improvement. However, the Advanced detail shows the distribution is right-skewed, and the P99 (410ns) is much closer to the System.Text.Json median than the FastJson median is.

For a deeper dive into how we handle the spikes that create those P99s, see The tail is a distribution, not a number.

What the table is telling you