If you benchmark a sorting algorithm with a list of 10 items, it will look incredibly fast. If you benchmark it with a million items, it will look slow. Neither number is "wrong," but neither number is useful on its own.

Performance is not a single value; it is a function of its inputs. To understand how your code actually behaves, you need to measure it across a spectrum of sizes, shapes, and types of data.

NBenchmark handles this through parameterized benchmarks, allowing you to run a single piece of logic across many different inputs without writing multiple benchmark methods.

Parameterization in Harness mode

When using a dedicated benchmark class (Harness mode), you can define inputs using the [Arguments] and [ArgumentsSource] attributes.

Inline literals with `[Arguments]`

For simple sweeps - like testing a few specific array sizes - [Arguments] is the fastest way. You can apply the attribute multiple times to the same method, with each attribute representing one test case.

public class ScalingBenchmark
{
    [Benchmark]
    [Arguments(10)]
    [Arguments(100)]
    [Arguments(1000)]
    public void SortData(int size)
    {
        var data = GenerateRandomData(size);
        data.Sort();
    }
}

Dynamic sweeps with `[ArgumentsSource]`

When your inputs are complex - such as a generated range of values or a set of files from a directory - use [ArgumentsSource]. This attribute points to a static method or property that returns an IEnumerable of arguments.

This is particularly powerful for creating logarithmic sweeps (e.g., 10, 100, 1000, 10000), which are essential for identifying where an algorithm's complexity (like $O(n \log n)$ vs $O(n^2)$) starts to dominate the execution time.

Note: The NBenchmark analyzer NB0003 ensures that the number of arguments provided by the source matches the method's parameter count, and NB0012 prevents you from accidentally combining [Arguments] and [ArgumentsSource] on the same method.

Parameterization in Suite mode

If you are building a comparison suite programmatically, you can use WithParameter. This tells the engine to run every benchmark in the suite against the provided value.

Alternatively, you can use the typed Add overloads to define exactly how the parameters are injected into your lambdas. This allows you to create a matrix of results where you can compare Implementation A vs Implementation B across five different input sizes in a single table.

When you read the resulting table, look for the scaling trend. A linear increase in time as input grows is expected; an exponential spike is a warning that your code will collapse under production loads.

Organizing the explosion with Categories

Parameterization is powerful, but it creates a "combinatorial explosion." If you have 10 benchmarks and each has 10 parameter sets, you now have 100 benchmarks to run. In a CI pipeline, running all 100 on every commit is a waste of time.

NBenchmark solves this with categories. You can mark a method or an entire class with [BenchmarkCategory("Slow")] or [BenchmarkCategory("SmokeTest")].

Categories allow you to tier your performance testing:

The Smoke Set: A tiny subset of parameters (e.g., $n=10$) that runs on every PR to catch catastrophic regressions.
The Deep Set: The full scaling sweep that runs nightly to monitor long-term performance trends.

You can filter these at the command line using --include-category and --exclude-category, or programmatically via WithCategories.

# Run only the fast smoke tests for a quick check
dotnet benchmark --include-category SmokeTest

# Run everything except the heavy nightly sweeps
dotnet benchmark --exclude-category Slow

Reading the scaling table

When you run a parameterized benchmark, NBenchmark names the resulting entries using the pattern MethodName(arg1, arg2, ...).

In your results table, this allows you to see the scaling curve directly. If you see that SortData(10) takes 100ns and SortData(100) takes 1,000ns, your code is scaling linearly. If SortData(100) suddenly jumps to 10,000ns, you've found a non-linear performance cliff.

By combining parameters with significance tests, you can answer high-precision questions: "Is Implementation B significantly faster than A for small inputs, but significantly slower for large ones?" This is the only way to make an informed decision about which algorithm to ship.

Go deeper: Now that you can measure how your code scales with input, there's one more variable to consider: the machine itself. A single run can be a fluke. To know if a change is truly a regression, you need to see how it moves across multiple launches.

One run cannot tell you how much it moves →

Benchmarks that take arguments