> ## Documentation Index > Fetch the complete documentation index at: https://codspeed.io/docs/llms.txt > Use this file to discover all available pages before exploring further. # Writing Benchmarks in Kotlin > Create benchmarks for your Kotlin codebase using Java Microbenchmark Harness > [!NOTE] > > The JVM integration is still in early development, and **only the [walltime > instrument](https://codspeed.io/docs/instruments/walltime.md) is currently supported**. > > If you have any feedback, please reach out to us via > [Discord](https://discord.gg/MxpaCfKSqF) or > [email our support](mailto:contact@codspeed.io). Integrating CodSpeed into your Kotlin project works through a fork of [JMH](https://github.com/openjdk/jmh) (Java Microbenchmark Harness). You write standard JMH benchmarks in Kotlin and swap in the CodSpeed JMH fork as a dependency. When running in CI with CodSpeed, the results are automatically collected and reported. JMH generates its measurement harness from the compiled `@Benchmark` classes, so the setup is close to the Java one. Kotlin only changes how benchmark classes have to be declared, and which generator a Maven build has to run. ## Installation CodSpeed provides a fork of JMH that collects walltime results and sends them to CodSpeed. **Gradle** Add the CodSpeed JMH fork as a Git submodule: ```bash theme={null} git submodule add https://github.com/CodSpeedHQ/codspeed-jvm.git third-party/codspeed-jvm git submodule update --init --recursive -- third-party/codspeed-jvm ``` Then include it as a composite build in your `settings.gradle.kts`, with dependency substitution to redirect JMH dependencies to the CodSpeed fork: ```kotlin settings.gradle.kts theme={null} includeBuild("third-party/codspeed-jvm/jmh-fork") { // [!code ++] dependencySubstitution { // [!code ++] substitute(module("org.openjdk.jmh:jmh-core")) // [!code ++] .using(project(":jmh-core")) // [!code ++] substitute(module("org.openjdk.jmh:jmh-generator-annprocess")) // [!code ++] .using(project(":jmh-generator-annprocess")) // [!code ++] } // [!code ++] } // [!code ++] ``` We recommend using the [JMH Gradle Plugin](https://github.com/melix/jmh-gradle-plugin) as it handles benchmark compilation and provides the `jmh` task. It generates the harness from the compiled Kotlin classes in `src/jmh/kotlin`, so no annotation processing setup is needed. Add it to your `build.gradle.kts`: ```kotlin build.gradle.kts theme={null} plugins { kotlin("jvm") version "2.1.20" id("me.champeau.jmh") version "0.7.3"// [!code ++] } kotlin { jvmToolchain(21)// [!code ++] } ``` **Maven** Add the CodSpeed JMH fork as a Git submodule and publish it to your local Maven repository: ```bash theme={null} git submodule add https://github.com/CodSpeedHQ/codspeed-jvm.git third-party/codspeed-jvm git submodule update --init --recursive -- third-party/codspeed-jvm cd third-party/codspeed-jvm ./gradlew -p jmh-fork publishToMavenLocal ``` We're planning to publish to Maven Central in the future. If you need this, please reach out via [Discord](https://discord.gg/MxpaCfKSqF) or [email](mailto:contact@codspeed.io). Then replace the JMH dependencies in your `pom.xml` with the CodSpeed fork. Maven resolves from `~/.m2/repository` by default. Kotlin projects use the JMH bytecode generator rather than the annotation processor, so the fork has to be substituted in two places: the `jmh-core` dependency and the generator used by `exec-maven-plugin`. ```xml pom.xml theme={null} 1.37// [!code --] 0.2.1// [!code ++] org.openjdk.jmh// [!code --] io.codspeed.jmh// [!code ++] jmh-core ${jmh.version} org.codehaus.mojo exec-maven-plugin org.openjdk.jmh// [!code --] io.codspeed.jmh// [!code ++] jmh-generator-bytecode ${jmh.version} ``` > [!NOTE] > > Your Maven project must be configured to compile Kotlin sources, run the JMH > generator over the compiled classes, and package an executable benchmark JAR. > The `jmh-kotlin-benchmark-archetype` from JMH generates a project with all of > this wired up: > > ```bash theme={null} > mvn archetype:generate -DinteractiveMode=false \ > -DarchetypeGroupId=org.openjdk.jmh \ > -DarchetypeArtifactId=jmh-kotlin-benchmark-archetype \ > -DarchetypeVersion=1.37 \ > -DgroupId=com.example -DartifactId=kotlin-benchmarks -Dversion=1.0 > ``` ## Creating benchmarks Write your benchmarks using standard JMH annotations: ```kotlin FibBenchmark.kt theme={null} package bench import org.openjdk.jmh.annotations.Benchmark open class FibBenchmark { @Benchmark fun fib(): Long = fib(30) private fun fib(n: Int): Long = if (n <= 1) n.toLong() else fib(n - 1) + fib(n - 2) } ``` > [!NOTE] > > JMH benchmarks must return their result or use `Blackhole.consume()` to > prevent the JVM from eliminating dead code. All examples on this page return > the computed value. ## Testing the benchmarks locally To run the benchmarks with CodSpeed locally, first install the `codspeed` runner: ```bash theme={null} curl -fsSL https://codspeed.io/install.sh | sh ``` Then run your benchmarks with CodSpeed: **Gradle** \$ codspeed run --mode walltime -- ./gradlew jmh ►►► Running the benchmarks \# JMH version: 0.2.1 \# VM version: JDK 21.0.9, OpenJDK 64-Bit Server VM, 21.0.9+10 \# Warmup: 3 iterations, 1 s each \# Measurement: 3 iterations, 1 s each \# Benchmark: bench.FibBenchmark.fib \# Run progress: 0.00% complete, ETA 00:00:18 \# Warmup Iteration 1: 284.114 ops/s ... Benchmark Mode Cnt Score Error Units FibBenchmark.fib thrpt 9 286.986 ± 1.660 ops/s **Maven** \$ mvn package -q \$ codspeed run --mode walltime -- java -jar target/benchmarks.jar ►►► Running the benchmarks \# JMH version: 0.2.1 \# VM version: JDK 21.0.9, OpenJDK 64-Bit Server VM, 21.0.9+10 \# Warmup: 3 iterations, 1 s each \# Measurement: 3 iterations, 1 s each \# Benchmark: com.example.MyBenchmark.fib \# Run progress: 0.00% complete, ETA 00:00:18 \# Warmup Iteration 1: 286.603 ops/s ... Benchmark Mode Cnt Score Error Units MyBenchmark.fib thrpt 9 286.967 ± 2.042 ops/s ## Running the benchmarks in your CI To generate performance reports, you need to run the benchmarks in your CI. This allows CodSpeed to automatically run benchmarks and warn you about regressions during development. > [!TIP] > > If you want more details on how to configure the CodSpeed action, you can check > out the [Continuous Reporting section](https://codspeed.io/docs/integrations/ci.md). Here is an example of a GitHub Actions workflow that runs the benchmarks and reports the results to CodSpeed on every push to the `main` branch and every pull request: **Gradle** ```yaml .github/workflows/codspeed.yml icon="github" theme={null} name: CodSpeed Benchmarks on: push: branches: - "main" # or "master" pull_request: # `workflow_dispatch` allows CodSpeed to trigger backtest # performance analysis in order to generate initial data. workflow_dispatch: jobs: benchmarks: name: Run benchmarks runs-on: codspeed-macro-arm64-graviton-ubuntu-22-04 permissions: # optional for public repositories contents: read # required for actions/checkout id-token: write # required for OIDC authentication with CodSpeed steps: - uses: actions/checkout@v7 with: submodules: recursive - uses: actions/setup-java@v4 with: distribution: temurin java-version: 21 - name: Run the benchmarks uses: CodSpeedHQ/action@v5 with: mode: walltime run: ./gradlew jmh ``` **Maven** ```yaml .github/workflows/codspeed.yml icon="github" theme={null} name: CodSpeed Benchmarks on: push: branches: - "main" # or "master" pull_request: # `workflow_dispatch` allows CodSpeed to trigger backtest # performance analysis in order to generate initial data. workflow_dispatch: jobs: benchmarks: name: Run benchmarks runs-on: codspeed-macro-arm64-graviton-ubuntu-22-04 permissions: # optional for public repositories contents: read # required for actions/checkout id-token: write # required for OIDC authentication with CodSpeed steps: - uses: actions/checkout@v7 with: submodules: recursive - uses: actions/setup-java@v4 with: distribution: temurin java-version: 21 - name: Install CodSpeed JMH fork run: cd third-party/codspeed-jvm && ./gradlew -p jmh-fork publishToMavenLocal - name: Build benchmark JAR run: mvn package -q - name: Run the benchmarks uses: CodSpeedHQ/action@v5 with: mode: walltime run: java -jar target/benchmarks.jar ``` ## Advanced usage JMH provides many features for writing expressive benchmarks. Below is a selection that can be useful in CodSpeed benchmarks. ### Parameterized benchmarks Use `@Param` to run the same benchmark with different input values. The annotated property has to be a `var`, since JMH assigns it before the benchmark runs and rejects final fields: ```kotlin ParamBenchmark.kt theme={null} package bench import org.openjdk.jmh.annotations.Benchmark import org.openjdk.jmh.annotations.Param import org.openjdk.jmh.annotations.Scope import org.openjdk.jmh.annotations.State @State(Scope.Benchmark) open class ParamBenchmark { @Param("10", "20", "30") var n: Int = 0 @Benchmark fun fib(): Long = fib(n) private fun fib(n: Int): Long = if (n <= 1) n.toLong() else fib(n - 1) + fib(n - 2) } ``` ### Shared state Use `@State` to share setup logic across benchmarks and control the scope of the state object: ```kotlin StateBenchmark.kt theme={null} package bench import org.openjdk.jmh.annotations.Benchmark import org.openjdk.jmh.annotations.Scope import org.openjdk.jmh.annotations.Setup import org.openjdk.jmh.annotations.State @State(Scope.Benchmark) open class StateBenchmark { private lateinit var numbers: List @Setup fun setup() { numbers = (0 until 1000).toList() } @Benchmark fun sum(): Int = numbers.sum() } ``` JMH annotations behave the same in Kotlin as in Java. For a deeper dive, see the dedicated guide: > **[How to Benchmark Java with JMH](https://codspeed.io/docs/guides/how-to-benchmark-java-with-jmh.md)** > > An in-depth guide to writing JMH benchmarks: project setup, annotations, > parameters, common pitfalls, and CodSpeed CI integration. ## Compatibility * **JDK 21 or later** is required. * Benchmark classes must not be final, since JMH subclasses them to generate the measurement harness. * All standard JMH annotations are supported. * Only the [Walltime instrument](https://codspeed.io/docs/instruments/walltime.md) is supported. * CodSpeed uses a custom mode for all benchmarks to collect statistically significant results. Any `@BenchmarkMode` annotations in your code are ignored. If you run into issues or require certain features, please [open an issue](https://github.com/CodSpeedHQ/codspeed-jvm/issues) or [join our Discord](https://discord.com/invite/MxpaCfKSqF) to get help. ## Next steps > **[Example repository](https://github.com/CodSpeedHQ/codspeed-jvm)** > > The CodSpeed JVM repository with example JMH benchmarks. > **[How to Benchmark Java with JMH](https://codspeed.io/docs/guides/how-to-benchmark-java-with-jmh.md)** > > An in-depth guide to writing JMH benchmarks with CodSpeed. > **[Walltime instrument](https://codspeed.io/docs/instruments/walltime.md)** > > Learn more about the Walltime instrument and how to use it. > **[Dive into performance changes](https://codspeed.io/docs/features/profiling.md)** > > Learn more about profiling and how to read flamegraphs. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.