๐ŸŒ AIๆœ็ดข & ไปฃ็† ไธป้กต
Skip to main content
CodSpeed is a performance testing tool helping you to consistently measure the performance of your codebase locally and in your CI/CD pipeline. More details on how CodSpeed works here: What is CodSpeed? This guide is a quick start to get a sample Python repository on GitHub up and running with CodSpeed:
  1. Create Performance Tests
  2. Connect your Repository
  3. Run the Tests in your CI
  4. Introduce a Performance Regression

โ€‹
Create Performance Tests

  1. Install the CodSpeed plugin for pytest: 
    pip install pytest-codspeed
  2. Write a performance test using the @pytest.mark.benchmark marker:
    tests/test_sum_squares.py
    import pytest

def sum_squares(arr):
    """Sum the squares of the numbers in an array."""
    total = 0
    for x in arr:
        total += x * x
    return total

# Your tests can also be benchmarks
@pytest.mark.benchmark
def test_sum_squares():
    assert sum_squares(range(1000)) == 332833500
  3. Run your performance tests locally: 
    $ pytest tests/ --codspeed
============================= test session starts ====================
platform darwin -- Python 3.13.0, pytest-7.4.4, pluggy-1.5.0
codspeed: 3.0.0 (enabled, mode: walltime, timer_resolution: 41.7ns)
rootdir: /home/user/codspeed-test, configfile: pytest.ini
plugins: codspeed-3.0.0
collected 1 items

tests/test_sum_squares.py .                                    [ 100%]

                         Benchmark Results
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ     Benchmark  โ”ƒ Time (best) โ”ƒ Rel. StdDev โ”ƒ Run time โ”ƒ Iters  โ”ƒ
โ”ฃโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‹โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‹โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‹โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‹โ”โ”โ”โ”โ”โ”โ”โ”โ”ซ
โ”ƒtest_sum_squaresโ”ƒ     1,873ns โ”ƒ        4.8% โ”ƒ    3.00s โ”ƒ 66,930 ๏ฟฝ๏ฟฝ๏ฟฝ
โ”—โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ปโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ปโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ปโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ปโ”โ”โ”โ”โ”โ”โ”โ”โ”›
=============================== 1 benchmarked ========================
=============================== 1 passed in 4.12s ====================
    Great! We set up our first performance test. Now, letโ€™s track the performance in the CI with CodSpeed!

โ€‹
Connect your Repository

  1. Login on CodSpeed
  2. Go to settings and install the CodSpeed GitHub App by clicking on the โ€œImportโ€ button.
    Adding a new repository from the settings page
  3. Select the organization or the user and add the repositories you want to use with CodSpeed: Github App organization selection
  4. Back in CodSpeed settings, configure a repository by clicking on the โ€œSetupโ€ button:
    Repository Setup button

โ€‹
Run the Tests in your CI

  1. Create a new GitHub Actions workflow file to run your benchmarks:
    .github/workflows/codspeed.yml
    name: CodSpeed Benchmarks

on:
  push:
    branches:
      - "main" # or "master"
  pull_request:

permissions: # optional for public repositories
  contents: read # required for actions/checkout
  id-token: write # required for OIDC authentication with CodSpeed
  
jobs:
  benchmarks:
    name: Run benchmarks
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      # ...
      # Setup your environment here:
      #  - Configure your Python/Rust/Node version
      #  - Install your dependencies
      #  - Build your benchmarks (if using a compiled language)
      # ...
      - name: Run the benchmarks
        uses: CodSpeedHQ/action@v4
        with:
          mode: simulation
          run: <Insert your benchmark command here>
  2. Create a Pull Request installing the workflow to the repository and wait for the report in the comments: Pull Request Result on Installation
  3. Merge it and congrats ๐ŸŽ‰, CodSpeed is installed!

โ€‹
Introduce a Performance Regression

  1. Letโ€™s change the implementation of the sum_squares with a more concise and elegant one:
    tests/test_sum_squares.py
    def sum_squares(arr):
    """Sum the squares of the numbers in an array."""
    total = 0
    for x in arr:
        total += x * x
    return total
    return sum(map(lambda x: x**2, arr))
  2. Open a Pull Request and wait for the CodSpeed report:
Pull Request Regression Result

Pull Request Performance Report showing the performance regression

Before merging, we can see that even if itโ€™s more concise, this new implementation is slower.
Pull Request Checks failing because of the performance regression

Pull Request Checks failing because of the performance regression

Merging it would have introduced a performance regression but we caught it before shipping it!
This behavior can be enforced by configuring performance checks.

โ€‹
Next Steps

What is CodSpeed?

Find out more details on how CodSpeed works

pytest-codspeed documentation

Learn more on how to create performance tests with pytest-codspeed

Explore the Performance Metrics

Understand the performance metrics generated by CodSpeed

Enforce Performance Checks

Make sure you or team members never merge unexpected performance regressions