Upgrade tests instead simulate the lifetime of a database: they chain together commands and version upgrades, verifying that database state and API responses remain consistent.

Basic example

{
  "type": "test",
  "name": "api-keys",
  "initialVersion": "1.19.0", // the first command will run on a brand new database of this version
  "commands": []
}

This example defines a no-op test (it does nothing).

If the file is saved at workloads/tests/example.json, you can run it with:

cargo xtask test workloads/tests/example.json

Commands

Commands represent API requests sent to Meilisearch endpoints during a test.

They are executed sequentially, and their responses can be validated to ensure consistent behavior across upgrades.


{
  "route": "keys",
  "method": "POST",
  "body": {
    "inline": {
      "actions": [
        "search",
        "documents.add"
      ],
      "description": "Test API Key",
      "expiresAt": null,
      "indexes": [ "movies" ]
    }
  }
}

This command issues a POST /keys request, creating an API key with permissions to search and add documents in the movies index.

Using assets in commands

To keep tests concise and reusable, you can define assets at the root of the workload file.

Assets are external data sources (such as datasets) that are cached between runs, making tests faster and easier to read.

{
  "type": "test",
  "name": "movies",
  "initialVersion": "1.12.0",
  "assets": {
    "movies.json": {
      "local_location": null,
      "remote_location": "https://milli-benchmarks.fra1.digitaloceanspaces.com/bench/datasets/movies.json",
      "sha256": "5b6e4cb660bc20327776e8a33ea197b43d9ec84856710ead1cc87ab24df77de1"
    }
  },
  "commands": [
    {
      "route": "indexes/movies/documents",
      "method": "POST",
      "body": {
        "asset": "movies.json"
      }
    }
  ]
}

In this example:

The movies.json dataset is defined as an asset, pointing to a remote URL.
The SHA-256 checksum ensures integrity.
The POST /indexes/movies/documents command uses this asset as the request body.

This makes the test much cleaner than inlining a large dataset directly into the command.

Asserting responses

Commands can specify both the expected status code and the expected response body.

{
  "route": "indexes/movies/documents",
  "method": "POST",
  "body": {
    "asset": "movies.json"
  },
  "expectedStatus": 202,
  "expectedResponse": {
    "enqueuedAt": "[timestamp]", // Set to a bracketed string to ignore the value
    "indexUid": "movies",
    "status": "enqueued",
    "taskUid": 1,
    "type": "documentAdditionOrUpdate"
  },
  "synchronous": "WaitForTask"
}

Manually writing expectedResponse fields can be tedious.

Instead, you can let the test runner populate them automatically:

# Run the workload to populate expected fields. Only adds the missing ones, doesn't change existing data
cargo xtask test workloads/tests/example.json --add-missing-responses

# OR

# Run the workload to populate expected fields. Updates all fields including existing ones
cargo xtask test workloads/tests/example.json --update-responses

This workflow is recommended:

Write the test without expected fields.
Run it with --add-missing-responses to capture the actual responses.
Review and commit the generated expectations.

Upgrade commands

Upgrade commands allow you to switch the Meilisearch instance from one version to another during a test.

When executed, an upgrade command will:

Stop the current Meilisearch server.
Upgrade the database to the specified version.
Restart the server with the new specified version.

Typical Usage

In most cases, you will:

Set up some data using commands on an older version.
Upgrade to the latest version.
Assert that the data and API behavior remain correct after the upgrade.

{
  "type": "test",
  "name": "movies",
  "initialVersion": "1.12.0", // An older version to start with
  "commands": [
    // Commands to populate the database
    {
      "upgrade": "latest" // Will build meilisearch locally and run it
    },
    // Commands to check the state of the database
  ]
}

This ensures backward compatibility: databases created with older Meilisearch versions should remain functional and consistent after an upgrade.

Advanced usage

As time goes on, tests may grow more complex as they evolve alongside new features and schema changes. A single test can chain together multiple upgrades, interleaving data population, API checks, and version transitions.

For example:

{
  "type": "test",
  "name": "movies",
  "initialVersion": "1.12.0",
  "commands": [
    // Commands to populate the database
    {
      "upgrade": "1.17.0"
    },
    // Commands on endpoints that were removed after 1.17
    {
      "upgrade": "latest"
    },
    // Check the state
  ]
}

Variables

Sometimes a command needs to use a value returned by a previous response. These values can be captured and reused using the register field.

{
  "route": "keys",
  "method": "POST",
  "body": {
    "inline": {
        "actions": [
        "search",
        "documents.add"
        ],
        "description": "Test API Key",
        "expiresAt": null,
        "indexes": [ "movies" ]
    }
  },
  "expectedResponse": {
      "key": "c6f64630bad2996b1f675007c8800168e14adf5d6a7bb1a400a6d2b158050eaf",
      // ...
  },
  "register": {
    "key": "/key"
  },
  "synchronous": "WaitForResponse"
}

The register field captures the value at the JSON path /key from the response. Paths follow the JavaScript Object Notation Pointer (RFC 6901) format. Registered variables are available for all subsequent commands.

Registered variables can be referenced by wrapping their name in double curly braces:

In the route/path:

{
  "route": "tasks/{{ task_id }}",
  "method": "GET"
}

In the request body:

{
  "route": "indexes/movies/documents",
  "method": "PATCH",
  "body": {
    "inline": {
      "id": "{{ document_id }}",
      "overview": "Shazam turns evil and the world is in danger.",
    }
  }
}

As an API-key:

{
  "route": "indexes/movies/documents",
  "method": "POST",
  "body": { /* ... */ },
  "apiKeyVariable": "key" // The content of the key variable will be used as an API key
}