Introduce more databases to measure

.github/ISSUE_TEMPLATE/new_feature_issue.md

@@ -24,11 +24,6 @@ TBD
 - [ ] If not, add the `no db change` label to your PR, and you're good to merge.
 - [ ] If yes, add the `db change` label to your PR. You'll receive a message explaining you what to do.
 
-### Reminders when adding features
-
-- [ ] Write unit tests using insta
-- [ ] Write declarative integration tests in [workloads/tests](https://github.com/meilisearch/meilisearch/tree/main/workloads/test). Specify the routes to call and then call `cargo xtask test workloads/tests/YOUR_TEST.json --update-responses` so that responses are automatically filled.
-
 ### Reminders when modifying the API
 
 - [ ] Update the openAPI file with utoipa:

.github/workflows/bench-pr.yml

@@ -67,6 +67,8 @@ jobs:
           ref: ${{ steps.comment-branch.outputs.head_ref }}
 
       - uses: dtolnay/rust-toolchain@1.89
+        with:
+          profile: minimal
 
       - name: Run benchmarks on PR ${{ github.event.issue.id }}
         run: |

.github/workflows/bench-push-indexing.yml

@@ -13,6 +13,8 @@ jobs:
     steps:
       - uses: actions/checkout@v5
       - uses: dtolnay/rust-toolchain@1.89
+        with:
+          profile: minimal
 
       # Run benchmarks
       - name: Run benchmarks - Dataset ${BENCH_NAME} - Branch main - Commit ${{ github.sha }}

.github/workflows/db-change-comments.yml

@@ -6,7 +6,7 @@ on:
 
 env:
   MESSAGE: |
-    ### Hello, I'm a bot 🤖
+    ### Hello, I'm a bot 🤖 
 
     You are receiving this message because you declared that this PR make changes to the Meilisearch database.
     Depending on the nature of the change, additional actions might be required on your part. The following sections detail the additional actions depending on the nature of the change, please copy the relevant section in the description of your PR, and make sure to perform the required actions.
@@ -19,7 +19,6 @@ env:
 
     - [ ] Detail the change to the DB format and why they are forward compatible
     - [ ] Forward-compatibility: A database created before this PR and using the features touched by this PR was able to be opened by a Meilisearch produced by the code of this PR.
-    - [ ] Declarative test: add a [declarative test containing a dumpless upgrade](https://github.com/meilisearch/meilisearch/blob/main/TESTING.md#typical-usage)
 
 
     ## This PR makes breaking changes
@@ -36,7 +35,8 @@ env:
       - [ ] Write the code to go from the old database to the new one
         - If the change happened in milli, the upgrade function should be written and called [here](https://github.com/meilisearch/meilisearch/blob/3fd86e8d76d7d468b0095d679adb09211ca3b6c0/crates/milli/src/update/upgrade/mod.rs#L24-L47)
         - If the change happened in the index-scheduler, we've never done it yet, but the right place to do it should be [here](https://github.com/meilisearch/meilisearch/blob/3fd86e8d76d7d468b0095d679adb09211ca3b6c0/crates/index-scheduler/src/scheduler/process_upgrade/mod.rs#L13)
-      - [ ] Declarative test: add a [declarative test containing a dumpless upgrade](https://github.com/meilisearch/meilisearch/blob/main/TESTING.md#typical-usage)
+      - [ ] Write an integration test [here](https://github.com/meilisearch/meilisearch/blob/main/crates/meilisearch/tests/upgrade/mod.rs) ensuring you can read the old database, upgrade to the new database, and read the new database as expected
+    
 
 jobs:
   add-comment:

.github/workflows/flaky-tests.yml

@@ -13,12 +13,6 @@ jobs:
       image: ubuntu:22.04
     steps:
       - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
-        run: |
-          sudo rm -rf "/opt/ghc" || true
-          sudo rm -rf "/usr/share/dotnet" || true
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - name: Install needed dependencies
         run: |
           apt-get update && apt-get install -y curl

.github/workflows/fuzzer-indexing.yml

@@ -13,6 +13,8 @@ jobs:
     steps:
       - uses: actions/checkout@v5
       - uses: dtolnay/rust-toolchain@1.89
+        with:
+          profile: minimal
 
       # Run benchmarks
       - name: Run the fuzzer

.github/workflows/publish-apt-brew-pkg.yml

@@ -25,12 +25,6 @@ jobs:
         run: |
           apt-get update && apt-get install -y curl
           apt-get install build-essential -y
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
-        run: |
-          sudo rm -rf "/opt/ghc" || true
-          sudo rm -rf "/usr/share/dotnet" || true
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - uses: dtolnay/rust-toolchain@1.89
       - name: Install cargo-deb
         run: cargo install cargo-deb

.github/workflows/publish-docker-images.yml

@@ -14,105 +14,10 @@ on:
   workflow_dispatch:
 
 jobs:
-  build:
+  docker:
-    runs-on: ${{ matrix.runner }}
+    runs-on: docker
-
-    strategy:
-      matrix:
-        platform: [amd64, arm64]
-        edition: [community, enterprise]
-        include:
-          - platform: amd64
-            runner: ubuntu-24.04
-          - platform: arm64
-            runner: ubuntu-24.04-arm
-          - edition: community
-            registry: getmeili/meilisearch
-            feature-flag: ""
-          - edition: enterprise
-            registry: getmeili/meilisearch-enterprise
-            feature-flag: "--features enterprise"
-
-    permissions: {}
-    steps:
-      - uses: actions/checkout@v5
-
-      - name: Prepare
-        run: |
-          platform=linux/${{ matrix.platform }}
-          echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-        with:
-          platforms: linux/${{ matrix.platform }}
-          install: true
-
-      - name: Login to Docker Hub
-        uses: docker/login-action@v3
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Docker meta
-        id: meta
-        uses: docker/metadata-action@v5
-        with:
-          images: ${{ matrix.registry }}
-          # Prevent `latest` to be updated for each new tag pushed.
-          # We need latest and `vX.Y` tags to only be pushed for the stable Meilisearch releases.
-          flavor: latest=false
-          tags: |
-            type=ref,event=tag
-            type=raw,value=nightly,enable=${{ github.event_name != 'push' }}
-            type=semver,pattern=v{{major}}.{{minor}},enable=${{ steps.check-tag-format.outputs.stable == 'true' }}
-            type=semver,pattern=v{{major}},enable=${{ steps.check-tag-format.outputs.stable == 'true' }}
-            type=raw,value=latest,enable=${{ steps.check-tag-format.outputs.stable == 'true' && steps.check-tag-format.outputs.latest == 'true' }}
-
-      - name: Build and push by digest
-        uses: docker/build-push-action@v6
-        id: build-and-push
-        with:
-          platforms: linux/${{ matrix.platform }}
-          labels: ${{ steps.meta.outputs.labels }}
-          tags: ${{ matrix.registry }}
-          outputs: type=image,push-by-digest=true,name-canonical=true,push=true
-          build-args: |
-            COMMIT_SHA=${{ github.sha }}
-            COMMIT_DATE=${{ steps.build-metadata.outputs.date }}
-            GIT_TAG=${{ github.ref_name }}
-            EXTRA_ARGS=${{ matrix.feature-flag }}
-
-      - name: Export digest
-        run: |
-          mkdir -p ${{ runner.temp }}/digests
-          digest="${{ steps.build-and-push.outputs.digest }}"
-          touch "${{ runner.temp }}/digests/${digest#sha256:}"
-
-      - name: Upload digest
-        uses: actions/upload-artifact@v4
-        with:
-          name: digests-${{ matrix.edition }}-${{ env.PLATFORM_PAIR }}
-          path: ${{ runner.temp }}/digests/*
-          if-no-files-found: error
-          retention-days: 1
-
-  merge:
-    runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        edition: [community, enterprise]
-        include:
-          - edition: community
-            registry: getmeili/meilisearch
-          - edition: enterprise
-            registry: getmeili/meilisearch-enterprise
-    needs:
-      - build
-
     permissions:
       id-token: write # This is needed to use Cosign in keyless mode
-
     steps:
       - uses: actions/checkout@v5
 
@@ -153,30 +58,26 @@ jobs:
 
           echo "date=$commit_date" >> $GITHUB_OUTPUT
 
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
       - name: Install cosign
         uses: sigstore/cosign-installer@d7543c93d881b35a8faa02e8e3605f69b7a1ce62 # tag=v3.10.0
 
-      - name: Download digests
-        uses: actions/download-artifact@v4
-        with:
-          path: ${{ runner.temp }}/digests
-          pattern: digests-${{ matrix.edition }}-*
-          merge-multiple: true
-
       - name: Login to Docker Hub
         uses: docker/login-action@v3
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
 
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-
       - name: Docker meta
         id: meta
         uses: docker/metadata-action@v5
         with:
-          images: ${{ matrix.registry }}
+          images: getmeili/meilisearch
           # Prevent `latest` to be updated for each new tag pushed.
           # We need latest and `vX.Y` tags to only be pushed for the stable Meilisearch releases.
           flavor: latest=false
@@ -187,31 +88,33 @@ jobs:
             type=semver,pattern=v{{major}},enable=${{ steps.check-tag-format.outputs.stable == 'true' }}
             type=raw,value=latest,enable=${{ steps.check-tag-format.outputs.stable == 'true' && steps.check-tag-format.outputs.latest == 'true' }}
 
-      - name: Create manifest list and push
+      - name: Build and push
-        working-directory: ${{ runner.temp }}/digests
+        uses: docker/build-push-action@v6
-        run: |
+        id: build-and-push
-          docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
+        with:
-            $(printf '${{ matrix.registry }}@sha256:%s ' *)
+          push: true
-
+          platforms: linux/amd64,linux/arm64
-      - name: Inspect image to fetch digest to sign
+          tags: ${{ steps.meta.outputs.tags }}
-        run: |
+          build-args: |
-          digest=$(docker buildx imagetools inspect --format='{{ json .Manifest }}' ${{ matrix.registry }}:${{ steps.meta.outputs.version }} | jq -r '.digest')
+            COMMIT_SHA=${{ github.sha }}
-          echo "DIGEST=${digest}" >> $GITHUB_ENV
+            COMMIT_DATE=${{ steps.build-metadata.outputs.date }}
+            GIT_TAG=${{ github.ref_name }}
 
       - name: Sign the images with GitHub OIDC Token
         env:
+          DIGEST: ${{ steps.build-and-push.outputs.digest }}
           TAGS: ${{ steps.meta.outputs.tags }}
         run: |
           images=""
           for tag in ${TAGS}; do
-            images+="${tag}@${{ env.DIGEST }} "
+            images+="${tag}@${DIGEST} "
           done
           cosign sign --yes ${images}
 
-      # /!\ Don't touch this without checking with engineers working on the Cloud code base on #discussion-engineering Slack channel
+      # /!\ Don't touch this without checking with Cloud team
-      - name: Notify meilisearch-cloud 
+      - name: Send CI information to Cloud team
         # Do not send if nightly build (i.e. 'schedule' or 'workflow_dispatch' event)
-        if: ${{ (github.event_name == 'push') && (matrix.edition == 'enterprise') }}
+        if: github.event_name == 'push'
         uses: peter-evans/repository-dispatch@v3
         with:
           token: ${{ secrets.MEILI_BOT_GH_PAT }}
@@ -219,13 +122,21 @@ jobs:
           event-type: cloud-docker-build
           client-payload: '{ "meilisearch_version": "${{ github.ref_name }}", "stable": "${{ steps.check-tag-format.outputs.stable }}" }'
 
-      # /!\ Don't touch this without checking with integration team members on #discussion-integrations Slack channel 
+      # Send notification to Swarmia to notify of a deployment: https://app.swarmia.com
-      - name: Notify meilisearch-kubernetes
+      # - name: 'Setup jq'
-        # Do not send if nightly build (i.e. 'schedule' or 'workflow_dispatch' event), or if not stable
+      #   uses: dcarbone/install-jq-action
-        if: ${{ github.event_name == 'push' && matrix.edition == 'community' && steps.check-tag-format.outputs.stable == 'true' }}
+      # - name: Send deployment to Swarmia
-        uses: peter-evans/repository-dispatch@v3
+      #   if: github.event_name == 'push' && success()
-        with:
+      #   run: |
-          token: ${{ secrets.MEILI_BOT_GH_PAT }}
+      #     JSON_STRING=$( jq --null-input --compact-output \
-          repository: meilisearch/meilisearch-kubernetes
+      #     --arg version "${{ github.ref_name }}" \
-          event-type: meilisearch-release
+      #     --arg appName "meilisearch" \
-          client-payload: '{ "version": "${{ github.ref_name }}" }'
+      #     --arg environment "production" \
+      #     --arg commitSha "${{ github.sha }}" \
+      #     --arg repositoryFullName "${{ github.repository }}" \
+      #     '{"version": $version, "appName": $appName, "environment": $environment, "commitSha": $commitSha, "repositoryFullName": $repositoryFullName}' )
+
+      #     curl -H "Authorization: ${{ secrets.SWARMIA_DEPLOYMENTS_AUTHORIZATION }}" \
+      #       -H "Content-Type: application/json" \
+      #       -d "$JSON_STRING" \
+      #       https://hook.swarmia.com/deployments

.github/workflows/publish-release-assets.yml

@@ -32,61 +32,157 @@ jobs:
         if: github.event_name == 'release' && steps.check-tag-format.outputs.stable == 'true'
         run: bash .github/scripts/check-release.sh
 
-  publish-binaries:
+  publish-linux:
-    name: Publish binary for ${{ matrix.release }} ${{ matrix.edition }} edition
+    name: Publish binary for Linux
-    runs-on: ${{ matrix.os }}
+    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        edition: [community, enterprise]
-        release:
-          [macos-amd64, macos-aarch64, windows, linux-amd64, linux-aarch64]
-        include:
-          - edition: "community"
-            feature-flag: ""
-            edition-suffix: ""
-          - edition: "enterprise"
-            feature-flag: "--features enterprise"
-            edition-suffix: "enterprise-"
-          - release: macos-amd64
-            os: macos-15-intel
-            binary_path: release/meilisearch
-            asset_name: macos-amd64
-            extra-args: ""
-          - release: macos-aarch64
-            os: macos-14
-            binary_path: aarch64-apple-darwin/release/meilisearch
-            asset_name: macos-apple-silicon
-            extra-args: "--target aarch64-apple-darwin"
-          - release: windows
-            os: windows-2022
-            binary_path: release/meilisearch.exe
-            asset_name: windows-amd64.exe
-            extra-args: ""
-          - release: linux-amd64
-            os: ubuntu-22.04
-            binary_path: x86_64-unknown-linux-gnu/release/meilisearch
-            asset_name: linux-amd64
-            extra-args: "--target x86_64-unknown-linux-gnu"
-          - release: linux-aarch64
-            os: ubuntu-22.04-arm
-            binary_path: aarch64-unknown-linux-gnu/release/meilisearch
-            asset_name: linux-aarch64
-            extra-args: "--target aarch64-unknown-linux-gnu"
     needs: check-version
+    container:
+      # Use ubuntu-22.04 to compile with glibc 2.35
+      image: ubuntu:22.04
     steps:
       - uses: actions/checkout@v5
+      - name: Install needed dependencies
+        run: |
+          apt-get update && apt-get install -y curl
+          apt-get install build-essential -y
       - uses: dtolnay/rust-toolchain@1.89
       - name: Build
-        run: cargo build --release --locked ${{ matrix.feature-flag }} ${{ matrix.extra-args }}
+        run: cargo build --release --locked
       # No need to upload binaries for dry run (cron or workflow_dispatch)
       - name: Upload binaries to release
         if: github.event_name == 'release'
         uses: svenstaro/upload-release-action@2.11.2
         with:
           repo_token: ${{ secrets.MEILI_BOT_GH_PAT }}
-          file: target/${{ matrix.binary_path }}
+          file: target/release/meilisearch
-          asset_name: meilisearch-${{ matrix.edition-suffix }}${{ matrix.asset_name }}
+          asset_name: meilisearch-linux-amd64
+          tag: ${{ github.ref }}
+
+  publish-macos-windows:
+    name: Publish binary for ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    needs: check-version
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [macos-13, windows-2022]
+        include:
+          - os: macos-13
+            artifact_name: meilisearch
+            asset_name: meilisearch-macos-amd64
+          - os: windows-2022
+            artifact_name: meilisearch.exe
+            asset_name: meilisearch-windows-amd64.exe
+    steps:
+      - uses: actions/checkout@v5
+      - uses: dtolnay/rust-toolchain@1.89
+      - name: Build
+        run: cargo build --release --locked
+      # No need to upload binaries for dry run (cron or workflow_dispatch)
+      - name: Upload binaries to release
+        if: github.event_name == 'release'
+        uses: svenstaro/upload-release-action@2.11.2
+        with:
+          repo_token: ${{ secrets.MEILI_BOT_GH_PAT }}
+          file: target/release/${{ matrix.artifact_name }}
+          asset_name: ${{ matrix.asset_name }}
+          tag: ${{ github.ref }}
+
+  publish-macos-apple-silicon:
+    name: Publish binary for macOS silicon
+    runs-on: macos-13
+    needs: check-version
+    strategy:
+      matrix:
+        include:
+          - target: aarch64-apple-darwin
+            asset_name: meilisearch-macos-apple-silicon
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v5
+      - name: Installing Rust toolchain
+        uses: dtolnay/rust-toolchain@1.89
+        with:
+          profile: minimal
+          target: ${{ matrix.target }}
+      - name: Cargo build
+        uses: actions-rs/cargo@v1
+        with:
+          command: build
+          args: --release --target ${{ matrix.target }}
+      - name: Upload the binary to release
+        # No need to upload binaries for dry run (cron or workflow_dispatch)
+        if: github.event_name == 'release'
+        uses: svenstaro/upload-release-action@2.11.2
+        with:
+          repo_token: ${{ secrets.MEILI_BOT_GH_PAT }}
+          file: target/${{ matrix.target }}/release/meilisearch
+          asset_name: ${{ matrix.asset_name }}
+          tag: ${{ github.ref }}
+
+  publish-aarch64:
+    name: Publish binary for aarch64
+    runs-on: ubuntu-latest
+    needs: check-version
+    env:
+      DEBIAN_FRONTEND: noninteractive
+    container:
+      # Use ubuntu-22.04 to compile with glibc 2.35
+      image: ubuntu:22.04
+    strategy:
+      matrix:
+        include:
+          - target: aarch64-unknown-linux-gnu
+            asset_name: meilisearch-linux-aarch64
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v5
+      - name: Install needed dependencies
+        run: |
+          apt-get update -y && apt upgrade -y
+          apt-get install -y curl build-essential gcc-aarch64-linux-gnu
+      - name: Set up Docker for cross compilation
+        run: |
+          apt-get install -y curl apt-transport-https ca-certificates software-properties-common
+          curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
+          add-apt-repository "deb [arch=$(dpkg --print-architecture)] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
+          apt-get update -y && apt-get install -y docker-ce
+      - name: Installing Rust toolchain
+        uses: dtolnay/rust-toolchain@1.89
+        with:
+          profile: minimal
+          target: ${{ matrix.target }}
+      - name: Configure target aarch64 GNU
+        ## Environment variable is not passed using env:
+        ## LD gold won't work with MUSL
+        # env:
+        #   JEMALLOC_SYS_WITH_LG_PAGE: 16
+        #   RUSTFLAGS: '-Clink-arg=-fuse-ld=gold'
+        run: |
+          echo '[target.aarch64-unknown-linux-gnu]' >> ~/.cargo/config
+          echo 'linker = "aarch64-linux-gnu-gcc"' >> ~/.cargo/config
+          echo 'JEMALLOC_SYS_WITH_LG_PAGE=16' >> $GITHUB_ENV
+      - name: Install a default toolchain that will be used to build cargo cross
+        run: |
+          rustup default stable
+      - name: Cargo build
+        uses: actions-rs/cargo@v1
+        with:
+          command: build
+          use-cross: true
+          args: --release --target ${{ matrix.target }}
+        env:
+          CROSS_DOCKER_IN_DOCKER: true
+      - name: List target output files
+        run: ls -lR ./target
+      - name: Upload the binary to release
+        # No need to upload binaries for dry run (cron or workflow_dispatch)
+        if: github.event_name == 'release'
+        uses: svenstaro/upload-release-action@2.11.2
+        with:
+          repo_token: ${{ secrets.MEILI_BOT_GH_PAT }}
+          file: target/${{ matrix.target }}/release/meilisearch
+          asset_name: ${{ matrix.asset_name }}
           tag: ${{ github.ref }}
 
   publish-openapi-file:

.github/workflows/sdks-tests.yml

@@ -68,7 +68,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -92,7 +92,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -122,7 +122,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -149,7 +149,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -184,7 +184,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -213,7 +213,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -238,7 +238,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -263,7 +263,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -284,7 +284,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -307,7 +307,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -338,7 +338,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}
@@ -370,7 +370,7 @@ jobs:
     runs-on: ubuntu-latest
     services:
       meilisearch:
-        image: getmeili/meilisearch-enterprise:${{ needs.define-docker-image.outputs.docker-image }}
+        image: getmeili/meilisearch:${{ needs.define-docker-image.outputs.docker-image }}
         env:
           MEILI_MASTER_KEY: ${{ env.MEILI_MASTER_KEY }}
           MEILI_NO_ANALYTICS: ${{ env.MEILI_NO_ANALYTICS }}

.github/workflows/test-suite.yml

@@ -15,40 +15,31 @@ env:
 
 jobs:
   test-linux:
-    name: Tests on Ubuntu
+    name: Tests on ubuntu-22.04
-    runs-on: ${{ matrix.runner }}
+    runs-on: ubuntu-latest
-    strategy:
+    container:
-      matrix:
+      # Use ubuntu-22.04 to compile with glibc 2.35
-        runner: [ubuntu-22.04, ubuntu-22.04-arm]
+      image: ubuntu:22.04
-        features: ["", "--features enterprise"]
     steps:
       - uses: actions/checkout@v5
-      - name: check free space before
+      - name: Install needed dependencies
-        run: df -h
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
         run: |
-          sudo rm -rf "/opt/ghc" || true
+          apt-get update && apt-get install -y curl
-          sudo rm -rf "/usr/share/dotnet" || true
+          apt-get install build-essential -y
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
-      - name: check free space after
-        run: df -h
       - name: Setup test with Rust stable
         uses: dtolnay/rust-toolchain@1.89
       - name: Cache dependencies
         uses: Swatinem/rust-cache@v2.8.0
-        with:
+      - name: Run cargo check without any default features
-          key: ${{ matrix.features }}
-      - name: Run cargo build without any default features
         uses: actions-rs/cargo@v1
         with:
           command: build
-          args: --locked --no-default-features --all
+          args: --locked --release --no-default-features --all
       - name: Run cargo test
         uses: actions-rs/cargo@v1
         with:
           command: test
-          args: --locked --all ${{ matrix.features }}
+          args: --locked --release --all
 
   test-others:
     name: Tests on ${{ matrix.os }}
@@ -56,58 +47,51 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [macos-14, windows-2022]
+        os: [macos-13, windows-2022]
-        features: ["", "--features enterprise"]
-    if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
     steps:
       - uses: actions/checkout@v5
       - name: Cache dependencies
         uses: Swatinem/rust-cache@v2.8.0
       - uses: dtolnay/rust-toolchain@1.89
-      - name: Run cargo build without any default features
+      - name: Run cargo check without any default features
         uses: actions-rs/cargo@v1
         with:
           command: build
-          args: --locked --no-default-features --all
+          args: --locked --release --no-default-features --all
       - name: Run cargo test
         uses: actions-rs/cargo@v1
         with:
           command: test
-          args: --locked --all ${{ matrix.features }}
+          args: --locked --release --all
 
   test-all-features:
     name: Tests almost all features
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-latest
+    container:
+      # Use ubuntu-22.04 to compile with glibc 2.35
+      image: ubuntu:22.04
     if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
     steps:
       - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
+      - name: Install needed dependencies
         run: |
-          sudo rm -rf "/opt/ghc" || true
+          apt-get update
-          sudo rm -rf "/usr/share/dotnet" || true
+          apt-get install --assume-yes build-essential curl
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - uses: dtolnay/rust-toolchain@1.89
       - name: Run cargo build with almost all features
         run: |
-          cargo build --workspace --locked --features "$(cargo xtask list-features --exclude-feature cuda,test-ollama)"
+          cargo build --workspace --locked --release --features "$(cargo xtask list-features --exclude-feature cuda,test-ollama)"
       - name: Run cargo test with almost all features
         run: |
-          cargo test --workspace --locked --features "$(cargo xtask list-features --exclude-feature cuda,test-ollama)"
+          cargo test --workspace --locked --release --features "$(cargo xtask list-features --exclude-feature cuda,test-ollama)"
 
   ollama-ubuntu:
     name: Test with Ollama
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-latest
     env:
       MEILI_TEST_OLLAMA_SERVER: "http://localhost:11434"
     steps:
       - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
-        run: |
-          sudo rm -rf "/opt/ghc" || true
-          sudo rm -rf "/usr/share/dotnet" || true
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - name: Install Ollama
         run: |
           curl -fsSL https://ollama.com/install.sh | sudo -E sh
@@ -131,20 +115,20 @@ jobs:
         uses: actions-rs/cargo@v1
         with:
           command: test
-          args: --locked -p meilisearch --features test-ollama ollama
+          args: --locked --release --all --features test-ollama ollama
 
   test-disabled-tokenization:
     name: Test disabled tokenization
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-latest
+    container:
+      image: ubuntu:22.04
     if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
     steps:
       - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
+      - name: Install needed dependencies
         run: |
-          sudo rm -rf "/opt/ghc" || true
+          apt-get update
-          sudo rm -rf "/usr/share/dotnet" || true
+          apt-get install --assume-yes build-essential curl
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - uses: dtolnay/rust-toolchain@1.89
       - name: Run cargo tree without default features and check lindera is not present
         run: |
@@ -156,39 +140,36 @@ jobs:
         run: |
           cargo tree -f '{p} {f}' -e normal | grep lindera -qz
 
-  build:
+  # We run tests in debug also, to make sure that the debug_assertions are hit
-    name: Build in release
+  test-debug:
-    runs-on: ubuntu-22.04
+    name: Run tests in debug
+    runs-on: ubuntu-latest
+    container:
+      # Use ubuntu-22.04 to compile with glibc 2.35
+      image: ubuntu:22.04
     steps:
       - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
+      - name: Install needed dependencies
         run: |
-          sudo rm -rf "/opt/ghc" || true
+          apt-get update && apt-get install -y curl
-          sudo rm -rf "/usr/share/dotnet" || true
+          apt-get install build-essential -y
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - uses: dtolnay/rust-toolchain@1.89
       - name: Cache dependencies
         uses: Swatinem/rust-cache@v2.8.0
-      - name: Build
+      - name: Run tests in debug
-        run: cargo build --release --locked --target x86_64-unknown-linux-gnu
+        uses: actions-rs/cargo@v1
+        with:
+          command: test
+          args: --locked --all
 
   clippy:
     name: Run Clippy
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        features: ["", "--features enterprise"]
     steps:
       - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
-        run: |
-          sudo rm -rf "/opt/ghc" || true
-          sudo rm -rf "/usr/share/dotnet" || true
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - uses: dtolnay/rust-toolchain@1.89
         with:
+          profile: minimal
           components: clippy
       - name: Cache dependencies
         uses: Swatinem/rust-cache@v2.8.0
@@ -196,21 +177,18 @@ jobs:
         uses: actions-rs/cargo@v1
         with:
           command: clippy
-          args: --all-targets ${{ matrix.features }} -- --deny warnings
+          args: --all-targets -- --deny warnings
 
   fmt:
     name: Run Rustfmt
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
-        run: |
-          sudo rm -rf "/opt/ghc" || true
-          sudo rm -rf "/usr/share/dotnet" || true
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - uses: dtolnay/rust-toolchain@1.89
         with:
+          profile: minimal
+          toolchain: nightly-2024-07-09
+          override: true
           components: rustfmt
       - name: Cache dependencies
         uses: Swatinem/rust-cache@v2.8.0
@@ -221,23 +199,3 @@ jobs:
         run: |
           echo -ne "\n" > crates/benchmarks/benches/datasets_paths.rs
           cargo fmt --all -- --check
-
-  declarative-tests:
-    name: Run declarative tests
-    runs-on: ubuntu-22.04-arm
-    permissions:
-      contents: read
-    steps:
-      - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
-        run: |
-          sudo rm -rf "/opt/ghc" || true
-          sudo rm -rf "/usr/share/dotnet" || true
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
-      - uses: dtolnay/rust-toolchain@1.89
-      - name: Cache dependencies
-        uses: Swatinem/rust-cache@v2.8.0
-      - name: Run declarative tests
-        run: |
-          cargo xtask test workloads/tests/*.json

.github/workflows/update-cargo-toml-version.yml

@@ -18,13 +18,9 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v5
-      - name: Clean space as per https://github.com/actions/virtual-environments/issues/709
-        run: |
-          sudo rm -rf "/opt/ghc" || true
-          sudo rm -rf "/usr/share/dotnet" || true
-          sudo rm -rf "/usr/local/lib/android" || true
-          sudo rm -rf "/usr/local/share/boost" || true
       - uses: dtolnay/rust-toolchain@1.89
+        with:
+          profile: minimal
       - name: Install sd
         run: cargo install sd
       - name: Update Cargo.toml file

BENCHMARKS.md

@@ -124,7 +124,6 @@ They are JSON files with the following structure (comments are not actually supp
 {
   // Name of the workload. Must be unique to the workload, as it will be used to group results on the dashboard.
   "name": "hackernews.ndjson_1M,no-threads",
-  "type": "bench",
   // Number of consecutive runs of the commands that should be performed.
   // Each run uses a fresh instance of Meilisearch and a fresh database.
   // Each run produces its own report file.

Cargo.toml

@@ -23,7 +23,7 @@ members = [
 ]
 
 [workspace.package]
-version = "1.29.0"
+version = "1.24.0"
 authors = [
     "Quentin de Quelen <quentin@dequelen.me>",
     "Clément Renault <clement@meilisearch.com>",
@@ -50,5 +50,3 @@ opt-level = 3
 opt-level = 3
 [profile.dev.package.roaring]
 opt-level = 3
-[profile.dev.package.gemm-f16]
-opt-level = 3

Cross.toml

@@ -0,0 +1,7 @@
+[build.env]
+passthrough = [
+    "RUST_BACKTRACE",
+    "CARGO_TERM_COLOR",
+    "RUSTFLAGS",
+    "JEMALLOC_SYS_WITH_LG_PAGE"
+]

Dockerfile

@@ -8,14 +8,16 @@ WORKDIR /
 ARG     COMMIT_SHA
 ARG     COMMIT_DATE
 ARG     GIT_TAG
-ARG     EXTRA_ARGS
 ENV     VERGEN_GIT_SHA=${COMMIT_SHA} VERGEN_GIT_COMMIT_TIMESTAMP=${COMMIT_DATE} VERGEN_GIT_DESCRIBE=${GIT_TAG}
 ENV     RUSTFLAGS="-C target-feature=-crt-static"
 
 COPY    . .
 RUN     set -eux; \
         apkArch="$(apk --print-arch)"; \
-        cargo build --release -p meilisearch -p meilitool ${EXTRA_ARGS}
+        if [ "$apkArch" = "aarch64" ]; then \
+            export JEMALLOC_SYS_WITH_LG_PAGE=16; \
+        fi && \
+        cargo build --release -p meilisearch -p meilitool
 
 # Run
 FROM    alpine:3.22

README.md

@@ -39,7 +39,6 @@
 ## 🖥 Examples
 
 - [**Movies**](https://where2watch.meilisearch.com/?utm_campaign=oss&utm_source=github&utm_medium=organization) — An application to help you find streaming platforms to watch movies using [hybrid search](https://www.meilisearch.com/solutions/hybrid-search?utm_campaign=oss&utm_source=github&utm_medium=meilisearch&utm_content=demos).
-- [**Flickr**](https://flickr.meilisearch.com/?utm_campaign=oss&utm_source=github&utm_medium=organization) — Search and explore one hundred million Flickr images with semantic search.
 - [**Ecommerce**](https://ecommerce.meilisearch.com/?utm_campaign=oss&utm_source=github&utm_medium=meilisearch&utm_content=demos) — Ecommerce website using disjunctive [facets](https://www.meilisearch.com/docs/learn/fine_tuning_results/faceted_search?utm_campaign=oss&utm_source=github&utm_medium=meilisearch&utm_content=demos), range and rating filtering, and pagination.
 - [**Songs**](https://music.meilisearch.com/?utm_campaign=oss&utm_source=github&utm_medium=meilisearch&utm_content=demos) — Search through 47 million of songs.
 - [**SaaS**](https://saas.meilisearch.com/?utm_campaign=oss&utm_source=github&utm_medium=meilisearch&utm_content=demos) — Search for contacts, deals, and companies in this [multi-tenant](https://www.meilisearch.com/docs/learn/security/multitenancy_tenant_tokens?utm_campaign=oss&utm_source=github&utm_medium=meilisearch&utm_content=demos) CRM application.

TESTING.md

@@ -1,326 +0,0 @@
-# Declarative tests
-
-Declarative tests ensure that Meilisearch features remain stable across versions.
-
-While we already have unit tests, those are run against **temporary databases** that are created fresh each time and therefore never risk corruption.
-
-Declarative tests instead **simulate the lifetime of a database**: they chain together commands and requests to change the binary, verifying that database state and API responses remain consistent.
-
-## Basic example
-
-```jsonc
-{
-  "type": "test",
-  "name": "api-keys",
-  "binary": { // the first command will run on the binary following this specification.
-    "source": "release", // get the binary as a release from GitHub
-    "version": "1.19.0", // version to fetch
-    "edition": "community" // edition to fetch
-  },
-  "commands": []
-}
-```
-
-This example defines a no-op test (it does nothing).
-
-If the file is saved at `workloads/tests/example.json`, you can run it with:
-
-```bash
-cargo xtask test workloads/tests/example.json
-```
-
-## Commands
-
-Commands represent API requests sent to Meilisearch endpoints during a test.
-
-They are executed sequentially, and their responses can be validated to ensure consistent behavior across upgrades.
-
-```jsonc
-
-{
-  "route": "keys",
-  "method": "POST",
-  "body": {
-    "inline": {
-      "actions": [
-        "search",
-        "documents.add"
-      ],
-      "description": "Test API Key",
-      "expiresAt": null,
-      "indexes": [ "movies" ]
-    }
-  }
-}
-```
-
-This command issues a `POST /keys` request, creating an API key with permissions to search and add documents in the `movies` index.
-
-### Using assets in commands
-
-To keep tests concise and reusable, you can define **assets** at the root of the workload file.
-
-Assets are external data sources (such as datasets) that are cached between runs, making tests faster and easier to read.
-
-```jsonc
-{
-  "type": "test",
-  "name": "movies",
-  "binary": {
-    "source": "release",
-    "version": "1.19.0",
-    "edition": "community"
-  },
-  "assets": {
-    "movies.json": {
-      "local_location": null,
-      "remote_location": "https://milli-benchmarks.fra1.digitaloceanspaces.com/bench/datasets/movies.json",
-      "sha256": "5b6e4cb660bc20327776e8a33ea197b43d9ec84856710ead1cc87ab24df77de1"
-    }
-  },
-  "commands": [
-    {
-      "route": "indexes/movies/documents",
-      "method": "POST",
-      "body": {
-        "asset": "movies.json"
-      }
-    }
-  ]
-}
-```
-
-In this example:
-- The `movies.json` dataset is defined as an asset, pointing to a remote URL.
-- The SHA-256 checksum ensures integrity.
-- The `POST /indexes/movies/documents` command uses this asset as the request body.
-
-This makes the test much cleaner than inlining a large dataset directly into the command.
-
-For asset handling, please refer to the [declarative benchmarks documentation](/BENCHMARKS.md#adding-new-assets).
-
-### Asserting responses
-
-Commands can specify both the **expected status code** and the **expected response body**.
-
-```jsonc
-{
-  "route": "indexes/movies/documents",
-  "method": "POST",
-  "body": {
-    "asset": "movies.json"
-  },
-  "expectedStatus": 202,
-  "expectedResponse": {
-    "enqueuedAt": "[timestamp]", // Set to a bracketed string to ignore the value
-    "indexUid": "movies",
-    "status": "enqueued",
-    "taskUid": 1,
-    "type": "documentAdditionOrUpdate"
-  },
-  "synchronous": "WaitForTask"
-}
-```
-
-Manually writing `expectedResponse` fields can be tedious.
-
-Instead, you can let the test runner populate them automatically:
-
-```bash
-# Run the workload to populate expected fields. Only adds the missing ones, doesn't change existing data
-cargo xtask test workloads/tests/example.json --add-missing-responses
-
-# OR
-
-# Run the workload to populate expected fields. Updates all fields including existing ones
-cargo xtask test workloads/tests/example.json --update-responses
-```
-
-This workflow is recommended:
-
-1. Write the test without expected fields.
-2. Run it with `--add-missing-responses` to capture the actual responses.
-3. Review and commit the generated expectations.
-
-## Changing binary
-
-It is possible to insert an instruction to change the current Meilisearch instance from one binary specification to another during a test.
-
-When executed, such an instruction will:
-1. Stop the current Meilisearch instance.
-2. Fetch the binary specified by the instruction.
-3. Restart the server with the specified binary on the same database.
-
-```jsonc
-{
-  "type": "test",
-  "name": "movies",
-  "binary": {
-    "source": "release",
-    "version": "1.19.0", // start with version v1.19.0
-    "edition": "community"
-  },
-  "assets": {
-    "movies.json": {
-      "local_location": null,
-      "remote_location": "https://milli-benchmarks.fra1.digitaloceanspaces.com/bench/datasets/movies.json",
-      "sha256": "5b6e4cb660bc20327776e8a33ea197b43d9ec84856710ead1cc87ab24df77de1"
-    }
-  },
-  "commands": [
-    // setup some data
-    {
-      "route": "indexes/movies/documents",
-      "method": "POST",
-      "body": {
-        "asset": "movies.json"
-      }
-    },
-    // switch binary to v1.24.0
-    {
-      "binary": {
-        "source": "release",
-        "version": "1.24.0",
-        "edition": "community"
-      }
-    }
-  ]
-}
-```
-
-### Typical Usage
-
-In most cases, the change binary instruction will be used to update a database.
-
-- **Set up** some data using commands on an older version.
-- **Upgrade** to the latest version.
-- **Assert** that the data and API behavior remain correct after the upgrade.
-
-To properly test the dumpless upgrade, one should typically:
-
-1. Open the database without processing the update task: Use a `binary` instruction to switch to the desired version, passing `--experimental-dumpless-upgrade` and `--experimental-max-number-of-batched-tasks=0` as extra CLI arguments
-2. Check that the search, stats and task queue still work.
-3. Open the database and process the update task: Use a `binary` instruction to switch to the desired version, passing `--experimental-dumpless-upgrade` as the extra CLI argument. Use a `health` command to wait for the upgrade task to finish.
-4. Check that the indexing, search, stats, and task queue still work.
-
-```jsonc
-{
-  "type": "test",
-  "name": "movies",
-  "binary": {
-    "source": "release",
-    "version": "1.12.0",
-    "edition": "community"
-  },
-  "commands": [
-    // 0. Run commands to populate the database
-    {
-      // ..
-    },
-    // 1. Open the database with new MS without processing the update task
-    {
-      "binary": {
-        "source": "build", // build the binary from the sources in the current git repository
-        "edition": "community",
-        "extraCliArgs": [
-          "--experimental-dumpless-upgrade", // allows to open with a newer MS
-          "--experimental-max-number-of-batched-tasks=0" // prevent processing of the update task
-        ]
-      }
-    },
-    // 2. Check the search etc.
-    {
-      // ..
-    },
-    // 3. Open the database with new MS and processing the update task
-    {
-      "binary": {
-        "source": "build", // build the binary from the sources in the current git repository
-        "edition": "community",
-        "extraCliArgs": [
-          "--experimental-dumpless-upgrade" // allows to open with a newer MS
-          // no `--experimental-max-number-of-batched-tasks=0`
-        ]
-      }
-    },
-    // 4. Check the indexing, search, etc.
-    {
-      // ..
-    }
-  ]
-}
-```
-
-This ensures backward compatibility: databases created with older Meilisearch versions should remain functional and consistent after an upgrade.
-
-## Variables
-
-Sometimes a command needs to use a value returned by a **previous response**.
-These values can be captured and reused using the register field.
-
-```jsonc
-{
-  "route": "keys",
-  "method": "POST",
-  "body": {
-    "inline": {
-        "actions": [
-        "search",
-        "documents.add"
-        ],
-        "description": "Test API Key",
-        "expiresAt": null,
-        "indexes": [ "movies" ]
-    }
-  },
-  "expectedResponse": {
-      "key": "c6f64630bad2996b1f675007c8800168e14adf5d6a7bb1a400a6d2b158050eaf",
-      // ...
-  },
-  "register": {
-    "key": "/key"
-  },
-  "synchronous": "WaitForResponse"
-}
-```
-
-The `register` field captures the value at the JSON path `/key` from the response.
-Paths follow the **JavaScript Object Notation Pointer (RFC 6901)** format.
-Registered variables are available for all subsequent commands.
-
-Registered variables can be referenced by wrapping their name in double curly braces:
-
-In the route/path:
-
-```jsonc
-{
-  "route": "tasks/{{ task_id }}",
-  "method": "GET"
-}
-```
-
-In the request body:
-
-```jsonc
-{
-  "route": "indexes/movies/documents",
-  "method": "PATCH",
-  "body": {
-    "inline": {
-      "id": "{{ document_id }}",
-      "overview": "Shazam turns evil and the world is in danger.",
-    }
-  }
-}
-```
-
-Or they can be referenced by their name (**without curly braces**) as an API key:
-
-```jsonc
-{
-  "route": "indexes/movies/documents",
-  "method": "POST",
-  "body": { /* ... */ },
-  "apiKeyVariable": "key" // The **content** of the key variable will be used as an API key
-}
-```

crates/benchmarks/Cargo.toml

@@ -11,27 +11,27 @@ edition.workspace = true
 license.workspace = true
 
 [dependencies]
-anyhow = "1.0.100"
+anyhow = "1.0.98"
-bumpalo = "3.19.0"
+bumpalo = "3.18.1"
-csv = "1.4.0"
+csv = "1.3.1"
-memmap2 = "0.9.9"
+memmap2 = "0.9.7"
 milli = { path = "../milli" }
-mimalloc = { version = "0.1.48", default-features = false }
+mimalloc = { version = "0.1.47", default-features = false }
-serde_json = { version = "1.0.145", features = ["preserve_order"] }
+serde_json = { version = "1.0.140", features = ["preserve_order"] }
-tempfile = "3.23.0"
+tempfile = "3.20.0"
 
 [dev-dependencies]
-criterion = { version = "0.7.0", features = ["html_reports"] }
+criterion = { version = "0.6.0", features = ["html_reports"] }
 rand = "0.8.5"
 rand_chacha = "0.3.1"
 roaring = "0.10.12"
 
 [build-dependencies]
-anyhow = "1.0.100"
+anyhow = "1.0.98"
-bytes = "1.11.0"
+bytes = "1.10.1"
-convert_case = "0.9.0"
+convert_case = "0.8.0"
-flate2 = "1.1.5"
+flate2 = "1.1.2"
-reqwest = { version = "0.12.24", features = ["blocking", "rustls-tls"], default-features = false }
+reqwest = { version = "0.12.20", features = ["blocking", "rustls-tls"], default-features = false }
 
 [features]
 default = ["milli/all-tokenizations"]

crates/benchmarks/benches/indexing.rs

@@ -21,10 +21,6 @@ use roaring::RoaringBitmap;
 #[global_allocator]
 static ALLOC: mimalloc::MiMalloc = mimalloc::MiMalloc;
 
-fn no_cancel() -> bool {
-    false
-}
-
 const BENCHMARK_ITERATION: usize = 10;
 
 fn setup_dir(path: impl AsRef<Path>) {
@@ -69,7 +65,7 @@ fn setup_settings<'t>(
     let sortable_fields = sortable_fields.iter().map(|s| s.to_string()).collect();
     builder.set_sortable_fields(sortable_fields);
 
-    builder.execute(&no_cancel, &Progress::default(), Default::default()).unwrap();
+    builder.execute(&|| false, &Progress::default(), Default::default()).unwrap();
 }
 
 fn setup_index_with_settings(
@@ -156,7 +152,7 @@ fn indexing_songs_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -172,7 +168,7 @@ fn indexing_songs_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -224,7 +220,7 @@ fn reindexing_songs_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -240,7 +236,7 @@ fn reindexing_songs_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -270,7 +266,7 @@ fn reindexing_songs_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -286,7 +282,7 @@ fn reindexing_songs_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -340,7 +336,7 @@ fn deleting_songs_in_batches_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -356,7 +352,7 @@ fn deleting_songs_in_batches_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -418,7 +414,7 @@ fn indexing_songs_in_three_batches_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -434,7 +430,7 @@ fn indexing_songs_in_three_batches_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -464,7 +460,7 @@ fn indexing_songs_in_three_batches_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -480,7 +476,7 @@ fn indexing_songs_in_three_batches_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -506,7 +502,7 @@ fn indexing_songs_in_three_batches_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -522,7 +518,7 @@ fn indexing_songs_in_three_batches_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -575,7 +571,7 @@ fn indexing_songs_without_faceted_numbers(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -591,7 +587,7 @@ fn indexing_songs_without_faceted_numbers(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -643,7 +639,7 @@ fn indexing_songs_without_faceted_fields(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -659,7 +655,7 @@ fn indexing_songs_without_faceted_fields(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -711,7 +707,7 @@ fn indexing_wiki(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -727,7 +723,7 @@ fn indexing_wiki(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -778,7 +774,7 @@ fn reindexing_wiki(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -794,7 +790,7 @@ fn reindexing_wiki(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -824,7 +820,7 @@ fn reindexing_wiki(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -840,7 +836,7 @@ fn reindexing_wiki(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -893,7 +889,7 @@ fn deleting_wiki_in_batches_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -909,7 +905,7 @@ fn deleting_wiki_in_batches_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -971,7 +967,7 @@ fn indexing_wiki_in_three_batches(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -987,7 +983,7 @@ fn indexing_wiki_in_three_batches(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1018,7 +1014,7 @@ fn indexing_wiki_in_three_batches(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1034,7 +1030,7 @@ fn indexing_wiki_in_three_batches(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1061,7 +1057,7 @@ fn indexing_wiki_in_three_batches(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1077,7 +1073,7 @@ fn indexing_wiki_in_three_batches(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1129,7 +1125,7 @@ fn indexing_movies_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1145,7 +1141,7 @@ fn indexing_movies_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1196,7 +1192,7 @@ fn reindexing_movies_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1212,7 +1208,7 @@ fn reindexing_movies_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1242,7 +1238,7 @@ fn reindexing_movies_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1258,7 +1254,7 @@ fn reindexing_movies_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1311,7 +1307,7 @@ fn deleting_movies_in_batches_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1327,7 +1323,7 @@ fn deleting_movies_in_batches_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1376,7 +1372,7 @@ fn delete_documents_from_ids(index: Index, document_ids_to_delete: Vec<RoaringBi
             Some(primary_key),
             &document_changes,
             RuntimeEmbedders::default(),
-            &no_cancel,
+            &|| false,
             &Progress::default(),
             &Default::default(),
         )
@@ -1426,7 +1422,7 @@ fn indexing_movies_in_three_batches(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1442,7 +1438,7 @@ fn indexing_movies_in_three_batches(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1472,7 +1468,7 @@ fn indexing_movies_in_three_batches(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1488,7 +1484,7 @@ fn indexing_movies_in_three_batches(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1514,7 +1510,7 @@ fn indexing_movies_in_three_batches(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1530,7 +1526,7 @@ fn indexing_movies_in_three_batches(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1605,7 +1601,7 @@ fn indexing_nested_movies_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1621,7 +1617,7 @@ fn indexing_nested_movies_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1697,7 +1693,7 @@ fn deleting_nested_movies_in_batches_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1713,7 +1709,7 @@ fn deleting_nested_movies_in_batches_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1781,7 +1777,7 @@ fn indexing_nested_movies_without_faceted_fields(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1797,7 +1793,7 @@ fn indexing_nested_movies_without_faceted_fields(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1849,7 +1845,7 @@ fn indexing_geo(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1865,7 +1861,7 @@ fn indexing_geo(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1916,7 +1912,7 @@ fn reindexing_geo(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1932,7 +1928,7 @@ fn reindexing_geo(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -1962,7 +1958,7 @@ fn reindexing_geo(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -1978,7 +1974,7 @@ fn reindexing_geo(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )
@@ -2031,7 +2027,7 @@ fn deleting_geo_in_batches_default(c: &mut Criterion) {
                         &rtxn,
                         None,
                         &mut new_fields_ids_map,
-                        &no_cancel,
+                        &|| false,
                         Progress::default(),
                         None,
                     )
@@ -2047,7 +2043,7 @@ fn deleting_geo_in_batches_default(c: &mut Criterion) {
                     primary_key,
                     &document_changes,
                     RuntimeEmbedders::default(),
-                    &no_cancel,
+                    &|| false,
                     &Progress::default(),
                     &Default::default(),
                 )

crates/build-info/Cargo.toml

@@ -11,8 +11,8 @@ license.workspace = true
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
 [dependencies]
-time = { version = "0.3.44", features = ["parsing"] }
+time = { version = "0.3.41", features = ["parsing"] }
 
 [build-dependencies]
-anyhow = "1.0.100"
+anyhow = "1.0.98"
-vergen-gitcl = "1.0.8"
+vergen-git2 = "1.0.7"

crates/build-info/build.rs

@@ -15,7 +15,7 @@ fn emit_git_variables() -> anyhow::Result<()> {
     // Note: any code that needs VERGEN_ environment variables should take care to define them manually in the Dockerfile and pass them
     // in the corresponding GitHub workflow (publish_docker.yml).
     // This is due to the Dockerfile building the binary outside of the git directory.
-    let mut builder = vergen_gitcl::GitclBuilder::default();
+    let mut builder = vergen_git2::Git2Builder::default();
 
     builder.branch(true);
     builder.commit_timestamp(true);
@@ -25,5 +25,5 @@ fn emit_git_variables() -> anyhow::Result<()> {
 
     let git2 = builder.build()?;
 
-    vergen_gitcl::Emitter::default().fail_on_error().add_instructions(&git2)?.emit()
+    vergen_git2::Emitter::default().fail_on_error().add_instructions(&git2)?.emit()
 }

crates/build-info/src/main.rs

@@ -1,6 +0,0 @@
-use build_info::BuildInfo;
-
-fn main() {
-    let info = BuildInfo::from_build();
-    dbg!(info);
-}

crates/dump/Cargo.toml

@@ -11,27 +11,24 @@ readme.workspace = true
 license.workspace = true
 
 [dependencies]
-anyhow = "1.0.100"
+anyhow = "1.0.98"
-flate2 = "1.1.5"
+flate2 = "1.1.2"
 http = "1.3.1"
 meilisearch-types = { path = "../meilisearch-types" }
 once_cell = "1.21.3"
-regex = "1.12.2"
+regex = "1.11.1"
 roaring = { version = "0.10.12", features = ["serde"] }
-serde = { version = "1.0.228", features = ["derive"] }
+serde = { version = "1.0.219", features = ["derive"] }
-serde_json = { version = "1.0.145", features = ["preserve_order"] }
+serde_json = { version = "1.0.140", features = ["preserve_order"] }
 tar = "0.4.44"
-tempfile = "3.23.0"
+tempfile = "3.20.0"
-thiserror = "2.0.17"
+thiserror = "2.0.12"
-time = { version = "0.3.44", features = ["serde-well-known", "formatting", "parsing", "macros"] }
+time = { version = "0.3.41", features = ["serde-well-known", "formatting", "parsing", "macros"] }
 tracing = "0.1.41"
-uuid = { version = "1.18.1", features = ["serde", "v4"] }
+uuid = { version = "1.17.0", features = ["serde", "v4"] }
 
 [dev-dependencies]
 big_s = "1.0.2"
 maplit = "1.0.2"
 meili-snap = { path = "../meili-snap" }
 meilisearch-types = { path = "../meilisearch-types" }
-
-[features]
-enterprise = ["meilisearch-types/enterprise"]

crates/dump/src/lib.rs

@@ -96,8 +96,6 @@ pub struct TaskDump {
     pub finished_at: Option<OffsetDateTime>,
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub network: Option<TaskNetwork>,
-    #[serde(default, skip_serializing_if = "Option::is_none")]
-    pub custom_metadata: Option<String>,
 }
 
 // A `Kind` specific version made for the dump. If modified you may break the dump.
@@ -180,7 +178,6 @@ impl From<Task> for TaskDump {
             started_at: task.started_at,
             finished_at: task.finished_at,
             network: task.network,
-            custom_metadata: task.custom_metadata,
         }
     }
 }
@@ -262,13 +259,13 @@ pub(crate) mod test {
     use big_s::S;
     use maplit::{btreemap, btreeset};
     use meilisearch_types::batches::{Batch, BatchEnqueuedAt, BatchStats};
+    use meilisearch_types::enterprise_edition::network::{Network, Remote};
     use meilisearch_types::facet_values_sort::FacetValuesSort;
     use meilisearch_types::features::RuntimeTogglableFeatures;
     use meilisearch_types::index_uid_pattern::IndexUidPattern;
     use meilisearch_types::keys::{Action, Key};
     use meilisearch_types::milli::update::Setting;
     use meilisearch_types::milli::{self, FilterableAttributesRule};
-    use meilisearch_types::network::{Network, Remote};
     use meilisearch_types::settings::{Checked, FacetingSettings, Settings};
     use meilisearch_types::task_view::DetailsView;
     use meilisearch_types::tasks::{BatchStopReason, Details, Kind, Status};
@@ -399,7 +396,6 @@ pub(crate) mod test {
                     started_at: Some(datetime!(2022-11-20 0:00 UTC)),
                     finished_at: Some(datetime!(2022-11-21 0:00 UTC)),
                     network: None,
-                    custom_metadata: None,
                 },
                 None,
             ),
@@ -425,7 +421,6 @@ pub(crate) mod test {
                     started_at: None,
                     finished_at: None,
                     network: None,
-                    custom_metadata: None,
                 },
                 Some(vec![
                     json!({ "id": 4, "race": "leonberg" }).as_object().unwrap().clone(),
@@ -446,7 +441,6 @@ pub(crate) mod test {
                     started_at: None,
                     finished_at: None,
                     network: None,
-                    custom_metadata: None,
                 },
                 None,
             ),

crates/dump/src/reader/compat/v5_to_v6.rs

@@ -164,7 +164,6 @@ impl CompatV5ToV6 {
                     started_at: task_view.started_at,
                     finished_at: task_view.finished_at,
                     network: None,
-                    custom_metadata: None,
                 };
 
                 (task, content_file)

crates/dump/src/reader/v6/mod.rs

@@ -24,7 +24,7 @@ pub type Batch = meilisearch_types::batches::Batch;
 pub type Key = meilisearch_types::keys::Key;
 pub type ChatCompletionSettings = meilisearch_types::features::ChatCompletionSettings;
 pub type RuntimeTogglableFeatures = meilisearch_types::features::RuntimeTogglableFeatures;
-pub type Network = meilisearch_types::network::Network;
+pub type Network = meilisearch_types::enterprise_edition::network::Network;
 pub type Webhooks = meilisearch_types::webhooks::WebhooksDumpView;
 
 // ===== Other types to clarify the code of the compat module

crates/dump/src/writer.rs

@@ -5,9 +5,9 @@ use std::path::PathBuf;
 use flate2::write::GzEncoder;
 use flate2::Compression;
 use meilisearch_types::batches::Batch;
+use meilisearch_types::enterprise_edition::network::Network;
 use meilisearch_types::features::{ChatCompletionSettings, RuntimeTogglableFeatures};
 use meilisearch_types::keys::Key;
-use meilisearch_types::network::Network;
 use meilisearch_types::settings::{Checked, Settings};
 use meilisearch_types::webhooks::WebhooksDumpView;
 use serde_json::{Map, Value};

crates/file-store/Cargo.toml

@@ -11,7 +11,7 @@ edition.workspace = true
 license.workspace = true
 
 [dependencies]
-tempfile = "3.23.0"
+tempfile = "3.20.0"
-thiserror = "2.0.17"
+thiserror = "2.0.12"
 tracing = "0.1.41"
-uuid = { version = "1.18.1", features = ["serde", "v4"] }
+uuid = { version = "1.17.0", features = ["serde", "v4"] }

crates/file-store/src/lib.rs

@@ -60,7 +60,7 @@ impl FileStore {
 
     /// Returns the file corresponding to the requested uuid.
     pub fn get_update(&self, uuid: Uuid) -> Result<StdFile> {
-        let path = self.update_path(uuid);
+        let path = self.get_update_path(uuid);
         let file = match StdFile::open(path) {
             Ok(file) => file,
             Err(e) => {
@@ -72,7 +72,7 @@ impl FileStore {
     }
 
     /// Returns the path that correspond to this uuid, the path could not exists.
-    pub fn update_path(&self, uuid: Uuid) -> PathBuf {
+    pub fn get_update_path(&self, uuid: Uuid) -> PathBuf {
         self.path.join(uuid.to_string())
     }
 

crates/flatten-serde-json/Cargo.toml

@@ -16,7 +16,7 @@ license.workspace = true
 serde_json = "1.0"
 
 [dev-dependencies]
-criterion = { version = "0.7.0", features = ["html_reports"] }
+criterion = { version = "0.6.0", features = ["html_reports"] }
 
 [[bench]]
 name = "benchmarks"

crates/fuzzers/Cargo.toml

@@ -11,12 +11,12 @@ edition.workspace = true
 license.workspace = true
 
 [dependencies]
-arbitrary = { version = "1.4.2", features = ["derive"] }
+arbitrary = { version = "1.4.1", features = ["derive"] }
-bumpalo = "3.19.0"
+bumpalo = "3.18.1"
-clap = { version = "4.5.52", features = ["derive"] }
+clap = { version = "4.5.40", features = ["derive"] }
 either = "1.15.0"
 fastrand = "2.3.0"
 milli = { path = "../milli" }
-serde = { version = "1.0.228", features = ["derive"] }
+serde = { version = "1.0.219", features = ["derive"] }
-serde_json = { version = "1.0.145", features = ["preserve_order"] }
+serde_json = { version = "1.0.140", features = ["preserve_order"] }
-tempfile = "3.23.0"
+tempfile = "3.20.0"

crates/index-scheduler/Cargo.toml

@@ -11,33 +11,31 @@ edition.workspace = true
 license.workspace = true
 
 [dependencies]
-anyhow = "1.0.100"
+anyhow = "1.0.98"
 bincode = "1.3.3"
 byte-unit = "5.1.6"
-bytes = "1.11.0"
+bumpalo = "3.18.1"
-bumpalo = "3.19.0"
 bumparaw-collections = "0.1.4"
-convert_case = "0.9.0"
+convert_case = "0.8.0"
-csv = "1.4.0"
+csv = "1.3.1"
 derive_builder = "0.20.2"
 dump = { path = "../dump" }
-enum-iterator = "2.3.0"
+enum-iterator = "2.1.0"
 file-store = { path = "../file-store" }
-flate2 = "1.1.5"
+flate2 = "1.1.2"
-indexmap = "2.12.0"
+indexmap = "2.9.0"
 meilisearch-auth = { path = "../meilisearch-auth" }
 meilisearch-types = { path = "../meilisearch-types" }
-memmap2 = "0.9.9"
+memmap2 = "0.9.7"
 page_size = "0.6.0"
-rayon = "1.11.0"
+rayon = "1.10.0"
 roaring = { version = "0.10.12", features = ["serde"] }
-serde = { version = "1.0.228", features = ["derive"] }
+serde = { version = "1.0.219", features = ["derive"] }
-serde_json = { version = "1.0.145", features = ["preserve_order"] }
+serde_json = { version = "1.0.140", features = ["preserve_order"] }
-tar = "0.4.44"
 synchronoise = "1.0.1"
-tempfile = "3.23.0"
+tempfile = "3.20.0"
-thiserror = "2.0.17"
+thiserror = "2.0.12"
-time = { version = "0.3.44", features = [
+time = { version = "0.3.41", features = [
     "serde-well-known",
     "formatting",
     "parsing",
@@ -45,11 +43,8 @@ time = { version = "0.3.44", features = [
 ] }
 tracing = "0.1.41"
 ureq = "2.12.1"
-uuid = { version = "1.18.1", features = ["serde", "v4"] }
+uuid = { version = "1.17.0", features = ["serde", "v4"] }
 backoff = "0.4.0"
-reqwest = { version = "0.12.24", features = ["rustls-tls", "http2"], default-features = false }
-rusty-s3 = "0.8.1"
-tokio = { version = "1.48.0", features = ["full"] }
 
 [dev-dependencies]
 big_s = "1.0.2"

crates/index-scheduler/src/dump.rs

@@ -150,7 +150,6 @@ impl<'a> Dump<'a> {
             details: task.details,
             status: task.status,
             network: task.network,
-            custom_metadata: task.custom_metadata,
             kind: match task.kind {
                 KindDump::DocumentImport {
                     primary_key,

crates/index-scheduler/src/error.rs

@@ -5,7 +5,6 @@ use meilisearch_types::error::{Code, ErrorCode};
 use meilisearch_types::milli::index::RollbackOutcome;
 use meilisearch_types::tasks::{Kind, Status};
 use meilisearch_types::{heed, milli};
-use reqwest::StatusCode;
 use thiserror::Error;
 
 use crate::TaskId;
@@ -128,14 +127,6 @@ pub enum Error {
     #[error("Aborted task")]
     AbortedTask,
 
-    #[error("S3 error: status: {status}, body: {body}")]
-    S3Error { status: StatusCode, body: String },
-    #[error("S3 HTTP error: {0}")]
-    S3HttpError(reqwest::Error),
-    #[error("S3 XML error: {0}")]
-    S3XmlError(Box<dyn std::error::Error + Send + Sync>),
-    #[error("S3 bucket error: {0}")]
-    S3BucketError(rusty_s3::BucketError),
     #[error(transparent)]
     Dump(#[from] dump::Error),
     #[error(transparent)]
@@ -235,10 +226,6 @@ impl Error {
             | Error::TaskCancelationWithEmptyQuery
             | Error::FromRemoteWhenExporting { .. }
             | Error::AbortedTask
-            | Error::S3Error { .. }
-            | Error::S3HttpError(_)
-            | Error::S3XmlError(_)
-            | Error::S3BucketError(_)
             | Error::Dump(_)
             | Error::Heed(_)
             | Error::Milli { .. }
@@ -306,14 +293,8 @@ impl ErrorCode for Error {
             Error::BatchNotFound(_) => Code::BatchNotFound,
             Error::TaskDeletionWithEmptyQuery => Code::MissingTaskFilters,
             Error::TaskCancelationWithEmptyQuery => Code::MissingTaskFilters,
+            // TODO: not sure of the Code to use
             Error::NoSpaceLeftInTaskQueue => Code::NoSpaceLeftOnDevice,
-            Error::S3Error { status, .. } if status.is_client_error() => {
-                Code::InvalidS3SnapshotRequest
-            }
-            Error::S3Error { .. } => Code::S3SnapshotServerError,
-            Error::S3HttpError(_) => Code::S3SnapshotServerError,
-            Error::S3XmlError(_) => Code::S3SnapshotServerError,
-            Error::S3BucketError(_) => Code::InvalidS3SnapshotParameters,
             Error::Dump(e) => e.error_code(),
             Error::Milli { error, .. } => error.error_code(),
             Error::ProcessBatchPanicked(_) => Code::Internal,

crates/index-scheduler/src/features.rs

@@ -1,9 +1,9 @@
 use std::sync::{Arc, RwLock};
 
+use meilisearch_types::enterprise_edition::network::Network;
 use meilisearch_types::features::{InstanceTogglableFeatures, RuntimeTogglableFeatures};
 use meilisearch_types::heed::types::{SerdeJson, Str};
 use meilisearch_types::heed::{Database, Env, RwTxn, WithoutTls};
-use meilisearch_types::network::Network;
 
 use crate::error::FeatureNotEnabledError;
 use crate::Result;

crates/index-scheduler/src/insta_snapshot.rs

@@ -6,7 +6,7 @@ use meilisearch_types::heed::types::{SerdeBincode, SerdeJson, Str};
 use meilisearch_types::heed::{Database, RoTxn};
 use meilisearch_types::milli::{CboRoaringBitmapCodec, RoaringBitmapCodec, BEU32};
 use meilisearch_types::tasks::{Details, Kind, Status, Task};
-use meilisearch_types::versioning::{self, VERSION_MAJOR, VERSION_MINOR, VERSION_PATCH};
+use meilisearch_types::versioning;
 use roaring::RoaringBitmap;
 
 use crate::index_mapper::IndexMapper;
@@ -36,7 +36,6 @@ pub fn snapshot_index_scheduler(scheduler: &IndexScheduler) -> String {
         run_loop_iteration: _,
         embedders: _,
         chat_settings: _,
-        runtime: _,
     } = scheduler;
 
     let rtxn = env.read_txn().unwrap();
@@ -232,7 +231,6 @@ pub fn snapshot_task(task: &Task) -> String {
         status,
         kind,
         network,
-        custom_metadata,
     } = task;
     snap.push('{');
     snap.push_str(&format!("uid: {uid}, "));
@@ -253,9 +251,6 @@ pub fn snapshot_task(task: &Task) -> String {
     if let Some(network) = network {
         snap.push_str(&format!("network: {network:?}, "))
     }
-    if let Some(custom_metadata) = custom_metadata {
-        snap.push_str(&format!("custom_metadata: {custom_metadata:?}"))
-    }
 
     snap.push('}');
     snap
@@ -320,11 +315,7 @@ fn snapshot_details(d: &Details) -> String {
             format!("{{ url: {url:?}, api_key: {api_key:?}, payload_size: {payload_size:?}, indexes: {indexes:?} }}")
         }
         Details::UpgradeDatabase { from, to } => {
-            if to == &(VERSION_MAJOR, VERSION_MINOR, VERSION_PATCH) {
+            format!("{{ from: {from:?}, to: {to:?} }}")
-                format!("{{ from: {from:?}, to: [current version] }}")
-            } else {
-                format!("{{ from: {from:?}, to: {to:?} }}")
-            }
         }
         Details::IndexCompaction { index_uid, pre_compaction_size, post_compaction_size } => {
             format!("{{ index_uid: {index_uid:?}, pre_compaction_size: {pre_compaction_size:?}, post_compaction_size: {post_compaction_size:?} }}")
@@ -404,21 +395,7 @@ pub fn snapshot_batch(batch: &Batch) -> String {
 
     snap.push('{');
     snap.push_str(&format!("uid: {uid}, "));
-    let details = if let Some(upgrade_to) = &details.upgrade_to {
+    snap.push_str(&format!("details: {}, ", serde_json::to_string(details).unwrap()));
-        if upgrade_to.as_str()
-            == format!("v{VERSION_MAJOR}.{VERSION_MINOR}.{VERSION_PATCH}").as_str()
-        {
-            let mut details = details.clone();
-
-            details.upgrade_to = Some("[current version]".into());
-            serde_json::to_string(&details).unwrap()
-        } else {
-            serde_json::to_string(details).unwrap()
-        }
-    } else {
-        serde_json::to_string(details).unwrap()
-    };
-    snap.push_str(&format!("details: {details}, "));
     snap.push_str(&format!("stats: {}, ", serde_json::to_string(&stats).unwrap()));
     if !embedder_stats.skip_serializing() {
         snap.push_str(&format!(

crates/index-scheduler/src/lib.rs

@@ -54,6 +54,7 @@ pub use features::RoFeatures;
 use flate2::bufread::GzEncoder;
 use flate2::Compression;
 use meilisearch_types::batches::Batch;
+use meilisearch_types::enterprise_edition::network::Network;
 use meilisearch_types::features::{
     ChatCompletionSettings, InstanceTogglableFeatures, RuntimeTogglableFeatures,
 };
@@ -66,7 +67,6 @@ use meilisearch_types::milli::vector::{
     Embedder, EmbedderOptions, RuntimeEmbedder, RuntimeEmbedders, RuntimeFragment,
 };
 use meilisearch_types::milli::{self, Index};
-use meilisearch_types::network::Network;
 use meilisearch_types::task_view::TaskView;
 use meilisearch_types::tasks::{KindWithContent, Task, TaskNetwork};
 use meilisearch_types::webhooks::{Webhook, WebhooksDumpView, WebhooksView};
@@ -216,9 +216,6 @@ pub struct IndexScheduler {
     /// A counter that is incremented before every call to [`tick`](IndexScheduler::tick)
     #[cfg(test)]
     run_loop_iteration: Arc<RwLock<usize>>,
-
-    /// The tokio runtime used for asynchronous tasks.
-    runtime: Option<tokio::runtime::Handle>,
 }
 
 impl IndexScheduler {
@@ -245,7 +242,6 @@ impl IndexScheduler {
             run_loop_iteration: self.run_loop_iteration.clone(),
             features: self.features.clone(),
             chat_settings: self.chat_settings,
-            runtime: self.runtime.clone(),
         }
     }
 
@@ -259,23 +255,13 @@ impl IndexScheduler {
     }
 
     /// Create an index scheduler and start its run loop.
+    #[allow(private_interfaces)] // because test_utils is private
     pub fn new(
         options: IndexSchedulerOptions,
         auth_env: Env<WithoutTls>,
         from_db_version: (u32, u32, u32),
-        runtime: Option<tokio::runtime::Handle>,
+        #[cfg(test)] test_breakpoint_sdr: crossbeam_channel::Sender<(test_utils::Breakpoint, bool)>,
-    ) -> Result<Self> {
+        #[cfg(test)] planned_failures: Vec<(usize, test_utils::FailureLocation)>,
-        let this = Self::new_without_run(options, auth_env, from_db_version, runtime)?;
-
-        this.run();
-        Ok(this)
-    }
-
-    fn new_without_run(
-        options: IndexSchedulerOptions,
-        auth_env: Env<WithoutTls>,
-        from_db_version: (u32, u32, u32),
-        runtime: Option<tokio::runtime::Handle>,
     ) -> Result<Self> {
         std::fs::create_dir_all(&options.tasks_path)?;
         std::fs::create_dir_all(&options.update_file_path)?;
@@ -330,7 +316,8 @@ impl IndexScheduler {
 
         wtxn.commit()?;
 
-        Ok(Self {
+        // allow unreachable_code to get rids of the warning in the case of a test build.
+        let this = Self {
             processing_tasks: Arc::new(RwLock::new(ProcessingTasks::new())),
             version,
             queue,
@@ -346,32 +333,15 @@ impl IndexScheduler {
             webhooks: Arc::new(webhooks),
             embedders: Default::default(),
 
-            #[cfg(test)] // Will be replaced in `new_tests` in test environments
+            #[cfg(test)]
-            test_breakpoint_sdr: crossbeam_channel::bounded(0).0,
+            test_breakpoint_sdr,
-            #[cfg(test)] // Will be replaced in `new_tests` in test environments
+            #[cfg(test)]
-            planned_failures: Default::default(),
+            planned_failures,
             #[cfg(test)]
             run_loop_iteration: Arc::new(RwLock::new(0)),
             features,
             chat_settings,
-            runtime,
+        };
-        })
-    }
-
-    /// Create an index scheduler and start its run loop.
-    #[cfg(test)]
-    fn new_test(
-        options: IndexSchedulerOptions,
-        auth_env: Env<WithoutTls>,
-        from_db_version: (u32, u32, u32),
-        runtime: Option<tokio::runtime::Handle>,
-        test_breakpoint_sdr: crossbeam_channel::Sender<(test_utils::Breakpoint, bool)>,
-        planned_failures: Vec<(usize, test_utils::FailureLocation)>,
-    ) -> Result<Self> {
-        let mut this = Self::new_without_run(options, auth_env, from_db_version, runtime)?;
-
-        this.test_breakpoint_sdr = test_breakpoint_sdr;
-        this.planned_failures = planned_failures;
 
         this.run();
         Ok(this)
@@ -756,19 +726,6 @@ impl IndexScheduler {
         kind: KindWithContent,
         task_id: Option<TaskId>,
         dry_run: bool,
-    ) -> Result<Task> {
-        self.register_with_custom_metadata(kind, task_id, None, dry_run)
-    }
-
-    /// Register a new task in the scheduler, with metadata.
-    ///
-    /// If it fails and data was associated with the task, it tries to delete the associated data.
-    pub fn register_with_custom_metadata(
-        &self,
-        kind: KindWithContent,
-        task_id: Option<TaskId>,
-        custom_metadata: Option<String>,
-        dry_run: bool,
     ) -> Result<Task> {
         // if the task doesn't delete or cancel anything and 40% of the task queue is full, we must refuse to enqueue the incoming task
         if !matches!(&kind, KindWithContent::TaskDeletion { tasks, .. } | KindWithContent::TaskCancelation { tasks, .. } if !tasks.is_empty())
@@ -779,7 +736,7 @@ impl IndexScheduler {
         }
 
         let mut wtxn = self.env.write_txn()?;
-        let task = self.queue.register(&mut wtxn, &kind, task_id, custom_metadata, dry_run)?;
+        let task = self.queue.register(&mut wtxn, &kind, task_id, dry_run)?;
 
         // If the registered task is a task cancelation
         // we inform the processing tasks to stop (if necessary).

crates/index-scheduler/src/queue/mod.rs

@@ -257,7 +257,6 @@ impl Queue {
         wtxn: &mut RwTxn,
         kind: &KindWithContent,
         task_id: Option<TaskId>,
-        custom_metadata: Option<String>,
         dry_run: bool,
     ) -> Result<Task> {
         let next_task_id = self.tasks.next_task_id(wtxn)?;
@@ -281,7 +280,6 @@ impl Queue {
             status: Status::Enqueued,
             kind: kind.clone(),
             network: None,
-            custom_metadata,
         };
         // For deletion and cancelation tasks, we want to make extra sure that they
         // don't attempt to delete/cancel tasks that are newer than themselves.
@@ -346,7 +344,6 @@ impl Queue {
                 tasks: to_delete,
             },
             None,
-            None,
             false,
         )?;
 

crates/index-scheduler/src/scheduler/mod.rs

@@ -25,7 +25,6 @@ use convert_case::{Case, Casing as _};
 use meilisearch_types::error::ResponseError;
 use meilisearch_types::heed::{Env, WithoutTls};
 use meilisearch_types::milli;
-use meilisearch_types::milli::update::S3SnapshotOptions;
 use meilisearch_types::tasks::Status;
 use process_batch::ProcessBatchInfo;
 use rayon::current_num_threads;
@@ -88,14 +87,11 @@ pub struct Scheduler {
 
     /// Snapshot compaction status.
     pub(crate) experimental_no_snapshot_compaction: bool,
-
-    /// S3 Snapshot options.
-    pub(crate) s3_snapshot_options: Option<S3SnapshotOptions>,
 }
 
 impl Scheduler {
-    pub(crate) fn private_clone(&self) -> Self {
+    pub(crate) fn private_clone(&self) -> Scheduler {
-        Self {
+        Scheduler {
             must_stop_processing: self.must_stop_processing.clone(),
             wake_up: self.wake_up.clone(),
             autobatching_enabled: self.autobatching_enabled,
@@ -107,52 +103,23 @@ impl Scheduler {
             version_file_path: self.version_file_path.clone(),
             embedding_cache_cap: self.embedding_cache_cap,
             experimental_no_snapshot_compaction: self.experimental_no_snapshot_compaction,
-            s3_snapshot_options: self.s3_snapshot_options.clone(),
         }
     }
 
     pub fn new(options: &IndexSchedulerOptions, auth_env: Env<WithoutTls>) -> Scheduler {
-        let IndexSchedulerOptions {
-            version_file_path,
-            auth_path: _,
-            tasks_path: _,
-            update_file_path: _,
-            indexes_path: _,
-            snapshots_path,
-            dumps_path,
-            cli_webhook_url: _,
-            cli_webhook_authorization: _,
-            task_db_size: _,
-            index_base_map_size: _,
-            enable_mdb_writemap: _,
-            index_growth_amount: _,
-            index_count: _,
-            indexer_config,
-            autobatching_enabled,
-            cleanup_enabled: _,
-            max_number_of_tasks: _,
-            max_number_of_batched_tasks,
-            batched_tasks_size_limit,
-            instance_features: _,
-            auto_upgrade: _,
-            embedding_cache_cap,
-            experimental_no_snapshot_compaction,
-        } = options;
-
         Scheduler {
             must_stop_processing: MustStopProcessing::default(),
             // we want to start the loop right away in case meilisearch was ctrl+Ced while processing things
             wake_up: Arc::new(SignalEvent::auto(true)),
-            autobatching_enabled: *autobatching_enabled,
+            autobatching_enabled: options.autobatching_enabled,
-            max_number_of_batched_tasks: *max_number_of_batched_tasks,
+            max_number_of_batched_tasks: options.max_number_of_batched_tasks,
-            batched_tasks_size_limit: *batched_tasks_size_limit,
+            batched_tasks_size_limit: options.batched_tasks_size_limit,
-            dumps_path: dumps_path.clone(),
+            dumps_path: options.dumps_path.clone(),
-            snapshots_path: snapshots_path.clone(),
+            snapshots_path: options.snapshots_path.clone(),
             auth_env,
-            version_file_path: version_file_path.clone(),
+            version_file_path: options.version_file_path.clone(),
-            embedding_cache_cap: *embedding_cache_cap,
+            embedding_cache_cap: options.embedding_cache_cap,
-            experimental_no_snapshot_compaction: *experimental_no_snapshot_compaction,
+            experimental_no_snapshot_compaction: options.experimental_no_snapshot_compaction,
-            s3_snapshot_options: indexer_config.s3_snapshot_options.clone(),
         }
     }
 }

crates/index-scheduler/src/scheduler/process_snapshot_creation.rs

@@ -12,8 +12,6 @@ use crate::processing::{AtomicUpdateFileStep, SnapshotCreationProgress};
 use crate::queue::TaskQueue;
 use crate::{Error, IndexScheduler, Result};
 
-const UPDATE_FILES_DIR_NAME: &str = "update_files";
-
 /// # Safety
 ///
 /// See [`EnvOpenOptions::open`].
@@ -80,32 +78,10 @@ impl IndexScheduler {
     pub(super) fn process_snapshot(
         &self,
         progress: Progress,
-        tasks: Vec<Task>,
+        mut tasks: Vec<Task>,
     ) -> Result<Vec<Task>> {
         progress.update_progress(SnapshotCreationProgress::StartTheSnapshotCreation);
 
-        match self.scheduler.s3_snapshot_options.clone() {
-            Some(options) => {
-                #[cfg(not(unix))]
-                {
-                    let _ = options;
-                    panic!("Non-unix platform does not support S3 snapshotting");
-                }
-                #[cfg(unix)]
-                self.runtime
-                    .as_ref()
-                    .expect("Runtime not initialized")
-                    .block_on(self.process_snapshot_to_s3(progress, options, tasks))
-            }
-            None => self.process_snapshots_to_disk(progress, tasks),
-        }
-    }
-
-    fn process_snapshots_to_disk(
-        &self,
-        progress: Progress,
-        mut tasks: Vec<Task>,
-    ) -> Result<Vec<Task>, Error> {
         fs::create_dir_all(&self.scheduler.snapshots_path)?;
         let temp_snapshot_dir = tempfile::tempdir()?;
 
@@ -152,7 +128,7 @@ impl IndexScheduler {
         let rtxn = self.env.read_txn()?;
 
         // 2.4 Create the update files directory
-        let update_files_dir = temp_snapshot_dir.path().join(UPDATE_FILES_DIR_NAME);
+        let update_files_dir = temp_snapshot_dir.path().join("update_files");
         fs::create_dir_all(&update_files_dir)?;
 
         // 2.5 Only copy the update files of the enqueued tasks
@@ -164,7 +140,7 @@ impl IndexScheduler {
             let task =
                 self.queue.tasks.get_task(&rtxn, task_id)?.ok_or(Error::CorruptedTaskQueue)?;
             if let Some(content_uuid) = task.content_uuid() {
-                let src = self.queue.file_store.update_path(content_uuid);
+                let src = self.queue.file_store.get_update_path(content_uuid);
                 let dst = update_files_dir.join(content_uuid.to_string());
                 fs::copy(src, dst)?;
             }
@@ -230,407 +206,4 @@ impl IndexScheduler {
 
         Ok(tasks)
     }
-
-    #[cfg(unix)]
-    pub(super) async fn process_snapshot_to_s3(
-        &self,
-        progress: Progress,
-        opts: meilisearch_types::milli::update::S3SnapshotOptions,
-        mut tasks: Vec<Task>,
-    ) -> Result<Vec<Task>> {
-        use meilisearch_types::milli::update::S3SnapshotOptions;
-
-        let S3SnapshotOptions {
-            s3_bucket_url,
-            s3_bucket_region,
-            s3_bucket_name,
-            s3_snapshot_prefix,
-            s3_access_key,
-            s3_secret_key,
-            s3_max_in_flight_parts,
-            s3_compression_level: level,
-            s3_signature_duration,
-            s3_multipart_part_size,
-        } = opts;
-
-        let must_stop_processing = self.scheduler.must_stop_processing.clone();
-        let retry_backoff = backoff::ExponentialBackoff::default();
-        let db_name = {
-            let mut base_path = self.env.path().to_owned();
-            base_path.pop();
-            base_path.file_name().and_then(OsStr::to_str).unwrap_or("data.ms").to_string()
-        };
-
-        let (reader, writer) = std::io::pipe()?;
-        let uploader_task = tokio::spawn(multipart_stream_to_s3(
-            s3_bucket_url,
-            s3_bucket_region,
-            s3_bucket_name,
-            s3_snapshot_prefix,
-            s3_access_key,
-            s3_secret_key,
-            s3_max_in_flight_parts,
-            s3_signature_duration,
-            s3_multipart_part_size,
-            must_stop_processing,
-            retry_backoff,
-            db_name,
-            reader,
-        ));
-
-        let index_scheduler = IndexScheduler::private_clone(self);
-        let builder_task = tokio::task::spawn_blocking(move || {
-            stream_tarball_into_pipe(progress, level, writer, index_scheduler)
-        });
-
-        let (uploader_result, builder_result) = tokio::join!(uploader_task, builder_task);
-
-        // Check uploader result first to early return on task abortion.
-        // safety: JoinHandle can return an error if the task was aborted, cancelled, or panicked.
-        uploader_result.unwrap()?;
-        builder_result.unwrap()?;
-
-        for task in &mut tasks {
-            task.status = Status::Succeeded;
-        }
-
-        Ok(tasks)
-    }
-}
-
-/// Streams a tarball of the database content into a pipe.
-#[cfg(unix)]
-fn stream_tarball_into_pipe(
-    progress: Progress,
-    level: u32,
-    writer: std::io::PipeWriter,
-    index_scheduler: IndexScheduler,
-) -> std::result::Result<(), Error> {
-    use std::io::Write as _;
-    use std::path::Path;
-
-    let writer = flate2::write::GzEncoder::new(writer, flate2::Compression::new(level));
-    let mut tarball = tar::Builder::new(writer);
-
-    // 1. Snapshot the version file
-    tarball
-        .append_path_with_name(&index_scheduler.scheduler.version_file_path, VERSION_FILE_NAME)?;
-
-    // 2. Snapshot the index scheduler LMDB env
-    progress.update_progress(SnapshotCreationProgress::SnapshotTheIndexScheduler);
-    let tasks_env_file = index_scheduler.env.try_clone_inner_file()?;
-    let path = Path::new("tasks").join("data.mdb");
-    append_file_to_tarball(&mut tarball, path, tasks_env_file)?;
-
-    // 2.3 Create a read transaction on the index-scheduler
-    let rtxn = index_scheduler.env.read_txn()?;
-
-    // 2.4 Create the update files directory
-    //     And only copy the update files of the enqueued tasks
-    progress.update_progress(SnapshotCreationProgress::SnapshotTheUpdateFiles);
-    let enqueued = index_scheduler.queue.tasks.get_status(&rtxn, Status::Enqueued)?;
-    let (atomic, update_file_progress) = AtomicUpdateFileStep::new(enqueued.len() as u32);
-    progress.update_progress(update_file_progress);
-
-    // We create the update_files directory so that it
-    // always exists even if there are no update files
-    let update_files_dir = Path::new(UPDATE_FILES_DIR_NAME);
-    let src_update_files_dir = {
-        let mut path = index_scheduler.env.path().to_path_buf();
-        path.pop();
-        path.join(UPDATE_FILES_DIR_NAME)
-    };
-    tarball.append_dir(update_files_dir, src_update_files_dir)?;
-
-    for task_id in enqueued {
-        let task = index_scheduler
-            .queue
-            .tasks
-            .get_task(&rtxn, task_id)?
-            .ok_or(Error::CorruptedTaskQueue)?;
-        if let Some(content_uuid) = task.content_uuid() {
-            use std::fs::File;
-
-            let src = index_scheduler.queue.file_store.update_path(content_uuid);
-            let mut update_file = File::open(src)?;
-            let path = update_files_dir.join(content_uuid.to_string());
-            tarball.append_file(path, &mut update_file)?;
-        }
-        atomic.fetch_add(1, Ordering::Relaxed);
-    }
-
-    // 3. Snapshot every indexes
-    progress.update_progress(SnapshotCreationProgress::SnapshotTheIndexes);
-    let index_mapping = index_scheduler.index_mapper.index_mapping;
-    let nb_indexes = index_mapping.len(&rtxn)? as u32;
-    let indexes_dir = Path::new("indexes");
-    let indexes_references: Vec<_> = index_scheduler
-        .index_mapper
-        .index_mapping
-        .iter(&rtxn)?
-        .map(|res| res.map_err(Error::from).map(|(name, uuid)| (name.to_string(), uuid)))
-        .collect::<Result<_, Error>>()?;
-
-    // It's prettier to use a for loop instead of the IndexMapper::try_for_each_index
-    // method, especially when we need to access the UUID, local path and index number.
-    for (i, (name, uuid)) in indexes_references.into_iter().enumerate() {
-        progress.update_progress(VariableNameStep::<SnapshotCreationProgress>::new(
-            &name, i as u32, nb_indexes,
-        ));
-        let path = indexes_dir.join(uuid.to_string()).join("data.mdb");
-        let index = index_scheduler.index_mapper.index(&rtxn, &name)?;
-        let index_file = index.try_clone_inner_file()?;
-        tracing::trace!("Appending index file for {name} in {}", path.display());
-        append_file_to_tarball(&mut tarball, path, index_file)?;
-    }
-
-    drop(rtxn);
-
-    // 4. Snapshot the auth LMDB env
-    progress.update_progress(SnapshotCreationProgress::SnapshotTheApiKeys);
-    let auth_env_file = index_scheduler.scheduler.auth_env.try_clone_inner_file()?;
-    let path = Path::new("auth").join("data.mdb");
-    append_file_to_tarball(&mut tarball, path, auth_env_file)?;
-
-    let mut gzencoder = tarball.into_inner()?;
-    gzencoder.flush()?;
-    gzencoder.try_finish()?;
-    let mut writer = gzencoder.finish()?;
-    writer.flush()?;
-
-    Result::<_, Error>::Ok(())
-}
-
-#[cfg(unix)]
-fn append_file_to_tarball<W, P>(
-    tarball: &mut tar::Builder<W>,
-    path: P,
-    mut auth_env_file: fs::File,
-) -> Result<(), Error>
-where
-    W: std::io::Write,
-    P: AsRef<std::path::Path>,
-{
-    use std::io::{Seek as _, SeekFrom};
-
-    // Note: A previous snapshot operation may have left the cursor
-    //       at the end of the file so we need to seek to the start.
-    auth_env_file.seek(SeekFrom::Start(0))?;
-    tarball.append_file(path, &mut auth_env_file)?;
-    Ok(())
-}
-
-/// Streams the content read from the given reader to S3.
-#[cfg(unix)]
-#[allow(clippy::too_many_arguments)]
-async fn multipart_stream_to_s3(
-    s3_bucket_url: String,
-    s3_bucket_region: String,
-    s3_bucket_name: String,
-    s3_snapshot_prefix: String,
-    s3_access_key: String,
-    s3_secret_key: String,
-    s3_max_in_flight_parts: std::num::NonZero<usize>,
-    s3_signature_duration: std::time::Duration,
-    s3_multipart_part_size: u64,
-    must_stop_processing: super::MustStopProcessing,
-    retry_backoff: backoff::exponential::ExponentialBackoff<backoff::SystemClock>,
-    db_name: String,
-    reader: std::io::PipeReader,
-) -> Result<(), Error> {
-    use std::collections::VecDeque;
-    use std::io;
-    use std::os::fd::OwnedFd;
-    use std::path::PathBuf;
-
-    use bytes::{Bytes, BytesMut};
-    use reqwest::{Client, Response};
-    use rusty_s3::actions::CreateMultipartUpload;
-    use rusty_s3::{Bucket, BucketError, Credentials, S3Action as _, UrlStyle};
-    use tokio::task::JoinHandle;
-
-    let reader = OwnedFd::from(reader);
-    let reader = tokio::net::unix::pipe::Receiver::from_owned_fd(reader)?;
-    let s3_snapshot_prefix = PathBuf::from(s3_snapshot_prefix);
-    let url =
-        s3_bucket_url.parse().map_err(BucketError::ParseError).map_err(Error::S3BucketError)?;
-    let bucket = Bucket::new(url, UrlStyle::Path, s3_bucket_name, s3_bucket_region)
-        .map_err(Error::S3BucketError)?;
-    let credential = Credentials::new(s3_access_key, s3_secret_key);
-
-    // Note for the future (rust 1.91+): use with_added_extension, it's prettier
-    let object_path = s3_snapshot_prefix.join(format!("{db_name}.snapshot"));
-    // Note: It doesn't work on Windows and if a port to this platform is needed,
-    //       use the slash-path crate or similar to get the correct path separator.
-    let object = object_path.display().to_string();
-
-    let action = bucket.create_multipart_upload(Some(&credential), &object);
-    let url = action.sign(s3_signature_duration);
-
-    let client = Client::new();
-    let resp = client.post(url).send().await.map_err(Error::S3HttpError)?;
-    let status = resp.status();
-
-    let body = match resp.error_for_status_ref() {
-        Ok(_) => resp.text().await.map_err(Error::S3HttpError)?,
-        Err(_) => {
-            return Err(Error::S3Error { status, body: resp.text().await.unwrap_or_default() })
-        }
-    };
-
-    let multipart =
-        CreateMultipartUpload::parse_response(&body).map_err(|e| Error::S3XmlError(Box::new(e)))?;
-    tracing::debug!("Starting the upload of the snapshot to {object}");
-
-    // We use this bumpalo for etags strings.
-    let bump = bumpalo::Bump::new();
-    let mut etags = Vec::<&str>::new();
-    let mut in_flight = VecDeque::<(JoinHandle<reqwest::Result<Response>>, Bytes)>::with_capacity(
-        s3_max_in_flight_parts.get(),
-    );
-
-    // Part numbers start at 1 and cannot be larger than 10k
-    for part_number in 1u16.. {
-        if must_stop_processing.get() {
-            return Err(Error::AbortedTask);
-        }
-
-        let part_upload =
-            bucket.upload_part(Some(&credential), &object, part_number, multipart.upload_id());
-        let url = part_upload.sign(s3_signature_duration);
-
-        // Wait for a buffer to be ready if there are in-flight parts that landed
-        let mut buffer = if in_flight.len() >= s3_max_in_flight_parts.get() {
-            let (handle, buffer) = in_flight.pop_front().expect("At least one in flight request");
-            let resp = join_and_map_error(handle).await?;
-            extract_and_append_etag(&bump, &mut etags, resp.headers())?;
-
-            let mut buffer = match buffer.try_into_mut() {
-                Ok(buffer) => buffer,
-                Err(_) => unreachable!("All bytes references were consumed in the task"),
-            };
-            buffer.clear();
-            buffer
-        } else {
-            BytesMut::with_capacity(s3_multipart_part_size as usize)
-        };
-
-        // If we successfully read enough bytes,
-        // we can continue and send the buffer/part
-        while buffer.len() < (s3_multipart_part_size as usize / 2) {
-            // Wait for the pipe to be readable
-
-            reader.readable().await?;
-
-            match reader.try_read_buf(&mut buffer) {
-                Ok(0) => break,
-                // We read some bytes but maybe not enough
-                Ok(_) => continue,
-                // The readiness event is a false positive.
-                Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,
-                Err(e) => return Err(e.into()),
-            }
-        }
-
-        if buffer.is_empty() {
-            // Break the loop if the buffer is
-            // empty after we tried to read bytes
-            break;
-        }
-
-        let body = buffer.freeze();
-        tracing::trace!("Sending part {part_number}");
-        let task = tokio::spawn({
-            let client = client.clone();
-            let body = body.clone();
-            backoff::future::retry(retry_backoff.clone(), move || {
-                let client = client.clone();
-                let url = url.clone();
-                let body = body.clone();
-                async move {
-                    match client.put(url).body(body).send().await {
-                        Ok(resp) if resp.status().is_client_error() => {
-                            resp.error_for_status().map_err(backoff::Error::Permanent)
-                        }
-                        Ok(resp) => Ok(resp),
-                        Err(e) => Err(backoff::Error::transient(e)),
-                    }
-                }
-            })
-        });
-        in_flight.push_back((task, body));
-    }
-
-    for (handle, _buffer) in in_flight {
-        let resp = join_and_map_error(handle).await?;
-        extract_and_append_etag(&bump, &mut etags, resp.headers())?;
-    }
-
-    tracing::debug!("Finalizing the multipart upload");
-
-    let action = bucket.complete_multipart_upload(
-        Some(&credential),
-        &object,
-        multipart.upload_id(),
-        etags.iter().map(AsRef::as_ref),
-    );
-    let url = action.sign(s3_signature_duration);
-    let body = action.body();
-    let resp = backoff::future::retry(retry_backoff, move || {
-        let client = client.clone();
-        let url = url.clone();
-        let body = body.clone();
-        async move {
-            match client.post(url).body(body).send().await {
-                Ok(resp) if resp.status().is_client_error() => {
-                    Err(backoff::Error::Permanent(Error::S3Error {
-                        status: resp.status(),
-                        body: resp.text().await.unwrap_or_default(),
-                    }))
-                }
-                Ok(resp) => Ok(resp),
-                Err(e) => Err(backoff::Error::transient(Error::S3HttpError(e))),
-            }
-        }
-    })
-    .await?;
-
-    let status = resp.status();
-    let body = resp.text().await.map_err(|e| Error::S3Error { status, body: e.to_string() })?;
-    if status.is_success() {
-        Ok(())
-    } else {
-        Err(Error::S3Error { status, body })
-    }
-}
-
-#[cfg(unix)]
-async fn join_and_map_error(
-    join_handle: tokio::task::JoinHandle<Result<reqwest::Response, reqwest::Error>>,
-) -> Result<reqwest::Response> {
-    // safety: Panic happens if the task (JoinHandle) was aborted, cancelled, or panicked
-    let request = join_handle.await.unwrap();
-    let resp = request.map_err(Error::S3HttpError)?;
-    match resp.error_for_status_ref() {
-        Ok(_) => Ok(resp),
-        Err(_) => Err(Error::S3Error {
-            status: resp.status(),
-            body: resp.text().await.unwrap_or_default(),
-        }),
-    }
-}
-
-#[cfg(unix)]
-fn extract_and_append_etag<'b>(
-    bump: &'b bumpalo::Bump,
-    etags: &mut Vec<&'b str>,
-    headers: &reqwest::header::HeaderMap,
-) -> Result<()> {
-    use reqwest::header::ETAG;
-
-    let etag = headers.get(ETAG).ok_or_else(|| Error::S3XmlError("Missing ETag header".into()))?;
-    let etag = etag.to_str().map_err(|e| Error::S3XmlError(Box::new(e)))?;
-    etags.push(bump.alloc_str(etag));
-
-    Ok(())
 }

crates/index-scheduler/src/scheduler/snapshots/test_failure.rs/upgrade_failure/after_processing_everything.snap

@@ -6,7 +6,7 @@ source: crates/index-scheduler/src/scheduler/test_failure.rs
 []
 ----------------------------------------------------------------------
 ### All Tasks:
-0 {uid: 0, batch_uid: 0, status: succeeded, details: { from: (1, 12, 0), to: [current version] }, kind: UpgradeDatabase { from: (1, 12, 0) }}
+0 {uid: 0, batch_uid: 0, status: succeeded, details: { from: (1, 12, 0), to: (1, 24, 0) }, kind: UpgradeDatabase { from: (1, 12, 0) }}
 1 {uid: 1, batch_uid: 1, status: succeeded, details: { primary_key: Some("mouse"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "catto", primary_key: Some("mouse") }}
 2 {uid: 2, batch_uid: 2, status: succeeded, details: { primary_key: Some("bone"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "doggo", primary_key: Some("bone") }}
 3 {uid: 3, batch_uid: 3, status: failed, error: ResponseError { code: 200, message: "Index `doggo` already exists.", error_code: "index_already_exists", error_type: "invalid_request", error_link: "https://docs.meilisearch.com/errors#index_already_exists" }, details: { primary_key: Some("bone"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "doggo", primary_key: Some("bone") }}
@@ -57,7 +57,7 @@ girafo: { number_of_documents: 0, field_distribution: {} }
 [timestamp] [4,]
 ----------------------------------------------------------------------
 ### All Batches:
-0 {uid: 0, details: {"upgradeFrom":"v1.12.0","upgradeTo":"[current version]"}, stats: {"totalNbTasks":1,"status":{"succeeded":1},"types":{"upgradeDatabase":1},"indexUids":{}}, stop reason: "stopped after the last task of type `upgradeDatabase` because they cannot be batched with tasks of any other type.", }
+0 {uid: 0, details: {"upgradeFrom":"v1.12.0","upgradeTo":"v1.24.0"}, stats: {"totalNbTasks":1,"status":{"succeeded":1},"types":{"upgradeDatabase":1},"indexUids":{}}, stop reason: "stopped after the last task of type `upgradeDatabase` because they cannot be batched with tasks of any other type.", }
 1 {uid: 1, details: {"primaryKey":"mouse"}, stats: {"totalNbTasks":1,"status":{"succeeded":1},"types":{"indexCreation":1},"indexUids":{"catto":1}}, stop reason: "created batch containing only task with id 1 of type `indexCreation` that cannot be batched with any other task.", }
 2 {uid: 2, details: {"primaryKey":"bone"}, stats: {"totalNbTasks":1,"status":{"succeeded":1},"types":{"indexCreation":1},"indexUids":{"doggo":1}}, stop reason: "created batch containing only task with id 2 of type `indexCreation` that cannot be batched with any other task.", }
 3 {uid: 3, details: {"primaryKey":"bone"}, stats: {"totalNbTasks":1,"status":{"failed":1},"types":{"indexCreation":1},"indexUids":{"doggo":1}}, stop reason: "created batch containing only task with id 3 of type `indexCreation` that cannot be batched with any other task.", }

crates/index-scheduler/src/scheduler/snapshots/test_failure.rs/upgrade_failure/register_automatic_upgrade_task.snap

@@ -6,7 +6,7 @@ source: crates/index-scheduler/src/scheduler/test_failure.rs
 []
 ----------------------------------------------------------------------
 ### All Tasks:
-0 {uid: 0, status: enqueued, details: { from: (1, 12, 0), to: [current version] }, kind: UpgradeDatabase { from: (1, 12, 0) }}
+0 {uid: 0, status: enqueued, details: { from: (1, 12, 0), to: (1, 24, 0) }, kind: UpgradeDatabase { from: (1, 12, 0) }}
 ----------------------------------------------------------------------
 ### Status:
 enqueued [0,]

crates/index-scheduler/src/scheduler/snapshots/test_failure.rs/upgrade_failure/registered_a_task_while_the_upgrade_task_is_enqueued.snap

@@ -6,7 +6,7 @@ source: crates/index-scheduler/src/scheduler/test_failure.rs
 []
 ----------------------------------------------------------------------
 ### All Tasks:
-0 {uid: 0, status: enqueued, details: { from: (1, 12, 0), to: [current version] }, kind: UpgradeDatabase { from: (1, 12, 0) }}
+0 {uid: 0, status: enqueued, details: { from: (1, 12, 0), to: (1, 24, 0) }, kind: UpgradeDatabase { from: (1, 12, 0) }}
 1 {uid: 1, status: enqueued, details: { primary_key: Some("mouse"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "catto", primary_key: Some("mouse") }}
 ----------------------------------------------------------------------
 ### Status:

crates/index-scheduler/src/scheduler/snapshots/test_failure.rs/upgrade_failure/upgrade_task_failed.snap

@@ -6,7 +6,7 @@ source: crates/index-scheduler/src/scheduler/test_failure.rs
 []
 ----------------------------------------------------------------------
 ### All Tasks:
-0 {uid: 0, batch_uid: 0, status: failed, error: ResponseError { code: 200, message: "Planned failure for tests.", error_code: "internal", error_type: "internal", error_link: "https://docs.meilisearch.com/errors#internal" }, details: { from: (1, 12, 0), to: [current version] }, kind: UpgradeDatabase { from: (1, 12, 0) }}
+0 {uid: 0, batch_uid: 0, status: failed, error: ResponseError { code: 200, message: "Planned failure for tests.", error_code: "internal", error_type: "internal", error_link: "https://docs.meilisearch.com/errors#internal" }, details: { from: (1, 12, 0), to: (1, 24, 0) }, kind: UpgradeDatabase { from: (1, 12, 0) }}
 1 {uid: 1, status: enqueued, details: { primary_key: Some("mouse"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "catto", primary_key: Some("mouse") }}
 ----------------------------------------------------------------------
 ### Status:
@@ -37,7 +37,7 @@ catto [1,]
 [timestamp] [0,]
 ----------------------------------------------------------------------
 ### All Batches:
-0 {uid: 0, details: {"upgradeFrom":"v1.12.0","upgradeTo":"[current version]"}, stats: {"totalNbTasks":1,"status":{"failed":1},"types":{"upgradeDatabase":1},"indexUids":{}}, stop reason: "stopped after the last task of type `upgradeDatabase` because they cannot be batched with tasks of any other type.", }
+0 {uid: 0, details: {"upgradeFrom":"v1.12.0","upgradeTo":"v1.24.0"}, stats: {"totalNbTasks":1,"status":{"failed":1},"types":{"upgradeDatabase":1},"indexUids":{}}, stop reason: "stopped after the last task of type `upgradeDatabase` because they cannot be batched with tasks of any other type.", }
 ----------------------------------------------------------------------
 ### Batch to tasks mapping:
 0 [0,]

crates/index-scheduler/src/scheduler/snapshots/test_failure.rs/upgrade_failure/upgrade_task_failed_again.snap

@@ -6,7 +6,7 @@ source: crates/index-scheduler/src/scheduler/test_failure.rs
 []
 ----------------------------------------------------------------------
 ### All Tasks:
-0 {uid: 0, batch_uid: 0, status: failed, error: ResponseError { code: 200, message: "Planned failure for tests.", error_code: "internal", error_type: "internal", error_link: "https://docs.meilisearch.com/errors#internal" }, details: { from: (1, 12, 0), to: [current version] }, kind: UpgradeDatabase { from: (1, 12, 0) }}
+0 {uid: 0, batch_uid: 0, status: failed, error: ResponseError { code: 200, message: "Planned failure for tests.", error_code: "internal", error_type: "internal", error_link: "https://docs.meilisearch.com/errors#internal" }, details: { from: (1, 12, 0), to: (1, 24, 0) }, kind: UpgradeDatabase { from: (1, 12, 0) }}
 1 {uid: 1, status: enqueued, details: { primary_key: Some("mouse"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "catto", primary_key: Some("mouse") }}
 2 {uid: 2, status: enqueued, details: { primary_key: Some("bone"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "doggo", primary_key: Some("bone") }}
 ----------------------------------------------------------------------
@@ -40,7 +40,7 @@ doggo [2,]
 [timestamp] [0,]
 ----------------------------------------------------------------------
 ### All Batches:
-0 {uid: 0, details: {"upgradeFrom":"v1.12.0","upgradeTo":"[current version]"}, stats: {"totalNbTasks":1,"status":{"failed":1},"types":{"upgradeDatabase":1},"indexUids":{}}, stop reason: "stopped after the last task of type `upgradeDatabase` because they cannot be batched with tasks of any other type.", }
+0 {uid: 0, details: {"upgradeFrom":"v1.12.0","upgradeTo":"v1.24.0"}, stats: {"totalNbTasks":1,"status":{"failed":1},"types":{"upgradeDatabase":1},"indexUids":{}}, stop reason: "stopped after the last task of type `upgradeDatabase` because they cannot be batched with tasks of any other type.", }
 ----------------------------------------------------------------------
 ### Batch to tasks mapping:
 0 [0,]

crates/index-scheduler/src/scheduler/snapshots/test_failure.rs/upgrade_failure/upgrade_task_succeeded.snap

@@ -6,7 +6,7 @@ source: crates/index-scheduler/src/scheduler/test_failure.rs
 []
 ----------------------------------------------------------------------
 ### All Tasks:
-0 {uid: 0, batch_uid: 0, status: succeeded, details: { from: (1, 12, 0), to: [current version] }, kind: UpgradeDatabase { from: (1, 12, 0) }}
+0 {uid: 0, batch_uid: 0, status: succeeded, details: { from: (1, 12, 0), to: (1, 24, 0) }, kind: UpgradeDatabase { from: (1, 12, 0) }}
 1 {uid: 1, status: enqueued, details: { primary_key: Some("mouse"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "catto", primary_key: Some("mouse") }}
 2 {uid: 2, status: enqueued, details: { primary_key: Some("bone"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "doggo", primary_key: Some("bone") }}
 3 {uid: 3, status: enqueued, details: { primary_key: Some("bone"), old_new_uid: None, new_index_uid: None }, kind: IndexCreation { index_uid: "doggo", primary_key: Some("bone") }}
@@ -43,7 +43,7 @@ doggo [2,3,]
 [timestamp] [0,]
 ----------------------------------------------------------------------
 ### All Batches:
-0 {uid: 0, details: {"upgradeFrom":"v1.12.0","upgradeTo":"[current version]"}, stats: {"totalNbTasks":1,"status":{"succeeded":1},"types":{"upgradeDatabase":1},"indexUids":{}}, stop reason: "stopped after the last task of type `upgradeDatabase` because they cannot be batched with tasks of any other type.", }
+0 {uid: 0, details: {"upgradeFrom":"v1.12.0","upgradeTo":"v1.24.0"}, stats: {"totalNbTasks":1,"status":{"succeeded":1},"types":{"upgradeDatabase":1},"indexUids":{}}, stop reason: "stopped after the last task of type `upgradeDatabase` because they cannot be batched with tasks of any other type.", }
 ----------------------------------------------------------------------
 ### Batch to tasks mapping:
 0 [0,]

crates/index-scheduler/src/test_utils.rs

@@ -126,7 +126,7 @@ impl IndexScheduler {
         std::fs::create_dir_all(&options.auth_path).unwrap();
         let auth_env = open_auth_store_env(&options.auth_path).unwrap();
         let index_scheduler =
-            Self::new_test(options, auth_env, version, None, sender, planned_failures).unwrap();
+            Self::new(options, auth_env, version, sender, planned_failures).unwrap();
 
         // To be 100% consistent between all test we're going to start the scheduler right now
         // and ensure it's in the expected starting state.

crates/index-scheduler/src/upgrade/mod.rs

@@ -1,7 +1,7 @@
 use anyhow::bail;
 use meilisearch_types::heed::{Env, RwTxn, WithoutTls};
 use meilisearch_types::tasks::{Details, KindWithContent, Status, Task};
-use meilisearch_types::versioning;
+use meilisearch_types::versioning::{VERSION_MAJOR, VERSION_MINOR, VERSION_PATCH};
 use time::OffsetDateTime;
 use tracing::info;
 
@@ -9,82 +9,79 @@ use crate::queue::TaskQueue;
 use crate::versioning::Versioning;
 
 trait UpgradeIndexScheduler {
-    fn upgrade(&self, env: &Env<WithoutTls>, wtxn: &mut RwTxn) -> anyhow::Result<()>;
+    fn upgrade(
-    /// Whether the migration should be applied, depending on the initial version of the index scheduler before
+        &self,
-    /// any migration was applied
+        env: &Env<WithoutTls>,
-    fn must_upgrade(&self, initial_version: (u32, u32, u32)) -> bool;
+        wtxn: &mut RwTxn,
-    /// A progress-centric description of the migration
+        original: (u32, u32, u32),
-    fn description(&self) -> &'static str;
+    ) -> anyhow::Result<()>;
+    fn target_version(&self) -> (u32, u32, u32);
 }
 
-/// Upgrade the index scheduler to the binary version.
-///
-/// # Warning
-///
-/// The current implementation uses a single wtxn to the index scheduler for the whole duration of the upgrade.
-/// If migrations start taking take a long time, it might prevent tasks from being registered.
-/// If this issue manifests, then it can be mitigated by adding a `fn target_version` to `UpgradeIndexScheduler`,
-/// to be able to write intermediate versions and drop the wtxn between applying migrations.
 pub fn upgrade_index_scheduler(
     env: &Env<WithoutTls>,
     versioning: &Versioning,
-    initial_version: (u32, u32, u32),
+    from: (u32, u32, u32),
+    to: (u32, u32, u32),
 ) -> anyhow::Result<()> {
-    let target_major: u32 = versioning::VERSION_MAJOR;
+    let current_major = to.0;
-    let target_minor: u32 = versioning::VERSION_MINOR;
+    let current_minor = to.1;
-    let target_patch: u32 = versioning::VERSION_PATCH;
+    let current_patch = to.2;
-    let target_version = (target_major, target_minor, target_patch);
-
-    if initial_version == target_version {
-        return Ok(());
-    }
 
     let upgrade_functions: &[&dyn UpgradeIndexScheduler] = &[
-        // List all upgrade functions to apply in order here.
+        // This is the last upgrade function, it will be called when the index is up to date.
+        // any other upgrade function should be added before this one.
+        &ToCurrentNoOp {},
     ];
 
-    let (initial_major, initial_minor, initial_patch) = initial_version;
+    let start = match from {
-
+        (1, 12, _) => 0,
-    if initial_version > target_version {
+        (1, 13, _) => 0,
-        bail!(
+        (1, 14, _) => 0,
-                "Database version {initial_major}.{initial_minor}.{initial_patch} is higher than the Meilisearch version {target_major}.{target_minor}.{target_patch}. Downgrade is not supported",
+        (1, 15, _) => 0,
+        (1, 16, _) => 0,
+        (1, 17, _) => 0,
+        (1, 18, _) => 0,
+        (1, 19, _) => 0,
+        (1, 20, _) => 0,
+        (1, 21, _) => 0,
+        (1, 22, _) => 0,
+        (1, 23, _) => 0,
+        (1, 24, _) => 0,
+        (major, minor, patch) => {
+            if major > current_major
+                || (major == current_major && minor > current_minor)
+                || (major == current_major && minor == current_minor && patch > current_patch)
+            {
+                bail!(
+                "Database version {major}.{minor}.{patch} is higher than the Meilisearch version {current_major}.{current_minor}.{current_patch}. Downgrade is not supported",
+                );
+            } else if major < 1 || (major == current_major && minor < 12) {
+                bail!(
+                "Database version {major}.{minor}.{patch} is too old for the experimental dumpless upgrade feature. Please generate a dump using the v{major}.{minor}.{patch} and import it in the v{current_major}.{current_minor}.{current_patch}",
             );
-    }
+            } else {
-
+                bail!("Unknown database version: v{major}.{minor}.{patch}");
-    if initial_version < (1, 12, 0) {
+            }
-        bail!(
+        }
-                "Database version {initial_major}.{initial_minor}.{initial_patch} is too old for the experimental dumpless upgrade feature. Please generate a dump using the v{initial_major}.{initial_minor}.{initial_patch} and import it in the v{target_major}.{target_minor}.{target_patch}",
+    };
-            );
-    }
 
     info!("Upgrading the task queue");
-    let mut wtxn = env.write_txn()?;
+    let mut local_from = from;
-    let migration_count = upgrade_functions.len();
+    for upgrade in upgrade_functions[start..].iter() {
-    for (migration_index, upgrade) in upgrade_functions.iter().enumerate() {
+        let target = upgrade.target_version();
-        if upgrade.must_upgrade(initial_version) {
+        info!(
-            info!(
+            "Upgrading from v{}.{}.{} to v{}.{}.{}",
-                "[{migration_index}/{migration_count}]Applying migration: {}",
+            local_from.0, local_from.1, local_from.2, target.0, target.1, target.2
-                upgrade.description()
+        );
-            );
+        let mut wtxn = env.write_txn()?;
-
+        upgrade.upgrade(env, &mut wtxn, local_from)?;
-            upgrade.upgrade(env, &mut wtxn)?;
+        versioning.set_version(&mut wtxn, target)?;
-
+        wtxn.commit()?;
-            info!(
+        local_from = target;
-                "[{}/{migration_count}]Migration applied: {}",
-                migration_index + 1,
-                upgrade.description()
-            )
-        } else {
-            info!(
-                "[{migration_index}/{migration_count}]Skipping unnecessary migration: {}",
-                upgrade.description()
-            )
-        }
     }
 
-    versioning.set_version(&mut wtxn, target_version)?;
+    let mut wtxn = env.write_txn()?;
-    info!("Task queue upgraded, spawning the upgrade database task");
-
     let queue = TaskQueue::new(env, &mut wtxn)?;
     let uid = queue.next_task_id(&wtxn)?;
     queue.register(
@@ -97,14 +94,31 @@ pub fn upgrade_index_scheduler(
             finished_at: None,
             error: None,
             canceled_by: None,
-            details: Some(Details::UpgradeDatabase { from: initial_version, to: target_version }),
+            details: Some(Details::UpgradeDatabase { from, to }),
             status: Status::Enqueued,
-            kind: KindWithContent::UpgradeDatabase { from: initial_version },
+            kind: KindWithContent::UpgradeDatabase { from },
             network: None,
-            custom_metadata: None,
         },
     )?;
     wtxn.commit()?;
 
     Ok(())
 }
+
+#[allow(non_camel_case_types)]
+struct ToCurrentNoOp {}
+
+impl UpgradeIndexScheduler for ToCurrentNoOp {
+    fn upgrade(
+        &self,
+        _env: &Env<WithoutTls>,
+        _wtxn: &mut RwTxn,
+        _original: (u32, u32, u32),
+    ) -> anyhow::Result<()> {
+        Ok(())
+    }
+
+    fn target_version(&self) -> (u32, u32, u32) {
+        (VERSION_MAJOR, VERSION_MINOR, VERSION_PATCH)
+    }
+}

crates/index-scheduler/src/utils.rs

@@ -379,7 +379,6 @@ impl crate::IndexScheduler {
                 status,
                 kind,
                 network: _,
-                custom_metadata: _,
             } = task;
             assert_eq!(uid, task.uid);
             if task.status != Status::Enqueued {

crates/index-scheduler/src/versioning.rs

@@ -64,7 +64,14 @@ impl Versioning {
         };
         wtxn.commit()?;
 
-        upgrade_index_scheduler(env, &this, from)?;
+        let bin_major: u32 = versioning::VERSION_MAJOR;
+        let bin_minor: u32 = versioning::VERSION_MINOR;
+        let bin_patch: u32 = versioning::VERSION_PATCH;
+        let to = (bin_major, bin_minor, bin_patch);
+
+        if from != to {
+            upgrade_index_scheduler(env, &this, from, to)?;
+        }
 
         // Once we reach this point it means the upgrade process, if there was one is entirely finished
         // we can safely say we reached the latest version of the index scheduler

crates/json-depth-checker/Cargo.toml

@@ -15,7 +15,7 @@ license.workspace = true
 serde_json = "1.0"
 
 [dev-dependencies]
-criterion = "0.7.0"
+criterion = "0.6.0"
 
 [[bench]]
 name = "depth"

crates/meili-snap/Cargo.toml

@@ -13,7 +13,7 @@ license.workspace = true
 [dependencies]
 # fixed version due to format breakages in v1.40
 insta = { version = "=1.39.0", features = ["json", "redactions"] }
-md5 = "0.8.0"
+md5 = "0.7.0"
 once_cell = "1.21"
-regex-lite = "0.1.8"
+regex-lite = "0.1.6"
-uuid = { version = "1.18.1", features = ["v4"] }
+uuid = { version = "1.17.0", features = ["v4"] }

crates/meilisearch-auth/Cargo.toml

@@ -12,15 +12,15 @@ license.workspace = true
 
 [dependencies]
 base64 = "0.22.1"
-enum-iterator = "2.3.0"
+enum-iterator = "2.1.0"
 hmac = "0.12.1"
 maplit = "1.0.2"
 meilisearch-types = { path = "../meilisearch-types" }
 rand = "0.8.5"
 roaring = { version = "0.10.12", features = ["serde"] }
-serde = { version = "1.0.228", features = ["derive"] }
+serde = { version = "1.0.219", features = ["derive"] }
-serde_json = { version = "1.0.145", features = ["preserve_order"] }
+serde_json = { version = "1.0.140", features = ["preserve_order"] }
 sha2 = "0.10.9"
-thiserror = "2.0.17"
+thiserror = "2.0.12"
-time = { version = "0.3.44", features = ["serde-well-known", "formatting", "parsing", "macros"] }
+time = { version = "0.3.41", features = ["serde-well-known", "formatting", "parsing", "macros"] }
-uuid = { version = "1.18.1", features = ["serde", "v4"] }
+uuid = { version = "1.17.0", features = ["serde", "v4"] }

crates/meilisearch-types/Cargo.toml

@@ -11,38 +11,38 @@ edition.workspace = true
 license.workspace = true
 
 [dependencies]
-actix-web = { version = "4.12.0", default-features = false }
+actix-web = { version = "4.11.0", default-features = false }
-anyhow = "1.0.100"
+anyhow = "1.0.98"
-bumpalo = "3.19.0"
+bumpalo = "3.18.1"
 bumparaw-collections = "0.1.4"
 byte-unit = { version = "5.1.6", features = ["serde"] }
-convert_case = "0.9.0"
+convert_case = "0.8.0"
-csv = "1.4.0"
+csv = "1.3.1"
-deserr = { version = "0.6.4", features = ["actix-web"] }
+deserr = { version = "0.6.3", features = ["actix-web"] }
 either = { version = "1.15.0", features = ["serde"] }
-enum-iterator = "2.3.0"
+enum-iterator = "2.1.0"
 file-store = { path = "../file-store" }
-flate2 = "1.1.5"
+flate2 = "1.1.2"
 fst = "0.4.7"
-memmap2 = "0.9.9"
+memmap2 = "0.9.7"
 milli = { path = "../milli" }
 roaring = { version = "0.10.12", features = ["serde"] }
 rustc-hash = "2.1.1"
-serde = { version = "1.0.228", features = ["derive"] }
+serde = { version = "1.0.219", features = ["derive"] }
 serde-cs = "0.2.4"
-serde_json = { version = "1.0.145", features = ["preserve_order"] }
+serde_json = { version = "1.0.140", features = ["preserve_order"] }
 tar = "0.4.44"
-tempfile = "3.23.0"
+tempfile = "3.20.0"
-thiserror = "2.0.17"
+thiserror = "2.0.12"
-time = { version = "0.3.44", features = [
+time = { version = "0.3.41", features = [
     "serde-well-known",
     "formatting",
     "parsing",
     "macros",
 ] }
-tokio = "1.48"
+tokio = "1.45"
 utoipa = { version = "5.4.0", features = ["macros"] }
-uuid = { version = "1.18.1", features = ["serde", "v4"] }
+uuid = { version = "1.17.0", features = ["serde", "v4"] }
 
 [dev-dependencies]
 # fixed version due to format breakages in v1.40
@@ -56,9 +56,6 @@ all-tokenizations = ["milli/all-tokenizations"]
 # chinese specialized tokenization
 chinese = ["milli/chinese"]
 chinese-pinyin = ["milli/chinese-pinyin"]
-
-enterprise = ["milli/enterprise"]
-
 # hebrew specialized tokenization
 hebrew = ["milli/hebrew"]
 # japanese specialized tokenization

crates/meilisearch-types/src/community_edition.rs

@@ -1,16 +0,0 @@
-pub mod network {
-    use milli::update::new::indexer::current_edition::sharding::Shards;
-
-    use crate::network::Network;
-
-    impl Network {
-        pub fn shards(&self) -> Option<Shards> {
-            None
-        }
-
-        pub fn sharding(&self) -> bool {
-            // always false in CE
-            false
-        }
-    }
-}

crates/meilisearch-types/src/enterprise_edition/network.rs

@@ -3,9 +3,21 @@
 // Use of this source code is governed by the Business Source License 1.1,
 // as found in the LICENSE-EE file or at <https://mariadb.com/bsl11>
 
-use milli::update::new::indexer::enterprise_edition::sharding::Shards;
+use std::collections::BTreeMap;
 
-use crate::network::Network;
+use milli::update::new::indexer::enterprise_edition::sharding::Shards;
+use serde::{Deserialize, Serialize};
+
+#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq, Default)]
+#[serde(rename_all = "camelCase")]
+pub struct Network {
+    #[serde(default, rename = "self")]
+    pub local: Option<String>,
+    #[serde(default)]
+    pub remotes: BTreeMap<String, Remote>,
+    #[serde(default)]
+    pub sharding: bool,
+}
 
 impl Network {
     pub fn shards(&self) -> Option<Shards> {
@@ -22,8 +34,14 @@ impl Network {
             None
         }
     }
-
+}
-    pub fn sharding(&self) -> bool {
+
-        self.sharding
+#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
-    }
+#[serde(rename_all = "camelCase")]
+pub struct Remote {
+    pub url: String,
+    #[serde(default)]
+    pub search_api_key: Option<String>,
+    #[serde(default)]
+    pub write_api_key: Option<String>,
 }

crates/meilisearch-types/src/error.rs

@@ -254,12 +254,10 @@ InvalidSearchHybridQuery                       , InvalidRequest       , BAD_REQU
 InvalidIndexLimit                              , InvalidRequest       , BAD_REQUEST ;
 InvalidIndexOffset                             , InvalidRequest       , BAD_REQUEST ;
 InvalidIndexPrimaryKey                         , InvalidRequest       , BAD_REQUEST ;
-InvalidIndexCustomMetadata                     , InvalidRequest       , BAD_REQUEST ;
 InvalidIndexUid                                , InvalidRequest       , BAD_REQUEST ;
 InvalidMultiSearchFacets                       , InvalidRequest       , BAD_REQUEST ;
 InvalidMultiSearchFacetsByIndex                , InvalidRequest       , BAD_REQUEST ;
 InvalidMultiSearchFacetOrder                   , InvalidRequest       , BAD_REQUEST ;
-InvalidMultiSearchQueryPersonalization         , InvalidRequest       , BAD_REQUEST ;
 InvalidMultiSearchFederated                    , InvalidRequest       , BAD_REQUEST ;
 InvalidMultiSearchFederationOptions            , InvalidRequest       , BAD_REQUEST ;
 InvalidMultiSearchMaxValuesPerFacet            , InvalidRequest       , BAD_REQUEST ;
@@ -317,8 +315,6 @@ InvalidSearchShowRankingScoreDetails           , InvalidRequest       , BAD_REQU
 InvalidSimilarShowRankingScoreDetails          , InvalidRequest       , BAD_REQUEST ;
 InvalidSearchSort                              , InvalidRequest       , BAD_REQUEST ;
 InvalidSearchDistinct                          , InvalidRequest       , BAD_REQUEST ;
-InvalidSearchPersonalize                       , InvalidRequest       , BAD_REQUEST ;
-InvalidSearchPersonalizeUserContext            , InvalidRequest       , BAD_REQUEST ;
 InvalidSearchMediaAndVector                    , InvalidRequest       , BAD_REQUEST ;
 InvalidSettingsDisplayedAttributes             , InvalidRequest       , BAD_REQUEST ;
 InvalidSettingsDistinctAttribute               , InvalidRequest       , BAD_REQUEST ;
@@ -394,9 +390,6 @@ TooManyVectors                                 , InvalidRequest       , BAD_REQU
 UnretrievableDocument                          , Internal             , BAD_REQUEST ;
 UnretrievableErrorCode                         , InvalidRequest       , BAD_REQUEST ;
 UnsupportedMediaType                           , InvalidRequest       , UNSUPPORTED_MEDIA_TYPE ;
-InvalidS3SnapshotRequest                       , Internal             , BAD_REQUEST ;
-InvalidS3SnapshotParameters                    , Internal             , BAD_REQUEST ;
-S3SnapshotServerError                          , Internal             , BAD_GATEWAY ;
 
 // Experimental features
 VectorEmbeddingError                           , InvalidRequest       , BAD_REQUEST ;
@@ -433,7 +426,6 @@ InvalidChatCompletionSearchQueryParamPrompt    , InvalidRequest       , BAD_REQU
 InvalidChatCompletionSearchFilterParamPrompt   , InvalidRequest       , BAD_REQUEST ;
 InvalidChatCompletionSearchIndexUidParamPrompt , InvalidRequest       , BAD_REQUEST ;
 InvalidChatCompletionPreQueryPrompt            , InvalidRequest       , BAD_REQUEST ;
-RequiresEnterpriseEdition                      , InvalidRequest       , UNAVAILABLE_FOR_LEGAL_REASONS ;
 // Webhooks
 InvalidWebhooks                                , InvalidRequest       , BAD_REQUEST ;
 InvalidWebhookUrl                              , InvalidRequest       , BAD_REQUEST ;
@@ -687,18 +679,6 @@ impl fmt::Display for deserr_codes::InvalidNetworkSearchApiKey {
     }
 }
 
-impl fmt::Display for deserr_codes::InvalidSearchPersonalize {
-    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        write!(f, "the value of `personalize` is invalid, expected a JSON object with `userContext` string.")
-    }
-}
-
-impl fmt::Display for deserr_codes::InvalidSearchPersonalizeUserContext {
-    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        write!(f, "the value of `userContext` is invalid, expected a string.")
-    }
-}
-
 #[macro_export]
 macro_rules! internal_error {
     ($target:ty : $($other:path), *) => {

crates/meilisearch-types/src/lib.rs

@@ -2,17 +2,10 @@
 
 pub mod batch_view;
 pub mod batches;
-#[cfg(not(feature = "enterprise"))]
-pub mod community_edition;
 pub mod compression;
 pub mod deserr;
 pub mod document_formats;
-#[cfg(feature = "enterprise")]
 pub mod enterprise_edition;
-#[cfg(not(feature = "enterprise"))]
-pub use community_edition as current_edition;
-#[cfg(feature = "enterprise")]
-pub use enterprise_edition as current_edition;
 pub mod error;
 pub mod facet_values_sort;
 pub mod features;
@@ -20,7 +13,6 @@ pub mod index_uid;
 pub mod index_uid_pattern;
 pub mod keys;
 pub mod locales;
-pub mod network;
 pub mod settings;
 pub mod star_or;
 pub mod task_view;

crates/meilisearch-types/src/network.rs

@@ -1,24 +0,0 @@
-use std::collections::BTreeMap;
-
-use serde::{Deserialize, Serialize};
-
-#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq, Default)]
-#[serde(rename_all = "camelCase")]
-pub struct Network {
-    #[serde(default, rename = "self")]
-    pub local: Option<String>,
-    #[serde(default)]
-    pub remotes: BTreeMap<String, Remote>,
-    #[serde(default)]
-    pub sharding: bool,
-}
-
-#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
-#[serde(rename_all = "camelCase")]
-pub struct Remote {
-    pub url: String,
-    #[serde(default)]
-    pub search_api_key: Option<String>,
-    #[serde(default)]
-    pub write_api_key: Option<String>,
-}

crates/meilisearch-types/src/settings.rs

@@ -346,26 +346,24 @@ impl<T> Settings<T> {
                 continue;
             };
 
-            hide_secret(api_key, 0);
+            Self::hide_secret(api_key);
         }
     }
-}
 
-/// Redact a secret string, starting from the `secret_offset`th byte.
+    fn hide_secret(secret: &mut String) {
-pub fn hide_secret(secret: &mut String, secret_offset: usize) {
+        match secret.len() {
-    match secret.len().checked_sub(secret_offset) {
+            x if x < 10 => {
-        None => (),
+                secret.replace_range(.., "XXX...");
-        Some(x) if x < 10 => {
+            }
-            secret.replace_range(secret_offset.., "XXX...");
+            x if x < 20 => {
-        }
+                secret.replace_range(2.., "XXXX...");
-        Some(x) if x < 20 => {
+            }
-            secret.replace_range((secret_offset + 2).., "XXXX...");
+            x if x < 30 => {
-        }
+                secret.replace_range(3.., "XXXXX...");
-        Some(x) if x < 30 => {
+            }
-            secret.replace_range((secret_offset + 3).., "XXXXX...");
+            _x => {
-        }
+                secret.replace_range(5.., "XXXXXX...");
-        Some(_x) => {
+            }
-            secret.replace_range((secret_offset + 5).., "XXXXXX...");
         }
     }
 }

crates/meilisearch-types/src/task_view.rs

@@ -55,9 +55,6 @@ pub struct TaskView {
 
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub network: Option<TaskNetwork>,
-
-    #[serde(default, skip_serializing_if = "Option::is_none")]
-    pub custom_metadata: Option<String>,
 }
 
 impl TaskView {
@@ -76,7 +73,6 @@ impl TaskView {
             started_at: task.started_at,
             finished_at: task.finished_at,
             network: task.network.clone(),
-            custom_metadata: task.custom_metadata.clone(),
         }
     }
 }

crates/meilisearch-types/src/tasks.rs

@@ -45,9 +45,6 @@ pub struct Task {
 
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub network: Option<TaskNetwork>,
-
-    #[serde(default, skip_serializing_if = "Option::is_none")]
-    pub custom_metadata: Option<String>,
 }
 
 impl Task {

crates/meilisearch-types/src/webhooks.rs

@@ -11,24 +11,6 @@ pub struct Webhook {
     pub headers: BTreeMap<String, String>,
 }
 
-impl Webhook {
-    pub fn redact_authorization_header(&mut self) {
-        // headers are case insensitive, so to make the redaction robust we iterate over qualifying headers
-        // rather than getting one canonical `Authorization` header.
-        for value in self
-            .headers
-            .iter_mut()
-            .filter_map(|(name, value)| name.eq_ignore_ascii_case("authorization").then_some(value))
-        {
-            if value.starts_with("Bearer ") {
-                crate::settings::hide_secret(value, "Bearer ".len());
-            } else {
-                crate::settings::hide_secret(value, 0);
-            }
-        }
-    }
-}
-
 #[derive(Debug, Serialize, Default, Clone, PartialEq)]
 #[serde(rename_all = "camelCase")]
 pub struct WebhooksView {

crates/meilisearch/Cargo.toml

@@ -14,91 +14,91 @@ default-run = "meilisearch"
 
 [dependencies]
 actix-cors = "0.7.1"
-actix-http = { version = "3.11.2", default-features = false, features = [
+actix-http = { version = "3.11.0", default-features = false, features = [
     "compress-brotli",
     "compress-gzip",
     "rustls-0_23",
 ] }
 actix-utils = "3.0.1"
-actix-web = { version = "4.12.0", default-features = false, features = [
+actix-web = { version = "4.11.0", default-features = false, features = [
     "macros",
     "compress-brotli",
     "compress-gzip",
     "cookies",
     "rustls-0_23",
 ] }
-anyhow = { version = "1.0.100", features = ["backtrace"] }
+anyhow = { version = "1.0.98", features = ["backtrace"] }
-bstr = "1.12.1"
+bstr = "1.12.0"
 byte-unit = { version = "5.1.6", features = ["serde"] }
-bytes = "1.11.0"
+bytes = "1.10.1"
-bumpalo = "3.19.0"
+bumpalo = "3.18.1"
-clap = { version = "4.5.52", features = ["derive", "env"] }
+clap = { version = "4.5.40", features = ["derive", "env"] }
 crossbeam-channel = "0.5.15"
-deserr = { version = "0.6.4", features = ["actix-web"] }
+deserr = { version = "0.6.3", features = ["actix-web"] }
 dump = { path = "../dump" }
 either = "1.15.0"
 file-store = { path = "../file-store" }
-flate2 = "1.1.5"
+flate2 = "1.1.2"
 fst = "0.4.7"
 futures = "0.3.31"
 futures-util = "0.3.31"
 index-scheduler = { path = "../index-scheduler" }
-indexmap = { version = "2.12.0", features = ["serde"] }
+indexmap = { version = "2.9.0", features = ["serde"] }
-is-terminal = "0.4.17"
+is-terminal = "0.4.16"
 itertools = "0.14.0"
 jsonwebtoken = "9.3.1"
 lazy_static = "1.5.0"
 meilisearch-auth = { path = "../meilisearch-auth" }
 meilisearch-types = { path = "../meilisearch-types" }
-memmap2 = "0.9.9"
+memmap2 = "0.9.7"
-mimalloc = { version = "0.1.48", default-features = false }
+mimalloc = { version = "0.1.47", default-features = false }
 mime = "0.3.17"
 num_cpus = "1.17.0"
 obkv = "0.3.0"
 once_cell = "1.21.3"
-ordered-float = "5.1.0"
+ordered-float = "5.0.0"
-parking_lot = "0.12.5"
+parking_lot = "0.12.4"
 permissive-json-pointer = { path = "../permissive-json-pointer" }
 pin-project-lite = "0.2.16"
 platform-dirs = "0.3.0"
 prometheus = { version = "0.14.0", features = ["process"] }
 rand = "0.8.5"
-rayon = "1.11.0"
+rayon = "1.10.0"
-regex = "1.12.2"
+regex = "1.11.1"
-reqwest = { version = "0.12.24", features = [
+reqwest = { version = "0.12.20", features = [
     "rustls-tls",
     "json",
 ], default-features = false }
-rustls = { version = "0.23.35", features = ["ring"], default-features = false }
+rustls = { version = "0.23.28", features = ["ring"], default-features = false }
-rustls-pki-types = { version = "1.13.0", features = ["alloc"] }
+rustls-pki-types = { version = "1.12.0", features = ["alloc"] }
 rustls-pemfile = "2.2.0"
 segment = { version = "0.2.6" }
-serde = { version = "1.0.228", features = ["derive"] }
+serde = { version = "1.0.219", features = ["derive"] }
-serde_json = { version = "1.0.145", features = ["preserve_order"] }
+serde_json = { version = "1.0.140", features = ["preserve_order"] }
 sha2 = "0.10.9"
 siphasher = "1.0.1"
 slice-group-by = "0.3.1"
-static-files = { version = "0.3.1", optional = true }
+static-files = { version = "0.2.5", optional = true }
-sysinfo = "0.37.2"
+sysinfo = "0.35.2"
 tar = "0.4.44"
-tempfile = "3.23.0"
+tempfile = "3.20.0"
-thiserror = "2.0.17"
+thiserror = "2.0.12"
-time = { version = "0.3.44", features = [
+time = { version = "0.3.41", features = [
     "serde-well-known",
     "formatting",
     "parsing",
     "macros",
 ] }
-tokio = { version = "1.48.0", features = ["full"] }
+tokio = { version = "1.45.1", features = ["full"] }
-toml = "0.9.8"
+toml = "0.8.23"
-uuid = { version = "1.18.1", features = ["serde", "v4", "v7"] }
+uuid = { version = "1.18.0", features = ["serde", "v4", "v7"] }
 serde_urlencoded = "0.7.1"
 termcolor = "1.4.1"
-url = { version = "2.5.7", features = ["serde"] }
+url = { version = "2.5.4", features = ["serde"] }
 tracing = "0.1.41"
 tracing-subscriber = { version = "0.3.20", features = ["json"] }
 tracing-trace = { version = "0.1.0", path = "../tracing-trace" }
-tracing-actix-web = "0.7.19"
+tracing-actix-web = "0.7.18"
 build-info = { version = "1.7.0", path = "../build-info" }
 roaring = "0.10.12"
 mopa-maintained = "0.2.3"
@@ -114,35 +114,35 @@ utoipa = { version = "5.4.0", features = [
 utoipa-scalar = { version = "0.3.0", optional = true, features = ["actix-web"] }
 async-openai = { git = "https://github.com/meilisearch/async-openai", branch = "better-error-handling" }
 secrecy = "0.10.3"
-actix-web-lab = { version = "0.24.3", default-features = false }
+actix-web-lab = { version = "0.24.1", default-features = false }
 urlencoding = "2.1.3"
 backoff = { version = "0.4.0", features = ["tokio"] }
-humantime = { version = "2.3.0", default-features = false }
+
 
 [dev-dependencies]
-actix-rt = "2.11.0"
+actix-rt = "2.10.0"
-brotli = "8.0.2"
+brotli = "8.0.1"
 # fixed version due to format breakages in v1.40
 insta = { version = "=1.39.0", features = ["redactions"] }
 manifest-dir-macros = "0.1.18"
 maplit = "1.0.2"
 meili-snap = { path = "../meili-snap" }
 temp-env = "0.3.6"
-wiremock = "0.6.5"
+wiremock = "0.6.3"
 yaup = "0.3.1"
 
 [build-dependencies]
-anyhow = { version = "1.0.100", optional = true }
+anyhow = { version = "1.0.98", optional = true }
-cargo_toml = { version = "0.22.3", optional = true }
+cargo_toml = { version = "0.22.1", optional = true }
 hex = { version = "0.4.3", optional = true }
-reqwest = { version = "0.12.24", features = [
+reqwest = { version = "0.12.20", features = [
     "blocking",
     "rustls-tls",
 ], default-features = false, optional = true }
 sha-1 = { version = "0.10.1", optional = true }
-static-files = { version = "0.3.1", optional = true }
+static-files = { version = "0.2.5", optional = true }
-tempfile = { version = "3.23.0", optional = true }
+tempfile = { version = "3.20.0", optional = true }
-zip = { version = "6.0.0", optional = true }
+zip = { version = "4.1.0", optional = true }
 
 [features]
 default = ["meilisearch-types/all-tokenizations", "mini-dashboard"]
@@ -160,7 +160,6 @@ mini-dashboard = [
 ]
 chinese = ["meilisearch-types/chinese"]
 chinese-pinyin = ["meilisearch-types/chinese-pinyin"]
-enterprise = ["meilisearch-types/enterprise"]
 hebrew = ["meilisearch-types/hebrew"]
 japanese = ["meilisearch-types/japanese"]
 korean = ["meilisearch-types/korean"]

crates/meilisearch/src/analytics/segment_analytics.rs

@@ -195,7 +195,7 @@ struct Infos {
     experimental_enable_logs_route: bool,
     experimental_reduce_indexing_memory_usage: bool,
     experimental_max_number_of_batched_tasks: usize,
-    experimental_limit_batched_tasks_total_size: Option<u64>,
+    experimental_limit_batched_tasks_total_size: u64,
     experimental_network: bool,
     experimental_multimodal: bool,
     experimental_chat_completions: bool,
@@ -208,7 +208,6 @@ struct Infos {
     experimental_no_edition_2024_for_prefix_post_processing: bool,
     experimental_no_edition_2024_for_facet_post_processing: bool,
     experimental_vector_store_setting: bool,
-    experimental_personalization: bool,
     gpu_enabled: bool,
     db_path: bool,
     import_dump: bool,
@@ -218,7 +217,6 @@ struct Infos {
     import_snapshot: bool,
     schedule_snapshot: Option<u64>,
     snapshot_dir: bool,
-    uses_s3_snapshots: bool,
     ignore_missing_snapshot: bool,
     ignore_snapshot_if_db_exists: bool,
     http_addr: bool,
@@ -287,8 +285,6 @@ impl Infos {
             indexer_options,
             config_file_path,
             no_analytics: _,
-            experimental_personalization_api_key,
-            s3_snapshot_options,
         } = options;
 
         let schedule_snapshot = match schedule_snapshot {
@@ -352,14 +348,13 @@ impl Infos {
             import_snapshot: import_snapshot.is_some(),
             schedule_snapshot,
             snapshot_dir: snapshot_dir != PathBuf::from("snapshots/"),
-            uses_s3_snapshots: s3_snapshot_options.is_some(),
             ignore_missing_snapshot,
             ignore_snapshot_if_db_exists,
             http_addr: http_addr != default_http_addr(),
             http_payload_size_limit,
             experimental_max_number_of_batched_tasks,
             experimental_limit_batched_tasks_total_size:
-                experimental_limit_batched_tasks_total_size.map(|size| size.as_u64()),
+                experimental_limit_batched_tasks_total_size.into(),
             task_queue_webhook: task_webhook_url.is_some(),
             task_webhook_authorization_header: task_webhook_authorization_header.is_some(),
             log_level: log_level.to_string(),
@@ -376,7 +371,6 @@ impl Infos {
             experimental_no_edition_2024_for_settings,
             experimental_no_edition_2024_for_prefix_post_processing,
             experimental_no_edition_2024_for_facet_post_processing,
-            experimental_personalization: experimental_personalization_api_key.is_some(),
         }
     }
 }

crates/meilisearch/src/error.rs

@@ -38,8 +38,6 @@ pub enum MeilisearchHttpError {
     PaginationInFederatedQuery(usize, &'static str),
     #[error("Inside `.queries[{0}]`: Using facet options is not allowed in federated queries.\n - Hint: remove `facets` from query #{0} or remove `federation` from the request\n - Hint: pass `federation.facetsByIndex.{1}: {2:?}` for facets in federated search")]
     FacetsInFederatedQuery(usize, String, Vec<String>),
-    #[error("Inside `.queries[{0}]`: Using `.personalize` is not allowed in federated queries.\n - Hint: remove `personalize` from query #{0} or remove `federation` from the request")]
-    PersonalizationInFederatedQuery(usize),
     #[error("Inconsistent order for values in facet `{facet}`: index `{previous_uid}` orders {previous_facet_order}, but index `{current_uid}` orders {index_facet_order}.\n - Hint: Remove `federation.mergeFacets` or change `faceting.sortFacetValuesBy` to be consistent in settings.")]
     InconsistentFacetOrder {
         facet: String,
@@ -139,9 +137,6 @@ impl ErrorCode for MeilisearchHttpError {
             MeilisearchHttpError::InconsistentFacetOrder { .. } => {
                 Code::InvalidMultiSearchFacetOrder
             }
-            MeilisearchHttpError::PersonalizationInFederatedQuery(_) => {
-                Code::InvalidMultiSearchQueryPersonalization
-            }
             MeilisearchHttpError::InconsistentOriginHeaders { .. } => {
                 Code::InconsistentDocumentChangeHeaders
             }

crates/meilisearch/src/lib.rs

@@ -11,7 +11,6 @@ pub mod middleware;
 pub mod option;
 #[cfg(test)]
 mod option_test;
-pub mod personalization;
 pub mod routes;
 pub mod search;
 pub mod search_queue;
@@ -59,7 +58,6 @@ use tracing::{error, info_span};
 use tracing_subscriber::filter::Targets;
 
 use crate::error::MeilisearchHttpError;
-use crate::personalization::PersonalizationService;
 
 /// Default number of simultaneously opened indexes.
 ///
@@ -130,8 +128,12 @@ pub type LogStderrType = tracing_subscriber::filter::Filtered<
 >;
 
 pub fn create_app(
-    services: ServicesData,
+    index_scheduler: Data<IndexScheduler>,
+    auth_controller: Data<AuthController>,
+    search_queue: Data<SearchQueue>,
     opt: Opt,
+    logs: (LogRouteHandle, LogStderrHandle),
+    analytics: Data<Analytics>,
     enable_dashboard: bool,
 ) -> actix_web::App<
     impl ServiceFactory<
@@ -143,7 +145,17 @@ pub fn create_app(
     >,
 > {
     let app = actix_web::App::new()
-        .configure(|s| configure_data(s, services, &opt))
+        .configure(|s| {
+            configure_data(
+                s,
+                index_scheduler.clone(),
+                auth_controller.clone(),
+                search_queue.clone(),
+                &opt,
+                logs,
+                analytics.clone(),
+            )
+        })
         .configure(routes::configure)
         .configure(|s| dashboard(s, enable_dashboard));
 
@@ -204,10 +216,7 @@ enum OnFailure {
     KeepDb,
 }
 
-pub fn setup_meilisearch(
+pub fn setup_meilisearch(opt: &Opt) -> anyhow::Result<(Arc<IndexScheduler>, Arc<AuthController>)> {
-    opt: &Opt,
-    handle: tokio::runtime::Handle,
-) -> anyhow::Result<(Arc<IndexScheduler>, Arc<AuthController>)> {
     let index_scheduler_opt = IndexSchedulerOptions {
         version_file_path: opt.db_path.join(VERSION_FILE_NAME),
         auth_path: opt.db_path.join("auth"),
@@ -221,26 +230,12 @@ pub fn setup_meilisearch(
         task_db_size: opt.max_task_db_size.as_u64() as usize,
         index_base_map_size: opt.max_index_size.as_u64() as usize,
         enable_mdb_writemap: opt.experimental_reduce_indexing_memory_usage,
-        indexer_config: Arc::new({
+        indexer_config: Arc::new((&opt.indexer_options).try_into()?),
-            let s3_snapshot_options =
-                opt.s3_snapshot_options.clone().map(|opt| opt.try_into()).transpose()?;
-            IndexerConfig { s3_snapshot_options, ..(&opt.indexer_options).try_into()? }
-        }),
         autobatching_enabled: true,
         cleanup_enabled: !opt.experimental_replication_parameters,
         max_number_of_tasks: 1_000_000,
         max_number_of_batched_tasks: opt.experimental_max_number_of_batched_tasks,
-        batched_tasks_size_limit: opt.experimental_limit_batched_tasks_total_size.map_or_else(
+        batched_tasks_size_limit: opt.experimental_limit_batched_tasks_total_size.into(),
-            || {
-                opt.indexer_options
-                    .max_indexing_memory
-                    // By default, we use half of the available memory to determine the size of batched tasks
-                    .map_or(u64::MAX, |mem| mem.as_u64() / 2)
-                    // And never exceed 10 GiB when we infer the limit
-                    .min(10 * 1024 * 1024 * 1024)
-            },
-            |size| size.as_u64(),
-        ),
         index_growth_amount: byte_unit::Byte::from_str("10GiB").unwrap().as_u64() as usize,
         index_count: DEFAULT_INDEX_COUNT,
         instance_features: opt.to_instance_features(),
@@ -261,7 +256,6 @@ pub fn setup_meilisearch(
                     index_scheduler_opt,
                     OnFailure::RemoveDb,
                     binary_version, // the db is empty
-                    handle,
                 )?,
                 Err(e) => {
                     std::fs::remove_dir_all(&opt.db_path)?;
@@ -279,7 +273,7 @@ pub fn setup_meilisearch(
             bail!("snapshot doesn't exist at {}", snapshot_path.display())
         // the snapshot and the db exist, and we can ignore the snapshot because of the ignore_snapshot_if_db_exists flag
         } else {
-            open_or_create_database(opt, index_scheduler_opt, empty_db, binary_version, handle)?
+            open_or_create_database(opt, index_scheduler_opt, empty_db, binary_version)?
         }
     } else if let Some(ref path) = opt.import_dump {
         let src_path_exists = path.exists();
@@ -290,7 +284,6 @@ pub fn setup_meilisearch(
                 index_scheduler_opt,
                 OnFailure::RemoveDb,
                 binary_version, // the db is empty
-                handle,
             )?;
             match import_dump(&opt.db_path, path, &mut index_scheduler, &mut auth_controller) {
                 Ok(()) => (index_scheduler, auth_controller),
@@ -311,10 +304,10 @@ pub fn setup_meilisearch(
         // the dump and the db exist and we can ignore the dump because of the ignore_dump_if_db_exists flag
         // or, the dump is missing but we can ignore that because of the ignore_missing_dump flag
         } else {
-            open_or_create_database(opt, index_scheduler_opt, empty_db, binary_version, handle)?
+            open_or_create_database(opt, index_scheduler_opt, empty_db, binary_version)?
         }
     } else {
-        open_or_create_database(opt, index_scheduler_opt, empty_db, binary_version, handle)?
+        open_or_create_database(opt, index_scheduler_opt, empty_db, binary_version)?
     };
 
     // We create a loop in a thread that registers snapshotCreation tasks
@@ -345,7 +338,6 @@ fn open_or_create_database_unchecked(
     index_scheduler_opt: IndexSchedulerOptions,
     on_failure: OnFailure,
     version: (u32, u32, u32),
-    handle: tokio::runtime::Handle,
 ) -> anyhow::Result<(IndexScheduler, AuthController)> {
     // we don't want to create anything in the data.ms yet, thus we
     // wrap our two builders in a closure that'll be executed later.
@@ -353,7 +345,7 @@ fn open_or_create_database_unchecked(
     let auth_env = open_auth_store_env(&index_scheduler_opt.auth_path).unwrap();
     let auth_controller = AuthController::new(auth_env.clone(), &opt.master_key);
     let index_scheduler_builder = || -> anyhow::Result<_> {
-        Ok(IndexScheduler::new(index_scheduler_opt, auth_env, version, Some(handle))?)
+        Ok(IndexScheduler::new(index_scheduler_opt, auth_env, version)?)
     };
 
     match (
@@ -460,7 +452,6 @@ fn open_or_create_database(
     index_scheduler_opt: IndexSchedulerOptions,
     empty_db: bool,
     binary_version: (u32, u32, u32),
-    handle: tokio::runtime::Handle,
 ) -> anyhow::Result<(IndexScheduler, AuthController)> {
     let version = if !empty_db {
         check_version(opt, &index_scheduler_opt, binary_version)?
@@ -468,7 +459,7 @@ fn open_or_create_database(
         binary_version
     };
 
-    open_or_create_database_unchecked(opt, index_scheduler_opt, OnFailure::KeepDb, version, handle)
+    open_or_create_database_unchecked(opt, index_scheduler_opt, OnFailure::KeepDb, version)
 }
 
 fn import_dump(
@@ -536,11 +527,7 @@ fn import_dump(
     let indexer_config = if base_config.max_threads.is_none() {
         let (thread_pool, _) = default_thread_pool_and_threads();
 
-        let _config = IndexerConfig {
+        let _config = IndexerConfig { thread_pool, ..*base_config };
-            thread_pool,
-            s3_snapshot_options: base_config.s3_snapshot_options.clone(),
-            ..*base_config
-        };
         backup_config = _config;
         &backup_config
     } else {
@@ -688,26 +675,23 @@ fn import_dump(
     Ok(index_scheduler_dump.finish()?)
 }
 
-pub fn configure_data(config: &mut web::ServiceConfig, services: ServicesData, opt: &Opt) {
+pub fn configure_data(
-    let ServicesData {
+    config: &mut web::ServiceConfig,
-        index_scheduler,
+    index_scheduler: Data<IndexScheduler>,
-        auth,
+    auth: Data<AuthController>,
-        search_queue,
+    search_queue: Data<SearchQueue>,
-        personalization_service,
+    opt: &Opt,
-        logs_route_handle,
+    (logs_route, logs_stderr): (LogRouteHandle, LogStderrHandle),
-        logs_stderr_handle,
+    analytics: Data<Analytics>,
-        analytics,
+) {
-    } = services;
-
     let http_payload_size_limit = opt.http_payload_size_limit.as_u64() as usize;
     config
         .app_data(index_scheduler)
         .app_data(auth)
         .app_data(search_queue)
         .app_data(analytics)
-        .app_data(personalization_service)
+        .app_data(web::Data::new(logs_route))
-        .app_data(logs_route_handle)
+        .app_data(web::Data::new(logs_stderr))
-        .app_data(logs_stderr_handle)
         .app_data(web::Data::new(opt.clone()))
         .app_data(
             web::JsonConfig::default()
@@ -768,14 +752,3 @@ pub fn dashboard(config: &mut web::ServiceConfig, enable_frontend: bool) {
 pub fn dashboard(config: &mut web::ServiceConfig, _enable_frontend: bool) {
     config.service(web::resource("/").route(web::get().to(routes::running)));
 }
-
-#[derive(Clone)]
-pub struct ServicesData {
-    pub index_scheduler: Data<IndexScheduler>,
-    pub auth: Data<AuthController>,
-    pub search_queue: Data<SearchQueue>,
-    pub personalization_service: Data<PersonalizationService>,
-    pub logs_route_handle: Data<LogRouteHandle>,
-    pub logs_stderr_handle: Data<LogStderrHandle>,
-    pub analytics: Data<Analytics>,
-}

crates/meilisearch/src/main.rs

@@ -14,11 +14,10 @@ use index_scheduler::IndexScheduler;
 use is_terminal::IsTerminal;
 use meilisearch::analytics::Analytics;
 use meilisearch::option::LogMode;
-use meilisearch::personalization::PersonalizationService;
 use meilisearch::search_queue::SearchQueue;
 use meilisearch::{
     analytics, create_app, setup_meilisearch, LogRouteHandle, LogRouteType, LogStderrHandle,
-    LogStderrType, Opt, ServicesData, SubscriberForSecondLayer,
+    LogStderrType, Opt, SubscriberForSecondLayer,
 };
 use meilisearch_auth::{generate_master_key, AuthController, MASTER_KEY_MIN_SIZE};
 use termcolor::{Color, ColorChoice, ColorSpec, StandardStream, WriteColor};
@@ -77,10 +76,7 @@ fn on_panic(info: &std::panic::PanicHookInfo) {
 
 #[actix_web::main]
 async fn main() -> anyhow::Result<()> {
-    // won't panic inside of tokio::main
+    try_main().await.inspect_err(|error| {
-    let runtime = tokio::runtime::Handle::current();
-
-    try_main(runtime).await.inspect_err(|error| {
         tracing::error!(%error);
         let mut current = error.source();
         let mut depth = 0;
@@ -92,7 +88,7 @@ async fn main() -> anyhow::Result<()> {
     })
 }
 
-async fn try_main(runtime: tokio::runtime::Handle) -> anyhow::Result<()> {
+async fn try_main() -> anyhow::Result<()> {
     let (opt, config_read_from) = Opt::try_build()?;
 
     std::panic::set_hook(Box::new(on_panic));
@@ -126,7 +122,7 @@ async fn try_main(runtime: tokio::runtime::Handle) -> anyhow::Result<()> {
         _ => (),
     }
 
-    let (index_scheduler, auth_controller) = setup_meilisearch(&opt, runtime)?;
+    let (index_scheduler, auth_controller) = setup_meilisearch(&opt)?;
 
     let analytics =
         analytics::Analytics::new(&opt, index_scheduler.clone(), auth_controller.clone()).await;
@@ -153,15 +149,8 @@ async fn run_http(
     let enable_dashboard = &opt.env == "development";
     let opt_clone = opt.clone();
     let index_scheduler = Data::from(index_scheduler);
-    let auth = Data::from(auth_controller);
+    let auth_controller = Data::from(auth_controller);
     let analytics = Data::from(analytics);
-    // Create personalization service with API key from options
-    let personalization_service = Data::new(
-        opt.experimental_personalization_api_key
-            .clone()
-            .map(PersonalizationService::cohere)
-            .unwrap_or_else(PersonalizationService::disabled),
-    );
     let search_queue = SearchQueue::new(
         opt.experimental_search_queue_size,
         available_parallelism()
@@ -173,25 +162,21 @@ async fn run_http(
         usize::from(opt.experimental_drop_search_after) as u64
     ));
     let search_queue = Data::new(search_queue);
-    let (logs_route_handle, logs_stderr_handle) = logs;
-    let logs_route_handle = Data::new(logs_route_handle);
-    let logs_stderr_handle = Data::new(logs_stderr_handle);
 
-    let services = ServicesData {
+    let http_server = HttpServer::new(move || {
-        index_scheduler,
+        create_app(
-        auth,
+            index_scheduler.clone(),
-        search_queue,
+            auth_controller.clone(),
-        personalization_service,
+            search_queue.clone(),
-        logs_route_handle,
+            opt.clone(),
-        logs_stderr_handle,
+            logs.clone(),
-        analytics,
+            analytics.clone(),
-    };
+            enable_dashboard,
-
+        )
-    let http_server =
+    })
-        HttpServer::new(move || create_app(services.clone(), opt.clone(), enable_dashboard))
+    // Disable signals allows the server to terminate immediately when a user enter CTRL-C
-            // Disable signals allows the server to terminate immediately when a user enter CTRL-C
+    .disable_signals()
-            .disable_signals()
+    .keep_alive(KeepAlive::Os);
-            .keep_alive(KeepAlive::Os);
 
     if let Some(config) = opt_clone.get_ssl_config()? {
         http_server.bind_rustls_0_23(opt_clone.http_addr, config)?.run().await?;

crates/meilisearch/src/metrics.rs

@@ -1,8 +1,7 @@
 use lazy_static::lazy_static;
 use prometheus::{
-    opts, register_gauge, register_gauge_vec, register_histogram_vec, register_int_counter_vec,
+    opts, register_gauge, register_histogram_vec, register_int_counter_vec, register_int_gauge,
-    register_int_gauge, register_int_gauge_vec, Gauge, GaugeVec, HistogramVec, IntCounterVec,
+    register_int_gauge_vec, Gauge, HistogramVec, IntCounterVec, IntGauge, IntGaugeVec,
-    IntGauge, IntGaugeVec,
 };
 
 lazy_static! {
@@ -74,20 +73,6 @@ lazy_static! {
         &["kind", "value"]
     )
     .expect("Can't create a metric");
-    pub static ref MEILISEARCH_BATCH_RUNNING_PROGRESS_TRACE: GaugeVec = register_gauge_vec!(
-        opts!("meilisearch_batch_running_progress_trace", "The currently running progress trace"),
-        &["batch_uid", "step_name"]
-    )
-    .expect("Can't create a metric");
-    pub static ref MEILISEARCH_LAST_FINISHED_BATCHES_PROGRESS_TRACE_MS: IntGaugeVec =
-        register_int_gauge_vec!(
-            opts!(
-                "meilisearch_last_finished_batches_progress_trace_ms",
-                "The last few batches progress trace in milliseconds"
-            ),
-            &["batch_uid", "step_name"]
-        )
-        .expect("Can't create a metric");
     pub static ref MEILISEARCH_LAST_UPDATE: IntGauge =
         register_int_gauge!(opts!("meilisearch_last_update", "Meilisearch Last Update"))
             .expect("Can't create a metric");
@@ -129,9 +114,4 @@ lazy_static! {
             "Meilisearch Task Queue Size Until Stop Registering",
         ))
         .expect("Can't create a metric");
-    pub static ref MEILISEARCH_PERSONALIZED_SEARCH_REQUESTS: IntGauge = register_int_gauge!(opts!(
-        "meilisearch_personalized_search_requests",
-        "Meilisearch number of search requests with personalization"
-    ))
-    .expect("Can't create a metric");
 }

crates/meilisearch/src/option.rs

@@ -7,13 +7,12 @@ use std::ops::Deref;
 use std::path::PathBuf;
 use std::str::FromStr;
 use std::sync::Arc;
-use std::time::Duration;
 use std::{env, fmt, fs};
 
 use byte_unit::{Byte, ParseError, UnitType};
 use clap::Parser;
 use meilisearch_types::features::InstanceTogglableFeatures;
-use meilisearch_types::milli::update::{IndexerConfig, S3SnapshotOptions};
+use meilisearch_types::milli::update::IndexerConfig;
 use meilisearch_types::milli::ThreadPoolNoAbortBuilder;
 use rustls::server::{ServerSessionMemoryCache, WebPkiClientVerifier};
 use rustls::RootCertStore;
@@ -75,22 +74,6 @@ const MEILI_EXPERIMENTAL_EMBEDDING_CACHE_ENTRIES: &str =
 const MEILI_EXPERIMENTAL_NO_SNAPSHOT_COMPACTION: &str = "MEILI_EXPERIMENTAL_NO_SNAPSHOT_COMPACTION";
 const MEILI_EXPERIMENTAL_NO_EDITION_2024_FOR_DUMPS: &str =
     "MEILI_EXPERIMENTAL_NO_EDITION_2024_FOR_DUMPS";
-const MEILI_EXPERIMENTAL_PERSONALIZATION_API_KEY: &str =
-    "MEILI_EXPERIMENTAL_PERSONALIZATION_API_KEY";
-
-// Related to S3 snapshots
-const MEILI_S3_BUCKET_URL: &str = "MEILI_S3_BUCKET_URL";
-const MEILI_S3_BUCKET_REGION: &str = "MEILI_S3_BUCKET_REGION";
-const MEILI_S3_BUCKET_NAME: &str = "MEILI_S3_BUCKET_NAME";
-const MEILI_S3_SNAPSHOT_PREFIX: &str = "MEILI_S3_SNAPSHOT_PREFIX";
-const MEILI_S3_ACCESS_KEY: &str = "MEILI_S3_ACCESS_KEY";
-const MEILI_S3_SECRET_KEY: &str = "MEILI_S3_SECRET_KEY";
-const MEILI_EXPERIMENTAL_S3_MAX_IN_FLIGHT_PARTS: &str = "MEILI_EXPERIMENTAL_S3_MAX_IN_FLIGHT_PARTS";
-const MEILI_EXPERIMENTAL_S3_COMPRESSION_LEVEL: &str = "MEILI_EXPERIMENTAL_S3_COMPRESSION_LEVEL";
-const MEILI_EXPERIMENTAL_S3_SIGNATURE_DURATION_SECONDS: &str =
-    "MEILI_EXPERIMENTAL_S3_SIGNATURE_DURATION_SECONDS";
-const MEILI_EXPERIMENTAL_S3_MULTIPART_PART_SIZE: &str = "MEILI_EXPERIMENTAL_S3_MULTIPART_PART_SIZE";
-
 const DEFAULT_CONFIG_FILE_PATH: &str = "./config.toml";
 const DEFAULT_DB_PATH: &str = "./data.ms";
 const DEFAULT_HTTP_ADDR: &str = "localhost:7700";
@@ -100,10 +83,6 @@ const DEFAULT_SNAPSHOT_DIR: &str = "snapshots/";
 const DEFAULT_SNAPSHOT_INTERVAL_SEC: u64 = 86400;
 const DEFAULT_SNAPSHOT_INTERVAL_SEC_STR: &str = "86400";
 const DEFAULT_DUMP_DIR: &str = "dumps/";
-const DEFAULT_S3_SNAPSHOT_MAX_IN_FLIGHT_PARTS: NonZeroUsize = NonZeroUsize::new(10).unwrap();
-const DEFAULT_S3_SNAPSHOT_COMPRESSION_LEVEL: u32 = 0;
-const DEFAULT_S3_SNAPSHOT_SIGNATURE_DURATION_SECONDS: u64 = 8 * 3600; // 8 hours
-const DEFAULT_S3_SNAPSHOT_MULTIPART_PART_SIZE: Byte = Byte::from_u64(375 * 1024 * 1024); // 375 MiB
 
 const MEILI_MAX_INDEXING_MEMORY: &str = "MEILI_MAX_INDEXING_MEMORY";
 const MEILI_MAX_INDEXING_THREADS: &str = "MEILI_MAX_INDEXING_THREADS";
@@ -473,14 +452,11 @@ pub struct Opt {
     #[serde(default = "default_limit_batched_tasks")]
     pub experimental_max_number_of_batched_tasks: usize,
 
-    /// Experimentally controls the maximum total size, in bytes, of tasks that will be processed
+    /// Experimentally reduces the maximum total size, in bytes, of tasks that will be processed at once,
-    /// simultaneously. When unspecified, defaults to half of the maximum indexing memory and
+    /// see: <https://github.com/orgs/meilisearch/discussions/801>
-    /// clamped to 10 GiB.
+    #[clap(long, env = MEILI_EXPERIMENTAL_LIMIT_BATCHED_TASKS_TOTAL_SIZE, default_value_t = default_limit_batched_tasks_total_size())]
-    ///
+    #[serde(default = "default_limit_batched_tasks_total_size")]
-    /// See: <https://github.com/orgs/meilisearch/discussions/801>
+    pub experimental_limit_batched_tasks_total_size: Byte,
-    #[clap(long, env = MEILI_EXPERIMENTAL_LIMIT_BATCHED_TASKS_TOTAL_SIZE)]
-    #[serde(default)]
-    pub experimental_limit_batched_tasks_total_size: Option<Byte>,
 
     /// Enables experimental caching of search query embeddings. The value represents the maximal number of entries in the cache of each
     /// distinct embedder.
@@ -499,20 +475,10 @@ pub struct Opt {
     #[serde(default)]
     pub experimental_no_snapshot_compaction: bool,
 
-    /// Experimental personalization API key feature.
-    ///
-    /// Sets the API key for personalization features.
-    #[clap(long, env = MEILI_EXPERIMENTAL_PERSONALIZATION_API_KEY)]
-    pub experimental_personalization_api_key: Option<String>,
-
     #[serde(flatten)]
     #[clap(flatten)]
     pub indexer_options: IndexerOpts,
 
-    #[serde(flatten)]
-    #[clap(flatten)]
-    pub s3_snapshot_options: Option<S3SnapshotOpts>,
-
     /// Set the path to a configuration file that should be used to setup the engine.
     /// Format must be TOML.
     #[clap(long)]
@@ -614,8 +580,6 @@ impl Opt {
             experimental_limit_batched_tasks_total_size,
             experimental_embedding_cache_entries,
             experimental_no_snapshot_compaction,
-            experimental_personalization_api_key,
-            s3_snapshot_options,
         } = self;
         export_to_env_if_not_present(MEILI_DB_PATH, db_path);
         export_to_env_if_not_present(MEILI_HTTP_ADDR, http_addr);
@@ -704,12 +668,10 @@ impl Opt {
             MEILI_EXPERIMENTAL_MAX_NUMBER_OF_BATCHED_TASKS,
             experimental_max_number_of_batched_tasks.to_string(),
         );
-        if let Some(limit) = experimental_limit_batched_tasks_total_size {
+        export_to_env_if_not_present(
-            export_to_env_if_not_present(
+            MEILI_EXPERIMENTAL_LIMIT_BATCHED_TASKS_TOTAL_SIZE,
-                MEILI_EXPERIMENTAL_LIMIT_BATCHED_TASKS_TOTAL_SIZE,
+            experimental_limit_batched_tasks_total_size.to_string(),
-                limit.to_string(),
+        );
-            );
-        }
         export_to_env_if_not_present(
             MEILI_EXPERIMENTAL_EMBEDDING_CACHE_ENTRIES,
             experimental_embedding_cache_entries.to_string(),
@@ -718,22 +680,7 @@ impl Opt {
             MEILI_EXPERIMENTAL_NO_SNAPSHOT_COMPACTION,
             experimental_no_snapshot_compaction.to_string(),
         );
-        if let Some(experimental_personalization_api_key) = experimental_personalization_api_key {
-            export_to_env_if_not_present(
-                MEILI_EXPERIMENTAL_PERSONALIZATION_API_KEY,
-                experimental_personalization_api_key,
-            );
-        }
         indexer_options.export_to_env();
-        if let Some(s3_snapshot_options) = s3_snapshot_options {
-            #[cfg(not(unix))]
-            {
-                let _ = s3_snapshot_options;
-                panic!("S3 snapshot options are not supported on Windows");
-            }
-            #[cfg(unix)]
-            s3_snapshot_options.export_to_env();
-        }
     }
 
     pub fn get_ssl_config(&self) -> anyhow::Result<Option<rustls::ServerConfig>> {
@@ -902,16 +849,6 @@ impl TryFrom<&IndexerOpts> for IndexerConfig {
     type Error = anyhow::Error;
 
     fn try_from(other: &IndexerOpts) -> Result<Self, Self::Error> {
-        let IndexerOpts {
-            max_indexing_memory,
-            max_indexing_threads,
-            skip_index_budget,
-            experimental_no_edition_2024_for_settings,
-            experimental_no_edition_2024_for_dumps,
-            experimental_no_edition_2024_for_prefix_post_processing,
-            experimental_no_edition_2024_for_facet_post_processing,
-        } = other;
-
         let thread_pool = ThreadPoolNoAbortBuilder::new_for_indexing()
             .num_threads(other.max_indexing_threads.unwrap_or_else(|| num_cpus::get() / 2))
             .build()?;
@@ -919,163 +856,21 @@ impl TryFrom<&IndexerOpts> for IndexerConfig {
         Ok(Self {
             thread_pool,
             log_every_n: Some(DEFAULT_LOG_EVERY_N),
-            max_memory: max_indexing_memory.map(|b| b.as_u64() as usize),
+            max_memory: other.max_indexing_memory.map(|b| b.as_u64() as usize),
-            max_threads: max_indexing_threads.0,
+            max_threads: *other.max_indexing_threads,
             max_positions_per_attributes: None,
-            skip_index_budget: *skip_index_budget,
+            skip_index_budget: other.skip_index_budget,
-            experimental_no_edition_2024_for_settings: *experimental_no_edition_2024_for_settings,
+            experimental_no_edition_2024_for_settings: other
-            experimental_no_edition_2024_for_dumps: *experimental_no_edition_2024_for_dumps,
+                .experimental_no_edition_2024_for_settings,
+            experimental_no_edition_2024_for_dumps: other.experimental_no_edition_2024_for_dumps,
             chunk_compression_type: Default::default(),
             chunk_compression_level: Default::default(),
             documents_chunk_size: Default::default(),
             max_nb_chunks: Default::default(),
-            experimental_no_edition_2024_for_prefix_post_processing:
+            experimental_no_edition_2024_for_prefix_post_processing: other
-                *experimental_no_edition_2024_for_prefix_post_processing,
+                .experimental_no_edition_2024_for_prefix_post_processing,
-            experimental_no_edition_2024_for_facet_post_processing:
+            experimental_no_edition_2024_for_facet_post_processing: other
-                *experimental_no_edition_2024_for_facet_post_processing,
+                .experimental_no_edition_2024_for_facet_post_processing,
-            s3_snapshot_options: None,
-        })
-    }
-}
-
-#[derive(Debug, Clone, Parser, Deserialize)]
-// This group is a bit tricky but makes it possible to require all listed fields if one of them
-// is specified. It lets us keep an Option for the S3SnapshotOpts configuration.
-// <https://github.com/clap-rs/clap/issues/5092#issuecomment-2616986075>
-#[group(requires_all = ["s3_bucket_url", "s3_bucket_region", "s3_bucket_name", "s3_snapshot_prefix", "s3_access_key", "s3_secret_key"])]
-pub struct S3SnapshotOpts {
-    /// The S3 bucket URL in the format https://s3.<region>.amazonaws.com.
-    #[clap(long, env = MEILI_S3_BUCKET_URL, required = false)]
-    #[serde(default)]
-    pub s3_bucket_url: String,
-
-    /// The region in the format us-east-1.
-    #[clap(long, env = MEILI_S3_BUCKET_REGION, required = false)]
-    #[serde(default)]
-    pub s3_bucket_region: String,
-
-    /// The bucket name.
-    #[clap(long, env = MEILI_S3_BUCKET_NAME, required = false)]
-    #[serde(default)]
-    pub s3_bucket_name: String,
-
-    /// The prefix path where to put the snapshot, uses normal slashes (/).
-    #[clap(long, env = MEILI_S3_SNAPSHOT_PREFIX, required = false)]
-    #[serde(default)]
-    pub s3_snapshot_prefix: String,
-
-    /// The S3 access key.
-    #[clap(long, env = MEILI_S3_ACCESS_KEY, required = false)]
-    #[serde(default)]
-    pub s3_access_key: String,
-
-    /// The S3 secret key.
-    #[clap(long, env = MEILI_S3_SECRET_KEY, required = false)]
-    #[serde(default)]
-    pub s3_secret_key: String,
-
-    /// The maximum number of parts that can be uploaded in parallel.
-    ///
-    /// For more information, see <https://github.com/orgs/meilisearch/discussions/869>.
-    #[clap(long, env = MEILI_EXPERIMENTAL_S3_MAX_IN_FLIGHT_PARTS, default_value_t = default_experimental_s3_snapshot_max_in_flight_parts())]
-    #[serde(default = "default_experimental_s3_snapshot_max_in_flight_parts")]
-    pub experimental_s3_max_in_flight_parts: NonZeroUsize,
-
-    /// The compression level. Defaults to no compression (0).
-    ///
-    /// For more information, see <https://github.com/orgs/meilisearch/discussions/869>.
-    #[clap(long, env = MEILI_EXPERIMENTAL_S3_COMPRESSION_LEVEL, default_value_t = default_experimental_s3_snapshot_compression_level())]
-    #[serde(default = "default_experimental_s3_snapshot_compression_level")]
-    pub experimental_s3_compression_level: u32,
-
-    /// The signature duration for the multipart upload.
-    ///
-    /// For more information, see <https://github.com/orgs/meilisearch/discussions/869>.
-    #[clap(long, env = MEILI_EXPERIMENTAL_S3_SIGNATURE_DURATION_SECONDS, default_value_t = default_experimental_s3_snapshot_signature_duration_seconds())]
-    #[serde(default = "default_experimental_s3_snapshot_signature_duration_seconds")]
-    pub experimental_s3_signature_duration_seconds: u64,
-
-    /// The size of the the multipart parts.
-    ///
-    /// Must not be less than 10MiB and larger than 8GiB. Yes,
-    /// twice the boundaries of the AWS S3 multipart upload
-    /// because we use it a bit differently internally.
-    ///
-    /// For more information, see <https://github.com/orgs/meilisearch/discussions/869>.
-    #[clap(long, env = MEILI_EXPERIMENTAL_S3_MULTIPART_PART_SIZE, default_value_t = default_experimental_s3_snapshot_multipart_part_size())]
-    #[serde(default = "default_experimental_s3_snapshot_multipart_part_size")]
-    pub experimental_s3_multipart_part_size: Byte,
-}
-
-impl S3SnapshotOpts {
-    /// Exports the values to their corresponding env vars if they are not set.
-    pub fn export_to_env(self) {
-        let S3SnapshotOpts {
-            s3_bucket_url,
-            s3_bucket_region,
-            s3_bucket_name,
-            s3_snapshot_prefix,
-            s3_access_key,
-            s3_secret_key,
-            experimental_s3_max_in_flight_parts,
-            experimental_s3_compression_level,
-            experimental_s3_signature_duration_seconds,
-            experimental_s3_multipart_part_size,
-        } = self;
-
-        export_to_env_if_not_present(MEILI_S3_BUCKET_URL, s3_bucket_url);
-        export_to_env_if_not_present(MEILI_S3_BUCKET_REGION, s3_bucket_region);
-        export_to_env_if_not_present(MEILI_S3_BUCKET_NAME, s3_bucket_name);
-        export_to_env_if_not_present(MEILI_S3_SNAPSHOT_PREFIX, s3_snapshot_prefix);
-        export_to_env_if_not_present(MEILI_S3_ACCESS_KEY, s3_access_key);
-        export_to_env_if_not_present(MEILI_S3_SECRET_KEY, s3_secret_key);
-        export_to_env_if_not_present(
-            MEILI_EXPERIMENTAL_S3_MAX_IN_FLIGHT_PARTS,
-            experimental_s3_max_in_flight_parts.to_string(),
-        );
-        export_to_env_if_not_present(
-            MEILI_EXPERIMENTAL_S3_COMPRESSION_LEVEL,
-            experimental_s3_compression_level.to_string(),
-        );
-        export_to_env_if_not_present(
-            MEILI_EXPERIMENTAL_S3_SIGNATURE_DURATION_SECONDS,
-            experimental_s3_signature_duration_seconds.to_string(),
-        );
-        export_to_env_if_not_present(
-            MEILI_EXPERIMENTAL_S3_MULTIPART_PART_SIZE,
-            experimental_s3_multipart_part_size.to_string(),
-        );
-    }
-}
-
-impl TryFrom<S3SnapshotOpts> for S3SnapshotOptions {
-    type Error = anyhow::Error;
-
-    fn try_from(other: S3SnapshotOpts) -> Result<Self, Self::Error> {
-        let S3SnapshotOpts {
-            s3_bucket_url,
-            s3_bucket_region,
-            s3_bucket_name,
-            s3_snapshot_prefix,
-            s3_access_key,
-            s3_secret_key,
-            experimental_s3_max_in_flight_parts,
-            experimental_s3_compression_level,
-            experimental_s3_signature_duration_seconds,
-            experimental_s3_multipart_part_size,
-        } = other;
-
-        Ok(S3SnapshotOptions {
-            s3_bucket_url,
-            s3_bucket_region,
-            s3_bucket_name,
-            s3_snapshot_prefix,
-            s3_access_key,
-            s3_secret_key,
-            s3_max_in_flight_parts: experimental_s3_max_in_flight_parts,
-            s3_compression_level: experimental_s3_compression_level,
-            s3_signature_duration: Duration::from_secs(experimental_s3_signature_duration_seconds),
-            s3_multipart_part_size: experimental_s3_multipart_part_size.as_u64(),
         })
     }
 }
@@ -1278,6 +1073,10 @@ fn default_limit_batched_tasks() -> usize {
     usize::MAX
 }
 
+fn default_limit_batched_tasks_total_size() -> Byte {
+    Byte::from_u64(u64::MAX)
+}
+
 fn default_embedding_cache_entries() -> usize {
     0
 }
@@ -1290,22 +1089,6 @@ fn default_snapshot_interval_sec() -> &'static str {
     DEFAULT_SNAPSHOT_INTERVAL_SEC_STR
 }
 
-fn default_experimental_s3_snapshot_max_in_flight_parts() -> NonZeroUsize {
-    DEFAULT_S3_SNAPSHOT_MAX_IN_FLIGHT_PARTS
-}
-
-fn default_experimental_s3_snapshot_compression_level() -> u32 {
-    DEFAULT_S3_SNAPSHOT_COMPRESSION_LEVEL
-}
-
-fn default_experimental_s3_snapshot_signature_duration_seconds() -> u64 {
-    DEFAULT_S3_SNAPSHOT_SIGNATURE_DURATION_SECONDS
-}
-
-fn default_experimental_s3_snapshot_multipart_part_size() -> Byte {
-    DEFAULT_S3_SNAPSHOT_MULTIPART_PART_SIZE
-}
-
 fn default_dump_dir() -> PathBuf {
     PathBuf::from(DEFAULT_DUMP_DIR)
 }

crates/meilisearch/src/personalization/mod.rs

@@ -1,366 +0,0 @@
-use std::time::Duration;
-
-use meilisearch_types::error::{Code, ErrorCode, ResponseError};
-use meilisearch_types::milli::TimeBudget;
-use rand::Rng;
-use reqwest::Client;
-use serde::{Deserialize, Serialize};
-use tracing::{debug, info, warn};
-
-use crate::search::{Personalize, SearchResult};
-
-const COHERE_API_URL: &str = "https://api.cohere.ai/v1/rerank";
-const MAX_RETRIES: u32 = 10;
-
-#[derive(Debug, thiserror::Error)]
-enum PersonalizationError {
-    #[error("Personalization service: HTTP request failed: {0}")]
-    Request(#[from] reqwest::Error),
-    #[error("Personalization service: Failed to parse response: {0}")]
-    Parse(String),
-    #[error("Personalization service: Cohere API error: {0}")]
-    Api(String),
-    #[error("Personalization service: Unauthorized: invalid API key")]
-    Unauthorized,
-    #[error("Personalization service: Rate limited: too many requests")]
-    RateLimited,
-    #[error("Personalization service: Bad request: {0}")]
-    BadRequest(String),
-    #[error("Personalization service: Internal server error: {0}")]
-    InternalServerError(String),
-    #[error("Personalization service: Network error: {0}")]
-    Network(String),
-    #[error("Personalization service: Deadline exceeded")]
-    DeadlineExceeded,
-    #[error(transparent)]
-    FeatureNotEnabled(#[from] index_scheduler::error::FeatureNotEnabledError),
-}
-
-impl ErrorCode for PersonalizationError {
-    fn error_code(&self) -> Code {
-        match self {
-            PersonalizationError::FeatureNotEnabled { .. } => Code::FeatureNotEnabled,
-            PersonalizationError::Unauthorized => Code::RemoteInvalidApiKey,
-            PersonalizationError::RateLimited => Code::TooManySearchRequests,
-            PersonalizationError::BadRequest(_) => Code::RemoteBadRequest,
-            PersonalizationError::InternalServerError(_) => Code::RemoteRemoteError,
-            PersonalizationError::Network(_) | PersonalizationError::Request(_) => {
-                Code::RemoteCouldNotSendRequest
-            }
-            PersonalizationError::Parse(_) | PersonalizationError::Api(_) => {
-                Code::RemoteBadResponse
-            }
-            PersonalizationError::DeadlineExceeded => Code::Internal, // should not be returned to the client
-        }
-    }
-}
-
-pub struct CohereService {
-    client: Client,
-    api_key: String,
-}
-
-impl CohereService {
-    pub fn new(api_key: String) -> Self {
-        info!("Personalization service initialized with Cohere API");
-        let client = Client::builder()
-            .timeout(Duration::from_secs(30))
-            .build()
-            .expect("Failed to create HTTP client");
-        Self { client, api_key }
-    }
-
-    pub async fn rerank_search_results(
-        &self,
-        search_result: SearchResult,
-        personalize: &Personalize,
-        query: Option<&str>,
-        time_budget: TimeBudget,
-    ) -> Result<SearchResult, ResponseError> {
-        if time_budget.exceeded() {
-            warn!("Could not rerank due to deadline");
-            // If the deadline is exceeded, return the original search result instead of an error
-            return Ok(search_result);
-        }
-
-        // Extract user context from personalization
-        let user_context = personalize.user_context.as_str();
-
-        // Build the prompt by merging query and user context
-        let prompt = match query {
-            Some(q) => format!("User Context: {user_context}\nQuery: {q}"),
-            None => format!("User Context: {user_context}"),
-        };
-
-        // Extract documents for reranking
-        let documents: Vec<String> = search_result
-            .hits
-            .iter()
-            .map(|hit| {
-                // Convert the document to a string representation for reranking
-                serde_json::to_string(&hit.document).unwrap_or_else(|_| "{}".to_string())
-            })
-            .collect();
-
-        if documents.is_empty() {
-            return Ok(search_result);
-        }
-
-        // Call Cohere's rerank API with retry logic
-        let reranked_indices =
-            match self.call_rerank_with_retry(&prompt, &documents, time_budget).await {
-                Ok(indices) => indices,
-                Err(PersonalizationError::DeadlineExceeded) => {
-                    // If the deadline is exceeded, return the original search result instead of an error
-                    return Ok(search_result);
-                }
-                Err(e) => return Err(e.into()),
-            };
-
-        debug!("Cohere rerank successful, reordering {} results", search_result.hits.len());
-
-        // Reorder the hits based on Cohere's reranking
-        let mut reranked_hits = Vec::new();
-        for index in reranked_indices.iter() {
-            if let Some(hit) = search_result.hits.get(*index) {
-                reranked_hits.push(hit.clone());
-            }
-        }
-
-        Ok(SearchResult { hits: reranked_hits, ..search_result })
-    }
-
-    async fn call_rerank_with_retry(
-        &self,
-        query: &str,
-        documents: &[String],
-        time_budget: TimeBudget,
-    ) -> Result<Vec<usize>, PersonalizationError> {
-        let request_body = CohereRerankRequest {
-            query: query.to_string(),
-            documents: documents.to_vec(),
-            model: "rerank-english-v3.0".to_string(),
-        };
-
-        // Retry loop similar to vector extraction
-        for attempt in 0..MAX_RETRIES {
-            let response_result = self.send_rerank_request(&request_body).await;
-
-            let retry_duration = match self.handle_response(response_result).await {
-                Ok(indices) => return Ok(indices),
-                Err(retry) => {
-                    warn!("Cohere rerank attempt #{} failed: {}", attempt, retry.error);
-
-                    if time_budget.exceeded() {
-                        warn!("Could not rerank due to deadline");
-                        return Err(PersonalizationError::DeadlineExceeded);
-                    } else {
-                        match retry.into_duration(attempt) {
-                            Ok(d) => d,
-                            Err(error) => return Err(error),
-                        }
-                    }
-                }
-            };
-
-            // randomly up to double the retry duration
-            let retry_duration = retry_duration
-                + rand::thread_rng().gen_range(std::time::Duration::ZERO..retry_duration);
-
-            warn!("Retrying after {}ms", retry_duration.as_millis());
-            tokio::time::sleep(retry_duration).await;
-        }
-
-        // Final attempt without retry
-        let response_result = self.send_rerank_request(&request_body).await;
-
-        match self.handle_response(response_result).await {
-            Ok(indices) => Ok(indices),
-            Err(retry) => Err(retry.into_error()),
-        }
-    }
-
-    async fn send_rerank_request(
-        &self,
-        request_body: &CohereRerankRequest,
-    ) -> Result<reqwest::Response, reqwest::Error> {
-        self.client
-            .post(COHERE_API_URL)
-            .header("Authorization", format!("Bearer {}", self.api_key))
-            .header("Content-Type", "application/json")
-            .json(request_body)
-            .send()
-            .await
-    }
-
-    async fn handle_response(
-        &self,
-        response_result: Result<reqwest::Response, reqwest::Error>,
-    ) -> Result<Vec<usize>, Retry> {
-        let response = match response_result {
-            Ok(r) => r,
-            Err(e) if e.is_timeout() => {
-                return Err(Retry::retry_later(PersonalizationError::Network(format!(
-                    "Request timeout: {}",
-                    e
-                ))));
-            }
-            Err(e) => {
-                return Err(Retry::retry_later(PersonalizationError::Network(format!(
-                    "Network error: {}",
-                    e
-                ))));
-            }
-        };
-
-        let status = response.status();
-        let status_code = status.as_u16();
-
-        if status.is_success() {
-            let rerank_response: CohereRerankResponse = match response.json().await {
-                Ok(r) => r,
-                Err(e) => {
-                    return Err(Retry::retry_later(PersonalizationError::Parse(format!(
-                        "Failed to parse response: {}",
-                        e
-                    ))));
-                }
-            };
-
-            // Extract indices from rerank results
-            let indices: Vec<usize> =
-                rerank_response.results.iter().map(|result| result.index as usize).collect();
-
-            return Ok(indices);
-        }
-
-        // Handle error status codes
-        let error_body = response.text().await.unwrap_or_else(|_| "Unknown error".to_string());
-
-        let retry = match status_code {
-            401 => Retry::give_up(PersonalizationError::Unauthorized),
-            429 => Retry::rate_limited(PersonalizationError::RateLimited),
-            400 => Retry::give_up(PersonalizationError::BadRequest(error_body)),
-            500..=599 => Retry::retry_later(PersonalizationError::InternalServerError(format!(
-                "Status {}: {}",
-                status_code, error_body
-            ))),
-            402..=499 => Retry::give_up(PersonalizationError::Api(format!(
-                "Status {}: {}",
-                status_code, error_body
-            ))),
-            _ => Retry::retry_later(PersonalizationError::Api(format!(
-                "Unexpected status {}: {}",
-                status_code, error_body
-            ))),
-        };
-
-        Err(retry)
-    }
-}
-
-#[derive(Serialize)]
-struct CohereRerankRequest {
-    query: String,
-    documents: Vec<String>,
-    model: String,
-}
-
-#[derive(Deserialize)]
-struct CohereRerankResponse {
-    results: Vec<CohereRerankResult>,
-}
-
-#[derive(Deserialize)]
-struct CohereRerankResult {
-    index: u32,
-}
-
-// Retry strategy similar to vector extraction
-struct Retry {
-    error: PersonalizationError,
-    strategy: RetryStrategy,
-}
-
-enum RetryStrategy {
-    GiveUp,
-    Retry,
-    RetryAfterRateLimit,
-}
-
-impl Retry {
-    fn give_up(error: PersonalizationError) -> Self {
-        Self { error, strategy: RetryStrategy::GiveUp }
-    }
-
-    fn retry_later(error: PersonalizationError) -> Self {
-        Self { error, strategy: RetryStrategy::Retry }
-    }
-
-    fn rate_limited(error: PersonalizationError) -> Self {
-        Self { error, strategy: RetryStrategy::RetryAfterRateLimit }
-    }
-
-    fn into_duration(self, attempt: u32) -> Result<Duration, PersonalizationError> {
-        match self.strategy {
-            RetryStrategy::GiveUp => Err(self.error),
-            RetryStrategy::Retry => {
-                // Exponential backoff: 10^attempt milliseconds
-                Ok(Duration::from_millis((10u64).pow(attempt)))
-            }
-            RetryStrategy::RetryAfterRateLimit => {
-                // Longer backoff for rate limits: 100ms + exponential
-                Ok(Duration::from_millis(100 + (10u64).pow(attempt)))
-            }
-        }
-    }
-
-    fn into_error(self) -> PersonalizationError {
-        self.error
-    }
-}
-
-pub enum PersonalizationService {
-    Cohere(CohereService),
-    Disabled,
-}
-
-impl PersonalizationService {
-    pub fn cohere(api_key: String) -> Self {
-        // If the API key is empty, consider the personalization service as disabled
-        if api_key.trim().is_empty() {
-            Self::disabled()
-        } else {
-            Self::Cohere(CohereService::new(api_key))
-        }
-    }
-
-    pub fn disabled() -> Self {
-        debug!("Personalization service disabled");
-        Self::Disabled
-    }
-
-    pub async fn rerank_search_results(
-        &self,
-        search_result: SearchResult,
-        personalize: &Personalize,
-        query: Option<&str>,
-        time_budget: TimeBudget,
-    ) -> Result<SearchResult, ResponseError> {
-        match self {
-            Self::Cohere(cohere_service) => {
-                cohere_service
-                    .rerank_search_results(search_result, personalize, query, time_budget)
-                    .await
-            }
-            Self::Disabled => Err(PersonalizationError::FeatureNotEnabled(
-                index_scheduler::error::FeatureNotEnabledError {
-                    disabled_action: "reranking search results",
-                    feature: "personalization",
-                    issue_link: "https://github.com/orgs/meilisearch/discussions/866",
-                },
-            )
-            .into()),
-        }
-    }
-}

crates/meilisearch/src/routes/indexes/community_edition.rs

@@ -1,39 +0,0 @@
-pub mod proxy {
-
-    use std::fs::File;
-
-    use actix_web::HttpRequest;
-    use index_scheduler::IndexScheduler;
-
-    use crate::error::MeilisearchHttpError;
-
-    pub enum Body<T: serde::Serialize> {
-        NdJsonPayload,
-        Inline(T),
-        None,
-    }
-
-    impl Body<()> {
-        pub fn with_ndjson_payload(_file: File) -> Self {
-            Self::NdJsonPayload
-        }
-
-        pub fn none() -> Self {
-            Self::None
-        }
-    }
-
-    pub const PROXY_ORIGIN_REMOTE_HEADER: &str = "Meili-Proxy-Origin-Remote";
-    pub const PROXY_ORIGIN_TASK_UID_HEADER: &str = "Meili-Proxy-Origin-TaskUid";
-
-    pub async fn proxy<T: serde::Serialize>(
-        _index_scheduler: &IndexScheduler,
-        _index_uid: &str,
-        _req: &HttpRequest,
-        _network: meilisearch_types::network::Network,
-        _body: Body<T>,
-        _task: &meilisearch_types::tasks::Task,
-    ) -> Result<(), MeilisearchHttpError> {
-        Ok(())
-    }
-}

crates/meilisearch/src/routes/indexes/documents.rs

@@ -45,7 +45,7 @@ use crate::extractors::authentication::policies::*;
 use crate::extractors::authentication::GuardedData;
 use crate::extractors::payload::Payload;
 use crate::extractors::sequential_extractor::SeqHandler;
-use crate::routes::indexes::current_edition::proxy::{proxy, Body};
+use crate::routes::indexes::enterprise_edition::proxy::{proxy, Body};
 use crate::routes::indexes::search::fix_sort_query_parameters;
 use crate::routes::{
     get_task_id, is_dry_run, PaginationView, SummarizedTaskView, PAGINATION_DEFAULT_LIMIT,
@@ -333,12 +333,10 @@ impl Aggregate for DocumentsDeletionAggregator {
 pub async fn delete_document(
     index_scheduler: GuardedData<ActionPolicy<{ actions::DOCUMENTS_DELETE }>, Data<IndexScheduler>>,
     path: web::Path<DocumentParam>,
-    params: AwebQueryParameter<CustomMetadataQuery, DeserrQueryParamError>,
     req: HttpRequest,
     opt: web::Data<Opt>,
     analytics: web::Data<Analytics>,
 ) -> Result<HttpResponse, ResponseError> {
-    let CustomMetadataQuery { custom_metadata } = params.into_inner();
     let DocumentParam { index_uid, document_id } = path.into_inner();
     let index_uid = IndexUid::try_from(index_uid)?;
     let network = index_scheduler.network();
@@ -361,13 +359,10 @@ pub async fn delete_document(
     let dry_run = is_dry_run(&req, &opt)?;
     let task = {
         let index_scheduler = index_scheduler.clone();
-        tokio::task::spawn_blocking(move || {
+        tokio::task::spawn_blocking(move || index_scheduler.register(task, uid, dry_run)).await??
-            index_scheduler.register_with_custom_metadata(task, uid, custom_metadata, dry_run)
-        })
-        .await??
     };
 
-    if network.sharding() && !dry_run {
+    if network.sharding && !dry_run {
         proxy(&index_scheduler, &index_uid, &req, network, Body::none(), &task).await?;
     }
 
@@ -683,19 +678,6 @@ pub struct UpdateDocumentsQuery {
     #[param(value_type = char, default = ",", example = ";")]
     #[deserr(default, try_from(char) = from_char_csv_delimiter -> DeserrQueryParamError<InvalidDocumentCsvDelimiter>, error = DeserrQueryParamError<InvalidDocumentCsvDelimiter>)]
     pub csv_delimiter: Option<u8>,
-
-    #[param(example = "custom")]
-    #[deserr(default, error = DeserrQueryParamError<InvalidIndexCustomMetadata>)]
-    pub custom_metadata: Option<String>,
-}
-
-#[derive(Deserialize, Debug, Deserr, IntoParams)]
-#[deserr(error = DeserrQueryParamError, rename_all = camelCase, deny_unknown_fields)]
-#[into_params(parameter_in = Query, rename_all = "camelCase")]
-pub struct CustomMetadataQuery {
-    #[param(example = "custom")]
-    #[deserr(default, error = DeserrQueryParamError<InvalidIndexCustomMetadata>)]
-    pub custom_metadata: Option<String>,
 }
 
 fn from_char_csv_delimiter(
@@ -837,7 +819,6 @@ pub async fn replace_documents(
         body,
         IndexDocumentsMethod::ReplaceDocuments,
         uid,
-        params.custom_metadata,
         dry_run,
         allow_index_creation,
         &req,
@@ -940,7 +921,6 @@ pub async fn update_documents(
         body,
         IndexDocumentsMethod::UpdateDocuments,
         uid,
-        params.custom_metadata,
         dry_run,
         allow_index_creation,
         &req,
@@ -960,7 +940,6 @@ async fn document_addition(
     body: Payload,
     method: IndexDocumentsMethod,
     task_id: Option<TaskId>,
-    custom_metadata: Option<String>,
     dry_run: bool,
     allow_index_creation: bool,
     req: &HttpRequest,
@@ -1086,10 +1065,8 @@ async fn document_addition(
     };
 
     let scheduler = index_scheduler.clone();
-    let task = match tokio::task::spawn_blocking(move || {
+    let task = match tokio::task::spawn_blocking(move || scheduler.register(task, task_id, dry_run))
-        scheduler.register_with_custom_metadata(task, task_id, custom_metadata, dry_run)
+        .await?
-    })
-    .await?
     {
         Ok(task) => task,
         Err(e) => {
@@ -1098,7 +1075,7 @@ async fn document_addition(
         }
     };
 
-    if network.sharding() {
+    if network.sharding {
         if let Some(file) = file {
             proxy(
                 &index_scheduler,
@@ -1153,7 +1130,7 @@ async fn copy_body_to_file(
 /// Delete a set of documents based on an array of document ids.
 #[utoipa::path(
     post,
-    path = "{indexUid}/documents/delete-batch",
+    path = "{indexUid}/delete-batch",
     tag = "Documents",
     security(("Bearer" = ["documents.delete", "documents.*", "*"])),
     params(
@@ -1184,14 +1161,11 @@ pub async fn delete_documents_batch(
     index_scheduler: GuardedData<ActionPolicy<{ actions::DOCUMENTS_DELETE }>, Data<IndexScheduler>>,
     index_uid: web::Path<String>,
     body: web::Json<Vec<Value>>,
-    params: AwebQueryParameter<CustomMetadataQuery, DeserrQueryParamError>,
     req: HttpRequest,
     opt: web::Data<Opt>,
     analytics: web::Data<Analytics>,
 ) -> Result<HttpResponse, ResponseError> {
     debug!(parameters = ?body, "Delete documents by batch");
-    let CustomMetadataQuery { custom_metadata } = params.into_inner();
-
     let index_uid = IndexUid::try_from(index_uid.into_inner())?;
     let network = index_scheduler.network();
 
@@ -1216,13 +1190,10 @@ pub async fn delete_documents_batch(
     let dry_run = is_dry_run(&req, &opt)?;
     let task = {
         let index_scheduler = index_scheduler.clone();
-        tokio::task::spawn_blocking(move || {
+        tokio::task::spawn_blocking(move || index_scheduler.register(task, uid, dry_run)).await??
-            index_scheduler.register_with_custom_metadata(task, uid, custom_metadata, dry_run)
-        })
-        .await??
     };
 
-    if network.sharding() && !dry_run {
+    if network.sharding && !dry_run {
         proxy(&index_scheduler, &index_uid, &req, network, Body::Inline(body), &task).await?;
     }
 
@@ -1273,15 +1244,12 @@ pub struct DocumentDeletionByFilter {
 pub async fn delete_documents_by_filter(
     index_scheduler: GuardedData<ActionPolicy<{ actions::DOCUMENTS_DELETE }>, Data<IndexScheduler>>,
     index_uid: web::Path<String>,
-    params: AwebQueryParameter<CustomMetadataQuery, DeserrQueryParamError>,
     body: AwebJson<DocumentDeletionByFilter, DeserrJsonError>,
     req: HttpRequest,
     opt: web::Data<Opt>,
     analytics: web::Data<Analytics>,
 ) -> Result<HttpResponse, ResponseError> {
     debug!(parameters = ?body, "Delete documents by filter");
-    let CustomMetadataQuery { custom_metadata } = params.into_inner();
-
     let index_uid = IndexUid::try_from(index_uid.into_inner())?;
     let index_uid = index_uid.into_inner();
     let filter = body.into_inner();
@@ -1314,13 +1282,10 @@ pub async fn delete_documents_by_filter(
     let dry_run = is_dry_run(&req, &opt)?;
     let task = {
         let index_scheduler = index_scheduler.clone();
-        tokio::task::spawn_blocking(move || {
+        tokio::task::spawn_blocking(move || index_scheduler.register(task, uid, dry_run)).await??
-            index_scheduler.register_with_custom_metadata(task, uid, custom_metadata, dry_run)
-        })
-        .await??
     };
 
-    if network.sharding() && !dry_run {
+    if network.sharding && !dry_run {
         proxy(&index_scheduler, &index_uid, &req, network, Body::Inline(filter), &task).await?;
     }
 
@@ -1407,14 +1372,12 @@ impl Aggregate for EditDocumentsByFunctionAggregator {
 pub async fn edit_documents_by_function(
     index_scheduler: GuardedData<ActionPolicy<{ actions::DOCUMENTS_ALL }>, Data<IndexScheduler>>,
     index_uid: web::Path<String>,
-    params: AwebQueryParameter<CustomMetadataQuery, DeserrQueryParamError>,
+    params: AwebJson<DocumentEditionByFunction, DeserrJsonError>,
-    body: AwebJson<DocumentEditionByFunction, DeserrJsonError>,
     req: HttpRequest,
     opt: web::Data<Opt>,
     analytics: web::Data<Analytics>,
 ) -> Result<HttpResponse, ResponseError> {
-    debug!(parameters = ?body, "Edit documents by function");
+    debug!(parameters = ?params, "Edit documents by function");
-    let CustomMetadataQuery { custom_metadata } = params.into_inner();
 
     index_scheduler
         .features()
@@ -1424,23 +1387,23 @@ pub async fn edit_documents_by_function(
 
     let index_uid = IndexUid::try_from(index_uid.into_inner())?;
     let index_uid = index_uid.into_inner();
-    let body = body.into_inner();
+    let params = params.into_inner();
 
     analytics.publish(
         EditDocumentsByFunctionAggregator {
-            filtered: body.filter.is_some(),
+            filtered: params.filter.is_some(),
-            with_context: body.context.is_some(),
+            with_context: params.context.is_some(),
             index_creation: index_scheduler.index(&index_uid).is_err(),
         },
         &req,
     );
 
     let engine = milli::rhai::Engine::new();
-    if let Err(e) = engine.compile(&body.function) {
+    if let Err(e) = engine.compile(&params.function) {
         return Err(ResponseError::from_msg(e.to_string(), Code::BadRequest));
     }
 
-    if let Some(ref filter) = body.filter {
+    if let Some(ref filter) = params.filter {
         // we ensure the filter is well formed before enqueuing it
         crate::search::parse_filter(
             filter,
@@ -1451,8 +1414,8 @@ pub async fn edit_documents_by_function(
     }
     let task = KindWithContent::DocumentEdition {
         index_uid: index_uid.clone(),
-        filter_expr: body.filter.clone(),
+        filter_expr: params.filter.clone(),
-        context: match body.context.clone() {
+        context: match params.context.clone() {
             Some(Value::Object(m)) => Some(m),
             None => None,
             _ => {
@@ -1462,21 +1425,18 @@ pub async fn edit_documents_by_function(
                 ))
             }
         },
-        function: body.function.clone(),
+        function: params.function.clone(),
     };
 
     let uid = get_task_id(&req, &opt)?;
     let dry_run = is_dry_run(&req, &opt)?;
     let task = {
         let index_scheduler = index_scheduler.clone();
-        tokio::task::spawn_blocking(move || {
+        tokio::task::spawn_blocking(move || index_scheduler.register(task, uid, dry_run)).await??
-            index_scheduler.register_with_custom_metadata(task, uid, custom_metadata, dry_run)
-        })
-        .await??
     };
 
-    if network.sharding() && !dry_run {
+    if network.sharding && !dry_run {
-        proxy(&index_scheduler, &index_uid, &req, network, Body::Inline(body), &task).await?;
+        proxy(&index_scheduler, &index_uid, &req, network, Body::Inline(params), &task).await?;
     }
 
     let task: SummarizedTaskView = task.into();
@@ -1517,14 +1477,12 @@ pub async fn edit_documents_by_function(
 pub async fn clear_all_documents(
     index_scheduler: GuardedData<ActionPolicy<{ actions::DOCUMENTS_DELETE }>, Data<IndexScheduler>>,
     index_uid: web::Path<String>,
-    params: AwebQueryParameter<CustomMetadataQuery, DeserrQueryParamError>,
     req: HttpRequest,
     opt: web::Data<Opt>,
     analytics: web::Data<Analytics>,
 ) -> Result<HttpResponse, ResponseError> {
     let index_uid = IndexUid::try_from(index_uid.into_inner())?;
     let network = index_scheduler.network();
-    let CustomMetadataQuery { custom_metadata } = params.into_inner();
 
     analytics.publish(
         DocumentsDeletionAggregator {
@@ -1543,13 +1501,10 @@ pub async fn clear_all_documents(
     let task = {
         let index_scheduler = index_scheduler.clone();
 
-        tokio::task::spawn_blocking(move || {
+        tokio::task::spawn_blocking(move || index_scheduler.register(task, uid, dry_run)).await??
-            index_scheduler.register_with_custom_metadata(task, uid, custom_metadata, dry_run)
-        })
-        .await??
     };
 
-    if network.sharding() && !dry_run {
+    if network.sharding && !dry_run {
         proxy(&index_scheduler, &index_uid, &req, network, Body::none(), &task).await?;
     }
 

crates/meilisearch/src/routes/indexes/enterprise_edition/proxy.rs

@@ -52,7 +52,7 @@ pub async fn proxy<T: serde::Serialize>(
     index_scheduler: &IndexScheduler,
     index_uid: &str,
     req: &HttpRequest,
-    network: meilisearch_types::network::Network,
+    network: meilisearch_types::enterprise_edition::network::Network,
     body: Body<T>,
     task: &meilisearch_types::tasks::Task,
 ) -> Result<(), MeilisearchHttpError> {

crates/meilisearch/src/routes/indexes/facet_search.rs

@@ -343,7 +343,6 @@ impl From<FacetSearchQuery> for SearchQuery {
             hybrid,
             ranking_score_threshold,
             locales,
-            personalize: None,
         }
     }
 }

crates/meilisearch/src/routes/indexes/mod.rs

@@ -30,16 +30,7 @@ use crate::Opt;
 
 pub mod compact;
 pub mod documents;
-
-#[cfg(not(feature = "enterprise"))]
-mod community_edition;
-#[cfg(feature = "enterprise")]
 mod enterprise_edition;
-#[cfg(not(feature = "enterprise"))]
-use community_edition as current_edition;
-#[cfg(feature = "enterprise")]
-use enterprise_edition as current_edition;
-
 pub mod facet_search;
 pub mod search;
 mod search_analytics;
@@ -50,7 +41,7 @@ mod settings_analytics;
 pub mod similar;
 mod similar_analytics;
 
-pub use current_edition::proxy::{PROXY_ORIGIN_REMOTE_HEADER, PROXY_ORIGIN_TASK_UID_HEADER};
+pub use enterprise_edition::proxy::{PROXY_ORIGIN_REMOTE_HEADER, PROXY_ORIGIN_TASK_UID_HEADER};
 
 #[derive(OpenApi)]
 #[openapi(

crates/meilisearch/src/routes/indexes/search.rs

@@ -24,9 +24,9 @@ use crate::metrics::MEILISEARCH_DEGRADED_SEARCH_REQUESTS;
 use crate::routes::indexes::search_analytics::{SearchAggregator, SearchGET, SearchPOST};
 use crate::routes::parse_include_metadata_header;
 use crate::search::{
-    add_search_rules, perform_search, HybridQuery, MatchingStrategy, Personalize,
+    add_search_rules, perform_search, HybridQuery, MatchingStrategy, RankingScoreThreshold,
-    RankingScoreThreshold, RetrieveVectors, SearchKind, SearchParams, SearchQuery, SearchResult,
+    RetrieveVectors, SearchKind, SearchParams, SearchQuery, SearchResult, SemanticRatio,
-    SemanticRatio, DEFAULT_CROP_LENGTH, DEFAULT_CROP_MARKER, DEFAULT_HIGHLIGHT_POST_TAG,
+    DEFAULT_CROP_LENGTH, DEFAULT_CROP_MARKER, DEFAULT_HIGHLIGHT_POST_TAG,
     DEFAULT_HIGHLIGHT_PRE_TAG, DEFAULT_SEARCH_LIMIT, DEFAULT_SEARCH_OFFSET, DEFAULT_SEMANTIC_RATIO,
 };
 use crate::search_queue::SearchQueue;
@@ -134,8 +134,6 @@ pub struct SearchQueryGet {
     #[deserr(default, error = DeserrQueryParamError<InvalidSearchLocales>)]
     #[param(value_type = Vec<Locale>, explode = false)]
     pub locales: Option<CS<Locale>>,
-    #[deserr(default, error = DeserrQueryParamError<InvalidSearchPersonalizeUserContext>)]
-    pub personalize_user_context: Option<String>,
 }
 
 #[derive(Debug, Clone, Copy, PartialEq, deserr::Deserr)]
@@ -207,9 +205,6 @@ impl TryFrom<SearchQueryGet> for SearchQuery {
             ));
         }
 
-        let personalize =
-            other.personalize_user_context.map(|user_context| Personalize { user_context });
-
         Ok(Self {
             q: other.q,
             // `media` not supported for `GET`
@@ -239,7 +234,6 @@ impl TryFrom<SearchQueryGet> for SearchQuery {
             hybrid,
             ranking_score_threshold: other.ranking_score_threshold.map(|o| o.0),
             locales: other.locales.map(|o| o.into_iter().collect()),
-            personalize,
         })
     }
 }
@@ -328,7 +322,6 @@ pub fn fix_sort_query_parameters(sort_query: &str) -> Vec<String> {
 pub async fn search_with_url_query(
     index_scheduler: GuardedData<ActionPolicy<{ actions::SEARCH }>, Data<IndexScheduler>>,
     search_queue: web::Data<SearchQueue>,
-    personalization_service: web::Data<crate::personalization::PersonalizationService>,
     index_uid: web::Path<String>,
     params: AwebQueryParameter<SearchQueryGet, DeserrQueryParamError>,
     req: HttpRequest,
@@ -349,16 +342,9 @@ pub async fn search_with_url_query(
 
     let index = index_scheduler.index(&index_uid)?;
 
-    // Extract personalization and query string before moving query
-    let personalize = query.personalize.take();
-
     let search_kind =
         search_kind(&query, index_scheduler.get_ref(), index_uid.to_string(), &index)?;
     let retrieve_vector = RetrieveVectors::new(query.retrieve_vectors);
-
-    // Save the query string for personalization if requested
-    let personalize_query = personalize.is_some().then(|| query.q.clone()).flatten();
-
     let permit = search_queue.try_get_search_permit().await?;
     let include_metadata = parse_include_metadata_header(&req);
 
@@ -379,24 +365,12 @@ pub async fn search_with_url_query(
     .await;
     permit.drop().await;
     let search_result = search_result?;
-    if let Ok((search_result, _)) = search_result.as_ref() {
+    if let Ok(ref search_result) = search_result {
         aggregate.succeed(search_result);
     }
     analytics.publish(aggregate, &req);
 
-    let (mut search_result, time_budget) = search_result?;
+    let search_result = search_result?;
-
-    // Apply personalization if requested
-    if let Some(personalize) = personalize.as_ref() {
-        search_result = personalization_service
-            .rerank_search_results(
-                search_result,
-                personalize,
-                personalize_query.as_deref(),
-                time_budget,
-            )
-            .await?;
-    }
 
     debug!(request_uid = ?request_uid, returns = ?search_result, "Search get");
     Ok(HttpResponse::Ok().json(search_result))
@@ -461,7 +435,6 @@ pub async fn search_with_url_query(
 pub async fn search_with_post(
     index_scheduler: GuardedData<ActionPolicy<{ actions::SEARCH }>, Data<IndexScheduler>>,
     search_queue: web::Data<SearchQueue>,
-    personalization_service: web::Data<crate::personalization::PersonalizationService>,
     index_uid: web::Path<String>,
     params: AwebJson<SearchQuery, DeserrJsonError>,
     req: HttpRequest,
@@ -482,18 +455,12 @@ pub async fn search_with_post(
 
     let index = index_scheduler.index(&index_uid)?;
 
-    // Extract personalization and query string before moving query
-    let personalize = query.personalize.take();
-
     let search_kind =
         search_kind(&query, index_scheduler.get_ref(), index_uid.to_string(), &index)?;
     let retrieve_vectors = RetrieveVectors::new(query.retrieve_vectors);
 
     let include_metadata = parse_include_metadata_header(&req);
 
-    // Save the query string for personalization if requested
-    let personalize_query = personalize.is_some().then(|| query.q.clone()).flatten();
-
     let permit = search_queue.try_get_search_permit().await?;
     let search_result = tokio::task::spawn_blocking(move || {
         perform_search(
@@ -512,7 +479,7 @@ pub async fn search_with_post(
     .await;
     permit.drop().await;
     let search_result = search_result?;
-    if let Ok((ref search_result, _)) = search_result {
+    if let Ok(ref search_result) = search_result {
         aggregate.succeed(search_result);
         if search_result.degraded {
             MEILISEARCH_DEGRADED_SEARCH_REQUESTS.inc();
@@ -520,19 +487,7 @@ pub async fn search_with_post(
     }
     analytics.publish(aggregate, &req);
 
-    let (mut search_result, time_budget) = search_result?;
+    let search_result = search_result?;
-
-    // Apply personalization if requested
-    if let Some(personalize) = personalize.as_ref() {
-        search_result = personalization_service
-            .rerank_search_results(
-                search_result,
-                personalize,
-                personalize_query.as_deref(),
-                time_budget,
-            )
-            .await?;
-    }
 
     debug!(request_uid = ?request_uid, returns = ?search_result, "Search post");
     Ok(HttpResponse::Ok().json(search_result))

crates/meilisearch/src/routes/indexes/search_analytics.rs

@@ -7,7 +7,6 @@ use serde_json::{json, Value};
 
 use crate::aggregate_methods;
 use crate::analytics::{Aggregate, AggregateMethod};
-use crate::metrics::MEILISEARCH_PERSONALIZED_SEARCH_REQUESTS;
 use crate::search::{
     SearchQuery, SearchResult, DEFAULT_CROP_LENGTH, DEFAULT_CROP_MARKER,
     DEFAULT_HIGHLIGHT_POST_TAG, DEFAULT_HIGHLIGHT_PRE_TAG, DEFAULT_SEARCH_LIMIT,
@@ -96,9 +95,6 @@ pub struct SearchAggregator<Method: AggregateMethod> {
     show_ranking_score_details: bool,
     ranking_score_threshold: bool,
 
-    // personalization
-    total_personalized: usize,
-
     marker: std::marker::PhantomData<Method>,
 }
 
@@ -133,7 +129,6 @@ impl<Method: AggregateMethod> SearchAggregator<Method> {
             hybrid,
             ranking_score_threshold,
             locales,
-            personalize,
         } = query;
 
         let mut ret = Self::default();
@@ -209,12 +204,6 @@ impl<Method: AggregateMethod> SearchAggregator<Method> {
             ret.locales = locales.iter().copied().collect();
         }
 
-        // personalization
-        if personalize.is_some() {
-            ret.total_personalized = 1;
-            MEILISEARCH_PERSONALIZED_SEARCH_REQUESTS.inc();
-        }
-
         ret.highlight_pre_tag = *highlight_pre_tag != DEFAULT_HIGHLIGHT_PRE_TAG();
         ret.highlight_post_tag = *highlight_post_tag != DEFAULT_HIGHLIGHT_POST_TAG();
         ret.crop_marker = *crop_marker != DEFAULT_CROP_MARKER();
@@ -307,7 +296,6 @@ impl<Method: AggregateMethod> Aggregate for SearchAggregator<Method> {
             total_used_negative_operator,
             ranking_score_threshold,
             mut locales,
-            total_personalized,
             marker: _,
         } = *new;
 
@@ -393,9 +381,6 @@ impl<Method: AggregateMethod> Aggregate for SearchAggregator<Method> {
         // locales
         self.locales.append(&mut locales);
 
-        // personalization
-        self.total_personalized = self.total_personalized.saturating_add(total_personalized);
-
         self
     }
 
@@ -441,7 +426,6 @@ impl<Method: AggregateMethod> Aggregate for SearchAggregator<Method> {
             total_used_negative_operator,
             ranking_score_threshold,
             locales,
-            total_personalized,
             marker: _,
         } = *self;
 
@@ -515,9 +499,6 @@ impl<Method: AggregateMethod> Aggregate for SearchAggregator<Method> {
                 "show_ranking_score_details": show_ranking_score_details,
                 "ranking_score_threshold": ranking_score_threshold,
             },
-            "personalization": {
-                "total_personalized": total_personalized,
-            },
         })
     }
 }

crates/meilisearch/src/routes/metrics.rs

@@ -4,7 +4,6 @@ use index_scheduler::{IndexScheduler, Query};
 use meilisearch_auth::AuthController;
 use meilisearch_types::error::ResponseError;
 use meilisearch_types::keys::actions;
-use meilisearch_types::milli::progress::ProgressStepView;
 use meilisearch_types::tasks::Status;
 use prometheus::{Encoder, TextEncoder};
 use time::OffsetDateTime;
@@ -39,12 +38,6 @@ pub fn configure(config: &mut web::ServiceConfig) {
 # HELP meilisearch_db_size_bytes Meilisearch DB Size In Bytes
 # TYPE meilisearch_db_size_bytes gauge
 meilisearch_db_size_bytes 1130496
-# HELP meilisearch_batch_running_progress_trace The currently running progress trace
-# TYPE meilisearch_batch_running_progress_trace gauge
-meilisearch_batch_running_progress_trace{batch_uid="0",step_name="document"} 0.710618582519409
-meilisearch_batch_running_progress_trace{batch_uid="0",step_name="extracting word proximity"} 0.2222222222222222
-meilisearch_batch_running_progress_trace{batch_uid="0",step_name="indexing"} 0.6666666666666666
-meilisearch_batch_running_progress_trace{batch_uid="0",step_name="processing tasks"} 0
 # HELP meilisearch_http_requests_total Meilisearch HTTP requests total
 # TYPE meilisearch_http_requests_total counter
 meilisearch_http_requests_total{method="GET",path="/metrics",status="400"} 1
@@ -68,13 +61,6 @@ meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="1
 meilisearch_http_response_time_seconds_bucket{method="GET",path="/metrics",le="+Inf"} 0
 meilisearch_http_response_time_seconds_sum{method="GET",path="/metrics"} 0
 meilisearch_http_response_time_seconds_count{method="GET",path="/metrics"} 0
-# HELP meilisearch_last_finished_batches_progress_trace_ms The last few batches progress trace in milliseconds
-# TYPE meilisearch_last_finished_batches_progress_trace_ms gauge
-meilisearch_last_finished_batches_progress_trace_ms{batch_uid="0",step_name="processing tasks"} 19360
-meilisearch_last_finished_batches_progress_trace_ms{batch_uid="0",step_name="processing tasks > computing document changes"} 368
-meilisearch_last_finished_batches_progress_trace_ms{batch_uid="0",step_name="processing tasks > computing document changes > preparing payloads"} 367
-meilisearch_last_finished_batches_progress_trace_ms{batch_uid="0",step_name="processing tasks > computing document changes > preparing payloads > payload"} 367
-meilisearch_last_finished_batches_progress_trace_ms{batch_uid="0",step_name="processing tasks > indexing"} 18970
 # HELP meilisearch_index_count Meilisearch Index Count
 # TYPE meilisearch_index_count gauge
 meilisearch_index_count 1
@@ -162,50 +148,6 @@ pub async fn get_metrics(
         }
     }
 
-    // Fetch and expose the current progressing step
-    crate::metrics::MEILISEARCH_BATCH_RUNNING_PROGRESS_TRACE.reset();
-    let (batches, _total) = index_scheduler.get_batches_from_authorized_indexes(
-        &Query { statuses: Some(vec![Status::Processing]), ..Query::default() },
-        auth_filters,
-    )?;
-    if let Some(batch) = batches.into_iter().next() {
-        let batch_uid = batch.uid.to_string();
-        if let Some(progress) = batch.progress {
-            for ProgressStepView { current_step, finished, total } in progress.steps {
-                crate::metrics::MEILISEARCH_BATCH_RUNNING_PROGRESS_TRACE
-                    .with_label_values(&[batch_uid.as_str(), current_step.as_ref()])
-                    // We return the completion ratio of the current step
-                    .set(finished as f64 / total as f64);
-            }
-        }
-    }
-
-    crate::metrics::MEILISEARCH_LAST_FINISHED_BATCHES_PROGRESS_TRACE_MS.reset();
-    let (batches, _total) = index_scheduler.get_batches_from_authorized_indexes(
-        // Fetch the finished batches...
-        &Query {
-            statuses: Some(vec![Status::Succeeded, Status::Failed]),
-            limit: Some(1),
-            ..Query::default()
-        },
-        auth_filters,
-    )?;
-    // ...and get the last batch only.
-    if let Some(batch) = batches.into_iter().next() {
-        let batch_uid = batch.uid.to_string();
-        for (step_name, duration_str) in batch.stats.progress_trace {
-            let Some(duration_str) = duration_str.as_str() else { continue };
-            match humantime::parse_duration(duration_str) {
-                Ok(duration) => {
-                    crate::metrics::MEILISEARCH_LAST_FINISHED_BATCHES_PROGRESS_TRACE_MS
-                        .with_label_values(&[&batch_uid, &step_name])
-                        .set(duration.as_millis() as i64);
-                }
-                Err(e) => tracing::error!("Failed to parse duration: {e}"),
-            }
-        }
-    }
-
     if let Some(last_update) = response.last_update {
         crate::metrics::MEILISEARCH_LAST_UPDATE.set(last_update.unix_timestamp());
     }

crates/meilisearch/src/routes/mod.rs

@@ -41,9 +41,7 @@ use crate::routes::indexes::IndexView;
 use crate::routes::multi_search::SearchResults;
 use crate::routes::network::{Network, Remote};
 use crate::routes::swap_indexes::SwapIndexesPayload;
-use crate::routes::webhooks::{
+use crate::routes::webhooks::{WebhookResults, WebhookSettings, WebhookWithMetadata};
-    WebhookResults, WebhookSettings, WebhookWithMetadataRedactedAuthorization,
-};
 use crate::search::{
     FederatedSearch, FederatedSearchResult, Federation, FederationOptions, MergeFacets,
     SearchQueryWithIndex, SearchResultWithIndex, SimilarQuery, SimilarResult,
@@ -105,7 +103,7 @@ mod webhooks;
         url = "/",
         description = "Local server",
     )),
-    components(schemas(PaginationView<KeyView>, PaginationView<IndexView>, IndexView, DocumentDeletionByFilter, AllBatches, BatchStats, ProgressStepView, ProgressView, BatchView, RuntimeTogglableFeatures, SwapIndexesPayload, DocumentEditionByFunction, MergeFacets, FederationOptions, SearchQueryWithIndex, Federation, FederatedSearch, FederatedSearchResult, SearchResults, SearchResultWithIndex, SimilarQuery, SimilarResult, PaginationView<serde_json::Value>, BrowseQuery, UpdateIndexRequest, IndexUid, IndexCreateRequest, KeyView, Action, CreateApiKey, UpdateStderrLogs, LogMode, GetLogs, IndexStats, Stats, HealthStatus, HealthResponse, VersionResponse, Code, ErrorType, AllTasks, TaskView, Status, DetailsView, ResponseError, Settings<Unchecked>, Settings<Checked>, TypoSettings, MinWordSizeTyposSetting, FacetingSettings, PaginationSettings, SummarizedTaskView, Kind, Network, Remote, FilterableAttributesRule, FilterableAttributesPatterns, AttributePatterns, FilterableAttributesFeatures, FilterFeatures, Export, WebhookSettings, WebhookResults, WebhookWithMetadataRedactedAuthorization, meilisearch_types::milli::vector::VectorStoreBackend))
+    components(schemas(PaginationView<KeyView>, PaginationView<IndexView>, IndexView, DocumentDeletionByFilter, AllBatches, BatchStats, ProgressStepView, ProgressView, BatchView, RuntimeTogglableFeatures, SwapIndexesPayload, DocumentEditionByFunction, MergeFacets, FederationOptions, SearchQueryWithIndex, Federation, FederatedSearch, FederatedSearchResult, SearchResults, SearchResultWithIndex, SimilarQuery, SimilarResult, PaginationView<serde_json::Value>, BrowseQuery, UpdateIndexRequest, IndexUid, IndexCreateRequest, KeyView, Action, CreateApiKey, UpdateStderrLogs, LogMode, GetLogs, IndexStats, Stats, HealthStatus, HealthResponse, VersionResponse, Code, ErrorType, AllTasks, TaskView, Status, DetailsView, ResponseError, Settings<Unchecked>, Settings<Checked>, TypoSettings, MinWordSizeTyposSetting, FacetingSettings, PaginationSettings, SummarizedTaskView, Kind, Network, Remote, FilterableAttributesRule, FilterableAttributesPatterns, AttributePatterns, FilterableAttributesFeatures, FilterFeatures, Export, WebhookSettings, WebhookResults, WebhookWithMetadata, meilisearch_types::milli::vector::VectorStoreBackend))
 )]
 pub struct MeilisearchApi;
 
@@ -218,8 +216,6 @@ pub struct SummarizedTaskView {
         deserialize_with = "time::serde::rfc3339::deserialize"
     )]
     enqueued_at: OffsetDateTime,
-    #[serde(default, skip_serializing_if = "Option::is_none")]
-    custom_metadata: Option<String>,
 }
 
 impl From<Task> for SummarizedTaskView {
@@ -230,7 +226,6 @@ impl From<Task> for SummarizedTaskView {
             status: task.status,
             kind: task.kind.as_kind(),
             enqueued_at: task.enqueued_at,
-            custom_metadata: task.custom_metadata,
         }
     }
 }

crates/meilisearch/src/routes/multi_search.rs

@@ -146,7 +146,6 @@ pub struct SearchResults {
 pub async fn multi_search_with_post(
     index_scheduler: GuardedData<ActionPolicy<{ actions::SEARCH }>, Data<IndexScheduler>>,
     search_queue: Data<SearchQueue>,
-    personalization_service: web::Data<crate::personalization::PersonalizationService>,
     params: AwebJson<FederatedSearch, DeserrJsonError>,
     req: HttpRequest,
     analytics: web::Data<Analytics>,
@@ -237,7 +236,7 @@ pub async fn multi_search_with_post(
             // changes.
             let search_results: Result<_, (ResponseError, usize)> = async {
                 let mut search_results = Vec::with_capacity(queries.len());
-                for (query_index, (index_uid, mut query, federation_options)) in queries
+                for (query_index, (index_uid, query, federation_options)) in queries
                     .into_iter()
                     .map(SearchQueryWithIndex::into_index_query_federation)
                     .enumerate()
@@ -270,13 +269,6 @@ pub async fn multi_search_with_post(
                         })
                         .with_index(query_index)?;
 
-                    // Extract personalization and query string before moving query
-                    let personalize = query.personalize.take();
-
-                    // Save the query string for personalization if requested
-                    let personalize_query =
-                        personalize.is_some().then(|| query.q.clone()).flatten();
-
                     let index_uid_str = index_uid.to_string();
 
                     let search_kind = search_kind(
@@ -288,7 +280,7 @@ pub async fn multi_search_with_post(
                     .with_index(query_index)?;
                     let retrieve_vector = RetrieveVectors::new(query.retrieve_vectors);
 
-                    let (mut search_result, time_budget) = tokio::task::spawn_blocking(move || {
+                    let search_result = tokio::task::spawn_blocking(move || {
                         perform_search(
                             SearchParams {
                                 index_uid: index_uid_str.clone(),
@@ -303,25 +295,11 @@ pub async fn multi_search_with_post(
                         )
                     })
                     .await
-                    .with_index(query_index)?
                     .with_index(query_index)?;
 
-                    // Apply personalization if requested
-                    if let Some(personalize) = personalize.as_ref() {
-                        search_result = personalization_service
-                            .rerank_search_results(
-                                search_result,
-                                personalize,
-                                personalize_query.as_deref(),
-                                time_budget,
-                            )
-                            .await
-                            .with_index(query_index)?;
-                    }
-
                     search_results.push(SearchResultWithIndex {
                         index_uid: index_uid.into_inner(),
-                        result: search_result,
+                        result: search_result.with_index(query_index)?,
                     });
                 }
                 Ok(search_results)

crates/meilisearch/src/routes/multi_search_analytics.rs

@@ -67,7 +67,6 @@ impl MultiSearchAggregator {
             hybrid: _,
             ranking_score_threshold: _,
             locales: _,
-            personalize: _,
         } in &federated_search.queries
         {
             if let Some(federation_options) = federation_options {

crates/meilisearch/src/routes/network.rs

@@ -7,6 +7,7 @@ use deserr::Deserr;
 use index_scheduler::IndexScheduler;
 use itertools::{EitherOrBoth, Itertools};
 use meilisearch_types::deserr::DeserrJsonError;
+use meilisearch_types::enterprise_edition::network::{Network as DbNetwork, Remote as DbRemote};
 use meilisearch_types::error::deserr_codes::{
     InvalidNetworkRemotes, InvalidNetworkSearchApiKey, InvalidNetworkSelf, InvalidNetworkSharding,
     InvalidNetworkUrl, InvalidNetworkWriteApiKey,
@@ -14,7 +15,6 @@ use meilisearch_types::error::deserr_codes::{
 use meilisearch_types::error::ResponseError;
 use meilisearch_types::keys::actions;
 use meilisearch_types::milli::update::Setting;
-use meilisearch_types::network::{Network as DbNetwork, Remote as DbRemote};
 use serde::Serialize;
 use tracing::debug;
 use utoipa::{OpenApi, ToSchema};
@@ -211,16 +211,6 @@ async fn patch_network(
     let old_network = index_scheduler.network();
     debug!(parameters = ?new_network, "Patch network");
 
-    #[cfg(not(feature = "enterprise"))]
-    if new_network.sharding.set().is_some() {
-        use meilisearch_types::error::Code;
-
-        return Err(ResponseError::from_msg(
-            "Meilisearch Enterprise Edition is required to set `network.sharding`".into(),
-            Code::RequiresEnterpriseEdition,
-        ));
-    }
-
     let merged_self = match new_network.local {
         Setting::Set(new_self) => Some(new_self),
         Setting::Reset => None,
@@ -322,7 +312,6 @@ async fn patch_network(
 
     let merged_network =
         DbNetwork { local: merged_self, remotes: merged_remotes, sharding: merged_sharding };
-
     index_scheduler.put_network(merged_network.clone())?;
     debug!(returns = ?merged_network, "Patch network");
     Ok(HttpResponse::Ok().json(merged_network))

crates/meilisearch/src/routes/webhooks.rs

@@ -90,7 +90,7 @@ fn deny_immutable_fields_webhook(
 #[derive(Debug, Serialize, ToSchema)]
 #[serde(rename_all = "camelCase")]
 #[schema(rename_all = "camelCase")]
-pub(super) struct WebhookWithMetadataRedactedAuthorization {
+pub(super) struct WebhookWithMetadata {
     uuid: Uuid,
     is_editable: bool,
     #[schema(value_type = WebhookSettings)]
@@ -98,9 +98,8 @@ pub(super) struct WebhookWithMetadataRedactedAuthorization {
     webhook: Webhook,
 }
 
-impl WebhookWithMetadataRedactedAuthorization {
+impl WebhookWithMetadata {
-    pub fn from(uuid: Uuid, mut webhook: Webhook) -> Self {
+    pub fn from(uuid: Uuid, webhook: Webhook) -> Self {
-        webhook.redact_authorization_header();
         Self { uuid, is_editable: uuid != Uuid::nil(), webhook }
     }
 }
@@ -108,7 +107,7 @@ impl WebhookWithMetadataRedactedAuthorization {
 #[derive(Debug, Serialize, ToSchema)]
 #[serde(rename_all = "camelCase")]
 pub(super) struct WebhookResults {
-    results: Vec<WebhookWithMetadataRedactedAuthorization>,
+    results: Vec<WebhookWithMetadata>,
 }
 
 #[utoipa::path(
@@ -151,7 +150,7 @@ async fn get_webhooks(
     let results = webhooks
         .webhooks
         .into_iter()
-        .map(|(uuid, webhook)| WebhookWithMetadataRedactedAuthorization::from(uuid, webhook))
+        .map(|(uuid, webhook)| WebhookWithMetadata::from(uuid, webhook))
         .collect::<Vec<_>>();
     let results = WebhookResults { results };
 
@@ -302,7 +301,7 @@ fn check_changed(uuid: Uuid, webhook: &Webhook) -> Result<(), WebhooksError> {
     tag = "Webhooks",
     security(("Bearer" = ["webhooks.get", "webhooks.*", "*.get", "*"])),
     responses(
-        (status = 200, description = "Webhook found", body = WebhookWithMetadataRedactedAuthorization, content_type = "application/json", example = json!({
+        (status = 200, description = "Webhook found", body = WebhookWithMetadata, content_type = "application/json", example = json!({
             "uuid": "550e8400-e29b-41d4-a716-446655440000",
             "url": "https://your.site/on-tasks-completed",
             "headers": {
@@ -325,7 +324,7 @@ async fn get_webhook(
     let mut webhooks = index_scheduler.webhooks_view();
 
     let webhook = webhooks.webhooks.remove(&uuid).ok_or(WebhookNotFound(uuid))?;
-    let webhook = WebhookWithMetadataRedactedAuthorization::from(uuid, webhook);
+    let webhook = WebhookWithMetadata::from(uuid, webhook);
 
     debug!(returns = ?webhook, "Get webhook");
     Ok(HttpResponse::Ok().json(webhook))
@@ -338,7 +337,7 @@ async fn get_webhook(
     request_body = WebhookSettings,
     security(("Bearer" = ["webhooks.create", "webhooks.*", "*"])),
     responses(
-        (status = 201, description = "Webhook created successfully", body = WebhookWithMetadataRedactedAuthorization, content_type = "application/json", example = json!({
+        (status = 201, description = "Webhook created successfully", body = WebhookWithMetadata, content_type = "application/json", example = json!({
             "uuid": "550e8400-e29b-41d4-a716-446655440000",
             "url": "https://your.site/on-tasks-completed",
             "headers": {
@@ -384,7 +383,7 @@ async fn post_webhook(
 
     analytics.publish(PostWebhooksAnalytics, &req);
 
-    let response = WebhookWithMetadataRedactedAuthorization::from(uuid, webhook);
+    let response = WebhookWithMetadata::from(uuid, webhook);
     debug!(returns = ?response, "Post webhook");
     Ok(HttpResponse::Created().json(response))
 }
@@ -396,7 +395,7 @@ async fn post_webhook(
     request_body = WebhookSettings,
     security(("Bearer" = ["webhooks.update", "webhooks.*", "*"])),
     responses(
-        (status = 200, description = "Webhook updated successfully", body = WebhookWithMetadataRedactedAuthorization, content_type = "application/json", example = json!({
+        (status = 200, description = "Webhook updated successfully", body = WebhookWithMetadata, content_type = "application/json", example = json!({
             "uuid": "550e8400-e29b-41d4-a716-446655440000",
             "url": "https://your.site/on-tasks-completed",
             "headers": {
@@ -436,7 +435,7 @@ async fn patch_webhook(
 
     analytics.publish(PatchWebhooksAnalytics, &req);
 
-    let response = WebhookWithMetadataRedactedAuthorization::from(uuid, webhook);
+    let response = WebhookWithMetadata::from(uuid, webhook);
     debug!(returns = ?response, "Patch webhook");
     Ok(HttpResponse::Ok().json(response))
 }

crates/meilisearch/src/search/federated/perform.rs

@@ -9,12 +9,12 @@ use std::vec::{IntoIter, Vec};
 use actix_http::StatusCode;
 use index_scheduler::{IndexScheduler, RoFeatures};
 use itertools::Itertools;
+use meilisearch_types::enterprise_edition::network::{Network, Remote};
 use meilisearch_types::error::ResponseError;
 use meilisearch_types::milli::order_by_map::OrderByMap;
 use meilisearch_types::milli::score_details::{ScoreDetails, WeightedScoreValue};
 use meilisearch_types::milli::vector::Embedding;
 use meilisearch_types::milli::{self, DocumentId, OrderBy, TimeBudget, DEFAULT_VALUES_PER_FACET};
-use meilisearch_types::network::{Network, Remote};
 use roaring::RoaringBitmap;
 use tokio::task::JoinHandle;
 use uuid::Uuid;
@@ -601,10 +601,6 @@ impl PartitionedQueries {
             .into());
         }
 
-        if federated_query.has_personalize() {
-            return Err(MeilisearchHttpError::PersonalizationInFederatedQuery(query_index).into());
-        }
-
         let (index_uid, query, federation_options) = federated_query.into_index_query_federation();
 
         let federation_options = federation_options.unwrap_or_default();

crates/meilisearch/src/search/federated/proxy.rs

@@ -1,6 +1,6 @@
 pub use error::ProxySearchError;
 use error::ReqwestErrorWithoutUrl;
-use meilisearch_types::network::Remote;
+use meilisearch_types::enterprise_edition::network::Remote;
 use rand::Rng as _;
 use reqwest::{Client, Response, StatusCode};
 use serde::de::DeserializeOwned;

crates/meilisearch/src/search/federated/types.rs

@@ -18,9 +18,10 @@ use serde::{Deserialize, Serialize};
 use utoipa::ToSchema;
 use uuid::Uuid;
 
+use crate::search::SearchMetadata;
+
 use super::super::{ComputedFacets, FacetStats, HitsInfo, SearchHit, SearchQueryWithIndex};
 use crate::milli::vector::Embedding;
-use crate::search::SearchMetadata;
 
 pub const DEFAULT_FEDERATED_WEIGHT: f64 = 1.0;
 

crates/meilisearch/src/search/mod.rs

@@ -59,13 +59,6 @@ pub const DEFAULT_HIGHLIGHT_POST_TAG: fn() -> String = || "</em>".to_string();
 pub const DEFAULT_SEMANTIC_RATIO: fn() -> SemanticRatio = || SemanticRatio(0.5);
 pub const INCLUDE_METADATA_HEADER: &str = "Meili-Include-Metadata";
 
-#[derive(Clone, Default, PartialEq, Deserr, ToSchema, Debug)]
-#[deserr(error = DeserrJsonError<InvalidSearchPersonalize>, rename_all = camelCase, deny_unknown_fields)]
-pub struct Personalize {
-    #[deserr(error = DeserrJsonError<InvalidSearchPersonalizeUserContext>)]
-    pub user_context: String,
-}
-
 #[derive(Clone, Default, PartialEq, Deserr, ToSchema)]
 #[deserr(error = DeserrJsonError, rename_all = camelCase, deny_unknown_fields)]
 pub struct SearchQuery {
@@ -129,8 +122,6 @@ pub struct SearchQuery {
     pub ranking_score_threshold: Option<RankingScoreThreshold>,
     #[deserr(default, error = DeserrJsonError<InvalidSearchLocales>)]
     pub locales: Option<Vec<Locale>>,
-    #[deserr(default, error = DeserrJsonError<InvalidSearchPersonalize>, default)]
-    pub personalize: Option<Personalize>,
 }
 
 impl From<SearchParameters> for SearchQuery {
@@ -178,7 +169,6 @@ impl From<SearchParameters> for SearchQuery {
             highlight_post_tag: DEFAULT_HIGHLIGHT_POST_TAG(),
             crop_marker: DEFAULT_CROP_MARKER(),
             locales: None,
-            personalize: None,
         }
     }
 }
@@ -260,7 +250,6 @@ impl fmt::Debug for SearchQuery {
             attributes_to_search_on,
             ranking_score_threshold,
             locales,
-            personalize,
         } = self;
 
         let mut debug = f.debug_struct("SearchQuery");
@@ -349,10 +338,6 @@ impl fmt::Debug for SearchQuery {
             debug.field("locales", &locales);
         }
 
-        if let Some(personalize) = personalize {
-            debug.field("personalize", &personalize);
-        }
-
         debug.finish()
     }
 }
@@ -558,9 +543,6 @@ pub struct SearchQueryWithIndex {
     pub ranking_score_threshold: Option<RankingScoreThreshold>,
     #[deserr(default, error = DeserrJsonError<InvalidSearchLocales>, default)]
     pub locales: Option<Vec<Locale>>,
-    #[deserr(default, error = DeserrJsonError<InvalidSearchPersonalize>, default)]
-    #[serde(skip)]
-    pub personalize: Option<Personalize>,
 
     #[deserr(default)]
     pub federation_options: Option<FederationOptions>,
@@ -585,10 +567,6 @@ impl SearchQueryWithIndex {
         self.facets.as_deref().filter(|v| !v.is_empty())
     }
 
-    pub fn has_personalize(&self) -> bool {
-        self.personalize.is_some()
-    }
-
     pub fn from_index_query_federation(
         index_uid: IndexUid,
         query: SearchQuery,
@@ -622,7 +600,6 @@ impl SearchQueryWithIndex {
             attributes_to_search_on,
             ranking_score_threshold,
             locales,
-            personalize,
         } = query;
 
         SearchQueryWithIndex {
@@ -654,7 +631,6 @@ impl SearchQueryWithIndex {
             attributes_to_search_on,
             ranking_score_threshold,
             locales,
-            personalize,
             federation_options,
         }
     }
@@ -690,7 +666,6 @@ impl SearchQueryWithIndex {
             hybrid,
             ranking_score_threshold,
             locales,
-            personalize,
         } = self;
         (
             index_uid,
@@ -722,7 +697,6 @@ impl SearchQueryWithIndex {
                 hybrid,
                 ranking_score_threshold,
                 locales,
-                personalize,
                 // do not use ..Default::default() here,
                 // rather add any missing field from `SearchQuery` to `SearchQueryWithIndex`
             },
@@ -1175,10 +1149,7 @@ pub struct SearchParams {
     pub include_metadata: bool,
 }
 
-pub fn perform_search(
+pub fn perform_search(params: SearchParams, index: &Index) -> Result<SearchResult, ResponseError> {
-    params: SearchParams,
-    index: &Index,
-) -> Result<(SearchResult, TimeBudget), ResponseError> {
     let SearchParams {
         index_uid,
         query,
@@ -1197,7 +1168,7 @@ pub fn perform_search(
     };
 
     let (search, is_finite_pagination, max_total_hits, offset) =
-        prepare_search(index, &rtxn, &query, &search_kind, time_budget.clone(), features)?;
+        prepare_search(index, &rtxn, &query, &search_kind, time_budget, features)?;
 
     let (
         milli::SearchResult {
@@ -1255,7 +1226,6 @@ pub fn perform_search(
         attributes_to_search_on: _,
         filter: _,
         distinct: _,
-        personalize: _,
     } = query;
 
     let format = AttributesFormat {
@@ -1321,7 +1291,7 @@ pub fn perform_search(
         request_uid: Some(request_uid),
         metadata,
     };
-    Ok((result, time_budget))
+    Ok(result)
 }
 
 #[derive(Debug, Clone, Default, Serialize, Deserialize, ToSchema)]

crates/meilisearch/tests/common/index.rs

@@ -91,16 +91,7 @@ impl<'a> Index<'a, Owned> {
         documents: Value,
         primary_key: Option<&str>,
     ) -> (Value, StatusCode) {
-        self._add_documents(documents, primary_key, None).await
+        self._add_documents(documents, primary_key).await
-    }
-
-    pub async fn add_documents_with_custom_metadata(
-        &self,
-        documents: Value,
-        primary_key: Option<&str>,
-        custom_metadata: Option<&str>,
-    ) -> (Value, StatusCode) {
-        self._add_documents(documents, primary_key, custom_metadata).await
     }
 
     pub async fn raw_add_documents(
@@ -361,25 +352,12 @@ impl<State> Index<'_, State> {
         &self,
         documents: Value,
         primary_key: Option<&str>,
-        custom_metadata: Option<&str>,
     ) -> (Value, StatusCode) {
-        let url = match (primary_key, custom_metadata) {
+        let url = match primary_key {
-            (Some(key), Some(meta)) => {
+            Some(key) => {
-                format!(
+                format!("/indexes/{}/documents?primaryKey={}", urlencode(self.uid.as_ref()), key)
-                    "/indexes/{}/documents?primaryKey={key}&customMetadata={meta}",
-                    urlencode(self.uid.as_ref()),
-                )
             }
-            (None, Some(meta)) => {
+            None => format!("/indexes/{}/documents", urlencode(self.uid.as_ref())),
-                format!(
-                    "/indexes/{}/documents?&customMetadata={meta}",
-                    urlencode(self.uid.as_ref()),
-                )
-            }
-            (Some(key), None) => {
-                format!("/indexes/{}/documents?&primaryKey={key}", urlencode(self.uid.as_ref()),)
-            }
-            (None, None) => format!("/indexes/{}/documents", urlencode(self.uid.as_ref())),
         };
         self.service.post_encoded(url, documents, self.encoder).await
     }

crates/meilisearch/tests/common/mod.rs

@@ -241,7 +241,7 @@ pub async fn shared_index_with_documents() -> &'static Index<'static, Shared> {
         let server = Server::new_shared();
         let index = server._index("SHARED_DOCUMENTS").to_shared();
         let documents = DOCUMENTS.clone();
-        let (response, _code) = index._add_documents(documents, None, None).await;
+        let (response, _code) = index._add_documents(documents, None).await;
         server.wait_task(response.uid()).await.succeeded();
         let (response, _code) = index
             ._update_settings(
@@ -284,7 +284,7 @@ pub async fn shared_index_with_score_documents() -> &'static Index<'static, Shar
         let server = Server::new_shared();
         let index = server._index("SHARED_SCORE_DOCUMENTS").to_shared();
         let documents = SCORE_DOCUMENTS.clone();
-        let (response, _code) = index._add_documents(documents, None, None).await;
+        let (response, _code) = index._add_documents(documents, None).await;
         server.wait_task(response.uid()).await.succeeded();
         let (response, _code) = index
             ._update_settings(
@@ -361,7 +361,7 @@ pub async fn shared_index_with_nested_documents() -> &'static Index<'static, Sha
         let server = Server::new_shared();
         let index = server._index("SHARED_NESTED_DOCUMENTS").to_shared();
         let documents = NESTED_DOCUMENTS.clone();
-        let (response, _code) = index._add_documents(documents, None, None).await;
+        let (response, _code) = index._add_documents(documents, None).await;
         server.wait_task(response.uid()).await.succeeded();
         let (response, _code) = index
             ._update_settings(
@@ -508,7 +508,7 @@ pub async fn shared_index_with_geo_documents() -> &'static Index<'static, Shared
         .get_or_init(|| async {
             let server = Server::new_shared();
             let index = server._index("SHARED_GEO_DOCUMENTS").to_shared();
-            let (response, _code) = index._add_documents(GEO_DOCUMENTS.clone(), None, None).await;
+            let (response, _code) = index._add_documents(GEO_DOCUMENTS.clone(), None).await;
             server.wait_task(response.uid()).await.succeeded();
 
             let (response, _code) = index
@@ -531,7 +531,7 @@ pub async fn shared_index_geojson_documents() -> &'static Index<'static, Shared>
             let index = server._index("SHARED_GEOJSON_DOCUMENTS").to_shared();
             let countries = include_str!("../documents/geojson/assets/countries.json");
             let lille = serde_json::from_str::<serde_json::Value>(countries).unwrap();
-            let (response, _code) = index._add_documents(Value(lille), Some("name"), None).await;
+            let (response, _code) = index._add_documents(Value(lille), Some("name")).await;
             server.wait_task(response.uid()).await.succeeded();
 
             let (response, _code) =

crates/meilisearch/tests/common/server.rs

@@ -49,8 +49,8 @@ impl Server<Owned> {
         }
 
         let options = default_settings(dir.path());
-        let handle = tokio::runtime::Handle::current();
+
-        let (index_scheduler, auth) = setup_meilisearch(&options, handle).unwrap();
+        let (index_scheduler, auth) = setup_meilisearch(&options).unwrap();
         let service = Service { index_scheduler, auth, options, api_key: None };
 
         Server { service, _dir: Some(dir), _marker: PhantomData }
@@ -65,9 +65,7 @@ impl Server<Owned> {
 
         options.master_key = Some("MASTER_KEY".to_string());
 
-        let handle = tokio::runtime::Handle::current();
+        let (index_scheduler, auth) = setup_meilisearch(&options).unwrap();
-
-        let (index_scheduler, auth) = setup_meilisearch(&options, handle).unwrap();
         let service = Service { index_scheduler, auth, options, api_key: None };
 
         Server { service, _dir: Some(dir), _marker: PhantomData }
@@ -80,9 +78,7 @@ impl Server<Owned> {
     }
 
     pub async fn new_with_options(options: Opt) -> Result<Self, anyhow::Error> {
-        let handle = tokio::runtime::Handle::current();
+        let (index_scheduler, auth) = setup_meilisearch(&options)?;
-
-        let (index_scheduler, auth) = setup_meilisearch(&options, handle)?;
         let service = Service { index_scheduler, auth, options, api_key: None };
 
         Ok(Server { service, _dir: None, _marker: PhantomData })
@@ -221,9 +217,8 @@ impl Server<Shared> {
         }
 
         let options = default_settings(dir.path());
-        let handle = tokio::runtime::Handle::current();
 
-        let (index_scheduler, auth) = setup_meilisearch(&options, handle).unwrap();
+        let (index_scheduler, auth) = setup_meilisearch(&options).unwrap();
         let service = Service { index_scheduler, auth, api_key: None, options };
 
         Server { service, _dir: Some(dir), _marker: PhantomData }

crates/meilisearch/tests/common/service.rs

@@ -10,9 +10,8 @@ use actix_web::test::TestRequest;
 use actix_web::web::Data;
 use index_scheduler::IndexScheduler;
 use meilisearch::analytics::Analytics;
-use meilisearch::personalization::PersonalizationService;
 use meilisearch::search_queue::SearchQueue;
-use meilisearch::{create_app, Opt, ServicesData, SubscriberForSecondLayer};
+use meilisearch::{create_app, Opt, SubscriberForSecondLayer};
 use meilisearch_auth::AuthController;
 use tracing::level_filters::LevelFilter;
 use tracing_subscriber::Layer;
@@ -136,24 +135,14 @@ impl Service {
             self.options.experimental_search_queue_size,
             NonZeroUsize::new(1).unwrap(),
         );
-        let personalization_service = self
-            .options
-            .experimental_personalization_api_key
-            .clone()
-            .map(PersonalizationService::cohere)
-            .unwrap_or_else(PersonalizationService::disabled);
 
         actix_web::test::init_service(create_app(
-            ServicesData {
+            self.index_scheduler.clone().into(),
-                index_scheduler: self.index_scheduler.clone().into(),
+            self.auth.clone().into(),
-                auth: self.auth.clone().into(),
+            Data::new(search_queue),
-                search_queue: Data::new(search_queue),
-                personalization_service: Data::new(personalization_service),
-                logs_route_handle: Data::new(route_layer_handle),
-                logs_stderr_handle: Data::new(stderr_layer_handle),
-                analytics: Data::new(Analytics::no_analytics()),
-            },
             self.options.clone(),
+            (route_layer_handle, stderr_layer_handle),
+            Data::new(Analytics::no_analytics()),
             true,
         ))
         .await

crates/meilisearch/tests/documents/get_documents.rs

@@ -1339,266 +1339,3 @@ async fn get_document_with_vectors() {
     }
     "###);
 }
-
-#[actix_rt::test]
-async fn test_fetch_documents_pagination_with_sorting() {
-    let server = Server::new_shared();
-    let index = server.unique_index();
-    let (task, _code) = index.create(None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    // Set name as sortable attribute
-    let (task, code) = index.update_settings_sortable_attributes(json!(["name"])).await;
-    assert_eq!(code, 202);
-    server.wait_task(task.uid()).await.succeeded();
-
-    let documents = json!((0..50)
-        .map(|i| json!({"id": i, "name": format!("doc_{:05}", std::cmp::min(i, 5))}))
-        .collect::<Vec<_>>());
-
-    // Add documents as described in the bug report
-    let (task, code) = index.add_documents(documents, None).await;
-    assert_eq!(code, 202);
-    server.wait_task(task.uid()).await.succeeded();
-
-    // Request 1 (first page): offset 0, limit 2
-    let (response, code) = index
-        .fetch_documents(json!({
-            "offset": 0,
-            "limit": 2,
-            "sort": ["name:asc"]
-        }))
-        .await;
-    assert_eq!(code, 200);
-    let results = response["results"].as_array().unwrap();
-    snapshot!(json_string!(results), @r###"
-    [
-      {
-        "id": 0,
-        "name": "doc_00000"
-      },
-      {
-        "id": 1,
-        "name": "doc_00001"
-      }
-    ]
-    "###);
-
-    // Request 2 (second page): offset 2, limit 2
-    let (response, code) = index
-        .fetch_documents(json!({
-            "offset": 2,
-            "limit": 2,
-            "sort": ["name:asc"]
-        }))
-        .await;
-    assert_eq!(code, 200);
-    let results = response["results"].as_array().unwrap();
-    snapshot!(json_string!(results), @r###"
-    [
-      {
-        "id": 2,
-        "name": "doc_00002"
-      },
-      {
-        "id": 3,
-        "name": "doc_00003"
-      }
-    ]
-    "###);
-
-    // Request 3 (third page): offset 4, limit 2
-    let (response, code) = index
-        .fetch_documents(json!({
-            "offset": 4,
-            "limit": 2,
-            "sort": ["name:asc"]
-        }))
-        .await;
-    assert_eq!(code, 200);
-    let results = response["results"].as_array().unwrap();
-    snapshot!(json_string!(results), @r###"
-    [
-      {
-        "id": 4,
-        "name": "doc_00004"
-      },
-      {
-        "id": 5,
-        "name": "doc_00005"
-      }
-    ]
-    "###);
-
-    // Request 4 (fourth page): offset 6, limit 2
-    let (response, code) = index
-        .fetch_documents(json!({
-            "offset": 6,
-            "limit": 2,
-            "sort": ["name:asc"]
-        }))
-        .await;
-    assert_eq!(code, 200);
-    let results = response["results"].as_array().unwrap();
-    snapshot!(json_string!(results), @r###"
-    [
-      {
-        "id": 6,
-        "name": "doc_00005"
-      },
-      {
-        "id": 7,
-        "name": "doc_00005"
-      }
-    ]
-    "###);
-}
-
-// <https://github.com/meilisearch/meilisearch/issues/5998>
-#[actix_rt::test]
-async fn get_document_sort_field_not_in_any_document() {
-    let server = Server::new_shared();
-    let index = server.unique_index();
-    let (task, _code) = index.create(None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (task, _code) = index.update_settings_sortable_attributes(json!(["created_at"])).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let documents = json!([
-        { "id": 1, "name": "Document 1" },
-        { "id": 2, "name": "Document 2" }
-    ]);
-    let (task, _code) = index.add_documents(documents, None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (response, code) = index
-        .fetch_documents(json!({
-            "sort": ["created_at:asc"]
-        }))
-        .await;
-
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "results": [
-        {
-          "id": 1,
-          "name": "Document 1"
-        },
-        {
-          "id": 2,
-          "name": "Document 2"
-        }
-      ],
-      "offset": 0,
-      "limit": 20,
-      "total": 2
-    }
-    "###);
-}
-
-#[actix_rt::test]
-async fn get_document_sort_includes_docs_without_field() {
-    let server = Server::new_shared();
-    let index = server.unique_index();
-    let (task, _code) = index.create(None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (task, _code) = index.update_settings_sortable_attributes(json!(["created_at"])).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let documents = json!([
-        { "id": 1, "name": "Doc without created_at" },
-        { "id": 2, "name": "Doc with created_at", "created_at": "2025-01-15" },
-        { "id": 3, "name": "Another doc without created_at" },
-        { "id": 4, "name": "Another doc with created_at", "created_at": "2025-01-10" }
-    ]);
-    let (task, _code) = index.add_documents(documents, None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (response, code) = index
-        .fetch_documents(json!({
-            "sort": ["created_at:asc"]
-        }))
-        .await;
-
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "results": [
-        {
-          "id": 4,
-          "name": "Another doc with created_at",
-          "created_at": "2025-01-10"
-        },
-        {
-          "id": 2,
-          "name": "Doc with created_at",
-          "created_at": "2025-01-15"
-        },
-        {
-          "id": 1,
-          "name": "Doc without created_at"
-        },
-        {
-          "id": 3,
-          "name": "Another doc without created_at"
-        }
-      ],
-      "offset": 0,
-      "limit": 20,
-      "total": 4
-    }
-    "###);
-}
-
-#[actix_rt::test]
-async fn get_document_sort_desc_includes_docs_without_field() {
-    let server = Server::new_shared();
-    let index = server.unique_index();
-    let (task, _code) = index.create(None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (task, _code) = index.update_settings_sortable_attributes(json!(["priority"])).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let documents = json!([
-        { "id": 1, "name": "Low priority", "priority": 1 },
-        { "id": 2, "name": "No priority" },
-        { "id": 3, "name": "High priority", "priority": 10 }
-    ]);
-    let (task, _code) = index.add_documents(documents, None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (response, code) = index
-        .fetch_documents(json!({
-            "sort": ["priority:desc"]
-        }))
-        .await;
-
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "results": [
-        {
-          "id": 3,
-          "name": "High priority",
-          "priority": 10
-        },
-        {
-          "id": 1,
-          "name": "Low priority",
-          "priority": 1
-        },
-        {
-          "id": 2,
-          "name": "No priority"
-        }
-      ],
-      "offset": 0,
-      "limit": 20,
-      "total": 3
-    }
-    "###);
-}

crates/meilisearch/tests/features/mod.rs

@@ -207,118 +207,3 @@ async fn errors() {
     }
     "###);
 }
-
-#[actix_rt::test]
-async fn search_with_personalization_without_enabling_the_feature() {
-    let server = Server::new().await;
-    let index = server.unique_index();
-
-    // Create the index and add some documents
-    let (task, _code) = index.create(None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (task, _code) = index
-        .add_documents(
-            json!([
-                {"id": 1, "title": "The Dark Knight", "genre": "Action"},
-                {"id": 2, "title": "Inception", "genre": "Sci-Fi"},
-                {"id": 3, "title": "The Matrix", "genre": "Sci-Fi"}
-            ]),
-            None,
-        )
-        .await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    // Try to search with personalization - should return feature_not_enabled error
-    let (response, code) = index
-        .search_post(json!({
-            "q": "movie",
-            "personalize": {
-                "userContext": "I love science fiction movies"
-            }
-        }))
-        .await;
-
-    meili_snap::snapshot!(code, @"400 Bad Request");
-    meili_snap::snapshot!(meili_snap::json_string!(response), @r###"
-    {
-      "message": "reranking search results requires enabling the `personalization` experimental feature. See https://github.com/orgs/meilisearch/discussions/866",
-      "code": "feature_not_enabled",
-      "type": "invalid_request",
-      "link": "https://docs.meilisearch.com/errors#feature_not_enabled"
-    }
-    "###);
-}
-
-#[actix_rt::test]
-async fn multi_search_with_personalization_without_enabling_the_feature() {
-    let server = Server::new().await;
-    let index = server.unique_index();
-
-    // Create the index and add some documents
-    let (task, _code) = index.create(None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (task, _code) = index
-        .add_documents(
-            json!([
-                {"id": 1, "title": "The Dark Knight", "genre": "Action"},
-                {"id": 2, "title": "Inception", "genre": "Sci-Fi"},
-                {"id": 3, "title": "The Matrix", "genre": "Sci-Fi"}
-            ]),
-            None,
-        )
-        .await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    // Try to multi-search with personalization - should return feature_not_enabled error
-    let (response, code) = server
-        .multi_search(json!({
-            "queries": [
-                {
-                    "indexUid": index.uid,
-                    "q": "movie",
-                    "personalize": {
-                        "userContext": "I love science fiction movies"
-                    }
-                }
-            ]
-        }))
-        .await;
-
-    meili_snap::snapshot!(code, @"400 Bad Request");
-    meili_snap::snapshot!(meili_snap::json_string!(response), @r###"
-    {
-      "message": "Inside `.queries[0]`: reranking search results requires enabling the `personalization` experimental feature. See https://github.com/orgs/meilisearch/discussions/866",
-      "code": "feature_not_enabled",
-      "type": "invalid_request",
-      "link": "https://docs.meilisearch.com/errors#feature_not_enabled"
-    }
-    "###);
-
-    // Try to federated search with personalization - should return feature_not_enabled error
-    let (response, code) = server
-        .multi_search(json!({
-          "federation": {},
-            "queries": [
-                {
-                    "indexUid": index.uid,
-                    "q": "movie",
-                    "personalize": {
-                        "userContext": "I love science fiction movies"
-                    }
-                }
-            ]
-        }))
-        .await;
-
-    meili_snap::snapshot!(code, @"400 Bad Request");
-    meili_snap::snapshot!(meili_snap::json_string!(response), @r###"
-    {
-      "message": "Inside `.queries[0]`: Using `.personalize` is not allowed in federated queries.\n - Hint: remove `personalize` from query #0 or remove `federation` from the request",
-      "code": "invalid_multi_search_query_personalization",
-      "type": "invalid_request",
-      "link": "https://docs.meilisearch.com/errors#invalid_multi_search_query_personalization"
-    }
-    "###);
-}

crates/meilisearch/tests/logs/mod.rs

@@ -8,9 +8,8 @@ use actix_web::http::header::ContentType;
 use actix_web::web::Data;
 use meili_snap::snapshot;
 use meilisearch::analytics::Analytics;
-use meilisearch::personalization::PersonalizationService;
 use meilisearch::search_queue::SearchQueue;
-use meilisearch::{create_app, Opt, ServicesData, SubscriberForSecondLayer};
+use meilisearch::{create_app, Opt, SubscriberForSecondLayer};
 use tracing::level_filters::LevelFilter;
 use tracing_subscriber::layer::SubscriberExt;
 use tracing_subscriber::Layer;
@@ -51,16 +50,12 @@ async fn basic_test_log_stream_route() {
     );
 
     let app = actix_web::test::init_service(create_app(
-        ServicesData {
+        server.service.index_scheduler.clone().into(),
-            index_scheduler: server.service.index_scheduler.clone().into(),
+        server.service.auth.clone().into(),
-            auth: server.service.auth.clone().into(),
+        Data::new(search_queue),
-            search_queue: Data::new(search_queue),
-            personalization_service: Data::new(PersonalizationService::disabled()),
-            logs_route_handle: Data::new(route_layer_handle),
-            logs_stderr_handle: Data::new(stderr_layer_handle),
-            analytics: Data::new(Analytics::no_analytics()),
-        },
         server.service.options.clone(),
+        (route_layer_handle, stderr_layer_handle),
+        Data::new(Analytics::no_analytics()),
         true,
     ))
     .await;

crates/meilisearch/tests/search/errors.rs

@@ -1,11 +1,7 @@
 use meili_snap::*;
-use meilisearch::Opt;
-use tempfile::TempDir;
 
 use super::test_settings_documents_indexing_swapping_and_search;
-use crate::common::{
+use crate::common::{shared_does_not_exists_index, Server, DOCUMENTS, NESTED_DOCUMENTS};
-    default_settings, shared_does_not_exists_index, Server, DOCUMENTS, NESTED_DOCUMENTS,
-};
 use crate::json;
 
 #[actix_rt::test]
@@ -1324,98 +1320,3 @@ async fn search_with_contains_without_enabling_the_feature() {
     }
     "#);
 }
-
-#[actix_rt::test]
-#[ignore]
-async fn search_with_personalization_invalid_api_key() {
-    // Create a server with a fake personalization API key
-    let dir = TempDir::new().unwrap();
-    let options = Opt {
-        experimental_personalization_api_key: Some("fake-api-key-12345".to_string()),
-        ..default_settings(dir.path())
-    };
-    let server = Server::new_with_options(options).await.unwrap();
-    let index = server.unique_index();
-
-    // Create the index and add some documents
-    let (task, _code) = index.create(None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (task, _code) = index
-        .add_documents(
-            json!([
-                {"id": 1, "title": "The Dark Knight", "genre": "Action"},
-                {"id": 2, "title": "Inception", "genre": "Sci-Fi"},
-                {"id": 3, "title": "The Matrix", "genre": "Sci-Fi"}
-            ]),
-            None,
-        )
-        .await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    // Try to search with personalization - should return remote_invalid_api_key error
-    let (response, code) = index
-        .search_post(json!({
-            "q": "the",
-            "personalize": {
-                "userContext": "I love science fiction movies"
-            }
-        }))
-        .await;
-
-    snapshot!(code, @"403 Forbidden");
-    snapshot!(json_string!(response), @r#"
-    {
-      "message": "Personalization service: Unauthorized: invalid API key",
-      "code": "remote_invalid_api_key",
-      "type": "auth",
-      "link": "https://docs.meilisearch.com/errors#remote_invalid_api_key"
-    }
-    "#);
-}
-
-#[actix_rt::test]
-async fn search_with_personalization_no_user_context() {
-    // Create a server with a fake personalization API key
-    let dir = TempDir::new().unwrap();
-    let options = Opt {
-        experimental_personalization_api_key: Some("fake-api-key-12345".to_string()),
-        ..default_settings(dir.path())
-    };
-    let server = Server::new_with_options(options).await.unwrap();
-    let index = server.unique_index();
-
-    // Create the index and add some documents
-    let (task, _code) = index.create(None).await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    let (task, _code) = index
-        .add_documents(
-            json!([
-                {"id": 1, "title": "The Dark Knight", "genre": "Action"},
-                {"id": 2, "title": "Inception", "genre": "Sci-Fi"},
-                {"id": 3, "title": "The Matrix", "genre": "Sci-Fi"}
-            ]),
-            None,
-        )
-        .await;
-    server.wait_task(task.uid()).await.succeeded();
-
-    // Try to search with personalization - should return remote_invalid_api_key error
-    let (response, code) = index
-        .search_post(json!({
-            "q": "the",
-            "personalize": {}
-        }))
-        .await;
-
-    snapshot!(code, @"400 Bad Request");
-    snapshot!(json_string!(response), @r###"
-    {
-      "message": "Missing field `userContext` inside `.personalize`",
-      "code": "invalid_search_personalize",
-      "type": "invalid_request",
-      "link": "https://docs.meilisearch.com/errors#invalid_search_personalize"
-    }
-    "###);
-}

crates/meilisearch/tests/search/hybrid.rs

@@ -137,60 +137,6 @@ static SIMPLE_SEARCH_DOCUMENTS: Lazy<Value> = Lazy::new(|| {
     }])
 });
 
-static MANY_DOCS: Lazy<Value> = Lazy::new(|| {
-    json!([
-    {
-        "title": "Shazam!",
-        "desc": "a Captain Marvel ersatz",
-        "id": "1",
-    },
-    {
-        "title": "Captain Planet",
-        "desc": "He's not part of the Marvel Cinematic Universe",
-        "id": "2",
-    },
-    {
-        "title": "Captain Marvel",
-        "desc": "a Shazam ersatz",
-        "id": "3",
-    },
-        {
-        "title": "Captain Marvel",
-        "desc": "a Shazam ersatz",
-        "id": "4",
-    },
-        {
-        "title": "Captain Marvel",
-        "desc": "a Shazam ersatz",
-        "id": "5",
-    },
-        {
-        "title": "Captain Marvel",
-        "desc": "a Shazam ersatz",
-        "id": "6",
-    },
-        {
-        "title": "Captain Marvel",
-        "desc": "a Shazam ersatz",
-        "id": "7",
-    },
-        {
-        "title": "Captain Marvel",
-        "desc": "a Shazam ersatz",
-        "id": "8",
-    },
-        {
-        "title": "Captain Marvel",
-        "desc": "a Shazam ersatz",
-        "id": "9",
-    },
-        {
-        "title": "Captain Marvel",
-        "desc": "a Shazam ersatz",
-        "id": "10",
-    }])
-});
-
 #[actix_rt::test]
 async fn simple_search() {
     let server = Server::new_shared();
@@ -503,38 +449,6 @@ async fn simple_search_hf() {
     snapshot!(response["semanticHitCount"], @"3");
 }
 
-#[actix_rt::test]
-async fn issue_5976_missing_docs_hf() {
-    let server = Server::new_shared();
-    let index = index_with_documents_hf(server, &MANY_DOCS).await;
-    let (response, code) = index
-        .search_post(
-            json!({"q": "Wonder replacement", "hybrid": {"embedder": "default", "semanticRatio": 1.0}, "retrieveVectors": true}),
-        )
-        .await;
-    snapshot!(code, @"200 OK");
-    let are_empty: Vec<_> = response["hits"]
-        .as_array()
-        .unwrap()
-        .iter()
-        .map(|hit| hit["_vectors"]["default"]["embeddings"].as_array().unwrap().is_empty())
-        .collect();
-    snapshot!(json!(are_empty), @r###"
-    [
-      false,
-      false,
-      false,
-      false,
-      false,
-      false,
-      false,
-      false,
-      false,
-      false
-    ]
-    "###);
-}
-
 #[actix_rt::test]
 async fn distribution_shift() {
     let server = Server::new_shared();

crates/meilisearch/tests/search/multi/proxy.rs

@@ -3141,537 +3141,3 @@ fn fail(override_response_body: Option<&str>) -> ResponseTemplate {
         response.set_body_json(json!({"error": "provoked error", "code": "test_error", "link": "https://docs.meilisearch.com/errors#test_error"}))
     }
 }
-
-#[cfg(feature = "enterprise")]
-#[actix_rt::test]
-async fn remote_auto_sharding() {
-    let ms0 = Server::new().await;
-    let ms1 = Server::new().await;
-    let ms2 = Server::new().await;
-
-    // enable feature
-
-    let (response, code) = ms0.set_features(json!({"network": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response["network"]), @"true");
-    let (response, code) = ms1.set_features(json!({"network": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response["network"]), @"true");
-    let (response, code) = ms2.set_features(json!({"network": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response["network"]), @"true");
-
-    // set self & sharding
-    let (response, code) = ms0.set_network(json!({"self": "ms0", "sharding": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "self": "ms0",
-      "remotes": {},
-      "sharding": true
-    }
-    "###);
-    let (response, code) = ms1.set_network(json!({"self": "ms1", "sharding": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "self": "ms1",
-      "remotes": {},
-      "sharding": true
-    }
-    "###);
-    let (response, code) = ms2.set_network(json!({"self": "ms2", "sharding": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "self": "ms2",
-      "remotes": {},
-      "sharding": true
-    }
-    "###);
-
-    // wrap servers
-    let ms0 = Arc::new(ms0);
-    let ms1 = Arc::new(ms1);
-    let ms2 = Arc::new(ms2);
-
-    let rms0 = LocalMeili::new(ms0.clone()).await;
-    let rms1 = LocalMeili::new(ms1.clone()).await;
-    let rms2 = LocalMeili::new(ms2.clone()).await;
-
-    // set network
-    let network = json!({"remotes": {
-        "ms0": {
-            "url": rms0.url()
-        },
-        "ms1": {
-            "url": rms1.url()
-        },
-        "ms2": {
-            "url": rms2.url()
-        }
-    }});
-
-    println!("{}", serde_json::to_string_pretty(&network).unwrap());
-
-    let (_response, status_code) = ms0.set_network(network.clone()).await;
-    snapshot!(status_code, @"200 OK");
-    let (_response, status_code) = ms1.set_network(network.clone()).await;
-    snapshot!(status_code, @"200 OK");
-    let (_response, status_code) = ms2.set_network(network.clone()).await;
-    snapshot!(status_code, @"200 OK");
-
-    // add documents
-    let documents = SCORE_DOCUMENTS.clone();
-    let documents = documents.as_array().unwrap();
-    let index0 = ms0.index("test");
-    let _index1 = ms1.index("test");
-    let _index2 = ms2.index("test");
-
-    let (task, _status_code) = index0.add_documents(json!(documents), None).await;
-
-    let t0 = task.uid();
-    let (t, _) = ms0.get_task(task.uid()).await;
-    let t1 = t["network"]["remote_tasks"]["ms1"]["taskUid"].as_u64().unwrap();
-    let t2 = t["network"]["remote_tasks"]["ms2"]["taskUid"].as_u64().unwrap();
-
-    ms0.wait_task(t0).await.succeeded();
-    ms1.wait_task(t1).await.succeeded();
-    ms2.wait_task(t2).await.succeeded();
-
-    // perform multi-search
-    let query = "badman returns";
-    let request = json!({
-        "federation": {},
-        "queries": [
-            {
-                "q": query,
-                "indexUid": "test",
-                "federationOptions": {
-                    "remote": "ms0"
-                }
-            },
-            {
-                "q": query,
-                "indexUid": "test",
-                "federationOptions": {
-                    "remote": "ms1"
-                }
-            },
-            {
-                "q": query,
-                "indexUid": "test",
-                "federationOptions": {
-                    "remote": "ms2"
-                }
-            },
-        ]
-    });
-
-    let (response, _status_code) = ms0.multi_search(request.clone()).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response, { ".processingTimeMs" => "[time]", ".requestUid" => "[uuid]" }), @r###"
-    {
-      "hits": [
-        {
-          "title": "Batman Returns",
-          "id": "C",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 2,
-            "weightedRankingScore": 0.8317901234567902,
-            "remote": "ms2"
-          }
-        },
-        {
-          "title": "Batman the dark knight returns: Part 1",
-          "id": "A",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 1,
-            "weightedRankingScore": 0.7028218694885362,
-            "remote": "ms1"
-          }
-        },
-        {
-          "title": "Batman the dark knight returns: Part 2",
-          "id": "B",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 1,
-            "weightedRankingScore": 0.7028218694885362,
-            "remote": "ms1"
-          }
-        },
-        {
-          "title": "Badman",
-          "id": "E",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 2,
-            "weightedRankingScore": 0.5,
-            "remote": "ms2"
-          }
-        },
-        {
-          "title": "Batman",
-          "id": "D",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 0,
-            "weightedRankingScore": 0.23106060606060605,
-            "remote": "ms0"
-          }
-        }
-      ],
-      "processingTimeMs": "[time]",
-      "limit": 20,
-      "offset": 0,
-      "estimatedTotalHits": 5,
-      "requestUid": "[uuid]",
-      "remoteErrors": {}
-    }
-    "###);
-    let (response, _status_code) = ms1.multi_search(request.clone()).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response, { ".processingTimeMs" => "[time]", ".requestUid" => "[uuid]" }), @r###"
-    {
-      "hits": [
-        {
-          "title": "Batman Returns",
-          "id": "C",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 2,
-            "weightedRankingScore": 0.8317901234567902,
-            "remote": "ms2"
-          }
-        },
-        {
-          "title": "Batman the dark knight returns: Part 1",
-          "id": "A",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 1,
-            "weightedRankingScore": 0.7028218694885362,
-            "remote": "ms1"
-          }
-        },
-        {
-          "title": "Batman the dark knight returns: Part 2",
-          "id": "B",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 1,
-            "weightedRankingScore": 0.7028218694885362,
-            "remote": "ms1"
-          }
-        },
-        {
-          "title": "Badman",
-          "id": "E",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 2,
-            "weightedRankingScore": 0.5,
-            "remote": "ms2"
-          }
-        },
-        {
-          "title": "Batman",
-          "id": "D",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 0,
-            "weightedRankingScore": 0.23106060606060605,
-            "remote": "ms0"
-          }
-        }
-      ],
-      "processingTimeMs": "[time]",
-      "limit": 20,
-      "offset": 0,
-      "estimatedTotalHits": 5,
-      "requestUid": "[uuid]",
-      "remoteErrors": {}
-    }
-    "###);
-    let (response, _status_code) = ms2.multi_search(request.clone()).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response, { ".processingTimeMs" => "[time]", ".requestUid" => "[uuid]" }), @r###"
-    {
-      "hits": [
-        {
-          "title": "Batman Returns",
-          "id": "C",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 2,
-            "weightedRankingScore": 0.8317901234567902,
-            "remote": "ms2"
-          }
-        },
-        {
-          "title": "Batman the dark knight returns: Part 1",
-          "id": "A",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 1,
-            "weightedRankingScore": 0.7028218694885362,
-            "remote": "ms1"
-          }
-        },
-        {
-          "title": "Batman the dark knight returns: Part 2",
-          "id": "B",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 1,
-            "weightedRankingScore": 0.7028218694885362,
-            "remote": "ms1"
-          }
-        },
-        {
-          "title": "Badman",
-          "id": "E",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 2,
-            "weightedRankingScore": 0.5,
-            "remote": "ms2"
-          }
-        },
-        {
-          "title": "Batman",
-          "id": "D",
-          "_federation": {
-            "indexUid": "test",
-            "queriesPosition": 0,
-            "weightedRankingScore": 0.23106060606060605,
-            "remote": "ms0"
-          }
-        }
-      ],
-      "processingTimeMs": "[time]",
-      "limit": 20,
-      "offset": 0,
-      "estimatedTotalHits": 5,
-      "requestUid": "[uuid]",
-      "remoteErrors": {}
-    }
-    "###);
-}
-
-#[cfg(not(feature = "enterprise"))]
-#[actix_rt::test]
-async fn sharding_not_enterprise() {
-    let ms0 = Server::new().await;
-
-    // enable feature
-
-    let (response, code) = ms0.set_features(json!({"network": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response["network"]), @"true");
-
-    let (response, code) = ms0.set_network(json!({"self": "ms0", "sharding": true})).await;
-    snapshot!(code, @"451 Unavailable For Legal Reasons");
-    snapshot!(json_string!(response), @r###"
-    {
-      "message": "Meilisearch Enterprise Edition is required to set `network.sharding`",
-      "code": "requires_enterprise_edition",
-      "type": "invalid_request",
-      "link": "https://docs.meilisearch.com/errors#requires_enterprise_edition"
-    }
-    "###);
-}
-
-#[cfg(feature = "enterprise")]
-#[actix_rt::test]
-async fn remote_auto_sharding_with_custom_metadata() {
-    let ms0 = Server::new().await;
-    let ms1 = Server::new().await;
-    let ms2 = Server::new().await;
-
-    // enable feature
-
-    let (response, code) = ms0.set_features(json!({"network": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response["network"]), @"true");
-    let (response, code) = ms1.set_features(json!({"network": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response["network"]), @"true");
-    let (response, code) = ms2.set_features(json!({"network": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response["network"]), @"true");
-
-    // set self & sharding
-
-    let (response, code) = ms0.set_network(json!({"self": "ms0", "sharding": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "self": "ms0",
-      "remotes": {},
-      "sharding": true
-    }
-    "###);
-    let (response, code) = ms1.set_network(json!({"self": "ms1", "sharding": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "self": "ms1",
-      "remotes": {},
-      "sharding": true
-    }
-    "###);
-    let (response, code) = ms2.set_network(json!({"self": "ms2", "sharding": true})).await;
-    snapshot!(code, @"200 OK");
-    snapshot!(json_string!(response), @r###"
-    {
-      "self": "ms2",
-      "remotes": {},
-      "sharding": true
-    }
-    "###);
-
-    // wrap servers
-    let ms0 = Arc::new(ms0);
-    let ms1 = Arc::new(ms1);
-    let ms2 = Arc::new(ms2);
-
-    let rms0 = LocalMeili::new(ms0.clone()).await;
-    let rms1 = LocalMeili::new(ms1.clone()).await;
-    let rms2 = LocalMeili::new(ms2.clone()).await;
-
-    // set network
-    let network = json!({"remotes": {
-        "ms0": {
-            "url": rms0.url()
-        },
-        "ms1": {
-            "url": rms1.url()
-        },
-        "ms2": {
-            "url": rms2.url()
-        }
-    }});
-
-    println!("{}", serde_json::to_string_pretty(&network).unwrap());
-
-    let (_response, status_code) = ms0.set_network(network.clone()).await;
-    snapshot!(status_code, @"200 OK");
-    let (_response, status_code) = ms1.set_network(network.clone()).await;
-    snapshot!(status_code, @"200 OK");
-    let (_response, status_code) = ms2.set_network(network.clone()).await;
-    snapshot!(status_code, @"200 OK");
-
-    // add documents
-    let documents = SCORE_DOCUMENTS.clone();
-    let documents = documents.as_array().unwrap();
-    let index0 = ms0.index("test");
-    let _index1 = ms1.index("test");
-    let _index2 = ms2.index("test");
-
-    let (task, _status_code) = index0
-        .add_documents_with_custom_metadata(
-            json!(documents),
-            None,
-            Some("remote_auto_sharding_with_custom_metadata"),
-        )
-        .await;
-
-    let t0 = task.uid();
-    let (t, _) = ms0.get_task(task.uid()).await;
-    let t1 = t["network"]["remote_tasks"]["ms1"]["taskUid"].as_u64().unwrap();
-    let t2 = t["network"]["remote_tasks"]["ms2"]["taskUid"].as_u64().unwrap();
-
-    let t = ms0.wait_task(t0).await.succeeded();
-    snapshot!(t, @r###"
-    {
-      "uid": "[uid]",
-      "batchUid": "[batch_uid]",
-      "indexUid": "test",
-      "status": "succeeded",
-      "type": "documentAdditionOrUpdate",
-      "canceledBy": null,
-      "details": {
-        "receivedDocuments": 5,
-        "indexedDocuments": 1
-      },
-      "error": null,
-      "duration": "[duration]",
-      "enqueuedAt": "[date]",
-      "startedAt": "[date]",
-      "finishedAt": "[date]",
-      "network": {
-        "remote_tasks": {
-          "ms1": {
-            "taskUid": 0,
-            "error": null
-          },
-          "ms2": {
-            "taskUid": 0,
-            "error": null
-          }
-        }
-      },
-      "customMetadata": "remote_auto_sharding_with_custom_metadata"
-    }
-    "###);
-
-    let t = ms1.wait_task(t1).await.succeeded();
-    snapshot!(t, @r###"
-    {
-      "uid": "[uid]",
-      "batchUid": "[batch_uid]",
-      "indexUid": "test",
-      "status": "succeeded",
-      "type": "documentAdditionOrUpdate",
-      "canceledBy": null,
-      "details": {
-        "receivedDocuments": 5,
-        "indexedDocuments": 2
-      },
-      "error": null,
-      "duration": "[duration]",
-      "enqueuedAt": "[date]",
-      "startedAt": "[date]",
-      "finishedAt": "[date]",
-      "network": {
-        "origin": {
-          "remoteName": "ms0",
-          "taskUid": 0
-        }
-      },
-      "customMetadata": "remote_auto_sharding_with_custom_metadata"
-    }
-    "###);
-
-    let t = ms2.wait_task(t2).await.succeeded();
-    snapshot!(t, @r###"
-    {
-      "uid": "[uid]",
-      "batchUid": "[batch_uid]",
-      "indexUid": "test",
-      "status": "succeeded",
-      "type": "documentAdditionOrUpdate",
-      "canceledBy": null,
-      "details": {
-        "receivedDocuments": 5,
-        "indexedDocuments": 2
-      },
-      "error": null,
-      "duration": "[duration]",
-      "enqueuedAt": "[date]",
-      "startedAt": "[date]",
-      "finishedAt": "[date]",
-      "network": {
-        "origin": {
-          "remoteName": "ms0",
-          "taskUid": 0
-        }
-      },
-      "customMetadata": "remote_auto_sharding_with_custom_metadata"
-    }
-    "###);
-}