Update milli dependency to filter/field-in

2564: Bump codecov/codecov-action from 1 to 3 r=curquiza a=dependabot[bot]

Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/codecov/codecov-action/releases">codecov/codecov-action's releases</a>.</em></p>
<blockquote>
<h2>v3.0.0</h2>
<h3>Breaking Changes</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/689">#689</a> Bump to node16 and small fixes</li>
</ul>
<h3>Features</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/688">#688</a> Incorporate <code>gcov</code> arguments for the Codecov uploader</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/548">#548</a> build(deps-dev): bump jest-junit from 12.2.0 to 13.0.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/603">#603</a> [Snyk] Upgrade <code>`@​actions/core</code>` from 1.5.0 to 1.6.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/628">#628</a> build(deps): bump node-fetch from 2.6.1 to 3.1.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/634">#634</a> build(deps): bump node-fetch from 3.1.1 to 3.2.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/636">#636</a> build(deps): bump openpgp from 5.0.1 to 5.1.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/652">#652</a> build(deps-dev): bump <code>`@​vercel/ncc</code>` from 0.30.0 to 0.33.3</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/653">#653</a> build(deps-dev): bump <code>`@​types/node</code>` from 16.11.21 to 17.0.18</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/659">#659</a> build(deps-dev): bump <code>`@​types/jest</code>` from 27.4.0 to 27.4.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/667">#667</a> build(deps): bump actions/checkout from 2 to 3</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/673">#673</a> build(deps): bump node-fetch from 3.2.0 to 3.2.3</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/683">#683</a> build(deps): bump minimist from 1.2.5 to 1.2.6</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/685">#685</a> build(deps): bump <code>`@​actions/github</code>` from 5.0.0 to 5.0.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/681">#681</a> build(deps-dev): bump <code>`@​types/node</code>` from 17.0.18 to 17.0.23</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/682">#682</a> build(deps-dev): bump typescript from 4.5.5 to 4.6.3</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/676">#676</a> build(deps): bump <code>`@​actions/exec</code>` from 1.1.0 to 1.1.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/675">#675</a> build(deps): bump openpgp from 5.1.0 to 5.2.1</li>
</ul>
<h2>v2.1.0</h2>
<h2>2.1.0</h2>
<h3>Features</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/515">#515</a> Allow specifying version of Codecov uploader</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/499">#499</a> build(deps-dev): bump <code>`@​vercel/ncc</code>` from 0.29.0 to 0.30.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/508">#508</a> build(deps): bump openpgp from 5.0.0-5 to 5.0.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/514">#514</a> build(deps-dev): bump <code>`@​types/node</code>` from 16.6.0 to 16.9.0</li>
</ul>
<h2>v2.0.3</h2>
<h2>2.0.3</h2>
<h3>Fixes</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/464">#464</a> Fix wrong link in the readme</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/485">#485</a> fix: Add override OS and linux default to platform</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/447">#447</a> build(deps): bump openpgp from 5.0.0-4 to 5.0.0-5</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/458">#458</a> build(deps-dev): bump eslint from 7.31.0 to 7.32.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/465">#465</a> build(deps-dev): bump <code>`@​typescript-eslint/eslint-plugin</code>` from 4.28.4 to 4.29.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/466">#466</a> build(deps-dev): bump <code>`@​typescript-eslint/parser</code>` from 4.28.4 to 4.29.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/468">#468</a> build(deps-dev): bump <code>`@​types/jest</code>` from 26.0.24 to 27.0.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/470">#470</a> build(deps-dev): bump <code>`@​types/node</code>` from 16.4.0 to 16.6.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/472">#472</a> build(deps): bump path-parse from 1.0.6 to 1.0.7</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/473">#473</a> build(deps-dev): bump <code>`@​types/jest</code>` from 27.0.0 to 27.0.1</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md">codecov/codecov-action's changelog</a>.</em></p>
<blockquote>
<h2>3.1.0</h2>
<h3>Features</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/699">#699</a> Incorporate <code>xcode</code> arguments for the Codecov uploader</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/694">#694</a> build(deps-dev): bump <code>`@​vercel/ncc</code>` from 0.33.3 to 0.33.4</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/696">#696</a> build(deps-dev): bump <code>`@​types/node</code>` from 17.0.23 to 17.0.25</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/698">#698</a> build(deps-dev): bump jest-junit from 13.0.0 to 13.2.0</li>
</ul>
<h2>3.0.0</h2>
<h3>Breaking Changes</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/689">#689</a> Bump to node16 and small fixes</li>
</ul>
<h3>Features</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/688">#688</a> Incorporate <code>gcov</code> arguments for the Codecov uploader</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/548">#548</a> build(deps-dev): bump jest-junit from 12.2.0 to 13.0.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/603">#603</a> [Snyk] Upgrade <code>`@​actions/core</code>` from 1.5.0 to 1.6.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/628">#628</a> build(deps): bump node-fetch from 2.6.1 to 3.1.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/634">#634</a> build(deps): bump node-fetch from 3.1.1 to 3.2.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/636">#636</a> build(deps): bump openpgp from 5.0.1 to 5.1.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/652">#652</a> build(deps-dev): bump <code>`@​vercel/ncc</code>` from 0.30.0 to 0.33.3</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/653">#653</a> build(deps-dev): bump <code>`@​types/node</code>` from 16.11.21 to 17.0.18</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/659">#659</a> build(deps-dev): bump <code>`@​types/jest</code>` from 27.4.0 to 27.4.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/667">#667</a> build(deps): bump actions/checkout from 2 to 3</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/673">#673</a> build(deps): bump node-fetch from 3.2.0 to 3.2.3</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/683">#683</a> build(deps): bump minimist from 1.2.5 to 1.2.6</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/685">#685</a> build(deps): bump <code>`@​actions/github</code>` from 5.0.0 to 5.0.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/681">#681</a> build(deps-dev): bump <code>`@​types/node</code>` from 17.0.18 to 17.0.23</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/682">#682</a> build(deps-dev): bump typescript from 4.5.5 to 4.6.3</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/676">#676</a> build(deps): bump <code>`@​actions/exec</code>` from 1.1.0 to 1.1.1</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/675">#675</a> build(deps): bump openpgp from 5.1.0 to 5.2.1</li>
</ul>
<h2>2.1.0</h2>
<h3>Features</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/515">#515</a> Allow specifying version of Codecov uploader</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/499">#499</a> build(deps-dev): bump <code>`@​vercel/ncc</code>` from 0.29.0 to 0.30.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/508">#508</a> build(deps): bump openpgp from 5.0.0-5 to 5.0.0</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/514">#514</a> build(deps-dev): bump <code>`@​types/node</code>` from 16.6.0 to 16.9.0</li>
</ul>
<h2>2.0.3</h2>
<h3>Fixes</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/464">#464</a> Fix wrong link in the readme</li>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/485">#485</a> fix: Add override OS and linux default to platform</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/447">#447</a> build(deps): bump openpgp from 5.0.0-4 to 5.0.0-5</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="81cd2dc814"><code>81cd2dc</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/699">#699</a> from codecov/feat-xcode</li>
<li><a href="a03184e530"><code>a03184e</code></a> feat: add xcode support</li>
<li><a href="6a6a9ae7b1"><code>6a6a9ae</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/694">#694</a> from codecov/dependabot/npm_and_yarn/vercel/ncc-0.33.4</li>
<li><a href="92a872a5e7"><code>92a872a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/696">#696</a> from codecov/dependabot/npm_and_yarn/types/node-17.0.25</li>
<li><a href="43a9c182dd"><code>43a9c18</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/698">#698</a> from codecov/dependabot/npm_and_yarn/jest-junit-13.2.0</li>
<li><a href="13ce822ccd"><code>13ce822</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/codecov/codecov-action/issues/690">#690</a> from codecov/ci-v3</li>
<li><a href="4d6dbaaea6"><code>4d6dbaa</code></a> build(deps-dev): bump jest-junit from 13.0.0 to 13.2.0</li>
<li><a href="98f0f19300"><code>98f0f19</code></a> build(deps-dev): bump <code>`@​types/node</code>` from 17.0.23 to 17.0.25</li>
<li><a href="d3021d9910"><code>d3021d9</code></a> build(deps-dev): bump <code>`@​vercel/ncc</code>` from 0.33.3 to 0.33.4</li>
<li><a href="2c83f35c20"><code>2c83f35</code></a> Update makefile to v3</li>
<li>Additional commits viewable in <a href="https://github.com/codecov/codecov-action/compare/v1...v3">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=codecov/codecov-action&package-manager=github_actions&previous-version=1&new-version=3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

2565: Bump docker/setup-qemu-action from 1 to 2 r=curquiza a=dependabot[bot]

Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 1 to 2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/docker/setup-qemu-action/releases">docker/setup-qemu-action's releases</a>.</em></p>
<blockquote>
<h2>v2.0.0</h2>
<ul>
<li>Node 16 as default runtime by <a href="https://github.com/crazy-max"><code>`@​crazy-max</code></a>` (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/48">#48</a>)
<ul>
<li>This requires a minimum <a href="https://github.com/actions/runner/releases/tag/v2.285.0">Actions Runner</a> version of v2.285.0, which is by default available in GHES 3.4 or later.</li>
</ul>
</li>
<li>chore: update dev dependencies and workflow by <a href="https://github.com/crazy-max"><code>`@​crazy-max</code></a>` (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/43">#43</a> <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/47">#47</a>)</li>
<li>Bump <code>`@​actions/core</code>` from 1.3.0 to 1.6.0 (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/37">#37</a> <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/39">#39</a> <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/41">#41</a>)</li>
<li>Bump <code>`@​actions/exec</code>` from 1.0.4 to 1.1.1 (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/38">#38</a> <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/46">#46</a>)</li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/docker/setup-qemu-action/compare/v1.2.0...v2.0.0">https://github.com/docker/setup-qemu-action/compare/v1.2.0...v2.0.0</a></p>
<h2>v1.2.0</h2>
<ul>
<li>Display image information (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/36">#36</a>)</li>
<li>Bump <code>`@​actions/core</code>` from 1.2.7 to 1.3.0 (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/35">#35</a>)</li>
</ul>
<h2>v1.1.0</h2>
<ul>
<li>Remove os limitation (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/30">#30</a>)</li>
<li>Bump <code>`@​actions/core</code>` from 1.2.6 to 1.2.7 (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/29">#29</a>)</li>
</ul>
<h2>v1.0.2</h2>
<ul>
<li>Enhance workflow (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/26">#26</a>)</li>
<li>Container based developer flow (<a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/19">#19</a> <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/20">#20</a>)</li>
</ul>
<h2>v1.0.1</h2>
<ul>
<li>Fix CVE-2020-15228</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="8b122486ce"><code>8b12248</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/48">#48</a> from crazy-max/node-16</li>
<li><a href="466d53193c"><code>466d531</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/50">#50</a> from crazy-max/update-readme</li>
<li><a href="607c1922b5"><code>607c192</code></a> simplify usage example</li>
<li><a href="d7849ecb9c"><code>d7849ec</code></a> Node 16 as default runtime</li>
<li><a href="2d4bfe71c9"><code>2d4bfe7</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/47">#47</a> from crazy-max/update-dev</li>
<li><a href="224b802eb3"><code>224b802</code></a> chore: update dev dependencies and workflow</li>
<li><a href="95bd865778"><code>95bd865</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/46">#46</a> from docker/dependabot/npm_and_yarn/actions/exec-1.1.1</li>
<li><a href="cfd091faa1"><code>cfd091f</code></a> Bump <code>`@​actions/exec</code>` from 1.1.0 to 1.1.1</li>
<li><a href="d2a60302b8"><code>d2a6030</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/docker/setup-qemu-action/issues/45">#45</a> from docker/dependabot/github_actions/actions/checkout-3</li>
<li><a href="97dc484a91"><code>97dc484</code></a> Bump actions/checkout from 2 to 3</li>
<li>Additional commits viewable in <a href="https://github.com/docker/setup-qemu-action/compare/v1...v2">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docker/setup-qemu-action&package-manager=github_actions&previous-version=1&new-version=2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

2566: Bump actions/checkout from 2 to 3 r=curquiza a=dependabot[bot]

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/actions/checkout/releases">actions/checkout's releases</a>.</em></p>
<blockquote>
<h2>v3.0.0</h2>
<ul>
<li>Updated to the node16 runtime by default
<ul>
<li>This requires a minimum <a href="https://github.com/actions/runner/releases/tag/v2.285.0">Actions Runner</a> version of v2.285.0 to run, which is by default available in GHES 3.4 or later.</li>
</ul>
</li>
</ul>
<h2>v2.4.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Add set-safe-directory input to allow customers to take control. (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/770">#770</a>) by <a href="https://github.com/TingluoHuang"><code>`@​TingluoHuang</code></a>` in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/776">actions/checkout#776</a></li>
<li>Prepare changelog for v2.4.2. by <a href="https://github.com/TingluoHuang"><code>`@​TingluoHuang</code></a>` in <a href="https://github-redirect.dependabot.com/actions/checkout/pull/778">actions/checkout#778</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/actions/checkout/compare/v2...v2.4.2">https://github.com/actions/checkout/compare/v2...v2.4.2</a></p>
<h2>v2.4.1</h2>
<ul>
<li>Fixed an issue where checkout failed to run in container jobs due to the new git setting <code>safe.directory</code></li>
</ul>
<h2>v2.4.0</h2>
<ul>
<li>Convert SSH URLs like <code>org-&lt;ORG_ID&gt;`@github.com:</code>` to <code>https://github.com/</code> - <a href="https://github-redirect.dependabot.com/actions/checkout/pull/621">pr</a></li>
</ul>
<h2>v2.3.5</h2>
<p>Update dependencies</p>
<h2>v2.3.4</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/379">Add missing <code>await</code>s</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/360">Swap to Environment Files</a></li>
</ul>
<h2>v2.3.3</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/345">Remove Unneeded commit information from build logs</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/326">Add Licensed to verify third party dependencies</a></li>
</ul>
<h2>v2.3.2</h2>
<p><a href="https://github-redirect.dependabot.com/actions/checkout/pull/320">Add Third Party License Information to Dist Files</a></p>
<h2>v2.3.1</h2>
<p><a href="https://github-redirect.dependabot.com/actions/checkout/pull/284">Fix default branch resolution for .wiki and when using SSH</a></p>
<h2>v2.3.0</h2>
<p><a href="https://github-redirect.dependabot.com/actions/checkout/pull/278">Fallback to the default branch</a></p>
<h2>v2.2.0</h2>
<p><a href="https://github-redirect.dependabot.com/actions/checkout/pull/258">Fetch all history for all tags and branches when fetch-depth=0</a></p>
<h2>v2.1.1</h2>
<p>Changes to support GHES (<a href="https://github-redirect.dependabot.com/actions/checkout/pull/236">here</a> and <a href="https://github-redirect.dependabot.com/actions/checkout/pull/248">here</a>)</p>
<h2>v2.1.0</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/191">Group output</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/199">Changes to support GHES alpha release</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/184">Persist core.sshCommand for submodules</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/163">Add support ssh</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/179">Convert submodule SSH URL to HTTPS, when not using SSH</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/actions/checkout/blob/main/CHANGELOG.md">actions/checkout's changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<h2>v3.0.2</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/770">Add input <code>set-safe-directory</code></a></li>
</ul>
<h2>v3.0.1</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/762">Fixed an issue where checkout failed to run in container jobs due to the new git setting <code>safe.directory</code></a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/744">Bumped various npm package versions</a></li>
</ul>
<h2>v3.0.0</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/689">Update to node 16</a></li>
</ul>
<h2>v2.3.1</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/284">Fix default branch resolution for .wiki and when using SSH</a></li>
</ul>
<h2>v2.3.0</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/278">Fallback to the default branch</a></li>
</ul>
<h2>v2.2.0</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/258">Fetch all history for all tags and branches when fetch-depth=0</a></li>
</ul>
<h2>v2.1.1</h2>
<ul>
<li>Changes to support GHES (<a href="https://github-redirect.dependabot.com/actions/checkout/pull/236">here</a> and <a href="https://github-redirect.dependabot.com/actions/checkout/pull/248">here</a>)</li>
</ul>
<h2>v2.1.0</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/191">Group output</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/199">Changes to support GHES alpha release</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/184">Persist core.sshCommand for submodules</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/163">Add support ssh</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/179">Convert submodule SSH URL to HTTPS, when not using SSH</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/157">Add submodule support</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/144">Follow proxy settings</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/141">Fix ref for pr closed event when a pr is merged</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/128">Fix issue checking detached when git less than 2.22</a></li>
</ul>
<h2>v2.0.0</h2>
<ul>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/108">Do not pass cred on command line</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/107">Add input persist-credentials</a></li>
<li><a href="https://github-redirect.dependabot.com/actions/checkout/pull/104">Fallback to REST API to download repo</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="2541b1294d"><code>2541b12</code></a> Prepare changelog for v3.0.2. (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/777">#777</a>)</li>
<li><a href="0ffe6f9c55"><code>0ffe6f9</code></a> Add set-safe-directory input to allow customers to take control. (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/770">#770</a>)</li>
<li><a href="dcd71f6466"><code>dcd71f6</code></a> Enforce safe directory (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/762">#762</a>)</li>
<li><a href="add3486cc3"><code>add3486</code></a> Patch to fix the dependbot alert. (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/744">#744</a>)</li>
<li><a href="5126516654"><code>5126516</code></a> Bump minimist from 1.2.5 to 1.2.6 (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/741">#741</a>)</li>
<li><a href="d50f8ea767"><code>d50f8ea</code></a> Add v3.0 release information to changelog (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/740">#740</a>)</li>
<li><a href="2d1c1198e7"><code>2d1c119</code></a> update test workflows to checkout v3 (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/709">#709</a>)</li>
<li><a href="a12a3943b4"><code>a12a394</code></a> update readme for v3 (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/708">#708</a>)</li>
<li><a href="8f9e05e482"><code>8f9e05e</code></a> Update to node 16 (<a href="https://github-redirect.dependabot.com/actions/checkout/issues/689">#689</a>)</li>
<li>See full diff in <a href="https://github.com/actions/checkout/compare/v2...v3">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/checkout&package-manager=github_actions&previous-version=2&new-version=3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

2567: Bump docker/setup-buildx-action from 1 to 2 r=curquiza a=dependabot[bot]

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1 to 2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/docker/setup-buildx-action/releases">docker/setup-buildx-action's releases</a>.</em></p>
<blockquote>
<h2>v2.0.0</h2>
<ul>
<li>Node 16 as default runtime by <a href="https://github.com/crazy-max"><code>`@​crazy-max</code></a>` (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/131">#131</a>)
<ul>
<li>This requires a minimum <a href="https://github.com/actions/runner/releases/tag/v2.285.0">Actions Runner</a> version of v2.285.0, which is by default available in GHES 3.4 or later.</li>
</ul>
</li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/docker/setup-buildx-action/compare/v1.7.0...v2.0.0">https://github.com/docker/setup-buildx-action/compare/v1.7.0...v2.0.0</a></p>
<h2>v1.7.0</h2>
<ul>
<li>Standalone mode by <a href="https://github.com/crazy-max"><code>`@​crazy-max</code></a>` in (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/119">#119</a>)</li>
<li>Update dev dependencies and workflow by <a href="https://github.com/crazy-max"><code>`@​crazy-max</code></a>` (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/114">#114</a> <a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/130">#130</a>)</li>
<li>Bump tmpl from 1.0.4 to 1.0.5 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/108">#108</a>)</li>
<li>Bump ansi-regex from 5.0.0 to 5.0.1 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/109">#109</a>)</li>
<li>Bump <code>`@​actions/core</code>` from 1.5.0 to 1.6.0 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/110">#110</a>)</li>
<li>Bump actions/checkout from 2 to 3 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/126">#126</a>)</li>
<li>Bump <code>`@​actions/tool-cache</code>` from 1.7.1 to 1.7.2 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/128">#128</a>)</li>
<li>Bump <code>`@​actions/exec</code>` from 1.1.0 to 1.1.1 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/129">#129</a>)</li>
<li>Bump minimist from 1.2.5 to 1.2.6 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/132">#132</a>)</li>
<li>Bump codecov/codecov-action from 2 to 3 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/133">#133</a>)</li>
<li>Bump semver from 7.3.5 to 7.3.7 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/136">#136</a>)</li>
</ul>
<h2>v1.6.0</h2>
<ul>
<li>Add <code>config-inline</code> input (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/106">#106</a>)</li>
<li>Bump <code>`@​actions/core</code>` from 1.4.0 to 1.5.0 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/104">#104</a>)</li>
<li>Bump codecov/codecov-action from 1 to 2 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/101">#101</a>)</li>
</ul>
<h2>v1.5.1</h2>
<ul>
<li>Explicit version spec for caching (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/100">#100</a>)</li>
</ul>
<h2>v1.5.0</h2>
<ul>
<li>Allow building buildx from source (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/99">#99</a>)</li>
</ul>
<h2>v1.4.1</h2>
<ul>
<li>Fix <code>docker: invalid reference format</code> (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/97">#97</a>)</li>
</ul>
<h2>v1.4.0</h2>
<ul>
<li>Update dev deps (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/95">#95</a>)</li>
<li>Use built-in <code>getExecOutput</code> (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/94">#94</a>)</li>
<li>Use <code>core.getBooleanInput</code> (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/93">#93</a>)</li>
<li>Bump <code>`@​actions/exec</code>` from 1.0.4 to 1.1.0 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/85">#85</a>)</li>
<li>Bump y18n from 4.0.0 to 4.0.3 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/91">#91</a>)</li>
<li>Bump hosted-git-info from 2.8.8 to 2.8.9 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/89">#89</a>)</li>
<li>Bump ws from 7.3.1 to 7.5.0 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/90">#90</a>)</li>
<li>Bump <code>`@​actions/tool-cache</code>` from 1.6.1 to 1.7.1 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/82">#82</a> <a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/86">#86</a>)</li>
<li>Bump <code>`@​actions/core</code>` from 1.2.7 to 1.4.0 (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/80">#80</a> <a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/87">#87</a>)</li>
</ul>
<h2>v1.3.0</h2>
<ul>
<li>Display BuildKit version (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/72">#72</a>)</li>
</ul>
<h2>v1.2.0</h2>
<ul>
<li>Remove os limitation (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/71">#71</a>)</li>
<li>Add test job for <code>config</code> input (<a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/68">#68</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="dc7b9719a9"><code>dc7b971</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/131">#131</a> from crazy-max/node16</li>
<li><a href="f55bc08278"><code>f55bc08</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/docker/setup-buildx-action/issues/141">#141</a> from crazy-max/fix-test</li>
<li><a href="aa877a9d36"><code>aa877a9</code></a> ci: fix standalone test</li>
<li><a href="130c56f342"><code>130c56f</code></a> Node 16 as default runtime</li>
<li>See full diff in <a href="https://github.com/docker/setup-buildx-action/compare/v1...v2">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docker/setup-buildx-action&package-manager=github_actions&previous-version=1&new-version=2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

.github/ISSUE_TEMPLATE/config.yml

@@ -1,7 +1,7 @@
 contact_links:
-  - name: Feature request
+  - name: Feature request & feedback
     url: https://github.com/meilisearch/product/discussions/categories/feedback-feature-proposal
-    about: The feature requests are not managed in this repository, please open a discussion in our dedicated product repository
+    about: The feature requests and feedback regarding the already existing features are not managed in this repository. Please open a discussion in our dedicated product repository
   - name: Documentation issue
     url: https://github.com/meilisearch/documentation/issues/new
     about: For documentation issues, open an issue or a PR in the documentation repository

.github/dependabot.yml

@@ -0,0 +1,9 @@
+# Set update schedule for GitHub Actions only
+
+version: 2
+updates:
+
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "monthly"

.github/workflows/README.md

@@ -1,20 +0,0 @@
-# GitHub Actions Workflow for Meilisearch
-
-> **Note:**
-
-> - We do not use [cache](https://github.com/actions/cache) yet but we could use it to speed up CI
-
-## Workflow
-
-- On each pull request, we trigger `cargo test`.
-- On each tag, we build:
-    - the tagged Docker image and publish it to Docker Hub
-    - the binaries for MacOS, Ubuntu, and Windows
-    - the Debian package
-- On each stable release (`v*.*.*` tag):
-    - we build the `latest` Docker image and publish it to Docker Hub
-    - we publish the binary to Hombrew and Gemfury
-
-## Problems
-
-- We do not test on Windows because we are unable to make it work, there is a disk space problem.

.github/workflows/coverage.yml

@@ -8,7 +8,7 @@ jobs:
   nightly-coverage:
     runs-on: ubuntu-18.04
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - uses: actions-rs/toolchain@v1
         with:
           toolchain: nightly
@@ -25,7 +25,7 @@ jobs:
           RUSTFLAGS: "-Zprofile -Ccodegen-units=1 -Cinline-threshold=0 -Clink-dead-code -Coverflow-checks=off -Cpanic=unwind -Zpanic_abort_tests"
       - uses: actions-rs/grcov@v0.1
       - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v1
+        uses: codecov/codecov-action@v3
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
           file: ${{ steps.coverage.outputs.report }}

.github/workflows/flaky.yml

@@ -8,7 +8,7 @@ jobs:
     runs-on: ubuntu-18.04
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: Install cargo-flaky
       run: cargo install cargo-flaky
     - name: Run cargo flaky 100 times

.github/workflows/publish-binaries.yml

@@ -6,7 +6,7 @@ name: Publish binaries to release
 
 jobs:
   publish:
-    name: Publish for ${{ matrix.os }}
+    name: Publish binary for ${{ matrix.os }}
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
@@ -27,7 +27,7 @@ jobs:
     - uses: hecrj/setup-rust-action@master
       with:
         rust-version: stable
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: Build
       run: cargo build --release --locked
     - name: Upload binaries to release
@@ -39,7 +39,7 @@ jobs:
         tag: ${{ github.ref }}
 
   publish-aarch64:
-    name: Publish to GitHub
+    name: Publish binary for aarch64
     runs-on: ${{ matrix.os }}
     continue-on-error: false
     strategy:
@@ -55,7 +55,7 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v2
+        uses: actions/checkout@v3
 
       - name: Installing Rust toolchain
         uses: actions-rs/toolchain@v1

.github/workflows/publish-deb-brew-pkg.yml

@@ -14,7 +14,7 @@ jobs:
         rust-version: stable
     - name: Install cargo-deb
       run: cargo install cargo-deb
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: Build deb package
       run: cargo deb -p meilisearch-http -o target/debian/meilisearch.deb
     - name: Upload debian pkg to release

.github/workflows/publish-docker-images.yml

@@ -0,0 +1,64 @@
+---
+on:
+  schedule:
+    - cron: '0 4 * * *' # Every day at 4:00am
+  push:
+    tags:
+      - '*'
+  release:
+    types: [released]
+
+name: Publish tagged images to Docker Hub
+
+jobs:
+  docker:
+    runs-on: docker
+    steps:
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Login to DockerHub
+        if: github.event_name != 'schedule'
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+
+      - name: Check tag format
+        id: check-tag-format
+        run: |
+          # Escape submitted tag name
+          escaped_tag=$(printf "%q" ${{ github.ref_name }})
+
+          # Check if tag has format v<nmumber>.<number>.<number> and set output.match
+          # to create a vX.Y (without patch version) Docker tag
+          if [[ $escaped_tag =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+            echo ::set-output name=match::true
+          else
+            echo ::set-output name=match::false
+          fi
+
+      - name: Docker meta
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          images: getmeili/meilisearch
+          # The lastest tag is only pushed for the official Meilisearch release
+          # See https://github.com/docker/metadata-action#latest-tag
+          flavor: latest=false
+          tags: |
+            type=ref,event=tag
+            type=semver,pattern=v{{major}}.{{minor}},enable=${{ steps.check-tag-format.outputs.match }}
+            type=raw,value=latest,enable=${{ github.event_name == 'release' }}
+
+      - name: Build and push
+        id: docker_build
+        uses: docker/build-push-action@v2
+        with:
+          # We do not push tags for the cron jobs, this is only for test purposes
+          push: ${{ github.event_name != 'schedule' }}
+          platforms: linux/amd64,linux/arm64
+          tags: ${{ steps.meta.outputs.tags }}

.github/workflows/publish-docker-latest.yml

@@ -1,30 +0,0 @@
----
-on:
-  release:
-    types: [released]
-
-name: Publish latest image to Docker Hub
-
-jobs:
-  docker-latest:
-    runs-on: docker
-    steps:
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v1
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
-
-      - name: Login to DockerHub
-        uses: docker/login-action@v1
-        with:
-          username: ${{ secrets.DOCKER_USERNAME }}
-          password: ${{ secrets.DOCKER_PASSWORD }}
-
-      - name: Build and push
-        id: docker_build
-        uses: docker/build-push-action@v2
-        with:
-          push: true
-          platforms: linux/amd64,linux/arm64
-          tags: getmeili/meilisearch:latest

.github/workflows/publish-docker-tag.yml

@@ -1,39 +0,0 @@
----
-on:
-  push:
-    tags:
-      - '*'
-
-name: Publish tagged image to Docker Hub
-
-jobs:
-  docker-tag:
-    runs-on: docker
-    steps:
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v1
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
-
-      - name: Login to DockerHub
-        uses: docker/login-action@v1
-        with:
-          username: ${{ secrets.DOCKER_USERNAME }}
-          password: ${{ secrets.DOCKER_PASSWORD }}
-
-      - name: Docker meta
-        id: meta
-        uses: docker/metadata-action@v3
-        with:
-          images: getmeili/meilisearch
-          flavor: latest=false
-          tags: type=ref,event=tag
-
-      - name: Build and push
-        id: docker_build
-        uses: docker/build-push-action@v2
-        with:
-          push: true
-          platforms: linux/amd64,linux/arm64
-          tags: ${{ steps.meta.outputs.tags }}

.github/workflows/rust.yml

@@ -12,6 +12,7 @@ on:
 env:
   CARGO_TERM_COLOR: always
   RUST_BACKTRACE: 1
+  RUSTFLAGS: "-D warnings"
 
 jobs:
   tests:
@@ -22,7 +23,7 @@ jobs:
       matrix:
         os: [ubuntu-18.04, macos-latest, windows-latest]
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: Cache dependencies
       uses: Swatinem/rust-cache@v1.3.0
     - name: Run cargo check without any default features
@@ -36,11 +37,30 @@ jobs:
         command: test
         args: --locked --release
 
+  # We run tests in debug also, to make sure that the debug_assertions are hit
+  test-debug:
+    name: Run tests in debug
+    runs-on: ubuntu-18.04
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions-rs/toolchain@v1
+        with:
+          profile: minimal
+          toolchain: stable
+          override: true
+      - name: Cache dependencies
+        uses: Swatinem/rust-cache@v1.3.0
+      - name: Run tests in debug
+        uses: actions-rs/cargo@v1
+        with:
+          command: test
+          args: --locked
+
   clippy:
     name: Run Clippy
     runs-on: ubuntu-18.04
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - uses: actions-rs/toolchain@v1
         with:
           profile: minimal
@@ -59,11 +79,11 @@ jobs:
     name: Run Rustfmt
     runs-on: ubuntu-18.04
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - uses: actions-rs/toolchain@v1
         with:
           profile: minimal
-          toolchain: nightly
+          toolchain: stable
           override: true
           components: rustfmt
       - name: Cache dependencies

CONTRIBUTING.md

@@ -5,14 +5,16 @@ First, thank you for contributing to Meilisearch! The goal of this document is t
 Remember that there are many ways to contribute other than writing code: writing [tutorials or blog posts](https://github.com/meilisearch/awesome-meilisearch), improving [the documentation](https://github.com/meilisearch/documentation), submitting [bug reports](https://github.com/meilisearch/meilisearch/issues/new?assignees=&labels=&template=bug_report.md&title=) and [feature requests](https://github.com/meilisearch/product/discussions/categories/feedback-feature-proposal)...
 
 ## Table of Contents
+
 - [Assumptions](#assumptions)
 - [How to Contribute](#how-to-contribute)
 - [Development Workflow](#development-workflow)
 - [Git Guidelines](#git-guidelines)
+- [Release Process (for internal team only)](#release-process-for-internal-team-only)
 
 ## Assumptions
 
-1. **You're familiar with [Github](https://github.com) and the [Pull Requests](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests)(PR) workflow.**
+1. **You're familiar with [GitHub](https://github.com) and the [Pull Requests](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests)(PR) workflow.**
 2. **You've read the Meilisearch [documentation](https://docs.meilisearch.com).**
 3. **You know about the [Meilisearch community](https://docs.meilisearch.com/learn/what_is_meilisearch/contact.html).
    Please use this for help.**
@@ -22,7 +24,7 @@ Remember that there are many ways to contribute other than writing code: writing
 1. Ensure your change has an issue! Find an
    [existing issue](https://github.com/meilisearch/meilisearch/issues/) or [open a new issue](https://github.com/meilisearch/meilisearch/issues/new).
    * This is where you can get a feel if the change will be accepted or not.
-2. Once approved, [fork the Meilisearch repository](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) in your own Github account.
+2. Once approved, [fork the Meilisearch repository](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) in your own GitHub account.
 3. [Create a new Git branch](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-and-deleting-branches-within-your-repository)
 4. Review the [Development Workflow](#development-workflow) section that describes the steps to maintain the repository.
 5. Make your changes on your branch.
@@ -44,6 +46,8 @@ We recommend using the `--release` flag to test the full performance of Meilisea
 cargo test
 ```
 
+This command will be triggered to each PR as a requirement for merging it.
+
 If you get a "Too many open files" error you might want to increase the open file limit using this command:
 
 ```bash
@@ -68,7 +72,7 @@ As minimal requirements, your commit message should:
 
 We don't follow any other convention, but if you want to use one, we recommend [the Chris Beams one](https://chris.beams.io/posts/git-commit/).
 
-### Github Pull Requests
+### GitHub Pull Requests
 
 Some notes on GitHub PRs:
 
@@ -78,6 +82,29 @@ Some notes on GitHub PRs:
   The draft PRs are recommended when you want to show that you are working on something and make your work visible.
 - The branch related to the PR must be **up-to-date with `main`** before merging. Fortunately, this project uses [Bors](https://github.com/bors-ng/bors-ng) to automatically enforce this requirement without the PR author having to rebase manually.
 
+## Release Process (for internal team only)
+
+Meilisearch tools follow the [Semantic Versioning Convention](https://semver.org/).
+
+### Automation to rebase and Merge the PRs
+
+This project integrates a bot that helps us manage pull requests merging.<br>
+_[Read more about this](https://github.com/meilisearch/integration-guides/blob/main/resources/bors.md)._
+
+### How to Publish a new Release
+
+The full Meilisearch release process is described in [this guide](https://github.com/meilisearch/core-team/blob/main/resources/meilisearch-release.md). Please follow it carefully before doing any release.
+
+### Release assets
+
+For each release, the following assets are created:
+- Binaries for differents platforms (Linux, MacOS, Windows and ARM architectures) are attached to the GitHub release
+- Binaries are pushed to HomeBrew and APT (not published for RC)
+- Docker tags are created/updated:
+  - `vX.Y.Z`
+  - `vX.Y` (not published for RC)
+  - `latest` (not published for RC)
+
 <hr>
 
 Thank you again for reading this through, we can not wait to begin to work with you if you made your way through this contributing guide ❤️

Cargo.toml

@@ -1,7 +1,15 @@
 [workspace]
+resolver = "2"
 members = [
     "meilisearch-http",
-    "meilisearch-error",
+    "meilisearch-types",
     "meilisearch-lib",
     "meilisearch-auth",
+    "permissive-json-pointer",
 ]
+
+[profile.dev.package.flate2]
+opt-level = 3
+
+[profile.dev.package.milli]
+opt-level = 3

Dockerfile

@@ -1,42 +1,22 @@
 # Compile
-FROM    alpine:3.14 AS compiler
+FROM    rust:alpine3.14 AS compiler
 
-RUN     apk update --quiet \
-        && apk add -q --no-cache curl build-base
-
-RUN     curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+RUN     apk add -q --update-cache --no-cache build-base openssl-dev
 
 WORKDIR /meilisearch
 
-COPY    Cargo.lock .
-COPY    Cargo.toml .
-
-COPY    meilisearch-auth/Cargo.toml meilisearch-auth/
-COPY    meilisearch-error/Cargo.toml meilisearch-error/
-COPY    meilisearch-http/Cargo.toml meilisearch-http/
-COPY    meilisearch-lib/Cargo.toml meilisearch-lib/
-
-ENV     RUSTFLAGS="-C target-feature=-crt-static"
-
-# Create dummy main.rs files for each workspace member to be able to compile all the dependencies
-RUN     find . -type d -name "meilisearch-*" | xargs -I{} sh -c 'mkdir {}/src; echo "fn main() { }" > {}/src/main.rs;'
-# Use `cargo build` instead of `cargo vendor` because we need to not only download but compile dependencies too
-RUN     if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
-            export JEMALLOC_SYS_WITH_LG_PAGE=16; \
-        fi && \
-        $HOME/.cargo/bin/cargo build --release
-# Cleanup dummy main.rs files
-RUN     find . -path "*/src/main.rs" -delete
-
 ARG     COMMIT_SHA
 ARG     COMMIT_DATE
 ENV     COMMIT_SHA=${COMMIT_SHA} COMMIT_DATE=${COMMIT_DATE}
+ENV     RUSTFLAGS="-C target-feature=-crt-static"
 
 COPY    . .
-RUN     if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
+RUN     set -eux; \
+        apkArch="$(apk --print-arch)"; \
+        if [ "$apkArch" = "aarch64" ]; then \
             export JEMALLOC_SYS_WITH_LG_PAGE=16; \
         fi && \
-        $HOME/.cargo/bin/cargo build --release
+        cargo build --release
 
 # Run
 FROM    alpine:3.14
@@ -47,9 +27,20 @@ ENV     MEILI_SERVER_PROVIDER docker
 RUN     apk update --quiet \
         && apk add -q --no-cache libgcc tini curl
 
-COPY    --from=compiler /meilisearch/target/release/meilisearch .
+# add meilisearch to the `/bin` so you can run it from anywhere and it's easy
+# to find.
+COPY    --from=compiler /meilisearch/target/release/meilisearch /bin/meilisearch
+# To stay compatible with the older version of the container (pre v0.27.0) we're
+# going to symlink the meilisearch binary in the path to `/meilisearch`
+RUN     ln -s /bin/meilisearch /meilisearch
+
+# This directory should hold all the data related to meilisearch so we're going
+# to move our PWD in there.
+# We don't want to put the meilisearch binary
+WORKDIR /meili_data
+
 
 EXPOSE  7700/tcp
 
 ENTRYPOINT ["tini", "--"]
-CMD     ./meilisearch
+CMD     /bin/meilisearch

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2019-2022 Meilisearch
+Copyright (c) 2019-2022 Meili SAS
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -58,7 +58,7 @@ meilisearch
 #### Docker
 
 ```bash
-docker run -p 7700:7700 -v "$(pwd)/data.ms:/data.ms" getmeili/meilisearch
+docker run -p 7700:7700 -v "$(pwd)/meili_data:/meili_data" getmeili/meilisearch
 ```
 
 #### Announcing a cloud-hosted Meilisearch
@@ -109,7 +109,7 @@ cargo run --release
 Let's create an index! If you need a sample dataset, use [this movie database](https://www.notion.so/meilisearch/A-movies-dataset-to-test-Meili-1cbf7c9cfa4247249c40edfa22d7ca87#b5ae399b81834705ba5420ac70358a65). You can also find it in the `datasets/` directory.
 
 ```bash
-curl -L 'https://bit.ly/2PAcw9l' -o movies.json
+curl -L https://docs.meilisearch.com/movies.json -o movies.json
 ```
 
 Now, you're ready to index some data.

SECURITY.md

@@ -4,7 +4,7 @@ Meilisearch takes the security of our software products and services seriously.
 
 If you believe you have found a security vulnerability in any Meilisearch-owned repository, please report it to us as described below.
 
-## Suported versions
+## Supported versions
 
 As long as we are pre-v1.0, only the latest version of Meilisearch will be supported with security updates.
 

bors.toml

@@ -3,7 +3,8 @@ status = [
     'Tests on macos-latest',
     'Tests on windows-latest',
     'Run Clippy',
-    'Run Rustfmt'
+    'Run Rustfmt',
+    'Run tests in debug',
 ]
 pr_status = ['Milestone Check']
 # 3 hours timeout

download-latest.sh

@@ -67,8 +67,8 @@ semverLT() {
     return 1
 }
 
-# Get a token from https://github.com/settings/tokens to increasae rate limit (from 60 to 5000), make sure the token scope is set to 'public_repo'
-# Create GITHUB_PAT enviroment variable once you aquired the token to start using it
+# Get a token from https://github.com/settings/tokens to increase rate limit (from 60 to 5000), make sure the token scope is set to 'public_repo'
+# Create GITHUB_PAT environment variable once you acquired the token to start using it
 # Returns the tag of the latest stable release (in terms of semver and not of release date)
 get_latest() {
     temp_file='temp_file' # temp_file needed because the grep would start before the download is over
@@ -89,7 +89,7 @@ get_latest() {
     latest=''
     current_tag=''
     for release_info in $releases; do
-        if [ $i -eq 0 ]; then # Cheking tag_name
+        if [ $i -eq 0 ]; then # Checking tag_name
             if echo "$release_info" | grep -q "$GREP_SEMVER_REGEXP"; then # If it's not an alpha or beta release
                 current_tag=$release_info
             else

meilisearch-auth/Cargo.toml

@@ -1,15 +1,18 @@
 [package]
 name = "meilisearch-auth"
-version = "0.26.0"
+version = "0.28.0"
 edition = "2021"
 
 [dependencies]
+base64 = "0.13.0"
 enum-iterator = "0.7.0"
-heed = { git = "https://github.com/Kerollmops/heed", tag = "v0.12.1" }
-sha2 = "0.9.6"
-meilisearch-error = { path = "../meilisearch-error" }
-serde_json = { version = "1.0.67", features = ["preserve_order"] }
-time = { version = "0.3.7", features = ["serde-well-known", "formatting", "parsing", "macros"] }
+hmac = "0.12.1"
+meilisearch-types = { path = "../meilisearch-types" }
+milli = { git = "https://github.com/meilisearch/milli.git", tag = "v0.29.3-filter.beta.0" }
 rand = "0.8.4"
-serde = { version = "1.0.130", features = ["derive"] }
-thiserror = "1.0.28"
+serde = { version = "1.0.136", features = ["derive"] }
+serde_json = { version = "1.0.79", features = ["preserve_order"] }
+sha2 = "0.10.2"
+thiserror = "1.0.30"
+time = { version = "0.3.7", features = ["serde-well-known", "formatting", "parsing", "macros"] }
+uuid = { version = "0.8.2", features = ["serde", "v4"] }

meilisearch-auth/src/action.rs

@@ -5,9 +5,11 @@ use serde::{Deserialize, Serialize};
 #[repr(u8)]
 pub enum Action {
     #[serde(rename = "*")]
-    All = 0,
+    All = actions::ALL,
     #[serde(rename = "search")]
     Search = actions::SEARCH,
+    #[serde(rename = "documents.*")]
+    DocumentsAll = actions::DOCUMENTS_ALL,
     #[serde(rename = "documents.add")]
     DocumentsAdd = actions::DOCUMENTS_ADD,
     #[serde(rename = "documents.get")]
@@ -32,18 +34,25 @@ pub enum Action {
     StatsGet = actions::STATS_GET,
     #[serde(rename = "dumps.create")]
     DumpsCreate = actions::DUMPS_CREATE,
-    #[serde(rename = "dumps.get")]
-    DumpsGet = actions::DUMPS_GET,
     #[serde(rename = "version")]
     Version = actions::VERSION,
+    #[serde(rename = "keys.create")]
+    KeysAdd = actions::KEYS_CREATE,
+    #[serde(rename = "keys.get")]
+    KeysGet = actions::KEYS_GET,
+    #[serde(rename = "keys.update")]
+    KeysUpdate = actions::KEYS_UPDATE,
+    #[serde(rename = "keys.delete")]
+    KeysDelete = actions::KEYS_DELETE,
 }
 
 impl Action {
     pub fn from_repr(repr: u8) -> Option<Self> {
         use actions::*;
         match repr {
-            0 => Some(Self::All),
+            ALL => Some(Self::All),
             SEARCH => Some(Self::Search),
+            DOCUMENTS_ALL => Some(Self::DocumentsAll),
             DOCUMENTS_ADD => Some(Self::DocumentsAdd),
             DOCUMENTS_GET => Some(Self::DocumentsGet),
             DOCUMENTS_DELETE => Some(Self::DocumentsDelete),
@@ -56,8 +65,11 @@ impl Action {
             SETTINGS_UPDATE => Some(Self::SettingsUpdate),
             STATS_GET => Some(Self::StatsGet),
             DUMPS_CREATE => Some(Self::DumpsCreate),
-            DUMPS_GET => Some(Self::DumpsGet),
             VERSION => Some(Self::Version),
+            KEYS_CREATE => Some(Self::KeysAdd),
+            KEYS_GET => Some(Self::KeysGet),
+            KEYS_UPDATE => Some(Self::KeysUpdate),
+            KEYS_DELETE => Some(Self::KeysDelete),
             _otherwise => None,
         }
     }
@@ -65,8 +77,9 @@ impl Action {
     pub fn repr(&self) -> u8 {
         use actions::*;
         match self {
-            Self::All => 0,
+            Self::All => ALL,
             Self::Search => SEARCH,
+            Self::DocumentsAll => DOCUMENTS_ALL,
             Self::DocumentsAdd => DOCUMENTS_ADD,
             Self::DocumentsGet => DOCUMENTS_GET,
             Self::DocumentsDelete => DOCUMENTS_DELETE,
@@ -79,26 +92,34 @@ impl Action {
             Self::SettingsUpdate => SETTINGS_UPDATE,
             Self::StatsGet => STATS_GET,
             Self::DumpsCreate => DUMPS_CREATE,
-            Self::DumpsGet => DUMPS_GET,
             Self::Version => VERSION,
+            Self::KeysAdd => KEYS_CREATE,
+            Self::KeysGet => KEYS_GET,
+            Self::KeysUpdate => KEYS_UPDATE,
+            Self::KeysDelete => KEYS_DELETE,
         }
     }
 }
 
 pub mod actions {
+    pub(crate) const ALL: u8 = 0;
     pub const SEARCH: u8 = 1;
-    pub const DOCUMENTS_ADD: u8 = 2;
-    pub const DOCUMENTS_GET: u8 = 3;
-    pub const DOCUMENTS_DELETE: u8 = 4;
-    pub const INDEXES_CREATE: u8 = 5;
-    pub const INDEXES_GET: u8 = 6;
-    pub const INDEXES_UPDATE: u8 = 7;
-    pub const INDEXES_DELETE: u8 = 8;
-    pub const TASKS_GET: u8 = 9;
-    pub const SETTINGS_GET: u8 = 10;
-    pub const SETTINGS_UPDATE: u8 = 11;
-    pub const STATS_GET: u8 = 12;
-    pub const DUMPS_CREATE: u8 = 13;
-    pub const DUMPS_GET: u8 = 14;
+    pub const DOCUMENTS_ALL: u8 = 2;
+    pub const DOCUMENTS_ADD: u8 = 3;
+    pub const DOCUMENTS_GET: u8 = 4;
+    pub const DOCUMENTS_DELETE: u8 = 5;
+    pub const INDEXES_CREATE: u8 = 6;
+    pub const INDEXES_GET: u8 = 7;
+    pub const INDEXES_UPDATE: u8 = 8;
+    pub const INDEXES_DELETE: u8 = 9;
+    pub const TASKS_GET: u8 = 10;
+    pub const SETTINGS_GET: u8 = 11;
+    pub const SETTINGS_UPDATE: u8 = 12;
+    pub const STATS_GET: u8 = 13;
+    pub const DUMPS_CREATE: u8 = 14;
     pub const VERSION: u8 = 15;
+    pub const KEYS_CREATE: u8 = 16;
+    pub const KEYS_GET: u8 = 17;
+    pub const KEYS_UPDATE: u8 = 18;
+    pub const KEYS_DELETE: u8 = 19;
 }

meilisearch-auth/src/dump.rs

@@ -1,5 +1,6 @@
+use serde_json::Deserializer;
+
 use std::fs::File;
-use std::io::BufRead;
 use std::io::BufReader;
 use std::io::Write;
 use std::path::Path;
@@ -36,10 +37,9 @@ impl AuthController {
             return Ok(());
         }
 
-        let mut reader = BufReader::new(File::open(&keys_file_path)?).lines();
-        while let Some(key) = reader.next().transpose()? {
-            let key = serde_json::from_str(&key)?;
-            store.put_api_key(key)?;
+        let reader = BufReader::new(File::open(&keys_file_path)?);
+        for key in Deserializer::from_reader(reader).into_iter() {
+            store.put_api_key(key?)?;
         }
 
         Ok(())

meilisearch-auth/src/error.rs

@@ -1,7 +1,7 @@
 use std::error::Error;
 
-use meilisearch_error::ErrorCode;
-use meilisearch_error::{internal_error, Code};
+use meilisearch_types::error::{Code, ErrorCode};
+use meilisearch_types::internal_error;
 use serde_json::Value;
 
 pub type Result<T> = std::result::Result<T, AuthControllerError>;
@@ -18,14 +18,24 @@ pub enum AuthControllerError {
     InvalidApiKeyExpiresAt(Value),
     #[error("`description` field value `{0}` is invalid. It should be a string or specified as a null value.")]
     InvalidApiKeyDescription(Value),
+    #[error(
+        "`name` field value `{0}` is invalid. It should be a string or specified as a null value."
+    )]
+    InvalidApiKeyName(Value),
+    #[error("`uid` field value `{0}` is invalid. It should be a valid UUID v4 string or omitted.")]
+    InvalidApiKeyUid(Value),
     #[error("API key `{0}` not found.")]
     ApiKeyNotFound(String),
+    #[error("`uid` field value `{0}` is already an existing API key.")]
+    ApiKeyAlreadyExists(String),
+    #[error("The `{0}` field cannot be modified for the given resource.")]
+    ImmutableField(String),
     #[error("Internal error: {0}")]
     Internal(Box<dyn Error + Send + Sync + 'static>),
 }
 
 internal_error!(
-    AuthControllerError: heed::Error,
+    AuthControllerError: milli::heed::Error,
     std::io::Error,
     serde_json::Error,
     std::str::Utf8Error
@@ -39,7 +49,11 @@ impl ErrorCode for AuthControllerError {
             Self::InvalidApiKeyIndexes(_) => Code::InvalidApiKeyIndexes,
             Self::InvalidApiKeyExpiresAt(_) => Code::InvalidApiKeyExpiresAt,
             Self::InvalidApiKeyDescription(_) => Code::InvalidApiKeyDescription,
+            Self::InvalidApiKeyName(_) => Code::InvalidApiKeyName,
             Self::ApiKeyNotFound(_) => Code::ApiKeyNotFound,
+            Self::InvalidApiKeyUid(_) => Code::InvalidApiKeyUid,
+            Self::ApiKeyAlreadyExists(_) => Code::ApiKeyAlreadyExists,
+            Self::ImmutableField(_) => Code::ImmutableField,
             Self::Internal(_) => Code::Internal,
         }
     }

meilisearch-auth/src/key.rs

@@ -1,20 +1,25 @@
 use crate::action::Action;
 use crate::error::{AuthControllerError, Result};
-use crate::store::{KeyId, KEY_ID_LENGTH};
-use rand::Rng;
+use crate::store::KeyId;
+
+use meilisearch_types::index_uid::IndexUid;
+use meilisearch_types::star_or::StarOr;
 use serde::{Deserialize, Serialize};
 use serde_json::{from_value, Value};
 use time::format_description::well_known::Rfc3339;
 use time::macros::{format_description, time};
 use time::{Date, OffsetDateTime, PrimitiveDateTime};
+use uuid::Uuid;
 
 #[derive(Debug, Deserialize, Serialize)]
 pub struct Key {
     #[serde(skip_serializing_if = "Option::is_none")]
     pub description: Option<String>,
-    pub id: KeyId,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub name: Option<String>,
+    pub uid: KeyId,
     pub actions: Vec<Action>,
-    pub indexes: Vec<String>,
+    pub indexes: Vec<StarOr<IndexUid>>,
     #[serde(with = "time::serde::rfc3339::option")]
     pub expires_at: Option<OffsetDateTime>,
     #[serde(with = "time::serde::rfc3339")]
@@ -25,16 +30,27 @@ pub struct Key {
 
 impl Key {
     pub fn create_from_value(value: Value) -> Result<Self> {
-        let description = match value.get("description") {
-            Some(Value::Null) => None,
-            Some(des) => Some(
-                from_value(des.clone())
-                    .map_err(|_| AuthControllerError::InvalidApiKeyDescription(des.clone()))?,
-            ),
-            None => None,
+        let name = match value.get("name") {
+            None | Some(Value::Null) => None,
+            Some(des) => from_value(des.clone())
+                .map(Some)
+                .map_err(|_| AuthControllerError::InvalidApiKeyName(des.clone()))?,
         };
 
-        let id = generate_id();
+        let description = match value.get("description") {
+            None | Some(Value::Null) => None,
+            Some(des) => from_value(des.clone())
+                .map(Some)
+                .map_err(|_| AuthControllerError::InvalidApiKeyDescription(des.clone()))?,
+        };
+
+        let uid = value.get("uid").map_or_else(
+            || Ok(Uuid::new_v4()),
+            |uid| {
+                from_value(uid.clone())
+                    .map_err(|_| AuthControllerError::InvalidApiKeyUid(uid.clone()))
+            },
+        )?;
 
         let actions = value
             .get("actions")
@@ -61,8 +77,9 @@ impl Key {
         let updated_at = created_at;
 
         Ok(Self {
+            name,
             description,
-            id,
+            uid,
             actions,
             indexes,
             expires_at,
@@ -78,20 +95,34 @@ impl Key {
             self.description = des?;
         }
 
-        if let Some(act) = value.get("actions") {
-            let act = from_value(act.clone())
-                .map_err(|_| AuthControllerError::InvalidApiKeyActions(act.clone()));
-            self.actions = act?;
+        if let Some(des) = value.get("name") {
+            let des = from_value(des.clone())
+                .map_err(|_| AuthControllerError::InvalidApiKeyName(des.clone()));
+            self.name = des?;
         }
 
-        if let Some(ind) = value.get("indexes") {
-            let ind = from_value(ind.clone())
-                .map_err(|_| AuthControllerError::InvalidApiKeyIndexes(ind.clone()));
-            self.indexes = ind?;
+        if value.get("uid").is_some() {
+            return Err(AuthControllerError::ImmutableField("uid".to_string()));
         }
 
-        if let Some(exp) = value.get("expiresAt") {
-            self.expires_at = parse_expiration_date(exp)?;
+        if value.get("actions").is_some() {
+            return Err(AuthControllerError::ImmutableField("actions".to_string()));
+        }
+
+        if value.get("indexes").is_some() {
+            return Err(AuthControllerError::ImmutableField("indexes".to_string()));
+        }
+
+        if value.get("expiresAt").is_some() {
+            return Err(AuthControllerError::ImmutableField("expiresAt".to_string()));
+        }
+
+        if value.get("createdAt").is_some() {
+            return Err(AuthControllerError::ImmutableField("createdAt".to_string()));
+        }
+
+        if value.get("updatedAt").is_some() {
+            return Err(AuthControllerError::ImmutableField("updatedAt".to_string()));
         }
 
         self.updated_at = OffsetDateTime::now_utc();
@@ -101,11 +132,13 @@ impl Key {
 
     pub(crate) fn default_admin() -> Self {
         let now = OffsetDateTime::now_utc();
+        let uid = Uuid::new_v4();
         Self {
-            description: Some("Default Admin API Key (Use it for all other operations. Caution! Do not use it on a public frontend)".to_string()),
-            id: generate_id(),
+            name: Some("Default Admin API Key".to_string()),
+            description: Some("Use it for anything that is not a search operation. Caution! Do not expose it on a public frontend".to_string()),
+            uid,
             actions: vec![Action::All],
-            indexes: vec!["*".to_string()],
+            indexes: vec![StarOr::Star],
             expires_at: None,
             created_at: now,
             updated_at: now,
@@ -114,13 +147,13 @@ impl Key {
 
     pub(crate) fn default_search() -> Self {
         let now = OffsetDateTime::now_utc();
+        let uid = Uuid::new_v4();
         Self {
-            description: Some(
-                "Default Search API Key (Use it to search from the frontend)".to_string(),
-            ),
-            id: generate_id(),
+            name: Some("Default Search API Key".to_string()),
+            description: Some("Use it to search from the frontend".to_string()),
+            uid,
             actions: vec![Action::Search],
-            indexes: vec!["*".to_string()],
+            indexes: vec![StarOr::Star],
             expires_at: None,
             created_at: now,
             updated_at: now,
@@ -128,19 +161,6 @@ impl Key {
     }
 }
 
-/// Generate a printable key of 64 characters using thread_rng.
-fn generate_id() -> [u8; KEY_ID_LENGTH] {
-    const CHARSET: &[u8] = b"abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
-
-    let mut rng = rand::thread_rng();
-    let mut bytes = [0; KEY_ID_LENGTH];
-    for byte in bytes.iter_mut() {
-        *byte = CHARSET[rng.gen_range(0..CHARSET.len())];
-    }
-
-    bytes
-}
-
 fn parse_expiration_date(value: &Value) -> Result<Option<OffsetDateTime>> {
     match value {
         Value::String(string) => OffsetDateTime::parse(string, &Rfc3339)

meilisearch-auth/src/lib.rs

@@ -5,18 +5,20 @@ mod key;
 mod store;
 
 use std::collections::{HashMap, HashSet};
+use std::ops::Deref;
 use std::path::Path;
-use std::str::from_utf8;
 use std::sync::Arc;
 
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
-use sha2::{Digest, Sha256};
 use time::OffsetDateTime;
+use uuid::Uuid;
 
 pub use action::{actions, Action};
 use error::{AuthControllerError, Result};
 pub use key::Key;
+use meilisearch_types::star_or::StarOr;
+use store::generate_key_as_base64;
 pub use store::open_auth_store_env;
 use store::HeedAuthStore;
 
@@ -42,62 +44,77 @@ impl AuthController {
 
     pub fn create_key(&self, value: Value) -> Result<Key> {
         let key = Key::create_from_value(value)?;
-        self.store.put_api_key(key)
+        match self.store.get_api_key(key.uid)? {
+            Some(_) => Err(AuthControllerError::ApiKeyAlreadyExists(
+                key.uid.to_string(),
+            )),
+            None => self.store.put_api_key(key),
+        }
     }
 
-    pub fn update_key(&self, key: impl AsRef<str>, value: Value) -> Result<Key> {
-        let mut key = self.get_key(key)?;
+    pub fn update_key(&self, uid: Uuid, value: Value) -> Result<Key> {
+        let mut key = self.get_key(uid)?;
         key.update_from_value(value)?;
         self.store.put_api_key(key)
     }
 
-    pub fn get_key(&self, key: impl AsRef<str>) -> Result<Key> {
+    pub fn get_key(&self, uid: Uuid) -> Result<Key> {
         self.store
-            .get_api_key(&key)?
-            .ok_or_else(|| AuthControllerError::ApiKeyNotFound(key.as_ref().to_string()))
+            .get_api_key(uid)?
+            .ok_or_else(|| AuthControllerError::ApiKeyNotFound(uid.to_string()))
+    }
+
+    pub fn get_optional_uid_from_encoded_key(&self, encoded_key: &[u8]) -> Result<Option<Uuid>> {
+        match &self.master_key {
+            Some(master_key) => self
+                .store
+                .get_uid_from_encoded_key(encoded_key, master_key.as_bytes()),
+            None => Ok(None),
+        }
+    }
+
+    pub fn get_uid_from_encoded_key(&self, encoded_key: &str) -> Result<Uuid> {
+        self.get_optional_uid_from_encoded_key(encoded_key.as_bytes())?
+            .ok_or_else(|| AuthControllerError::ApiKeyNotFound(encoded_key.to_string()))
     }
 
     pub fn get_key_filters(
         &self,
-        key: impl AsRef<str>,
+        uid: Uuid,
         search_rules: Option<SearchRules>,
     ) -> Result<AuthFilter> {
         let mut filters = AuthFilter::default();
-        if self
-            .master_key
-            .as_ref()
-            .map_or(false, |master_key| master_key != key.as_ref())
-        {
-            let key = self
-                .store
-                .get_api_key(&key)?
-                .ok_or_else(|| AuthControllerError::ApiKeyNotFound(key.as_ref().to_string()))?;
+        let key = self
+            .store
+            .get_api_key(uid)?
+            .ok_or_else(|| AuthControllerError::ApiKeyNotFound(uid.to_string()))?;
 
-            if !key.indexes.iter().any(|i| i.as_str() == "*") {
-                filters.search_rules = match search_rules {
-                    // Intersect search_rules with parent key authorized indexes.
-                    Some(search_rules) => SearchRules::Map(
-                        key.indexes
-                            .into_iter()
-                            .filter_map(|index| {
-                                search_rules
-                                    .get_index_search_rules(&index)
-                                    .map(|index_search_rules| (index, Some(index_search_rules)))
-                            })
-                            .collect(),
-                    ),
-                    None => SearchRules::Set(key.indexes.into_iter().collect()),
-                };
-            } else if let Some(search_rules) = search_rules {
-                filters.search_rules = search_rules;
-            }
-
-            filters.allow_index_creation = key
-                .actions
-                .iter()
-                .any(|&action| action == Action::IndexesAdd || action == Action::All);
+        if !key.indexes.iter().any(|i| i == &StarOr::Star) {
+            filters.search_rules = match search_rules {
+                // Intersect search_rules with parent key authorized indexes.
+                Some(search_rules) => SearchRules::Map(
+                    key.indexes
+                        .into_iter()
+                        .filter_map(|index| {
+                            search_rules.get_index_search_rules(index.deref()).map(
+                                |index_search_rules| {
+                                    (String::from(index), Some(index_search_rules))
+                                },
+                            )
+                        })
+                        .collect(),
+                ),
+                None => SearchRules::Set(key.indexes.into_iter().map(String::from).collect()),
+            };
+        } else if let Some(search_rules) = search_rules {
+            filters.search_rules = search_rules;
         }
 
+        filters.allow_index_creation = key
+            .actions
+            .iter()
+            .any(|&action| action == Action::IndexesAdd || action == Action::All);
+
         Ok(filters)
     }
 
@@ -105,13 +122,11 @@ impl AuthController {
         self.store.list_api_keys()
     }
 
-    pub fn delete_key(&self, key: impl AsRef<str>) -> Result<()> {
-        if self.store.delete_api_key(&key)? {
+    pub fn delete_key(&self, uid: Uuid) -> Result<()> {
+        if self.store.delete_api_key(uid)? {
             Ok(())
         } else {
-            Err(AuthControllerError::ApiKeyNotFound(
-                key.as_ref().to_string(),
-            ))
+            Err(AuthControllerError::ApiKeyNotFound(uid.to_string()))
         }
     }
 
@@ -121,32 +136,32 @@ impl AuthController {
 
     /// Generate a valid key from a key id using the current master key.
     /// Returns None if no master key has been set.
-    pub fn generate_key(&self, id: &str) -> Option<String> {
+    pub fn generate_key(&self, uid: Uuid) -> Option<String> {
         self.master_key
             .as_ref()
-            .map(|master_key| generate_key(master_key.as_bytes(), id))
+            .map(|master_key| generate_key_as_base64(uid.as_bytes(), master_key.as_bytes()))
     }
 
     /// Check if the provided key is authorized to make a specific action
     /// without checking if the key is valid.
     pub fn is_key_authorized(
         &self,
-        key: &[u8],
+        uid: Uuid,
         action: Action,
         index: Option<&str>,
     ) -> Result<bool> {
         match self
             .store
             // check if the key has access to all indexes.
-            .get_expiration_date(key, action, None)?
+            .get_expiration_date(uid, action, None)?
             .or(match index {
                 // else check if the key has access to the requested index.
                 Some(index) => {
                     self.store
-                        .get_expiration_date(key, action, Some(index.as_bytes()))?
+                        .get_expiration_date(uid, action, Some(index.as_bytes()))?
                 }
                 // or to any index if no index has been requested.
-                None => self.store.prefix_first_expiration_date(key, action)?,
+                None => self.store.prefix_first_expiration_date(uid, action)?,
             }) {
             // check expiration date.
             Some(Some(exp)) => Ok(OffsetDateTime::now_utc() < exp),
@@ -156,29 +171,6 @@ impl AuthController {
             None => Ok(false),
         }
     }
-
-    /// Check if the provided key is valid
-    /// without checking if the key is authorized to make a specific action.
-    pub fn is_key_valid(&self, key: &[u8]) -> Result<bool> {
-        if let Some(id) = self.store.get_key_id(key) {
-            let id = from_utf8(&id)?;
-            if let Some(generated) = self.generate_key(id) {
-                return Ok(generated.as_bytes() == key);
-            }
-        }
-
-        Ok(false)
-    }
-
-    /// Check if the provided key is valid
-    /// and is authorized to make a specific action.
-    pub fn authenticate(&self, key: &[u8], action: Action, index: Option<&str>) -> Result<bool> {
-        if self.is_key_authorized(key, action, index)? {
-            self.is_key_valid(key)
-        } else {
-            Ok(false)
-        }
-    }
 }
 
 pub struct AuthFilter {
@@ -258,12 +250,6 @@ pub struct IndexSearchRules {
     pub filter: Option<serde_json::Value>,
 }
 
-fn generate_key(master_key: &[u8], keyid: &str) -> String {
-    let key = [keyid.as_bytes(), master_key].concat();
-    let sha = Sha256::digest(&key);
-    format!("{}{:x}", keyid, sha)
-}
-
 fn generate_default_keys(store: &HeedAuthStore) -> Result<()> {
     store.put_api_key(Key::default_admin())?;
     store.put_api_key(Key::default_search())?;

meilisearch-auth/src/store.rs

@@ -1,27 +1,31 @@
-use enum_iterator::IntoEnumIterator;
 use std::borrow::Cow;
 use std::cmp::Reverse;
 use std::convert::TryFrom;
 use std::convert::TryInto;
 use std::fs::create_dir_all;
+use std::ops::Deref;
 use std::path::Path;
 use std::str;
 use std::sync::Arc;
 
-use heed::types::{ByteSlice, DecodeIgnore, SerdeJson};
-use heed::{Database, Env, EnvOpenOptions, RwTxn};
+use enum_iterator::IntoEnumIterator;
+use hmac::{Hmac, Mac};
+use meilisearch_types::star_or::StarOr;
+use milli::heed::types::{ByteSlice, DecodeIgnore, SerdeJson};
+use milli::heed::{Database, Env, EnvOpenOptions, RwTxn};
+use sha2::{Digest, Sha256};
 use time::OffsetDateTime;
+use uuid::Uuid;
 
 use super::error::Result;
 use super::{Action, Key};
 
 const AUTH_STORE_SIZE: usize = 1_073_741_824; //1GiB
-pub const KEY_ID_LENGTH: usize = 8;
 const AUTH_DB_PATH: &str = "auth";
 const KEY_DB_NAME: &str = "api-keys";
 const KEY_ID_ACTION_INDEX_EXPIRATION_DB_NAME: &str = "keyid-action-index-expiration";
 
-pub type KeyId = [u8; KEY_ID_LENGTH];
+pub type KeyId = Uuid;
 
 #[derive(Clone)]
 pub struct HeedAuthStore {
@@ -39,7 +43,7 @@ impl Drop for HeedAuthStore {
     }
 }
 
-pub fn open_auth_store_env(path: &Path) -> heed::Result<heed::Env> {
+pub fn open_auth_store_env(path: &Path) -> milli::heed::Result<milli::heed::Env> {
     let mut options = EnvOpenOptions::new();
     options.map_size(AUTH_STORE_SIZE); // 1GB
     options.max_dbs(2);
@@ -73,33 +77,43 @@ impl HeedAuthStore {
     }
 
     pub fn put_api_key(&self, key: Key) -> Result<Key> {
+        let uid = key.uid;
         let mut wtxn = self.env.write_txn()?;
-        self.keys.put(&mut wtxn, &key.id, &key)?;
 
-        let id = key.id;
+        self.keys.put(&mut wtxn, uid.as_bytes(), &key)?;
+
         // delete key from inverted database before refilling it.
-        self.delete_key_from_inverted_db(&mut wtxn, &id)?;
+        self.delete_key_from_inverted_db(&mut wtxn, &uid)?;
         // create inverted database.
         let db = self.action_keyid_index_expiration;
 
         let actions = if key.actions.contains(&Action::All) {
             // if key.actions contains All, we iterate over all actions.
             Action::into_enum_iter().collect()
+        } else if key.actions.contains(&Action::DocumentsAll) {
+            // if key.actions.contains.DocumentsAll add all actions related to documents.
+            let mut actions = key.actions.clone();
+            actions.append(&mut vec![
+                Action::DocumentsAdd,
+                Action::DocumentsGet,
+                Action::DocumentsDelete,
+            ]);
+            actions
         } else {
             key.actions.clone()
         };
 
-        let no_index_restriction = key.indexes.contains(&"*".to_owned());
+        let no_index_restriction = key.indexes.contains(&StarOr::Star);
         for action in actions {
             if no_index_restriction {
                 // If there is no index restriction we put None.
-                db.put(&mut wtxn, &(&id, &action, None), &key.expires_at)?;
+                db.put(&mut wtxn, &(&uid, &action, None), &key.expires_at)?;
             } else {
                 // else we create a key for each index.
                 for index in key.indexes.iter() {
                     db.put(
                         &mut wtxn,
-                        &(&id, &action, Some(index.as_bytes())),
+                        &(&uid, &action, Some(index.deref().as_bytes())),
                         &key.expires_at,
                     )?;
                 }
@@ -111,24 +125,39 @@ impl HeedAuthStore {
         Ok(key)
     }
 
-    pub fn get_api_key(&self, key: impl AsRef<str>) -> Result<Option<Key>> {
+    pub fn get_api_key(&self, uid: Uuid) -> Result<Option<Key>> {
         let rtxn = self.env.read_txn()?;
-        match self.get_key_id(key.as_ref().as_bytes()) {
-            Some(id) => self.keys.get(&rtxn, &id).map_err(|e| e.into()),
-            None => Ok(None),
-        }
+        self.keys.get(&rtxn, uid.as_bytes()).map_err(|e| e.into())
     }
 
-    pub fn delete_api_key(&self, key: impl AsRef<str>) -> Result<bool> {
+    pub fn get_uid_from_encoded_key(
+        &self,
+        encoded_key: &[u8],
+        master_key: &[u8],
+    ) -> Result<Option<Uuid>> {
+        let rtxn = self.env.read_txn()?;
+        let uid = self
+            .keys
+            .remap_data_type::<DecodeIgnore>()
+            .iter(&rtxn)?
+            .filter_map(|res| match res {
+                Ok((uid, _))
+                    if generate_key_as_base64(uid, master_key).as_bytes() == encoded_key =>
+                {
+                    let (uid, _) = try_split_array_at(uid)?;
+                    Some(Uuid::from_bytes(*uid))
+                }
+                _ => None,
+            })
+            .next();
+
+        Ok(uid)
+    }
+
+    pub fn delete_api_key(&self, uid: Uuid) -> Result<bool> {
         let mut wtxn = self.env.write_txn()?;
-        let existing = match self.get_key_id(key.as_ref().as_bytes()) {
-            Some(id) => {
-                let existing = self.keys.delete(&mut wtxn, &id)?;
-                self.delete_key_from_inverted_db(&mut wtxn, &id)?;
-                existing
-            }
-            None => false,
-        };
+        let existing = self.keys.delete(&mut wtxn, uid.as_bytes())?;
+        self.delete_key_from_inverted_db(&mut wtxn, &uid)?;
         wtxn.commit()?;
 
         Ok(existing)
@@ -147,49 +176,37 @@ impl HeedAuthStore {
 
     pub fn get_expiration_date(
         &self,
-        key: &[u8],
+        uid: Uuid,
         action: Action,
         index: Option<&[u8]>,
     ) -> Result<Option<Option<OffsetDateTime>>> {
         let rtxn = self.env.read_txn()?;
-        match self.get_key_id(key) {
-            Some(id) => {
-                let tuple = (&id, &action, index);
-                Ok(self.action_keyid_index_expiration.get(&rtxn, &tuple)?)
-            }
-            None => Ok(None),
-        }
+        let tuple = (&uid, &action, index);
+        Ok(self.action_keyid_index_expiration.get(&rtxn, &tuple)?)
     }
 
     pub fn prefix_first_expiration_date(
         &self,
-        key: &[u8],
+        uid: Uuid,
         action: Action,
     ) -> Result<Option<Option<OffsetDateTime>>> {
         let rtxn = self.env.read_txn()?;
-        match self.get_key_id(key) {
-            Some(id) => {
-                let tuple = (&id, &action, None);
-                Ok(self
-                    .action_keyid_index_expiration
-                    .prefix_iter(&rtxn, &tuple)?
-                    .next()
-                    .transpose()?
-                    .map(|(_, expiration)| expiration))
-            }
-            None => Ok(None),
-        }
-    }
+        let tuple = (&uid, &action, None);
+        let exp = self
+            .action_keyid_index_expiration
+            .prefix_iter(&rtxn, &tuple)?
+            .next()
+            .transpose()?
+            .map(|(_, expiration)| expiration);
 
-    pub fn get_key_id(&self, key: &[u8]) -> Option<KeyId> {
-        try_split_array_at::<_, KEY_ID_LENGTH>(key).map(|(id, _)| *id)
+        Ok(exp)
     }
 
     fn delete_key_from_inverted_db(&self, wtxn: &mut RwTxn, key: &KeyId) -> Result<()> {
         let mut iter = self
             .action_keyid_index_expiration
             .remap_types::<ByteSlice, DecodeIgnore>()
-            .prefix_iter_mut(wtxn, key)?;
+            .prefix_iter_mut(wtxn, key.as_bytes())?;
         while iter.next().transpose()?.is_some() {
             // safety: we don't keep references from inside the LMDB database.
             unsafe { iter.del_current()? };
@@ -200,31 +217,32 @@ impl HeedAuthStore {
 }
 
 /// Codec allowing to retrieve the expiration date of an action,
-/// optionnally on a spcific index, for a given key.
+/// optionally on a specific index, for a given key.
 pub struct KeyIdActionCodec;
 
-impl<'a> heed::BytesDecode<'a> for KeyIdActionCodec {
+impl<'a> milli::heed::BytesDecode<'a> for KeyIdActionCodec {
     type DItem = (KeyId, Action, Option<&'a [u8]>);
 
     fn bytes_decode(bytes: &'a [u8]) -> Option<Self::DItem> {
-        let (key_id, action_bytes) = try_split_array_at(bytes)?;
+        let (key_id_bytes, action_bytes) = try_split_array_at(bytes)?;
         let (action_bytes, index) = match try_split_array_at(action_bytes)? {
             (action, []) => (action, None),
             (action, index) => (action, Some(index)),
         };
+        let key_id = Uuid::from_bytes(*key_id_bytes);
         let action = Action::from_repr(u8::from_be_bytes(*action_bytes))?;
 
-        Some((*key_id, action, index))
+        Some((key_id, action, index))
     }
 }
 
-impl<'a> heed::BytesEncode<'a> for KeyIdActionCodec {
+impl<'a> milli::heed::BytesEncode<'a> for KeyIdActionCodec {
     type EItem = (&'a KeyId, &'a Action, Option<&'a [u8]>);
 
     fn bytes_encode((key_id, action, index): &Self::EItem) -> Option<Cow<[u8]>> {
         let mut bytes = Vec::new();
 
-        bytes.extend_from_slice(*key_id);
+        bytes.extend_from_slice(key_id.as_bytes());
         let action_bytes = u8::to_be_bytes(action.repr());
         bytes.extend_from_slice(&action_bytes);
         if let Some(index) = index {
@@ -235,6 +253,15 @@ impl<'a> heed::BytesEncode<'a> for KeyIdActionCodec {
     }
 }
 
+pub fn generate_key_as_base64(uid: &[u8], master_key: &[u8]) -> String {
+    let master_key_sha = Sha256::digest(master_key);
+    let mut mac = Hmac::<Sha256>::new_from_slice(master_key_sha.as_slice()).unwrap();
+    mac.update(uid);
+
+    let result = mac.finalize();
+    base64::encode_config(result.into_bytes(), base64::URL_SAFE_NO_PAD)
+}
+
 /// Divides one slice into two at an index, returns `None` if mid is out of bounds.
 pub fn try_split_at<T>(slice: &[T], mid: usize) -> Option<(&[T], &[T])> {
     if mid <= slice.len() {

meilisearch-http/Cargo.toml

@@ -4,90 +4,91 @@ description = "Meilisearch HTTP server"
 edition = "2021"
 license = "MIT"
 name = "meilisearch-http"
-version = "0.26.0"
+version = "0.28.0"
 
 [[bin]]
 name = "meilisearch"
 path = "src/main.rs"
 
 [build-dependencies]
-static-files = { version = "0.2.1", optional = true }
-anyhow = { version = "1.0.43", optional = true }
-cargo_toml = { version = "0.9", optional = true }
+anyhow = { version = "1.0.56", optional = true }
+cargo_toml = { version = "0.11.4", optional = true }
 hex = { version = "0.4.3", optional = true }
-reqwest = { version = "0.11.4", features = ["blocking", "rustls-tls"], default-features = false, optional = true }
-sha-1 = { version = "0.9.8", optional = true }
-tempfile = { version = "3.2.0", optional = true }
-vergen = { version = "5.1.15", default-features = false, features = ["git"] }
+reqwest = { version = "0.11.9", features = ["blocking", "rustls-tls"], default-features = false, optional = true }
+sha-1 = { version = "0.10.0", optional = true }
+static-files = { version = "0.2.3", optional = true }
+tempfile = { version = "3.3.0", optional = true }
+vergen = { version = "7.0.0", default-features = false, features = ["git"] }
 zip = { version = "0.5.13", optional = true }
 
 [dependencies]
-actix-cors = "0.6"
-actix-web = { version = "4", features = ["rustls"] }
+actix-cors = "0.6.1"
+actix-web = { version = "4.0.1", default-features = false, features = ["macros", "compress-brotli", "compress-gzip", "cookies", "rustls"] }
 actix-web-static-files = { git = "https://github.com/kilork/actix-web-static-files.git", rev = "2d3b6160", optional = true }
-anyhow = { version = "1.0.43", features = ["backtrace"] }
-arc-swap = "1.3.2"
-async-stream = "0.3.2"
-async-trait = "0.1.51"
+anyhow = { version = "1.0.56", features = ["backtrace"] }
+async-stream = "0.3.3"
+async-trait = "0.1.52"
 bstr = "0.2.17"
-byte-unit = { version = "4.0.12", default-features = false, features = ["std", "serde"] }
+byte-unit = { version = "4.0.14", default-features = false, features = ["std", "serde"] }
 bytes = "1.1.0"
-crossbeam-channel = "0.5.1"
+clap = { version = "3.1.6", features = ["derive", "env"] }
+crossbeam-channel = "0.5.2"
 either = "1.6.1"
 env_logger = "0.9.0"
-flate2 = "1.0.21"
+flate2 = "1.0.22"
 fst = "0.4.7"
-futures = "0.3.17"
-futures-util = "0.3.17"
-heed = { git = "https://github.com/Kerollmops/heed", tag = "v0.12.1" }
-http = "0.2.4"
-indexmap = { version = "1.7.0", features = ["serde-1"] }
-iso8601-duration = "0.1.0"
-itertools = "0.10.1"
-jsonwebtoken = "7"
+futures = "0.3.21"
+futures-util = "0.3.21"
+http = "0.2.6"
+indexmap = { version = "1.8.0", features = ["serde-1"] }
+itertools = "0.10.3"
+jsonwebtoken = "8.0.1"
 log = "0.4.14"
 meilisearch-auth = { path = "../meilisearch-auth" }
-meilisearch-error = { path = "../meilisearch-error" }
+meilisearch-types = { path = "../meilisearch-types" }
 meilisearch-lib = { path = "../meilisearch-lib" }
 mime = "0.3.16"
-num_cpus = "1.13.0"
+num_cpus = "1.13.1"
 obkv = "0.2.0"
-once_cell = "1.8.0"
-parking_lot = "0.11.2"
-platform-dirs = "0.3.0"
-rand = "0.8.4"
-rayon = "1.5.1"
-regex = "1.5.4"
-rustls = "0.20.2"
-rustls-pemfile = "0.2"
-segment = { version = "0.2.0", optional = true }
-serde = { version = "1.0.130", features = ["derive"] }
-serde_json = { version = "1.0.67", features = ["preserve_order"] }
-sha2 = "0.9.6"
-siphasher = "0.3.7"
-slice-group-by = "0.2.6"
-static-files = { version = "0.2.1", optional = true }
-clap = { version = "3.0", features = ["derive", "env"] }
-sysinfo = "0.20.2"
-tar = "0.4.37"
-tempfile = "3.2.0"
-thiserror = "1.0.28"
-time = { version = "0.3.7", features = ["serde-well-known", "formatting", "parsing", "macros"] }
-tokio = { version = "1.11.0", features = ["full"] }
-tokio-stream = "0.1.7"
-uuid = { version = "0.8.2", features = ["serde"] }
-walkdir = "2.3.2"
+once_cell = "1.10.0"
+parking_lot = "0.12.0"
 pin-project-lite = "0.2.8"
+platform-dirs = "0.3.0"
+rand = "0.8.5"
+rayon = "1.5.1"
+regex = "1.5.5"
+reqwest = { version = "0.11.4", features = ["rustls-tls", "json"], default-features = false }
+rustls = "0.20.4"
+rustls-pemfile = "0.3.0"
+segment = { version = "0.2.0", optional = true }
+serde = { version = "1.0.136", features = ["derive"] }
+serde-cs = "0.2.3"
+serde_json = { version = "1.0.79", features = ["preserve_order"] }
+sha2 = "0.10.2"
+siphasher = "0.3.10"
+slice-group-by = "0.3.0"
+static-files = { version = "0.2.3", optional = true }
+sysinfo = "0.23.5"
+tar = "0.4.38"
+tempfile = "3.3.0"
+thiserror = "1.0.30"
+time = { version = "0.3.7", features = ["serde-well-known", "formatting", "parsing", "macros"] }
+tokio = { version = "1.17.0", features = ["full"] }
+tokio-stream = "0.1.8"
+uuid = { version = "0.8.2", features = ["serde", "v4"] }
+walkdir = "2.3.2"
 
 [dev-dependencies]
-actix-rt = "2.2.0"
+actix-rt = "2.7.0"
 assert-json-diff = "2.0.1"
+manifest-dir-macros = "0.1.14"
 maplit = "1.0.2"
-paste = "1.0.5"
 serde_url_params = "0.2.1"
 urlencoding = "2.1.0"
 
 [features]
+default = ["analytics", "mini-dashboard"]
+analytics = ["segment"]
 mini-dashboard = [
     "actix-web-static-files",
     "static-files",
@@ -99,12 +100,10 @@ mini-dashboard = [
     "tempfile",
     "zip",
 ]
-analytics = ["segment"]
-default = ["analytics", "mini-dashboard"]
 
 [target.'cfg(target_os = "linux")'.dependencies]
-tikv-jemallocator = "0.4.1"
+tikv-jemallocator = "0.4.3"
 
 [package.metadata.mini-dashboard]
-assets-url = "https://github.com/meilisearch/mini-dashboard/releases/download/v0.1.9/build.zip"
-sha1 = "b1833c3e5dc6b5d9d519ae4834935ae6c8a47024"
+assets-url = "https://github.com/meilisearch/mini-dashboard/releases/download/v0.1.10/build.zip"
+sha1 = "1adf96592c267425c110bfefc36b7fc6bfb0f93d"

meilisearch-http/src/analytics/mod.rs

@@ -61,7 +61,7 @@ pub trait Analytics: Sync + Send {
     /// The method used to publish most analytics that do not need to be batched every hours
     fn publish(&self, event_name: String, send: Value, request: Option<&HttpRequest>);
 
-    /// This method should be called to aggergate a get search
+    /// This method should be called to aggregate a get search
     fn get_search(&self, aggregate: SearchAggregator);
 
     /// This method should be called to aggregate a post search

meilisearch-http/src/analytics/segment_analytics.rs

@@ -8,7 +8,10 @@ use actix_web::http::header::USER_AGENT;
 use actix_web::HttpRequest;
 use http::header::CONTENT_TYPE;
 use meilisearch_auth::SearchRules;
-use meilisearch_lib::index::{SearchQuery, SearchResult};
+use meilisearch_lib::index::{
+    SearchQuery, SearchResult, DEFAULT_CROP_LENGTH, DEFAULT_CROP_MARKER,
+    DEFAULT_HIGHLIGHT_POST_TAG, DEFAULT_HIGHLIGHT_PRE_TAG,
+};
 use meilisearch_lib::index_controller::Stats;
 use meilisearch_lib::MeiliSearch;
 use once_cell::sync::Lazy;
@@ -28,6 +31,8 @@ use crate::Opt;
 
 use super::{config_user_id_path, MEILISEARCH_CONFIG_PATH};
 
+const ANALYTICS_HEADER: &str = "X-Meilisearch-Client";
+
 /// Write the instance-uid in the `data.ms` and in `~/.config/MeiliSearch/path-to-db-instance-uid`. Ignore the errors.
 fn write_user_id(db_path: &Path, user_id: &str) {
     let _ = fs::write(db_path.join("instance-uid"), user_id.as_bytes());
@@ -45,7 +50,8 @@ const SEGMENT_API_KEY: &str = "P3FWhhEsJiEDCuEHpmcN9DHcK4hVfBvb";
 pub fn extract_user_agents(request: &HttpRequest) -> Vec<String> {
     request
         .headers()
-        .get(USER_AGENT)
+        .get(ANALYTICS_HEADER)
+        .or_else(|| request.headers().get(USER_AGENT))
         .map(|header| header.to_str().ok())
         .flatten()
         .unwrap_or("unknown")
@@ -75,7 +81,19 @@ impl SegmentAnalytics {
         let user_id = user_id.unwrap_or_else(|| Uuid::new_v4().to_string());
         write_user_id(&opt.db_path, &user_id);
 
-        let client = HttpClient::default();
+        let client = reqwest::Client::builder()
+            .connect_timeout(Duration::from_secs(10))
+            .build();
+
+        // if reqwest throws an error we won't be able to send analytics
+        if client.is_err() {
+            return super::MockAnalytics::new(opt);
+        }
+
+        let client = HttpClient::new(
+            client.unwrap(),
+            "https://telemetry.meilisearch.com".to_string(),
+        );
         let user = User::UserId { user_id };
         let mut batcher = AutoBatcher::new(client, Batcher::new(None), SEGMENT_API_KEY.to_string());
 
@@ -127,11 +145,7 @@ impl SegmentAnalytics {
 
 impl super::Analytics for SegmentAnalytics {
     fn publish(&self, event_name: String, mut send: Value, request: Option<&HttpRequest>) {
-        let user_agent = request
-            .map(|req| req.headers().get(USER_AGENT))
-            .flatten()
-            .map(|header| header.to_str().unwrap_or("unknown"))
-            .map(|s| s.split(';').map(str::trim).collect::<Vec<&str>>());
+        let user_agent = request.map(|req| extract_user_agents(req));
 
         send["user-agent"] = json!(user_agent);
         let event = Track {
@@ -355,6 +369,13 @@ pub struct SearchAggregator {
     // pagination
     max_limit: usize,
     max_offset: usize,
+
+    // formatting
+    highlight_pre_tag: bool,
+    highlight_post_tag: bool,
+    crop_marker: bool,
+    show_matches_position: bool,
+    crop_length: bool,
 }
 
 impl SearchAggregator {
@@ -405,6 +426,12 @@ impl SearchAggregator {
         ret.max_limit = query.limit;
         ret.max_offset = query.offset.unwrap_or_default();
 
+        ret.highlight_pre_tag = query.highlight_pre_tag != DEFAULT_HIGHLIGHT_PRE_TAG();
+        ret.highlight_post_tag = query.highlight_post_tag != DEFAULT_HIGHLIGHT_POST_TAG();
+        ret.crop_marker = query.crop_marker != DEFAULT_CROP_MARKER();
+        ret.crop_length = query.crop_length != DEFAULT_CROP_LENGTH();
+        ret.show_matches_position = query.show_matches_position;
+
         ret
     }
 
@@ -452,6 +479,12 @@ impl SearchAggregator {
         // pagination
         self.max_limit = self.max_limit.max(other.max_limit);
         self.max_offset = self.max_offset.max(other.max_offset);
+
+        self.highlight_pre_tag |= other.highlight_pre_tag;
+        self.highlight_post_tag |= other.highlight_post_tag;
+        self.crop_marker |= other.crop_marker;
+        self.show_matches_position |= other.show_matches_position;
+        self.crop_length |= other.crop_length;
     }
 
     pub fn into_event(self, user: &User, event_name: &str) -> Option<Track> {
@@ -462,7 +495,7 @@ impl SearchAggregator {
             let percentile_99th = 0.99 * (self.total_succeeded as f64 - 1.) + 1.;
             // we get all the values in a sorted manner
             let time_spent = self.time_spent.into_sorted_vec();
-            // We are only intersted by the slowest value of the 99th fastest results
+            // We are only interested by the slowest value of the 99th fastest results
             let time_spent = time_spent.get(percentile_99th as usize);
 
             let properties = json!({
@@ -489,6 +522,13 @@ impl SearchAggregator {
                    "max_limit": self.max_limit,
                    "max_offset": self.max_offset,
                 },
+                "formatting": {
+                    "highlight_pre_tag": self.highlight_pre_tag,
+                    "highlight_post_tag": self.highlight_post_tag,
+                    "crop_marker": self.crop_marker,
+                    "show_matches_position": self.show_matches_position,
+                    "crop_length": self.crop_length,
+                },
             });
 
             Some(Track {
@@ -534,8 +574,8 @@ impl DocumentsAggregator {
         let content_type = request
             .headers()
             .get(CONTENT_TYPE)
-            .map(|s| s.to_str().unwrap_or("unkown"))
-            .unwrap()
+            .map(|s| s.to_str().unwrap_or("unknown"))
+            .unwrap_or("unknown")
             .to_string();
         ret.content_types.insert(content_type);
         ret.index_creation = index_creation;

meilisearch-http/src/error.rs

@@ -1,6 +1,6 @@
 use actix_web as aweb;
 use aweb::error::{JsonPayloadError, QueryPayloadError};
-use meilisearch_error::{Code, ErrorCode, ResponseError};
+use meilisearch_types::error::{Code, ErrorCode, ResponseError};
 
 #[derive(Debug, thiserror::Error)]
 pub enum MeilisearchHttpError {

meilisearch-http/src/extractors/authentication/error.rs

@@ -1,4 +1,4 @@
-use meilisearch_error::{Code, ErrorCode};
+use meilisearch_types::error::{Code, ErrorCode};
 
 #[derive(Debug, thiserror::Error)]
 pub enum AuthenticationError {

meilisearch-http/src/extractors/authentication/mod.rs

@@ -5,12 +5,11 @@ use std::ops::Deref;
 use std::pin::Pin;
 
 use actix_web::FromRequest;
+use error::AuthenticationError;
 use futures::future::err;
 use futures::Future;
-use meilisearch_error::{Code, ResponseError};
-
-use error::AuthenticationError;
 use meilisearch_auth::{AuthController, AuthFilter};
+use meilisearch_types::error::{Code, ResponseError};
 
 pub struct GuardedData<P, D> {
     data: D,
@@ -70,11 +69,9 @@ impl<P, D> GuardedData<P, D> {
     where
         P: Policy + 'static,
     {
-        Ok(tokio::task::spawn_blocking(move || {
-            P::authenticate(auth, token.as_ref(), index.as_deref())
-        })
-        .await
-        .map_err(|e| ResponseError::from_msg(e.to_string(), Code::Internal))?)
+        tokio::task::spawn_blocking(move || P::authenticate(auth, token.as_ref(), index.as_deref()))
+            .await
+            .map_err(|e| ResponseError::from_msg(e.to_string(), Code::Internal))
     }
 }
 
@@ -131,38 +128,39 @@ pub trait Policy {
 }
 
 pub mod policies {
-    use jsonwebtoken::{dangerous_insecure_decode, decode, Algorithm, DecodingKey, Validation};
-    use once_cell::sync::Lazy;
+    use jsonwebtoken::{decode, Algorithm, DecodingKey, Validation};
     use serde::{Deserialize, Serialize};
     use time::OffsetDateTime;
+    use uuid::Uuid;
 
     use crate::extractors::authentication::Policy;
     use meilisearch_auth::{Action, AuthController, AuthFilter, SearchRules};
     // reexport actions in policies in order to be used in routes configuration.
     pub use meilisearch_auth::actions;
 
-    pub static TENANT_TOKEN_VALIDATION: Lazy<Validation> = Lazy::new(|| Validation {
-        validate_exp: false,
-        algorithms: vec![Algorithm::HS256, Algorithm::HS384, Algorithm::HS512],
-        ..Default::default()
-    });
+    fn tenant_token_validation() -> Validation {
+        let mut validation = Validation::default();
+        validation.validate_exp = false;
+        validation.required_spec_claims.remove("exp");
+        validation.algorithms = vec![Algorithm::HS256, Algorithm::HS384, Algorithm::HS512];
+        validation
+    }
 
-    pub struct MasterPolicy;
+    /// Extracts the key id used to sign the payload, without performing any validation.
+    fn extract_key_id(token: &str) -> Option<Uuid> {
+        let mut validation = tenant_token_validation();
+        validation.insecure_disable_signature_validation();
+        let dummy_key = DecodingKey::from_secret(b"secret");
+        let token_data = decode::<Claims>(token, &dummy_key, &validation).ok()?;
 
-    impl Policy for MasterPolicy {
-        fn authenticate(
-            auth: AuthController,
-            token: &str,
-            _index: Option<&str>,
-        ) -> Option<AuthFilter> {
-            if let Some(master_key) = auth.get_master_key() {
-                if master_key == token {
-                    return Some(AuthFilter::default());
-                }
-            }
+        // get token fields without validating it.
+        let Claims { api_key_uid, .. } = token_data.claims;
+        Some(api_key_uid)
+    }
 
-            None
-        }
+    fn is_keys_action(action: u8) -> bool {
+        use actions::*;
+        matches!(action, KEYS_GET | KEYS_CREATE | KEYS_UPDATE | KEYS_DELETE)
     }
 
     pub struct ActionPolicy<const A: u8>;
@@ -174,7 +172,12 @@ pub mod policies {
             index: Option<&str>,
         ) -> Option<AuthFilter> {
             // authenticate if token is the master key.
-            if auth.get_master_key().map_or(true, |mk| mk == token) {
+            // master key can only have access to keys routes.
+            // if master key is None only keys routes are inaccessible.
+            if auth
+                .get_master_key()
+                .map_or_else(|| !is_keys_action(A), |mk| mk == token)
+            {
                 return Some(AuthFilter::default());
             }
 
@@ -184,8 +187,10 @@ pub mod policies {
                 return Some(filters);
             } else if let Some(action) = Action::from_repr(A) {
                 // API key
-                if let Ok(true) = auth.authenticate(token.as_bytes(), action, index) {
-                    return auth.get_key_filters(token, None).ok();
+                if let Ok(Some(uid)) = auth.get_optional_uid_from_encoded_key(token.as_bytes()) {
+                    if let Ok(true) = auth.is_key_authorized(uid, action, index) {
+                        return auth.get_key_filters(uid, None).ok();
+                    }
                 }
             }
 
@@ -204,43 +209,34 @@ pub mod policies {
                 return None;
             }
 
-            // get token fields without validating it.
-            let Claims {
-                search_rules,
-                exp,
-                api_key_prefix,
-            } = dangerous_insecure_decode::<Claims>(token).ok()?.claims;
-
-            // Check index access if an index restriction is provided.
-            if let Some(index) = index {
-                if !search_rules.is_index_authorized(index) {
-                    return None;
-                }
-            }
-
-            // Check if token is expired.
-            if let Some(exp) = exp {
-                if OffsetDateTime::now_utc().unix_timestamp() > exp {
-                    return None;
-                }
-            }
-
+            let uid = extract_key_id(token)?;
             // check if parent key is authorized to do the action.
-            if auth
-                .is_key_authorized(api_key_prefix.as_bytes(), Action::Search, index)
-                .ok()?
-            {
+            if auth.is_key_authorized(uid, Action::Search, index).ok()? {
                 // Check if tenant token is valid.
-                let key = auth.generate_key(&api_key_prefix)?;
-                decode::<Claims>(
+                let key = auth.generate_key(uid)?;
+                let data = decode::<Claims>(
                     token,
                     &DecodingKey::from_secret(key.as_bytes()),
-                    &TENANT_TOKEN_VALIDATION,
+                    &tenant_token_validation(),
                 )
                 .ok()?;
 
+                // Check index access if an index restriction is provided.
+                if let Some(index) = index {
+                    if !data.claims.search_rules.is_index_authorized(index) {
+                        return None;
+                    }
+                }
+
+                // Check if token is expired.
+                if let Some(exp) = data.claims.exp {
+                    if OffsetDateTime::now_utc().unix_timestamp() > exp {
+                        return None;
+                    }
+                }
+
                 return auth
-                    .get_key_filters(api_key_prefix, Some(search_rules))
+                    .get_key_filters(uid, Some(data.claims.search_rules))
                     .ok();
             }
 
@@ -253,6 +249,6 @@ pub mod policies {
     struct Claims {
         search_rules: SearchRules,
         exp: Option<i64>,
-        api_key_prefix: String,
+        api_key_uid: Uuid,
     }
 }

meilisearch-http/src/helpers/env.rs

@@ -1,10 +1,11 @@
+use meilisearch_lib::heed::Env;
 use walkdir::WalkDir;
 
 pub trait EnvSizer {
     fn size(&self) -> u64;
 }
 
-impl EnvSizer for heed::Env {
+impl EnvSizer for Env {
     fn size(&self) -> u64 {
         WalkDir::new(self.path())
             .into_iter()

meilisearch-http/src/lib.rs

@@ -2,7 +2,7 @@
 #[macro_use]
 pub mod error;
 pub mod analytics;
-mod task;
+pub mod task;
 #[macro_use]
 pub mod extractors;
 pub mod helpers;
@@ -148,10 +148,10 @@ macro_rules! create_app {
         use actix_web::middleware::TrailingSlash;
         use actix_web::App;
         use actix_web::{middleware, web};
-        use meilisearch_error::ResponseError;
         use meilisearch_http::error::MeilisearchHttpError;
         use meilisearch_http::routes;
         use meilisearch_http::{configure_data, dashboard};
+        use meilisearch_types::error::ResponseError;
 
         App::new()
             .configure(|s| configure_data(s, $data.clone(), $auth.clone(), &$opt, $analytics))

meilisearch-http/src/main.rs

@@ -1,6 +1,7 @@
 use std::env;
 use std::sync::Arc;
 
+use actix_web::http::KeepAlive;
 use actix_web::HttpServer;
 use clap::Parser;
 use meilisearch_auth::AuthController;
@@ -83,7 +84,8 @@ async fn run_http(
         )
     })
     // Disable signals allows the server to terminate immediately when a user enter CTRL-C
-    .disable_signals();
+    .disable_signals()
+    .keep_alive(KeepAlive::Os);
 
     if let Some(config) = opt.get_ssl_config()? {
         http_server

meilisearch-http/src/option.rs

@@ -19,6 +19,7 @@ use serde::Serialize;
 const POSSIBLE_ENV: [&str; 2] = ["development", "production"];
 
 #[derive(Debug, Clone, Parser, Serialize)]
+#[clap(version)]
 pub struct Opt {
     /// The destination where the database must be created.
     #[clap(long, env = "MEILI_DB_PATH", default_value = "./data.ms")]
@@ -145,8 +146,8 @@ pub struct Opt {
     #[clap(long, env = "MEILI_LOG_LEVEL", default_value = "info")]
     pub log_level: String,
 
-    #[serde(skip)]
-    #[clap(skip)]
+    #[serde(flatten)]
+    #[clap(flatten)]
     pub indexer_options: IndexerOpts,
 
     #[serde(flatten)]
@@ -258,3 +259,13 @@ fn load_ocsp(filename: &Option<PathBuf>) -> anyhow::Result<Vec<u8>> {
 
     Ok(ret)
 }
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn test_valid_opt() {
+        assert!(Opt::try_parse_from(Some("")).is_ok());
+    }
+}

meilisearch-http/src/routes/api_key.rs

@@ -1,17 +1,19 @@
 use std::str;
 
 use actix_web::{web, HttpRequest, HttpResponse};
-
-use meilisearch_auth::{error::AuthControllerError, Action, AuthController, Key};
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
 use time::OffsetDateTime;
+use uuid::Uuid;
+
+use meilisearch_auth::{error::AuthControllerError, Action, AuthController, Key};
+use meilisearch_types::error::{Code, ResponseError};
 
 use crate::extractors::{
     authentication::{policies::*, GuardedData},
     sequential_extractor::SeqHandler,
 };
-use meilisearch_error::{Code, ResponseError};
+use crate::routes::Pagination;
 
 pub fn configure(cfg: &mut web::ServiceConfig) {
     cfg.service(
@@ -20,7 +22,7 @@ pub fn configure(cfg: &mut web::ServiceConfig) {
             .route(web::get().to(SeqHandler(list_api_keys))),
     )
     .service(
-        web::resource("/{api_key}")
+        web::resource("/{key}")
             .route(web::get().to(SeqHandler(get_api_key)))
             .route(web::patch().to(SeqHandler(patch_api_key)))
             .route(web::delete().to(SeqHandler(delete_api_key))),
@@ -28,7 +30,7 @@ pub fn configure(cfg: &mut web::ServiceConfig) {
 }
 
 pub async fn create_api_key(
-    auth_controller: GuardedData<MasterPolicy, AuthController>,
+    auth_controller: GuardedData<ActionPolicy<{ actions::KEYS_CREATE }>, AuthController>,
     body: web::Json<Value>,
     _req: HttpRequest,
 ) -> Result<HttpResponse, ResponseError> {
@@ -44,30 +46,35 @@ pub async fn create_api_key(
 }
 
 pub async fn list_api_keys(
-    auth_controller: GuardedData<MasterPolicy, AuthController>,
-    _req: HttpRequest,
+    auth_controller: GuardedData<ActionPolicy<{ actions::KEYS_GET }>, AuthController>,
+    paginate: web::Query<Pagination>,
 ) -> Result<HttpResponse, ResponseError> {
-    let res = tokio::task::spawn_blocking(move || -> Result<_, AuthControllerError> {
+    let page_view = tokio::task::spawn_blocking(move || -> Result<_, AuthControllerError> {
         let keys = auth_controller.list_keys()?;
-        let res: Vec<_> = keys
-            .into_iter()
-            .map(|k| KeyView::from_key(k, &auth_controller))
-            .collect();
-        Ok(res)
+        let page_view = paginate.auto_paginate_sized(
+            keys.into_iter()
+                .map(|k| KeyView::from_key(k, &auth_controller)),
+        );
+
+        Ok(page_view)
     })
     .await
     .map_err(|e| ResponseError::from_msg(e.to_string(), Code::Internal))??;
 
-    Ok(HttpResponse::Ok().json(KeyListView::from(res)))
+    Ok(HttpResponse::Ok().json(page_view))
 }
 
 pub async fn get_api_key(
-    auth_controller: GuardedData<MasterPolicy, AuthController>,
+    auth_controller: GuardedData<ActionPolicy<{ actions::KEYS_GET }>, AuthController>,
     path: web::Path<AuthParam>,
 ) -> Result<HttpResponse, ResponseError> {
-    let api_key = path.into_inner().api_key;
+    let key = path.into_inner().key;
+
     let res = tokio::task::spawn_blocking(move || -> Result<_, AuthControllerError> {
-        let key = auth_controller.get_key(&api_key)?;
+        let uid =
+            Uuid::parse_str(&key).or_else(|_| auth_controller.get_uid_from_encoded_key(&key))?;
+        let key = auth_controller.get_key(uid)?;
+
         Ok(KeyView::from_key(key, &auth_controller))
     })
     .await
@@ -77,14 +84,17 @@ pub async fn get_api_key(
 }
 
 pub async fn patch_api_key(
-    auth_controller: GuardedData<MasterPolicy, AuthController>,
+    auth_controller: GuardedData<ActionPolicy<{ actions::KEYS_UPDATE }>, AuthController>,
     body: web::Json<Value>,
     path: web::Path<AuthParam>,
 ) -> Result<HttpResponse, ResponseError> {
-    let api_key = path.into_inner().api_key;
+    let key = path.into_inner().key;
     let body = body.into_inner();
     let res = tokio::task::spawn_blocking(move || -> Result<_, AuthControllerError> {
-        let key = auth_controller.update_key(&api_key, body)?;
+        let uid =
+            Uuid::parse_str(&key).or_else(|_| auth_controller.get_uid_from_encoded_key(&key))?;
+        let key = auth_controller.update_key(uid, body)?;
+
         Ok(KeyView::from_key(key, &auth_controller))
     })
     .await
@@ -94,27 +104,33 @@ pub async fn patch_api_key(
 }
 
 pub async fn delete_api_key(
-    auth_controller: GuardedData<MasterPolicy, AuthController>,
+    auth_controller: GuardedData<ActionPolicy<{ actions::KEYS_DELETE }>, AuthController>,
     path: web::Path<AuthParam>,
 ) -> Result<HttpResponse, ResponseError> {
-    let api_key = path.into_inner().api_key;
-    tokio::task::spawn_blocking(move || auth_controller.delete_key(&api_key))
-        .await
-        .map_err(|e| ResponseError::from_msg(e.to_string(), Code::Internal))??;
+    let key = path.into_inner().key;
+    tokio::task::spawn_blocking(move || {
+        let uid =
+            Uuid::parse_str(&key).or_else(|_| auth_controller.get_uid_from_encoded_key(&key))?;
+        auth_controller.delete_key(uid)
+    })
+    .await
+    .map_err(|e| ResponseError::from_msg(e.to_string(), Code::Internal))??;
 
     Ok(HttpResponse::NoContent().finish())
 }
 
 #[derive(Deserialize)]
 pub struct AuthParam {
-    api_key: String,
+    key: String,
 }
 
 #[derive(Debug, Serialize)]
 #[serde(rename_all = "camelCase")]
 struct KeyView {
+    name: Option<String>,
     description: Option<String>,
     key: String,
+    uid: Uuid,
     actions: Vec<Action>,
     indexes: Vec<String>,
     #[serde(serialize_with = "time::serde::rfc3339::option::serialize")]
@@ -127,28 +143,18 @@ struct KeyView {
 
 impl KeyView {
     fn from_key(key: Key, auth: &AuthController) -> Self {
-        let key_id = str::from_utf8(&key.id).unwrap();
-        let generated_key = auth.generate_key(key_id).unwrap_or_default();
+        let generated_key = auth.generate_key(key.uid).unwrap_or_default();
 
         KeyView {
+            name: key.name,
             description: key.description,
             key: generated_key,
+            uid: key.uid,
             actions: key.actions,
-            indexes: key.indexes,
+            indexes: key.indexes.into_iter().map(String::from).collect(),
             expires_at: key.expires_at,
             created_at: key.created_at,
             updated_at: key.updated_at,
         }
     }
 }
-
-#[derive(Debug, Serialize)]
-struct KeyListView {
-    results: Vec<KeyView>,
-}
-
-impl From<Vec<KeyView>> for KeyListView {
-    fn from(results: Vec<KeyView>) -> Self {
-        Self { results }
-    }
-}

meilisearch-http/src/routes/dump.rs

@@ -1,19 +1,16 @@
 use actix_web::{web, HttpRequest, HttpResponse};
 use log::debug;
-use meilisearch_error::ResponseError;
 use meilisearch_lib::MeiliSearch;
-use serde::{Deserialize, Serialize};
+use meilisearch_types::error::ResponseError;
 use serde_json::json;
 
 use crate::analytics::Analytics;
 use crate::extractors::authentication::{policies::*, GuardedData};
 use crate::extractors::sequential_extractor::SeqHandler;
+use crate::task::SummarizedTaskView;
 
 pub fn configure(cfg: &mut web::ServiceConfig) {
-    cfg.service(web::resource("").route(web::post().to(SeqHandler(create_dump))))
-        .service(
-            web::resource("/{dump_uid}/status").route(web::get().to(SeqHandler(get_dump_status))),
-        );
+    cfg.service(web::resource("").route(web::post().to(SeqHandler(create_dump))));
 }
 
 pub async fn create_dump(
@@ -23,29 +20,8 @@ pub async fn create_dump(
 ) -> Result<HttpResponse, ResponseError> {
     analytics.publish("Dump Created".to_string(), json!({}), Some(&req));
 
-    let res = meilisearch.create_dump().await?;
+    let res: SummarizedTaskView = meilisearch.register_dump_task().await?.into();
 
     debug!("returns: {:?}", res);
     Ok(HttpResponse::Accepted().json(res))
 }
-
-#[derive(Debug, Serialize)]
-#[serde(rename_all = "camelCase")]
-struct DumpStatusResponse {
-    status: String,
-}
-
-#[derive(Deserialize)]
-struct DumpParam {
-    dump_uid: String,
-}
-
-async fn get_dump_status(
-    meilisearch: GuardedData<ActionPolicy<{ actions::DUMPS_GET }>, MeiliSearch>,
-    path: web::Path<DumpParam>,
-) -> Result<HttpResponse, ResponseError> {
-    let res = meilisearch.dump_info(path.dump_uid.clone()).await?;
-
-    debug!("returns: {:?}", res);
-    Ok(HttpResponse::Ok().json(res))
-}

meilisearch-http/src/routes/indexes/documents.rs

@@ -6,13 +6,15 @@ use actix_web::{web, HttpRequest, HttpResponse};
 use bstr::ByteSlice;
 use futures::{Stream, StreamExt};
 use log::debug;
-use meilisearch_error::ResponseError;
 use meilisearch_lib::index_controller::{DocumentAdditionFormat, Update};
 use meilisearch_lib::milli::update::IndexDocumentsMethod;
 use meilisearch_lib::MeiliSearch;
+use meilisearch_types::error::ResponseError;
+use meilisearch_types::star_or::StarOr;
 use mime::Mime;
 use once_cell::sync::Lazy;
 use serde::Deserialize;
+use serde_cs::vec::CS;
 use serde_json::Value;
 use tokio::sync::mpsc;
 
@@ -21,11 +23,9 @@ use crate::error::MeilisearchHttpError;
 use crate::extractors::authentication::{policies::*, GuardedData};
 use crate::extractors::payload::Payload;
 use crate::extractors::sequential_extractor::SeqHandler;
+use crate::routes::{fold_star_or, PaginationView};
 use crate::task::SummarizedTaskView;
 
-const DEFAULT_RETRIEVE_DOCUMENTS_OFFSET: usize = 0;
-const DEFAULT_RETRIEVE_DOCUMENTS_LIMIT: usize = 20;
-
 static ACCEPTED_CONTENT_TYPE: Lazy<Vec<String>> = Lazy::new(|| {
     vec![
         "application/json".to_string(),
@@ -86,14 +86,24 @@ pub fn configure(cfg: &mut web::ServiceConfig) {
     );
 }
 
+#[derive(Deserialize, Debug)]
+#[serde(rename_all = "camelCase", deny_unknown_fields)]
+pub struct GetDocument {
+    fields: Option<CS<StarOr<String>>>,
+}
+
 pub async fn get_document(
     meilisearch: GuardedData<ActionPolicy<{ actions::DOCUMENTS_GET }>, MeiliSearch>,
     path: web::Path<DocumentParam>,
+    params: web::Query<GetDocument>,
 ) -> Result<HttpResponse, ResponseError> {
     let index = path.index_uid.clone();
     let id = path.document_id.clone();
+    let GetDocument { fields } = params.into_inner();
+    let attributes_to_retrieve = fields.and_then(fold_star_or);
+
     let document = meilisearch
-        .document(index, id, None as Option<Vec<String>>)
+        .document(index, id, attributes_to_retrieve)
         .await?;
     debug!("returns: {:?}", document);
     Ok(HttpResponse::Ok().json(document))
@@ -116,9 +126,11 @@ pub async fn delete_document(
 #[derive(Deserialize, Debug)]
 #[serde(rename_all = "camelCase", deny_unknown_fields)]
 pub struct BrowseQuery {
-    offset: Option<usize>,
-    limit: Option<usize>,
-    attributes_to_retrieve: Option<String>,
+    #[serde(default)]
+    offset: usize,
+    #[serde(default = "crate::routes::PAGINATION_DEFAULT_LIMIT")]
+    limit: usize,
+    fields: Option<CS<StarOr<String>>>,
 }
 
 pub async fn get_all_documents(
@@ -127,27 +139,21 @@ pub async fn get_all_documents(
     params: web::Query<BrowseQuery>,
 ) -> Result<HttpResponse, ResponseError> {
     debug!("called with params: {:?}", params);
-    let attributes_to_retrieve = params.attributes_to_retrieve.as_ref().and_then(|attrs| {
-        let mut names = Vec::new();
-        for name in attrs.split(',').map(String::from) {
-            if name == "*" {
-                return None;
-            }
-            names.push(name);
-        }
-        Some(names)
-    });
+    let BrowseQuery {
+        limit,
+        offset,
+        fields,
+    } = params.into_inner();
+    let attributes_to_retrieve = fields.and_then(fold_star_or);
 
-    let documents = meilisearch
-        .documents(
-            path.into_inner(),
-            params.offset.unwrap_or(DEFAULT_RETRIEVE_DOCUMENTS_OFFSET),
-            params.limit.unwrap_or(DEFAULT_RETRIEVE_DOCUMENTS_LIMIT),
-            attributes_to_retrieve,
-        )
+    let (total, documents) = meilisearch
+        .documents(path.into_inner(), offset, limit, attributes_to_retrieve)
         .await?;
-    debug!("returns: {:?}", documents);
-    Ok(HttpResponse::Ok().json(documents))
+
+    let ret = PaginationView::new(offset, limit, total as usize, documents);
+
+    debug!("returns: {:?}", ret);
+    Ok(HttpResponse::Ok().json(ret))
 }
 
 #[derive(Deserialize, Debug)]

meilisearch-http/src/routes/indexes/mod.rs

@@ -1,8 +1,8 @@
 use actix_web::{web, HttpRequest, HttpResponse};
 use log::debug;
-use meilisearch_error::ResponseError;
 use meilisearch_lib::index_controller::Update;
 use meilisearch_lib::MeiliSearch;
+use meilisearch_types::error::ResponseError;
 use serde::{Deserialize, Serialize};
 use serde_json::json;
 use time::OffsetDateTime;
@@ -12,10 +12,11 @@ use crate::extractors::authentication::{policies::*, GuardedData};
 use crate::extractors::sequential_extractor::SeqHandler;
 use crate::task::SummarizedTaskView;
 
+use super::Pagination;
+
 pub mod documents;
 pub mod search;
 pub mod settings;
-pub mod tasks;
 
 pub fn configure(cfg: &mut web::ServiceConfig) {
     cfg.service(
@@ -28,30 +29,32 @@ pub fn configure(cfg: &mut web::ServiceConfig) {
             .service(
                 web::resource("")
                     .route(web::get().to(SeqHandler(get_index)))
-                    .route(web::put().to(SeqHandler(update_index)))
+                    .route(web::patch().to(SeqHandler(update_index)))
                     .route(web::delete().to(SeqHandler(delete_index))),
             )
             .service(web::resource("/stats").route(web::get().to(SeqHandler(get_index_stats))))
             .service(web::scope("/documents").configure(documents::configure))
             .service(web::scope("/search").configure(search::configure))
-            .service(web::scope("/tasks").configure(tasks::configure))
             .service(web::scope("/settings").configure(settings::configure)),
     );
 }
 
 pub async fn list_indexes(
     data: GuardedData<ActionPolicy<{ actions::INDEXES_GET }>, MeiliSearch>,
+    paginate: web::Query<Pagination>,
 ) -> Result<HttpResponse, ResponseError> {
     let search_rules = &data.filters().search_rules;
-    let indexes: Vec<_> = data
-        .list_indexes()
-        .await?
+    let indexes: Vec<_> = data.list_indexes().await?;
+    let nb_indexes = indexes.len();
+    let iter = indexes
         .into_iter()
-        .filter(|i| search_rules.is_index_authorized(&i.uid))
-        .collect();
+        .filter(|i| search_rules.is_index_authorized(&i.uid));
+    let ret = paginate
+        .into_inner()
+        .auto_paginate_unsized(nb_indexes, iter);
 
-    debug!("returns: {:?}", indexes);
-    Ok(HttpResponse::Ok().json(indexes))
+    debug!("returns: {:?}", ret);
+    Ok(HttpResponse::Ok().json(ret))
 }
 
 #[derive(Debug, Deserialize)]

meilisearch-http/src/routes/indexes/search.rs

@@ -1,15 +1,20 @@
 use actix_web::{web, HttpRequest, HttpResponse};
 use log::debug;
 use meilisearch_auth::IndexSearchRules;
-use meilisearch_error::ResponseError;
-use meilisearch_lib::index::{default_crop_length, SearchQuery, DEFAULT_SEARCH_LIMIT};
+use meilisearch_lib::index::{
+    SearchQuery, DEFAULT_CROP_LENGTH, DEFAULT_CROP_MARKER, DEFAULT_HIGHLIGHT_POST_TAG,
+    DEFAULT_HIGHLIGHT_PRE_TAG, DEFAULT_SEARCH_LIMIT,
+};
 use meilisearch_lib::MeiliSearch;
+use meilisearch_types::error::ResponseError;
 use serde::Deserialize;
+use serde_cs::vec::CS;
 use serde_json::Value;
 
 use crate::analytics::{Analytics, SearchAggregator};
 use crate::extractors::authentication::{policies::*, GuardedData};
 use crate::extractors::sequential_extractor::SeqHandler;
+use crate::routes::{fold_star_or, StarOr};
 
 pub fn configure(cfg: &mut web::ServiceConfig) {
     cfg.service(
@@ -25,36 +30,26 @@ pub struct SearchQueryGet {
     q: Option<String>,
     offset: Option<usize>,
     limit: Option<usize>,
-    attributes_to_retrieve: Option<String>,
-    attributes_to_crop: Option<String>,
-    #[serde(default = "default_crop_length")]
+    attributes_to_retrieve: Option<CS<StarOr<String>>>,
+    attributes_to_crop: Option<CS<StarOr<String>>>,
+    #[serde(default = "DEFAULT_CROP_LENGTH")]
     crop_length: usize,
-    attributes_to_highlight: Option<String>,
+    attributes_to_highlight: Option<CS<StarOr<String>>>,
     filter: Option<String>,
     sort: Option<String>,
     #[serde(default = "Default::default")]
-    matches: bool,
-    facets_distribution: Option<String>,
+    show_matches_position: bool,
+    facets: Option<CS<StarOr<String>>>,
+    #[serde(default = "DEFAULT_HIGHLIGHT_PRE_TAG")]
+    highlight_pre_tag: String,
+    #[serde(default = "DEFAULT_HIGHLIGHT_POST_TAG")]
+    highlight_post_tag: String,
+    #[serde(default = "DEFAULT_CROP_MARKER")]
+    crop_marker: String,
 }
 
 impl From<SearchQueryGet> for SearchQuery {
     fn from(other: SearchQueryGet) -> Self {
-        let attributes_to_retrieve = other
-            .attributes_to_retrieve
-            .map(|attrs| attrs.split(',').map(String::from).collect());
-
-        let attributes_to_crop = other
-            .attributes_to_crop
-            .map(|attrs| attrs.split(',').map(String::from).collect());
-
-        let attributes_to_highlight = other
-            .attributes_to_highlight
-            .map(|attrs| attrs.split(',').map(String::from).collect());
-
-        let facets_distribution = other
-            .facets_distribution
-            .map(|attrs| attrs.split(',').map(String::from).collect());
-
         let filter = match other.filter {
             Some(f) => match serde_json::from_str(&f) {
                 Ok(v) => Some(v),
@@ -63,20 +58,21 @@ impl From<SearchQueryGet> for SearchQuery {
             None => None,
         };
 
-        let sort = other.sort.map(|attr| fix_sort_query_parameters(&attr));
-
         Self {
             q: other.q,
             offset: other.offset,
-            limit: other.limit.unwrap_or(DEFAULT_SEARCH_LIMIT),
-            attributes_to_retrieve,
-            attributes_to_crop,
+            limit: other.limit.unwrap_or_else(DEFAULT_SEARCH_LIMIT),
+            attributes_to_retrieve: other.attributes_to_retrieve.and_then(fold_star_or),
+            attributes_to_crop: other.attributes_to_crop.and_then(fold_star_or),
             crop_length: other.crop_length,
-            attributes_to_highlight,
+            attributes_to_highlight: other.attributes_to_highlight.and_then(fold_star_or),
             filter,
-            sort,
-            matches: other.matches,
-            facets_distribution,
+            sort: other.sort.map(|attr| fix_sort_query_parameters(&attr)),
+            show_matches_position: other.show_matches_position,
+            facets: other.facets.and_then(fold_star_or),
+            highlight_pre_tag: other.highlight_pre_tag,
+            highlight_post_tag: other.highlight_post_tag,
+            crop_marker: other.crop_marker,
         }
     }
 }
@@ -157,10 +153,6 @@ pub async fn search_with_url_query(
 
     let search_result = search_result?;
 
-    // Tests that the nb_hits is always set to false
-    #[cfg(test)]
-    assert!(!search_result.exhaustive_nb_hits);
-
     debug!("returns: {:?}", search_result);
     Ok(HttpResponse::Ok().json(search_result))
 }
@@ -195,10 +187,6 @@ pub async fn search_with_post(
 
     let search_result = search_result?;
 
-    // Tests that the nb_hits is always set to false
-    #[cfg(test)]
-    assert!(!search_result.exhaustive_nb_hits);
-
     debug!("returns: {:?}", search_result);
     Ok(HttpResponse::Ok().json(search_result))
 }

meilisearch-http/src/routes/indexes/settings.rs

@@ -1,10 +1,10 @@
 use log::debug;
 
 use actix_web::{web, HttpRequest, HttpResponse};
-use meilisearch_error::ResponseError;
 use meilisearch_lib::index::{Settings, Unchecked};
 use meilisearch_lib::index_controller::Update;
 use meilisearch_lib::MeiliSearch;
+use meilisearch_types::error::ResponseError;
 use serde_json::json;
 
 use crate::analytics::Analytics;
@@ -13,7 +13,7 @@ use crate::task::SummarizedTaskView;
 
 #[macro_export]
 macro_rules! make_setting_route {
-    ($route:literal, $type:ty, $attr:ident, $camelcase_attr:literal, $analytics_var:ident, $analytics:expr) => {
+    ($route:literal, $update_verb:ident, $type:ty, $attr:ident, $camelcase_attr:literal, $analytics_var:ident, $analytics:expr) => {
         pub mod $attr {
             use actix_web::{web, HttpRequest, HttpResponse, Resource};
             use log::debug;
@@ -21,11 +21,11 @@ macro_rules! make_setting_route {
             use meilisearch_lib::milli::update::Setting;
             use meilisearch_lib::{index::Settings, index_controller::Update, MeiliSearch};
 
-            use crate::analytics::Analytics;
-            use crate::extractors::authentication::{policies::*, GuardedData};
-            use crate::extractors::sequential_extractor::SeqHandler;
-            use crate::task::SummarizedTaskView;
-            use meilisearch_error::ResponseError;
+            use meilisearch_types::error::ResponseError;
+            use $crate::analytics::Analytics;
+            use $crate::extractors::authentication::{policies::*, GuardedData};
+            use $crate::extractors::sequential_extractor::SeqHandler;
+            use $crate::task::SummarizedTaskView;
 
             pub async fn delete(
                 meilisearch: GuardedData<ActionPolicy<{ actions::SETTINGS_UPDATE }>, MeiliSearch>,
@@ -100,18 +100,27 @@ macro_rules! make_setting_route {
             pub fn resources() -> Resource {
                 Resource::new($route)
                     .route(web::get().to(SeqHandler(get)))
-                    .route(web::post().to(SeqHandler(update)))
+                    .route(web::$update_verb().to(SeqHandler(update)))
                     .route(web::delete().to(SeqHandler(delete)))
             }
         }
     };
-    ($route:literal, $type:ty, $attr:ident, $camelcase_attr:literal) => {
-        make_setting_route!($route, $type, $attr, $camelcase_attr, _analytics, |_, _| {});
+    ($route:literal, $update_verb:ident, $type:ty, $attr:ident, $camelcase_attr:literal) => {
+        make_setting_route!(
+            $route,
+            $update_verb,
+            $type,
+            $attr,
+            $camelcase_attr,
+            _analytics,
+            |_, _| {}
+        );
     };
 }
 
 make_setting_route!(
     "/filterable-attributes",
+    put,
     std::collections::BTreeSet<String>,
     filterable_attributes,
     "filterableAttributes",
@@ -134,6 +143,7 @@ make_setting_route!(
 
 make_setting_route!(
     "/sortable-attributes",
+    put,
     std::collections::BTreeSet<String>,
     sortable_attributes,
     "sortableAttributes",
@@ -145,8 +155,8 @@ make_setting_route!(
             "SortableAttributes Updated".to_string(),
             json!({
                 "sortable_attributes": {
-                    "total": setting.as_ref().map(|sort| sort.len()).unwrap_or(0),
-                    "has_geo": setting.as_ref().map(|sort| sort.contains("_geo")).unwrap_or(false),
+                    "total": setting.as_ref().map(|sort| sort.len()),
+                    "has_geo": setting.as_ref().map(|sort| sort.contains("_geo")),
                 },
             }),
             Some(req),
@@ -156,13 +166,57 @@ make_setting_route!(
 
 make_setting_route!(
     "/displayed-attributes",
+    put,
     Vec<String>,
     displayed_attributes,
     "displayedAttributes"
 );
 
+make_setting_route!(
+    "/typo-tolerance",
+    patch,
+    meilisearch_lib::index::updates::TypoSettings,
+    typo_tolerance,
+    "typoTolerance",
+    analytics,
+    |setting: &Option<meilisearch_lib::index::updates::TypoSettings>, req: &HttpRequest| {
+        use serde_json::json;
+
+        analytics.publish(
+            "TypoTolerance Updated".to_string(),
+            json!({
+                "typo_tolerance": {
+                    "enabled": setting.as_ref().map(|s| !matches!(s.enabled, Setting::Set(false))),
+                    "disable_on_attributes": setting
+                        .as_ref()
+                        .and_then(|s| s.disable_on_attributes.as_ref().set().map(|m| !m.is_empty())),
+                    "disable_on_words": setting
+                        .as_ref()
+                        .and_then(|s| s.disable_on_words.as_ref().set().map(|m| !m.is_empty())),
+                    "min_word_size_for_one_typo": setting
+                        .as_ref()
+                        .and_then(|s| s.min_word_size_for_typos
+                            .as_ref()
+                            .set()
+                            .map(|s| s.one_typo.set()))
+                        .flatten(),
+                    "min_word_size_for_two_typos": setting
+                        .as_ref()
+                        .and_then(|s| s.min_word_size_for_typos
+                            .as_ref()
+                            .set()
+                            .map(|s| s.two_typos.set()))
+                        .flatten(),
+                },
+            }),
+            Some(req),
+        );
+    }
+);
+
 make_setting_route!(
     "/searchable-attributes",
+    put,
     Vec<String>,
     searchable_attributes,
     "searchableAttributes",
@@ -174,7 +228,7 @@ make_setting_route!(
             "SearchableAttributes Updated".to_string(),
             json!({
                 "searchable_attributes": {
-                    "total": setting.as_ref().map(|searchable| searchable.len()).unwrap_or(0),
+                    "total": setting.as_ref().map(|searchable| searchable.len()),
                 },
             }),
             Some(req),
@@ -184,6 +238,7 @@ make_setting_route!(
 
 make_setting_route!(
     "/stop-words",
+    put,
     std::collections::BTreeSet<String>,
     stop_words,
     "stopWords"
@@ -191,6 +246,7 @@ make_setting_route!(
 
 make_setting_route!(
     "/synonyms",
+    put,
     std::collections::BTreeMap<String, Vec<String>>,
     synonyms,
     "synonyms"
@@ -198,6 +254,7 @@ make_setting_route!(
 
 make_setting_route!(
     "/distinct-attribute",
+    put,
     String,
     distinct_attribute,
     "distinctAttribute"
@@ -205,6 +262,7 @@ make_setting_route!(
 
 make_setting_route!(
     "/ranking-rules",
+    put,
     Vec<String>,
     ranking_rules,
     "rankingRules",
@@ -224,13 +282,57 @@ make_setting_route!(
     }
 );
 
+make_setting_route!(
+    "/faceting",
+    patch,
+    meilisearch_lib::index::updates::FacetingSettings,
+    faceting,
+    "faceting",
+    analytics,
+    |setting: &Option<meilisearch_lib::index::updates::FacetingSettings>, req: &HttpRequest| {
+        use serde_json::json;
+
+        analytics.publish(
+            "Faceting Updated".to_string(),
+            json!({
+                "faceting": {
+                    "max_values_per_facet": setting.as_ref().and_then(|s| s.max_values_per_facet.set()),
+                },
+            }),
+            Some(req),
+        );
+    }
+);
+
+make_setting_route!(
+    "/pagination",
+    patch,
+    meilisearch_lib::index::updates::PaginationSettings,
+    pagination,
+    "pagination",
+    analytics,
+    |setting: &Option<meilisearch_lib::index::updates::PaginationSettings>, req: &HttpRequest| {
+        use serde_json::json;
+
+        analytics.publish(
+            "Pagination Updated".to_string(),
+            json!({
+                "pagination": {
+                    "limited_to": setting.as_ref().and_then(|s| s.limited_to.set()),
+                },
+            }),
+            Some(req),
+        );
+    }
+);
+
 macro_rules! generate_configure {
     ($($mod:ident),*) => {
         pub fn configure(cfg: &mut web::ServiceConfig) {
             use crate::extractors::sequential_extractor::SeqHandler;
             cfg.service(
                 web::resource("")
-                .route(web::post().to(SeqHandler(update_all)))
+                .route(web::patch().to(SeqHandler(update_all)))
                 .route(web::get().to(SeqHandler(get_all)))
                 .route(web::delete().to(SeqHandler(delete_all))))
                 $(.service($mod::resources()))*;
@@ -246,7 +348,8 @@ generate_configure!(
     distinct_attribute,
     stop_words,
     synonyms,
-    ranking_rules
+    ranking_rules,
+    typo_tolerance
 );
 
 pub async fn update_all(
@@ -265,15 +368,46 @@ pub async fn update_all(
                 "sort_position": settings.ranking_rules.as_ref().set().map(|sort| sort.iter().position(|s| s == "sort")),
             },
             "searchable_attributes": {
-                "total": settings.searchable_attributes.as_ref().set().map(|searchable| searchable.len()).unwrap_or(0),
+                "total": settings.searchable_attributes.as_ref().set().map(|searchable| searchable.len()),
             },
            "sortable_attributes": {
-                "total": settings.sortable_attributes.as_ref().set().map(|sort| sort.len()).unwrap_or(0),
-                "has_geo": settings.sortable_attributes.as_ref().set().map(|sort| sort.iter().any(|s| s == "_geo")).unwrap_or(false),
+                "total": settings.sortable_attributes.as_ref().set().map(|sort| sort.len()),
+                "has_geo": settings.sortable_attributes.as_ref().set().map(|sort| sort.iter().any(|s| s == "_geo")),
             },
            "filterable_attributes": {
-                "total": settings.filterable_attributes.as_ref().set().map(|filter| filter.len()).unwrap_or(0),
-                "has_geo": settings.filterable_attributes.as_ref().set().map(|filter| filter.iter().any(|s| s == "_geo")).unwrap_or(false),
+                "total": settings.filterable_attributes.as_ref().set().map(|filter| filter.len()),
+                "has_geo": settings.filterable_attributes.as_ref().set().map(|filter| filter.iter().any(|s| s == "_geo")),
+            },
+            "typo_tolerance": {
+                "enabled": settings.typo_tolerance
+                    .as_ref()
+                    .set()
+                    .and_then(|s| s.enabled.as_ref().set())
+                    .copied(),
+                "disable_on_attributes": settings.typo_tolerance
+                    .as_ref()
+                    .set()
+                    .and_then(|s| s.disable_on_attributes.as_ref().set().map(|m| !m.is_empty())),
+                "disable_on_words": settings.typo_tolerance
+                    .as_ref()
+                    .set()
+                    .and_then(|s| s.disable_on_words.as_ref().set().map(|m| !m.is_empty())),
+                "min_word_size_for_one_typo": settings.typo_tolerance
+                    .as_ref()
+                    .set()
+                    .and_then(|s| s.min_word_size_for_typos
+                        .as_ref()
+                        .set()
+                        .map(|s| s.one_typo.set()))
+                    .flatten(),
+                "min_word_size_for_two_typos": settings.typo_tolerance
+                    .as_ref()
+                    .set()
+                    .and_then(|s| s.min_word_size_for_typos
+                        .as_ref()
+                        .set()
+                        .map(|s| s.two_typos.set()))
+                    .flatten(),
             },
         }),
         Some(&req),

meilisearch-http/src/routes/indexes/tasks.rs

@@ -1,80 +0,0 @@
-use actix_web::{web, HttpRequest, HttpResponse};
-use log::debug;
-use meilisearch_error::ResponseError;
-use meilisearch_lib::MeiliSearch;
-use serde::{Deserialize, Serialize};
-use serde_json::json;
-use time::OffsetDateTime;
-
-use crate::analytics::Analytics;
-use crate::extractors::authentication::{policies::*, GuardedData};
-use crate::extractors::sequential_extractor::SeqHandler;
-use crate::task::{TaskListView, TaskView};
-
-pub fn configure(cfg: &mut web::ServiceConfig) {
-    cfg.service(web::resource("").route(web::get().to(SeqHandler(get_all_tasks_status))))
-        .service(web::resource("{task_id}").route(web::get().to(SeqHandler(get_task_status))));
-}
-
-#[derive(Debug, Serialize)]
-#[serde(rename_all = "camelCase")]
-pub struct UpdateIndexResponse {
-    name: String,
-    uid: String,
-    #[serde(serialize_with = "time::serde::rfc3339::serialize")]
-    created_at: OffsetDateTime,
-    #[serde(serialize_with = "time::serde::rfc3339::serialize")]
-    updated_at: OffsetDateTime,
-    #[serde(serialize_with = "time::serde::rfc3339::serialize")]
-    primary_key: OffsetDateTime,
-}
-
-#[derive(Deserialize)]
-pub struct UpdateParam {
-    index_uid: String,
-    task_id: u64,
-}
-
-pub async fn get_task_status(
-    meilisearch: GuardedData<ActionPolicy<{ actions::TASKS_GET }>, MeiliSearch>,
-    index_uid: web::Path<UpdateParam>,
-    req: HttpRequest,
-    analytics: web::Data<dyn Analytics>,
-) -> Result<HttpResponse, ResponseError> {
-    analytics.publish(
-        "Index Tasks Seen".to_string(),
-        json!({ "per_task_uid": true }),
-        Some(&req),
-    );
-
-    let UpdateParam { index_uid, task_id } = index_uid.into_inner();
-
-    let task: TaskView = meilisearch.get_index_task(index_uid, task_id).await?.into();
-
-    debug!("returns: {:?}", task);
-    Ok(HttpResponse::Ok().json(task))
-}
-
-pub async fn get_all_tasks_status(
-    meilisearch: GuardedData<ActionPolicy<{ actions::TASKS_GET }>, MeiliSearch>,
-    index_uid: web::Path<String>,
-    req: HttpRequest,
-    analytics: web::Data<dyn Analytics>,
-) -> Result<HttpResponse, ResponseError> {
-    analytics.publish(
-        "Index Tasks Seen".to_string(),
-        json!({ "per_task_uid": false }),
-        Some(&req),
-    );
-
-    let tasks: TaskListView = meilisearch
-        .list_index_task(index_uid.into_inner(), None, None)
-        .await?
-        .into_iter()
-        .map(TaskView::from)
-        .collect::<Vec<_>>()
-        .into();
-
-    debug!("returns: {:?}", tasks);
-    Ok(HttpResponse::Ok().json(tasks))
-}

meilisearch-http/src/routes/mod.rs

@@ -1,11 +1,13 @@
 use actix_web::{web, HttpResponse};
 use log::debug;
 use serde::{Deserialize, Serialize};
+
 use time::OffsetDateTime;
 
-use meilisearch_error::ResponseError;
 use meilisearch_lib::index::{Settings, Unchecked};
 use meilisearch_lib::MeiliSearch;
+use meilisearch_types::error::ResponseError;
+use meilisearch_types::star_or::StarOr;
 
 use crate::extractors::authentication::{policies::*, GuardedData};
 
@@ -24,6 +26,101 @@ pub fn configure(cfg: &mut web::ServiceConfig) {
         .service(web::scope("/indexes").configure(indexes::configure));
 }
 
+/// Extracts the raw values from the `StarOr` types and
+/// return None if a `StarOr::Star` is encountered.
+pub fn fold_star_or<T, O>(content: impl IntoIterator<Item = StarOr<T>>) -> Option<O>
+where
+    O: FromIterator<T>,
+{
+    content
+        .into_iter()
+        .map(|value| match value {
+            StarOr::Star => None,
+            StarOr::Other(val) => Some(val),
+        })
+        .collect()
+}
+
+const PAGINATION_DEFAULT_LIMIT: fn() -> usize = || 20;
+
+#[derive(Debug, Clone, Copy, Deserialize)]
+#[serde(rename_all = "camelCase", deny_unknown_fields)]
+pub struct Pagination {
+    #[serde(default)]
+    pub offset: usize,
+    #[serde(default = "PAGINATION_DEFAULT_LIMIT")]
+    pub limit: usize,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct PaginationView<T> {
+    pub results: Vec<T>,
+    pub offset: usize,
+    pub limit: usize,
+    pub total: usize,
+}
+
+impl Pagination {
+    /// Given the full data to paginate, returns the selected section.
+    pub fn auto_paginate_sized<T>(
+        self,
+        content: impl IntoIterator<Item = T> + ExactSizeIterator,
+    ) -> PaginationView<T>
+    where
+        T: Serialize,
+    {
+        let total = content.len();
+        let content: Vec<_> = content
+            .into_iter()
+            .skip(self.offset)
+            .take(self.limit)
+            .collect();
+        self.format_with(total, content)
+    }
+
+    /// Given an iterator and the total number of elements, returns the selected section.
+    pub fn auto_paginate_unsized<T>(
+        self,
+        total: usize,
+        content: impl IntoIterator<Item = T>,
+    ) -> PaginationView<T>
+    where
+        T: Serialize,
+    {
+        let content: Vec<_> = content
+            .into_iter()
+            .skip(self.offset)
+            .take(self.limit)
+            .collect();
+        self.format_with(total, content)
+    }
+
+    /// Given the data already paginated + the total number of elements, it stores
+    /// everything in a [PaginationResult].
+    pub fn format_with<T>(self, total: usize, results: Vec<T>) -> PaginationView<T>
+    where
+        T: Serialize,
+    {
+        PaginationView {
+            results,
+            offset: self.offset,
+            limit: self.limit,
+            total,
+        }
+    }
+}
+
+impl<T> PaginationView<T> {
+    pub fn new(offset: usize, limit: usize, total: usize, results: Vec<T>) -> Self {
+        Self {
+            offset,
+            limit,
+            results,
+            total,
+        }
+    }
+}
+
 #[derive(Debug, Clone, Serialize, Deserialize)]
 #[allow(clippy::large_enum_variant)]
 #[serde(tag = "name")]

meilisearch-http/src/routes/tasks.rs

@@ -1,22 +1,67 @@
 use actix_web::{web, HttpRequest, HttpResponse};
-use meilisearch_error::ResponseError;
-use meilisearch_lib::tasks::task::TaskId;
+use meilisearch_lib::tasks::task::{TaskContent, TaskEvent, TaskId};
 use meilisearch_lib::tasks::TaskFilter;
 use meilisearch_lib::MeiliSearch;
+use meilisearch_types::error::ResponseError;
+use meilisearch_types::index_uid::IndexUid;
+use meilisearch_types::star_or::StarOr;
+use serde::Deserialize;
+use serde_cs::vec::CS;
 use serde_json::json;
 
 use crate::analytics::Analytics;
 use crate::extractors::authentication::{policies::*, GuardedData};
 use crate::extractors::sequential_extractor::SeqHandler;
-use crate::task::{TaskListView, TaskView};
+use crate::task::{TaskListView, TaskStatus, TaskType, TaskView};
+
+use super::fold_star_or;
+
+const DEFAULT_LIMIT: fn() -> usize = || 20;
 
 pub fn configure(cfg: &mut web::ServiceConfig) {
     cfg.service(web::resource("").route(web::get().to(SeqHandler(get_tasks))))
         .service(web::resource("/{task_id}").route(web::get().to(SeqHandler(get_task))));
 }
 
+#[derive(Deserialize, Debug)]
+#[serde(rename_all = "camelCase", deny_unknown_fields)]
+pub struct TaskFilterQuery {
+    #[serde(rename = "type")]
+    type_: Option<CS<StarOr<TaskType>>>,
+    status: Option<CS<StarOr<TaskStatus>>>,
+    index_uid: Option<CS<StarOr<IndexUid>>>,
+    #[serde(default = "DEFAULT_LIMIT")]
+    limit: usize,
+    from: Option<TaskId>,
+}
+
+#[rustfmt::skip]
+fn task_type_matches_content(type_: &TaskType, content: &TaskContent) -> bool {
+    matches!((type_, content),
+          (TaskType::IndexCreation, TaskContent::IndexCreation { .. })
+        | (TaskType::IndexUpdate, TaskContent::IndexUpdate { .. })
+        | (TaskType::IndexDeletion, TaskContent::IndexDeletion { .. })
+        | (TaskType::DocumentAdditionOrUpdate, TaskContent::DocumentAddition { .. })
+        | (TaskType::DocumentDeletion, TaskContent::DocumentDeletion{ .. })
+        | (TaskType::SettingsUpdate, TaskContent::SettingsUpdate { .. })
+    )
+}
+
+#[rustfmt::skip]
+fn task_status_matches_events(status: &TaskStatus, events: &[TaskEvent]) -> bool {
+    events.last().map_or(false, |event| {
+        matches!((status, event),
+              (TaskStatus::Enqueued, TaskEvent::Created(_))
+            | (TaskStatus::Processing, TaskEvent::Processing(_) | TaskEvent::Batched { .. })
+            | (TaskStatus::Succeeded, TaskEvent::Succeeded { .. })
+            | (TaskStatus::Failed, TaskEvent::Failed { .. }),
+        )
+    })
+}
+
 async fn get_tasks(
     meilisearch: GuardedData<ActionPolicy<{ actions::TASKS_GET }>, MeiliSearch>,
+    params: web::Query<TaskFilterQuery>,
     req: HttpRequest,
     analytics: web::Data<dyn Analytics>,
 ) -> Result<HttpResponse, ResponseError> {
@@ -26,24 +71,98 @@ async fn get_tasks(
         Some(&req),
     );
 
+    let TaskFilterQuery {
+        type_,
+        status,
+        index_uid,
+        limit,
+        from,
+    } = params.into_inner();
+
     let search_rules = &meilisearch.filters().search_rules;
-    let filters = if search_rules.is_index_authorized("*") {
-        None
-    } else {
-        let mut filters = TaskFilter::default();
-        for (index, _policy) in search_rules.clone() {
-            filters.filter_index(index);
+
+    // We first transform a potential indexUid=* into a "not specified indexUid filter"
+    // for every one of the filters: type, status, and indexUid.
+    let type_: Option<Vec<_>> = type_.and_then(fold_star_or);
+    let status: Option<Vec<_>> = status.and_then(fold_star_or);
+    let index_uid: Option<Vec<_>> = index_uid.and_then(fold_star_or);
+
+    // Then we filter on potential indexes and make sure that the search filter
+    // restrictions are also applied.
+    let indexes_filters = match index_uid {
+        Some(indexes) => {
+            let mut filters = TaskFilter::default();
+            for name in indexes {
+                if search_rules.is_index_authorized(&name) {
+                    filters.filter_index(name.to_string());
+                }
+            }
+            Some(filters)
+        }
+        None => {
+            if search_rules.is_index_authorized("*") {
+                None
+            } else {
+                let mut filters = TaskFilter::default();
+                for (index, _policy) in search_rules.clone() {
+                    filters.filter_index(index);
+                }
+                Some(filters)
+            }
         }
-        Some(filters)
     };
 
-    let tasks: TaskListView = meilisearch
-        .list_tasks(filters, None, None)
+    // Then we complete the task filter with other potential status and types filters.
+    let filters = if type_.is_some() || status.is_some() {
+        let mut filters = indexes_filters.unwrap_or_default();
+        filters.filter_fn(move |task| {
+            let matches_type = match &type_ {
+                Some(types) => types
+                    .iter()
+                    .any(|t| task_type_matches_content(t, &task.content)),
+                None => true,
+            };
+
+            let matches_status = match &status {
+                Some(statuses) => statuses
+                    .iter()
+                    .any(|t| task_status_matches_events(t, &task.events)),
+                None => true,
+            };
+
+            matches_type && matches_status
+        });
+        Some(filters)
+    } else {
+        indexes_filters
+    };
+
+    // We +1 just to know if there is more after this "page" or not.
+    let limit = limit.saturating_add(1);
+
+    let mut tasks_results: Vec<_> = meilisearch
+        .list_tasks(filters, Some(limit), from)
         .await?
         .into_iter()
         .map(TaskView::from)
-        .collect::<Vec<_>>()
-        .into();
+        .collect();
+
+    // If we were able to fetch the number +1 tasks we asked
+    // it means that there is more to come.
+    let next = if tasks_results.len() == limit {
+        tasks_results.pop().map(|t| t.uid)
+    } else {
+        None
+    };
+
+    let from = tasks_results.first().map(|t| t.uid);
+
+    let tasks = TaskListView {
+        results: tasks_results,
+        limit: limit.saturating_sub(1),
+        from,
+        next,
+    };
 
     Ok(HttpResponse::Ok().json(tasks))
 }

meilisearch-http/src/task.rs

@@ -1,62 +1,137 @@
-use std::fmt::Write;
+use std::error::Error;
+use std::fmt::{self, Write};
+use std::str::FromStr;
 use std::write;
 
-use meilisearch_error::ResponseError;
 use meilisearch_lib::index::{Settings, Unchecked};
-use meilisearch_lib::milli::update::IndexDocumentsMethod;
 use meilisearch_lib::tasks::batch::BatchId;
 use meilisearch_lib::tasks::task::{
     DocumentDeletion, Task, TaskContent, TaskEvent, TaskId, TaskResult,
 };
-use serde::{Serialize, Serializer};
+use meilisearch_types::error::ResponseError;
+use serde::{Deserialize, Serialize, Serializer};
 use time::{Duration, OffsetDateTime};
 
 use crate::AUTOBATCHING_ENABLED;
 
-#[derive(Debug, Serialize)]
+#[derive(Debug, Serialize, Deserialize)]
 #[serde(rename_all = "camelCase")]
-enum TaskType {
+pub enum TaskType {
     IndexCreation,
     IndexUpdate,
     IndexDeletion,
-    DocumentAddition,
-    DocumentPartial,
+    DocumentAdditionOrUpdate,
     DocumentDeletion,
     SettingsUpdate,
-    ClearAll,
+    DumpCreation,
 }
 
 impl From<TaskContent> for TaskType {
     fn from(other: TaskContent) -> Self {
         match other {
-            TaskContent::DocumentAddition {
-                merge_strategy: IndexDocumentsMethod::ReplaceDocuments,
-                ..
-            } => TaskType::DocumentAddition,
-            TaskContent::DocumentAddition {
-                merge_strategy: IndexDocumentsMethod::UpdateDocuments,
-                ..
-            } => TaskType::DocumentPartial,
-            TaskContent::DocumentDeletion(DocumentDeletion::Clear) => TaskType::ClearAll,
-            TaskContent::DocumentDeletion(DocumentDeletion::Ids(_)) => TaskType::DocumentDeletion,
-            TaskContent::SettingsUpdate { .. } => TaskType::SettingsUpdate,
-            TaskContent::IndexDeletion => TaskType::IndexDeletion,
             TaskContent::IndexCreation { .. } => TaskType::IndexCreation,
             TaskContent::IndexUpdate { .. } => TaskType::IndexUpdate,
-            _ => unreachable!("unexpected task type"),
+            TaskContent::IndexDeletion { .. } => TaskType::IndexDeletion,
+            TaskContent::DocumentAddition { .. } => TaskType::DocumentAdditionOrUpdate,
+            TaskContent::DocumentDeletion { .. } => TaskType::DocumentDeletion,
+            TaskContent::SettingsUpdate { .. } => TaskType::SettingsUpdate,
+            TaskContent::Dump { .. } => TaskType::DumpCreation,
         }
     }
 }
 
-#[derive(Debug, Serialize)]
+#[derive(Debug)]
+pub struct TaskTypeError {
+    invalid_type: String,
+}
+
+impl fmt::Display for TaskTypeError {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        write!(
+            f,
+            "invalid task type `{}`, expecting one of: \
+            indexCreation, indexUpdate, indexDeletion, documentAdditionOrUpdate, \
+            documentDeletion, settingsUpdate, dumpCreation",
+            self.invalid_type
+        )
+    }
+}
+
+impl Error for TaskTypeError {}
+
+impl FromStr for TaskType {
+    type Err = TaskTypeError;
+
+    fn from_str(type_: &str) -> Result<Self, TaskTypeError> {
+        if type_.eq_ignore_ascii_case("indexCreation") {
+            Ok(TaskType::IndexCreation)
+        } else if type_.eq_ignore_ascii_case("indexUpdate") {
+            Ok(TaskType::IndexUpdate)
+        } else if type_.eq_ignore_ascii_case("indexDeletion") {
+            Ok(TaskType::IndexDeletion)
+        } else if type_.eq_ignore_ascii_case("documentAdditionOrUpdate") {
+            Ok(TaskType::DocumentAdditionOrUpdate)
+        } else if type_.eq_ignore_ascii_case("documentDeletion") {
+            Ok(TaskType::DocumentDeletion)
+        } else if type_.eq_ignore_ascii_case("settingsUpdate") {
+            Ok(TaskType::SettingsUpdate)
+        } else if type_.eq_ignore_ascii_case("dumpCreation") {
+            Ok(TaskType::DumpCreation)
+        } else {
+            Err(TaskTypeError {
+                invalid_type: type_.to_string(),
+            })
+        }
+    }
+}
+
+#[derive(Debug, Serialize, Deserialize)]
 #[serde(rename_all = "camelCase")]
-enum TaskStatus {
+pub enum TaskStatus {
     Enqueued,
     Processing,
     Succeeded,
     Failed,
 }
 
+#[derive(Debug)]
+pub struct TaskStatusError {
+    invalid_status: String,
+}
+
+impl fmt::Display for TaskStatusError {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        write!(
+            f,
+            "invalid task status `{}`, expecting one of: \
+            enqueued, processing, succeeded, or failed",
+            self.invalid_status,
+        )
+    }
+}
+
+impl Error for TaskStatusError {}
+
+impl FromStr for TaskStatus {
+    type Err = TaskStatusError;
+
+    fn from_str(status: &str) -> Result<Self, TaskStatusError> {
+        if status.eq_ignore_ascii_case("enqueued") {
+            Ok(TaskStatus::Enqueued)
+        } else if status.eq_ignore_ascii_case("processing") {
+            Ok(TaskStatus::Processing)
+        } else if status.eq_ignore_ascii_case("succeeded") {
+            Ok(TaskStatus::Succeeded)
+        } else if status.eq_ignore_ascii_case("failed") {
+            Ok(TaskStatus::Failed)
+        } else {
+            Err(TaskStatusError {
+                invalid_status: status.to_string(),
+            })
+        }
+    }
+}
+
 #[derive(Debug, Serialize)]
 #[serde(untagged)]
 #[allow(clippy::large_enum_variant)]
@@ -80,6 +155,8 @@ enum TaskDetails {
     },
     #[serde(rename_all = "camelCase")]
     ClearAll { deleted_documents: Option<u64> },
+    #[serde(rename_all = "camelCase")]
+    Dump { dump_uid: String },
 }
 
 /// Serialize a `time::Duration` as a best effort ISO 8601 while waiting for
@@ -136,8 +213,8 @@ fn serialize_duration<S: Serializer>(
 #[derive(Debug, Serialize)]
 #[serde(rename_all = "camelCase")]
 pub struct TaskView {
-    uid: TaskId,
-    index_uid: String,
+    pub uid: TaskId,
+    index_uid: Option<String>,
     status: TaskStatus,
     #[serde(rename = "type")]
     task_type: TaskType,
@@ -159,46 +236,44 @@ pub struct TaskView {
 
 impl From<Task> for TaskView {
     fn from(task: Task) -> Self {
+        let index_uid = task.index_uid().map(String::from);
         let Task {
             id,
-            index_uid,
             content,
             events,
         } = task;
 
         let (task_type, mut details) = match content {
             TaskContent::DocumentAddition {
-                merge_strategy,
-                documents_count,
-                ..
+                documents_count, ..
             } => {
                 let details = TaskDetails::DocumentAddition {
                     received_documents: documents_count,
                     indexed_documents: None,
                 };
 
-                let task_type = match merge_strategy {
-                    IndexDocumentsMethod::UpdateDocuments => TaskType::DocumentPartial,
-                    IndexDocumentsMethod::ReplaceDocuments => TaskType::DocumentAddition,
-                    _ => unreachable!("Unexpected document merge strategy."),
-                };
-
-                (task_type, Some(details))
+                (TaskType::DocumentAdditionOrUpdate, Some(details))
             }
-            TaskContent::DocumentDeletion(DocumentDeletion::Ids(ids)) => (
+            TaskContent::DocumentDeletion {
+                deletion: DocumentDeletion::Ids(ids),
+                ..
+            } => (
                 TaskType::DocumentDeletion,
                 Some(TaskDetails::DocumentDeletion {
                     received_document_ids: ids.len(),
                     deleted_documents: None,
                 }),
             ),
-            TaskContent::DocumentDeletion(DocumentDeletion::Clear) => (
-                TaskType::ClearAll,
+            TaskContent::DocumentDeletion {
+                deletion: DocumentDeletion::Clear,
+                ..
+            } => (
+                TaskType::DocumentDeletion,
                 Some(TaskDetails::ClearAll {
                     deleted_documents: None,
                 }),
             ),
-            TaskContent::IndexDeletion => (
+            TaskContent::IndexDeletion { .. } => (
                 TaskType::IndexDeletion,
                 Some(TaskDetails::ClearAll {
                     deleted_documents: None,
@@ -208,14 +283,18 @@ impl From<Task> for TaskView {
                 TaskType::SettingsUpdate,
                 Some(TaskDetails::Settings { settings }),
             ),
-            TaskContent::IndexCreation { primary_key } => (
+            TaskContent::IndexCreation { primary_key, .. } => (
                 TaskType::IndexCreation,
                 Some(TaskDetails::IndexInfo { primary_key }),
             ),
-            TaskContent::IndexUpdate { primary_key } => (
+            TaskContent::IndexUpdate { primary_key, .. } => (
                 TaskType::IndexUpdate,
                 Some(TaskDetails::IndexInfo { primary_key }),
             ),
+            TaskContent::Dump { uid } => (
+                TaskType::DumpCreation,
+                Some(TaskDetails::Dump { dump_uid: uid }),
+            ),
         };
 
         // An event always has at least one event: "Created"
@@ -223,7 +302,7 @@ impl From<Task> for TaskView {
             TaskEvent::Created(_) => (TaskStatus::Enqueued, None, None),
             TaskEvent::Batched { .. } => (TaskStatus::Enqueued, None, None),
             TaskEvent::Processing(_) => (TaskStatus::Processing, None, None),
-            TaskEvent::Succeded { timestamp, result } => {
+            TaskEvent::Succeeded { timestamp, result } => {
                 match (result, &mut details) {
                     (
                         TaskResult::DocumentAddition {
@@ -313,7 +392,7 @@ impl From<Task> for TaskView {
 
         Self {
             uid: id,
-            index_uid: index_uid.into_inner(),
+            index_uid,
             status,
             task_type,
             details,
@@ -329,20 +408,17 @@ impl From<Task> for TaskView {
 
 #[derive(Debug, Serialize)]
 pub struct TaskListView {
-    results: Vec<TaskView>,
-}
-
-impl From<Vec<TaskView>> for TaskListView {
-    fn from(results: Vec<TaskView>) -> Self {
-        Self { results }
-    }
+    pub results: Vec<TaskView>,
+    pub limit: usize,
+    pub from: Option<TaskId>,
+    pub next: Option<TaskId>,
 }
 
 #[derive(Debug, Serialize)]
 #[serde(rename_all = "camelCase")]
 pub struct SummarizedTaskView {
-    uid: TaskId,
-    index_uid: String,
+    task_uid: TaskId,
+    index_uid: Option<String>,
     status: TaskStatus,
     #[serde(rename = "type")]
     task_type: TaskType,
@@ -364,8 +440,8 @@ impl From<Task> for SummarizedTaskView {
         };
 
         Self {
-            uid: other.id,
-            index_uid: other.index_uid.to_string(),
+            task_uid: other.id,
+            index_uid: other.index_uid().map(String::from),
             status: TaskStatus::Enqueued,
             task_type: other.content.into(),
             enqueued_at,

meilisearch-http/tests/auth/authorization.rs

@@ -11,14 +11,14 @@ pub static AUTHORIZATIONS: Lazy<HashMap<(&'static str, &'static str), HashSet<&'
         hashmap! {
             ("POST",    "/indexes/products/search") =>                         hashset!{"search", "*"},
             ("GET",     "/indexes/products/search") =>                         hashset!{"search", "*"},
-            ("POST",    "/indexes/products/documents") =>                      hashset!{"documents.add", "*"},
-            ("GET",     "/indexes/products/documents") =>                      hashset!{"documents.get", "*"},
-            ("GET",     "/indexes/products/documents/0") =>                    hashset!{"documents.get", "*"},
-            ("DELETE",  "/indexes/products/documents/0") =>                    hashset!{"documents.delete", "*"},
+            ("POST",    "/indexes/products/documents") =>                      hashset!{"documents.add", "documents.*", "*"},
+            ("GET",     "/indexes/products/documents") =>                      hashset!{"documents.get", "documents.*", "*"},
+            ("GET",     "/indexes/products/documents/0") =>                    hashset!{"documents.get", "documents.*", "*"},
+            ("DELETE",  "/indexes/products/documents/0") =>                    hashset!{"documents.delete", "documents.*", "*"},
             ("GET",     "/tasks") =>                                           hashset!{"tasks.get", "*"},
-            ("GET",     "/indexes/products/tasks") =>                          hashset!{"tasks.get", "*"},
-            ("GET",     "/indexes/products/tasks/0") =>                        hashset!{"tasks.get", "*"},
-            ("PUT",     "/indexes/products/") =>                               hashset!{"indexes.update", "*"},
+            ("GET",     "/tasks?indexUid=products") =>                         hashset!{"tasks.get", "*"},
+            ("GET",     "/tasks/0") =>                                         hashset!{"tasks.get", "*"},
+            ("PATCH",   "/indexes/products/") =>                               hashset!{"indexes.update", "*"},
             ("GET",     "/indexes/products/") =>                               hashset!{"indexes.get", "*"},
             ("DELETE",  "/indexes/products/") =>                               hashset!{"indexes.delete", "*"},
             ("POST",    "/indexes") =>                                         hashset!{"indexes.create", "*"},
@@ -33,20 +33,25 @@ pub static AUTHORIZATIONS: Lazy<HashMap<(&'static str, &'static str), HashSet<&'
             ("GET",     "/indexes/products/settings/stop-words") =>            hashset!{"settings.get", "*"},
             ("GET",     "/indexes/products/settings/synonyms") =>              hashset!{"settings.get", "*"},
             ("DELETE",  "/indexes/products/settings") =>                       hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings") =>                       hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings/displayed-attributes") =>  hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings/distinct-attribute") =>    hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings/filterable-attributes") => hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings/ranking-rules") =>         hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings/searchable-attributes") => hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings/sortable-attributes") =>   hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings/stop-words") =>            hashset!{"settings.update", "*"},
-            ("POST",    "/indexes/products/settings/synonyms") =>              hashset!{"settings.update", "*"},
+            ("PATCH",   "/indexes/products/settings") =>                       hashset!{"settings.update", "*"},
+            ("PATCH",   "/indexes/products/settings/typo-tolerance") =>        hashset!{"settings.update", "*"},
+            ("PUT",     "/indexes/products/settings/displayed-attributes") =>  hashset!{"settings.update", "*"},
+            ("PUT",     "/indexes/products/settings/distinct-attribute") =>    hashset!{"settings.update", "*"},
+            ("PUT",     "/indexes/products/settings/filterable-attributes") => hashset!{"settings.update", "*"},
+            ("PUT",     "/indexes/products/settings/ranking-rules") =>         hashset!{"settings.update", "*"},
+            ("PUT",     "/indexes/products/settings/searchable-attributes") => hashset!{"settings.update", "*"},
+            ("PUT",     "/indexes/products/settings/sortable-attributes") =>   hashset!{"settings.update", "*"},
+            ("PUT",     "/indexes/products/settings/stop-words") =>            hashset!{"settings.update", "*"},
+            ("PUT",     "/indexes/products/settings/synonyms") =>              hashset!{"settings.update", "*"},
             ("GET",     "/indexes/products/stats") =>                          hashset!{"stats.get", "*"},
             ("GET",     "/stats") =>                                           hashset!{"stats.get", "*"},
             ("POST",    "/dumps") =>                                           hashset!{"dumps.create", "*"},
-            ("GET",     "/dumps/0/status") =>                                  hashset!{"dumps.get", "*"},
             ("GET",     "/version") =>                                         hashset!{"version", "*"},
+            ("PATCH",   "/keys/mykey/") =>                                     hashset!{"keys.update", "*"},
+            ("GET",     "/keys/mykey/") =>                                     hashset!{"keys.get", "*"},
+            ("DELETE",  "/keys/mykey/") =>                                     hashset!{"keys.delete", "*"},
+            ("POST",    "/keys") =>                                            hashset!{"keys.create", "*"},
+            ("GET",     "/keys") =>                                            hashset!{"keys.get", "*"},
         }
     });
 
@@ -81,7 +86,7 @@ async fn error_access_expired_key() {
     });
 
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     let key = response["key"].as_str().unwrap();
@@ -93,8 +98,14 @@ async fn error_access_expired_key() {
     for (method, route) in AUTHORIZATIONS.keys() {
         let (response, code) = server.dummy_request(method, route).await;
 
-        assert_eq!(response, INVALID_RESPONSE.clone());
-        assert_eq!(code, 403);
+        assert_eq!(
+            response,
+            INVALID_RESPONSE.clone(),
+            "on route: {:?} - {:?}",
+            method,
+            route
+        );
+        assert_eq!(403, code, "{:?}", &response);
     }
 }
 
@@ -111,7 +122,7 @@ async fn error_access_unauthorized_index() {
     });
 
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     let key = response["key"].as_str().unwrap();
@@ -124,8 +135,14 @@ async fn error_access_unauthorized_index() {
     {
         let (response, code) = server.dummy_request(method, route).await;
 
-        assert_eq!(response, INVALID_RESPONSE.clone());
-        assert_eq!(code, 403);
+        assert_eq!(
+            response,
+            INVALID_RESPONSE.clone(),
+            "on route: {:?} - {:?}",
+            method,
+            route
+        );
+        assert_eq!(403, code, "{:?}", &response);
     }
 }
 
@@ -133,36 +150,54 @@ async fn error_access_unauthorized_index() {
 #[cfg_attr(target_os = "windows", ignore)]
 async fn error_access_unauthorized_action() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
-
-    let content = json!({
-        "indexes": ["products"],
-        "actions": [],
-        "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
-    });
-
-    let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
-    assert!(response["key"].is_string());
-
-    let key = response["key"].as_str().unwrap();
-    server.use_api_key(&key);
 
     for ((method, route), action) in AUTHORIZATIONS.iter() {
+        // create a new API key letting only the needed action.
         server.use_api_key("MASTER_KEY");
 
-        // Patch API key letting all rights but the needed one.
         let content = json!({
+            "indexes": ["products"],
             "actions": ALL_ACTIONS.difference(action).collect::<Vec<_>>(),
+            "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
         });
-        let (_, code) = server.patch_api_key(&key, content).await;
-        assert_eq!(code, 200);
 
+        let (response, code) = server.add_api_key(content).await;
+        assert_eq!(201, code, "{:?}", &response);
+        assert!(response["key"].is_string());
+
+        let key = response["key"].as_str().unwrap();
         server.use_api_key(&key);
         let (response, code) = server.dummy_request(method, route).await;
 
-        assert_eq!(response, INVALID_RESPONSE.clone());
-        assert_eq!(code, 403);
+        assert_eq!(
+            response,
+            INVALID_RESPONSE.clone(),
+            "on route: {:?} - {:?}",
+            method,
+            route
+        );
+        assert_eq!(403, code, "{:?}", &response);
+    }
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn access_authorized_master_key() {
+    let mut server = Server::new_auth().await;
+    server.use_api_key("MASTER_KEY");
+
+    // master key must have access to all routes.
+    for ((method, route), _) in AUTHORIZATIONS.iter() {
+        let (response, code) = server.dummy_request(method, route).await;
+
+        assert_ne!(
+            response,
+            INVALID_RESPONSE.clone(),
+            "on route: {:?} - {:?}",
+            method,
+            route
+        );
+        assert_ne!(code, 403);
     }
 }
 
@@ -170,36 +205,34 @@ async fn error_access_unauthorized_action() {
 #[cfg_attr(target_os = "windows", ignore)]
 async fn access_authorized_restricted_index() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
-
-    let content = json!({
-        "indexes": ["products"],
-        "actions": [],
-        "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
-    });
-
-    let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
-    assert!(response["key"].is_string());
-
-    let key = response["key"].as_str().unwrap();
-    server.use_api_key(&key);
-
     for ((method, route), actions) in AUTHORIZATIONS.iter() {
         for action in actions {
-            // Patch API key letting only the needed action.
+            // create a new API key letting only the needed action.
+            server.use_api_key("MASTER_KEY");
+
             let content = json!({
+                "indexes": ["products"],
                 "actions": [action],
+                "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
             });
 
-            server.use_api_key("MASTER_KEY");
-            let (_, code) = server.patch_api_key(&key, content).await;
-            assert_eq!(code, 200);
+            let (response, code) = server.add_api_key(content).await;
+            assert_eq!(201, code, "{:?}", &response);
+            assert!(response["key"].is_string());
 
+            let key = response["key"].as_str().unwrap();
             server.use_api_key(&key);
+
             let (response, code) = server.dummy_request(method, route).await;
 
-            assert_ne!(response, INVALID_RESPONSE.clone());
+            assert_ne!(
+                response,
+                INVALID_RESPONSE.clone(),
+                "on route: {:?} - {:?} with action: {:?}",
+                method,
+                route,
+                action
+            );
             assert_ne!(code, 403);
         }
     }
@@ -209,36 +242,35 @@ async fn access_authorized_restricted_index() {
 #[cfg_attr(target_os = "windows", ignore)]
 async fn access_authorized_no_index_restriction() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
-
-    let content = json!({
-        "indexes": ["*"],
-        "actions": [],
-        "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
-    });
-
-    let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
-    assert!(response["key"].is_string());
-
-    let key = response["key"].as_str().unwrap();
-    server.use_api_key(&key);
 
     for ((method, route), actions) in AUTHORIZATIONS.iter() {
         for action in actions {
+            // create a new API key letting only the needed action.
             server.use_api_key("MASTER_KEY");
 
-            // Patch API key letting only the needed action.
             let content = json!({
+                "indexes": ["*"],
                 "actions": [action],
+                "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
             });
-            let (_, code) = server.patch_api_key(&key, content).await;
-            assert_eq!(code, 200);
 
+            let (response, code) = server.add_api_key(content).await;
+            assert_eq!(201, code, "{:?}", &response);
+            assert!(response["key"].is_string());
+
+            let key = response["key"].as_str().unwrap();
             server.use_api_key(&key);
+
             let (response, code) = server.dummy_request(method, route).await;
 
-            assert_ne!(response, INVALID_RESPONSE.clone());
+            assert_ne!(
+                response,
+                INVALID_RESPONSE.clone(),
+                "on route: {:?} - {:?} with action: {:?}",
+                method,
+                route,
+                action
+            );
             assert_ne!(code, 403);
         }
     }
@@ -248,16 +280,16 @@ async fn access_authorized_no_index_restriction() {
 #[cfg_attr(target_os = "windows", ignore)]
 async fn access_authorized_stats_restricted_index() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
+    server.use_admin_key("MASTER_KEY").await;
 
     // create index `test`
     let index = server.index("test");
-    let (_, code) = index.create(Some("id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     // create index `products`
     let index = server.index("products");
-    let (_, code) = index.create(Some("product_id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("product_id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     index.wait_task(0).await;
 
     // create key with access on `products` index only.
@@ -267,7 +299,7 @@ async fn access_authorized_stats_restricted_index() {
         "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
     });
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     // use created key.
@@ -275,7 +307,7 @@ async fn access_authorized_stats_restricted_index() {
     server.use_api_key(&key);
 
     let (response, code) = server.stats().await;
-    assert_eq!(code, 200);
+    assert_eq!(200, code, "{:?}", &response);
 
     // key should have access on `products` index.
     assert!(response["indexes"].get("products").is_some());
@@ -288,16 +320,16 @@ async fn access_authorized_stats_restricted_index() {
 #[cfg_attr(target_os = "windows", ignore)]
 async fn access_authorized_stats_no_index_restriction() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
+    server.use_admin_key("MASTER_KEY").await;
 
     // create index `test`
     let index = server.index("test");
-    let (_, code) = index.create(Some("id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     // create index `products`
     let index = server.index("products");
-    let (_, code) = index.create(Some("product_id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("product_id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     index.wait_task(0).await;
 
     // create key with access on all indexes.
@@ -307,7 +339,7 @@ async fn access_authorized_stats_no_index_restriction() {
         "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
     });
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     // use created key.
@@ -315,7 +347,7 @@ async fn access_authorized_stats_no_index_restriction() {
     server.use_api_key(&key);
 
     let (response, code) = server.stats().await;
-    assert_eq!(code, 200);
+    assert_eq!(200, code, "{:?}", &response);
 
     // key should have access on `products` index.
     assert!(response["indexes"].get("products").is_some());
@@ -328,16 +360,16 @@ async fn access_authorized_stats_no_index_restriction() {
 #[cfg_attr(target_os = "windows", ignore)]
 async fn list_authorized_indexes_restricted_index() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
+    server.use_admin_key("MASTER_KEY").await;
 
     // create index `test`
     let index = server.index("test");
-    let (_, code) = index.create(Some("id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     // create index `products`
     let index = server.index("products");
-    let (_, code) = index.create(Some("product_id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("product_id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     index.wait_task(0).await;
 
     // create key with access on `products` index only.
@@ -347,17 +379,17 @@ async fn list_authorized_indexes_restricted_index() {
         "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
     });
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     // use created key.
     let key = response["key"].as_str().unwrap();
     server.use_api_key(&key);
 
-    let (response, code) = server.list_indexes().await;
-    assert_eq!(code, 200);
+    let (response, code) = server.list_indexes(None, None).await;
+    assert_eq!(200, code, "{:?}", &response);
 
-    let response = response.as_array().unwrap();
+    let response = response["results"].as_array().unwrap();
     // key should have access on `products` index.
     assert!(response.iter().any(|index| index["uid"] == "products"));
 
@@ -369,16 +401,16 @@ async fn list_authorized_indexes_restricted_index() {
 #[cfg_attr(target_os = "windows", ignore)]
 async fn list_authorized_indexes_no_index_restriction() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
+    server.use_admin_key("MASTER_KEY").await;
 
     // create index `test`
     let index = server.index("test");
-    let (_, code) = index.create(Some("id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     // create index `products`
     let index = server.index("products");
-    let (_, code) = index.create(Some("product_id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("product_id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     index.wait_task(0).await;
 
     // create key with access on all indexes.
@@ -388,17 +420,17 @@ async fn list_authorized_indexes_no_index_restriction() {
         "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
     });
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     // use created key.
     let key = response["key"].as_str().unwrap();
     server.use_api_key(&key);
 
-    let (response, code) = server.list_indexes().await;
-    assert_eq!(code, 200);
+    let (response, code) = server.list_indexes(None, None).await;
+    assert_eq!(200, code, "{:?}", &response);
 
-    let response = response.as_array().unwrap();
+    let response = response["results"].as_array().unwrap();
     // key should have access on `products` index.
     assert!(response.iter().any(|index| index["uid"] == "products"));
 
@@ -409,16 +441,16 @@ async fn list_authorized_indexes_no_index_restriction() {
 #[actix_rt::test]
 async fn list_authorized_tasks_restricted_index() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
+    server.use_admin_key("MASTER_KEY").await;
 
     // create index `test`
     let index = server.index("test");
-    let (_, code) = index.create(Some("id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     // create index `products`
     let index = server.index("products");
-    let (_, code) = index.create(Some("product_id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("product_id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     index.wait_task(0).await;
 
     // create key with access on `products` index only.
@@ -428,7 +460,7 @@ async fn list_authorized_tasks_restricted_index() {
         "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
     });
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     // use created key.
@@ -436,7 +468,7 @@ async fn list_authorized_tasks_restricted_index() {
     server.use_api_key(&key);
 
     let (response, code) = server.service.get("/tasks").await;
-    assert_eq!(code, 200);
+    assert_eq!(200, code, "{:?}", &response);
     println!("{}", response);
     let response = response["results"].as_array().unwrap();
     // key should have access on `products` index.
@@ -449,16 +481,16 @@ async fn list_authorized_tasks_restricted_index() {
 #[actix_rt::test]
 async fn list_authorized_tasks_no_index_restriction() {
     let mut server = Server::new_auth().await;
-    server.use_api_key("MASTER_KEY");
+    server.use_admin_key("MASTER_KEY").await;
 
     // create index `test`
     let index = server.index("test");
-    let (_, code) = index.create(Some("id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     // create index `products`
     let index = server.index("products");
-    let (_, code) = index.create(Some("product_id")).await;
-    assert_eq!(code, 202);
+    let (response, code) = index.create(Some("product_id")).await;
+    assert_eq!(202, code, "{:?}", &response);
     index.wait_task(0).await;
 
     // create key with access on all indexes.
@@ -468,7 +500,7 @@ async fn list_authorized_tasks_no_index_restriction() {
         "expiresAt": (OffsetDateTime::now_utc() + Duration::hours(1)).format(&Rfc3339).unwrap(),
     });
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     // use created key.
@@ -476,7 +508,7 @@ async fn list_authorized_tasks_no_index_restriction() {
     server.use_api_key(&key);
 
     let (response, code) = server.service.get("/tasks").await;
-    assert_eq!(code, 200);
+    assert_eq!(200, code, "{:?}", &response);
 
     let response = response["results"].as_array().unwrap();
     // key should have access on `products` index.
@@ -499,7 +531,7 @@ async fn error_creating_index_without_action() {
         "expiresAt": "2050-11-13T00:00:00Z"
     });
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     // use created key.
@@ -523,8 +555,8 @@ async fn error_creating_index_without_action() {
     ]);
 
     let (response, code) = index.add_documents(documents, None).await;
-    assert_eq!(code, 202, "{:?}", response);
-    let task_id = response["uid"].as_u64().unwrap();
+    assert_eq!(202, code, "{:?}", &response);
+    let task_id = response["taskUid"].as_u64().unwrap();
 
     let response = index.wait_task(task_id).await;
     assert_eq!(response["status"], "failed");
@@ -534,8 +566,8 @@ async fn error_creating_index_without_action() {
     let settings = json!({ "distinctAttribute": "test"});
 
     let (response, code) = index.update_settings(settings).await;
-    assert_eq!(code, 202);
-    let task_id = response["uid"].as_u64().unwrap();
+    assert_eq!(202, code, "{:?}", &response);
+    let task_id = response["taskUid"].as_u64().unwrap();
 
     let response = index.wait_task(task_id).await;
 
@@ -544,8 +576,8 @@ async fn error_creating_index_without_action() {
 
     // try to create a index via add specialized settings route
     let (response, code) = index.update_distinct_attribute(json!("test")).await;
-    assert_eq!(code, 202);
-    let task_id = response["uid"].as_u64().unwrap();
+    assert_eq!(202, code, "{:?}", &response);
+    let task_id = response["taskUid"].as_u64().unwrap();
 
     let response = index.wait_task(task_id).await;
 
@@ -566,7 +598,7 @@ async fn lazy_create_index() {
     });
 
     let (response, code) = server.add_api_key(content).await;
-    assert_eq!(code, 201);
+    assert_eq!(201, code, "{:?}", &response);
     assert!(response["key"].is_string());
 
     // use created key.
@@ -583,13 +615,13 @@ async fn lazy_create_index() {
     ]);
 
     let (response, code) = index.add_documents(documents, None).await;
-    assert_eq!(code, 202, "{:?}", response);
-    let task_id = response["uid"].as_u64().unwrap();
+    assert_eq!(202, code, "{:?}", &response);
+    let task_id = response["taskUid"].as_u64().unwrap();
 
     index.wait_task(task_id).await;
 
     let (response, code) = index.get_task(task_id).await;
-    assert_eq!(code, 200);
+    assert_eq!(200, code, "{:?}", &response);
     assert_eq!(response["status"], "succeeded");
 
     // try to create a index via add settings route
@@ -597,24 +629,24 @@ async fn lazy_create_index() {
     let settings = json!({ "distinctAttribute": "test"});
 
     let (response, code) = index.update_settings(settings).await;
-    assert_eq!(code, 202);
-    let task_id = response["uid"].as_u64().unwrap();
+    assert_eq!(202, code, "{:?}", &response);
+    let task_id = response["taskUid"].as_u64().unwrap();
 
     index.wait_task(task_id).await;
 
     let (response, code) = index.get_task(task_id).await;
-    assert_eq!(code, 200);
+    assert_eq!(200, code, "{:?}", &response);
     assert_eq!(response["status"], "succeeded");
 
     // try to create a index via add specialized settings route
     let index = server.index("test2");
     let (response, code) = index.update_distinct_attribute(json!("test")).await;
-    assert_eq!(code, 202);
-    let task_id = response["uid"].as_u64().unwrap();
+    assert_eq!(202, code, "{:?}", &response);
+    let task_id = response["taskUid"].as_u64().unwrap();
 
     index.wait_task(task_id).await;
 
     let (response, code) = index.get_task(task_id).await;
-    assert_eq!(code, 200);
+    assert_eq!(200, code, "{:?}", &response);
     assert_eq!(response["status"], "succeeded");
 }

meilisearch-http/tests/auth/mod.rs

@@ -13,6 +13,15 @@ impl Server {
         self.service.api_key = Some(api_key.as_ref().to_string());
     }
 
+    /// Fetch and use the default admin key for nexts http requests.
+    pub async fn use_admin_key(&mut self, master_key: impl AsRef<str>) {
+        self.use_api_key(master_key);
+        let (response, code) = self.list_api_keys().await;
+        assert_eq!(200, code, "{:?}", response);
+        let admin_key = &response["results"][1]["key"];
+        self.use_api_key(admin_key.as_str().unwrap());
+    }
+
     pub async fn add_api_key(&self, content: Value) -> (Value, StatusCode) {
         let url = "/keys";
         self.service.post(url, content).await

meilisearch-http/tests/auth/tenant_token.rs

@@ -8,11 +8,15 @@ use time::{Duration, OffsetDateTime};
 
 use super::authorization::{ALL_ACTIONS, AUTHORIZATIONS};
 
-fn generate_tenant_token(parent_key: impl AsRef<str>, mut body: HashMap<&str, Value>) -> String {
+fn generate_tenant_token(
+    parent_uid: impl AsRef<str>,
+    parent_key: impl AsRef<str>,
+    mut body: HashMap<&str, Value>,
+) -> String {
     use jsonwebtoken::{encode, EncodingKey, Header};
 
-    let key_id = &parent_key.as_ref()[..8];
-    body.insert("apiKeyPrefix", json!(key_id));
+    let parent_uid = parent_uid.as_ref();
+    body.insert("apiKeyUid", json!(parent_uid));
     encode(
         &Header::default(),
         &body,
@@ -114,7 +118,7 @@ static REFUSED_KEYS: Lazy<Vec<Value>> = Lazy::new(|| {
 macro_rules! compute_autorized_search {
     ($tenant_tokens:expr, $filter:expr, $expected_count:expr) => {
         let mut server = Server::new_auth().await;
-        server.use_api_key("MASTER_KEY");
+        server.use_admin_key("MASTER_KEY").await;
         let index = server.index("sales");
         let documents = DOCUMENTS.clone();
         index.add_documents(documents, None).await;
@@ -130,9 +134,10 @@ macro_rules! compute_autorized_search {
             let (response, code) = server.add_api_key(key_content.clone()).await;
             assert_eq!(code, 201);
             let key = response["key"].as_str().unwrap();
+            let uid = response["uid"].as_str().unwrap();
 
             for tenant_token in $tenant_tokens.iter() {
-                let web_token = generate_tenant_token(&key, tenant_token.clone());
+                let web_token = generate_tenant_token(&uid, &key, tenant_token.clone());
                 server.use_api_key(&web_token);
                 let index = server.index("sales");
                 index
@@ -160,7 +165,7 @@ macro_rules! compute_autorized_search {
 macro_rules! compute_forbidden_search {
     ($tenant_tokens:expr, $parent_keys:expr) => {
         let mut server = Server::new_auth().await;
-        server.use_api_key("MASTER_KEY");
+        server.use_admin_key("MASTER_KEY").await;
         let index = server.index("sales");
         let documents = DOCUMENTS.clone();
         index.add_documents(documents, None).await;
@@ -172,9 +177,10 @@ macro_rules! compute_forbidden_search {
             let (response, code) = server.add_api_key(key_content.clone()).await;
             assert_eq!(code, 201, "{:?}", response);
             let key = response["key"].as_str().unwrap();
+            let uid = response["uid"].as_str().unwrap();
 
             for tenant_token in $tenant_tokens.iter() {
-                let web_token = generate_tenant_token(&key, tenant_token.clone());
+                let web_token = generate_tenant_token(&uid, &key, tenant_token.clone());
                 server.use_api_key(&web_token);
                 let index = server.index("sales");
                 index
@@ -461,12 +467,13 @@ async fn error_access_forbidden_routes() {
     assert!(response["key"].is_string());
 
     let key = response["key"].as_str().unwrap();
+    let uid = response["uid"].as_str().unwrap();
 
     let tenant_token = hashmap! {
         "searchRules" => json!(["*"]),
         "exp" => json!((OffsetDateTime::now_utc() + Duration::hours(1)).unix_timestamp())
     };
-    let web_token = generate_tenant_token(&key, tenant_token);
+    let web_token = generate_tenant_token(&uid, &key, tenant_token);
     server.use_api_key(&web_token);
 
     for ((method, route), actions) in AUTHORIZATIONS.iter() {
@@ -496,12 +503,13 @@ async fn error_access_expired_parent_key() {
     assert!(response["key"].is_string());
 
     let key = response["key"].as_str().unwrap();
+    let uid = response["uid"].as_str().unwrap();
 
     let tenant_token = hashmap! {
         "searchRules" => json!(["*"]),
         "exp" => json!((OffsetDateTime::now_utc() + Duration::hours(1)).unix_timestamp())
     };
-    let web_token = generate_tenant_token(&key, tenant_token);
+    let web_token = generate_tenant_token(&uid, &key, tenant_token);
     server.use_api_key(&web_token);
 
     // test search request while parent_key is not expired
@@ -538,12 +546,13 @@ async fn error_access_modified_token() {
     assert!(response["key"].is_string());
 
     let key = response["key"].as_str().unwrap();
+    let uid = response["uid"].as_str().unwrap();
 
     let tenant_token = hashmap! {
         "searchRules" => json!(["products"]),
         "exp" => json!((OffsetDateTime::now_utc() + Duration::hours(1)).unix_timestamp())
     };
-    let web_token = generate_tenant_token(&key, tenant_token);
+    let web_token = generate_tenant_token(&uid, &key, tenant_token);
     server.use_api_key(&web_token);
 
     // test search request while web_token is valid
@@ -558,7 +567,7 @@ async fn error_access_modified_token() {
         "exp" => json!((OffsetDateTime::now_utc() + Duration::hours(1)).unix_timestamp())
     };
 
-    let alt = generate_tenant_token(&key, tenant_token);
+    let alt = generate_tenant_token(&uid, &key, tenant_token);
     let altered_token = [
         web_token.split('.').next().unwrap(),
         alt.split('.').nth(1).unwrap(),

meilisearch-http/tests/common/index.rs

@@ -4,29 +4,12 @@ use std::{
 };
 
 use actix_web::http::StatusCode;
-use paste::paste;
 use serde_json::{json, Value};
 use tokio::time::sleep;
 use urlencoding::encode;
 
 use super::service::Service;
 
-macro_rules! make_settings_test_routes {
-    ($($name:ident),+) => {
-        $(paste! {
-            pub async fn [<update_$name>](&self, value: Value) -> (Value, StatusCode) {
-                let url = format!("/indexes/{}/settings/{}", encode(self.uid.as_ref()).to_string(), stringify!($name).replace("_", "-"));
-                self.service.post(url, value).await
-            }
-
-            pub async fn [<get_$name>](&self) -> (Value, StatusCode) {
-                let url = format!("/indexes/{}/settings/{}", encode(self.uid.as_ref()).to_string(), stringify!($name).replace("_", "-"));
-                self.service.get(url).await
-            }
-        })*
-    };
-}
-
 pub struct Index<'a> {
     pub uid: String,
     pub service: &'a Service,
@@ -46,7 +29,7 @@ impl Index<'_> {
             .post_str(url, include_str!("../assets/test_set.json"))
             .await;
         assert_eq!(code, 202);
-        let update_id = response["uid"].as_i64().unwrap();
+        let update_id = response["taskUid"].as_i64().unwrap();
         self.wait_task(update_id as u64).await;
         update_id as u64
     }
@@ -65,7 +48,7 @@ impl Index<'_> {
         });
         let url = format!("/indexes/{}", encode(self.uid.as_ref()));
 
-        self.service.put(url, body).await
+        self.service.patch(url, body).await
     }
 
     pub async fn delete(&self) -> (Value, StatusCode) {
@@ -106,37 +89,52 @@ impl Index<'_> {
     }
 
     pub async fn wait_task(&self, update_id: u64) -> Value {
-        // try 10 times to get status, or panic to not wait forever
+        // try several times to get status, or panic to not wait forever
         let url = format!("/tasks/{}", update_id);
-        for _ in 0..10 {
+        for _ in 0..100 {
             let (response, status_code) = self.service.get(&url).await;
-            assert_eq!(status_code, 200, "response: {}", response);
+            assert_eq!(200, status_code, "response: {}", response);
 
             if response["status"] == "succeeded" || response["status"] == "failed" {
                 return response;
             }
 
-            sleep(Duration::from_secs(1)).await;
+            // wait 0.5 second.
+            sleep(Duration::from_millis(500)).await;
         }
         panic!("Timeout waiting for update id");
     }
 
     pub async fn get_task(&self, update_id: u64) -> (Value, StatusCode) {
-        let url = format!("/indexes/{}/tasks/{}", self.uid, update_id);
+        let url = format!("/tasks/{}", update_id);
         self.service.get(url).await
     }
 
     pub async fn list_tasks(&self) -> (Value, StatusCode) {
-        let url = format!("/indexes/{}/tasks", self.uid);
+        let url = format!("/tasks?indexUid={}", self.uid);
+        self.service.get(url).await
+    }
+
+    pub async fn filtered_tasks(&self, type_: &[&str], status: &[&str]) -> (Value, StatusCode) {
+        let mut url = format!("/tasks?indexUid={}", self.uid);
+        if !type_.is_empty() {
+            url += &format!("&type={}", type_.join(","));
+        }
+        if !status.is_empty() {
+            url += &format!("&status={}", status.join(","));
+        }
         self.service.get(url).await
     }
 
     pub async fn get_document(
         &self,
         id: u64,
-        _options: Option<GetDocumentOptions>,
+        options: Option<GetDocumentOptions>,
     ) -> (Value, StatusCode) {
-        let url = format!("/indexes/{}/documents/{}", encode(self.uid.as_ref()), id);
+        let mut url = format!("/indexes/{}/documents/{}", encode(self.uid.as_ref()), id);
+        if let Some(fields) = options.and_then(|o| o.fields) {
+            url.push_str(&format!("?fields={}", fields.join(",")));
+        }
         self.service.get(url).await
     }
 
@@ -151,10 +149,7 @@ impl Index<'_> {
         }
 
         if let Some(attributes_to_retrieve) = options.attributes_to_retrieve {
-            url.push_str(&format!(
-                "attributesToRetrieve={}&",
-                attributes_to_retrieve.join(",")
-            ));
+            url.push_str(&format!("fields={}&", attributes_to_retrieve.join(",")));
         }
 
         self.service.get(url).await
@@ -187,7 +182,7 @@ impl Index<'_> {
 
     pub async fn update_settings(&self, settings: Value) -> (Value, StatusCode) {
         let url = format!("/indexes/{}/settings", encode(self.uid.as_ref()));
-        self.service.post(url, settings).await
+        self.service.patch(url, settings).await
     }
 
     pub async fn delete_settings(&self) -> (Value, StatusCode) {
@@ -231,10 +226,28 @@ impl Index<'_> {
         self.service.get(url).await
     }
 
-    make_settings_test_routes!(distinct_attribute);
+    pub async fn update_distinct_attribute(&self, value: Value) -> (Value, StatusCode) {
+        let url = format!(
+            "/indexes/{}/settings/{}",
+            encode(self.uid.as_ref()),
+            "distinct-attribute"
+        );
+        self.service.put(url, value).await
+    }
+
+    pub async fn get_distinct_attribute(&self) -> (Value, StatusCode) {
+        let url = format!(
+            "/indexes/{}/settings/{}",
+            encode(self.uid.as_ref()),
+            "distinct-attribute"
+        );
+        self.service.get(url).await
+    }
 }
 
-pub struct GetDocumentOptions;
+pub struct GetDocumentOptions {
+    pub fields: Option<Vec<&'static str>>,
+}
 
 #[derive(Debug, Default)]
 pub struct GetAllDocumentsOptions {

meilisearch-http/tests/common/mod.rs

@@ -3,7 +3,7 @@ pub mod server;
 pub mod service;
 
 pub use index::{GetAllDocumentsOptions, GetDocumentOptions};
-pub use server::Server;
+pub use server::{default_settings, Server};
 
 /// Performs a search test on both post and get routes
 #[macro_export]

meilisearch-http/tests/common/server.rs

@@ -1,4 +1,6 @@
 #![allow(dead_code)]
+
+use clap::Parser;
 use std::path::Path;
 
 use actix_web::http::StatusCode;
@@ -50,16 +52,13 @@ impl Server {
         }
     }
 
-    pub async fn new_auth() -> Self {
-        let dir = TempDir::new().unwrap();
-
+    pub async fn new_auth_with_options(mut options: Opt, dir: TempDir) -> Self {
         if cfg!(windows) {
             std::env::set_var("TMP", TEST_TEMP_DIR.path());
         } else {
             std::env::set_var("TMPDIR", TEST_TEMP_DIR.path());
         }
 
-        let mut options = default_settings(dir.path());
         options.master_key = Some("MASTER_KEY".to_string());
 
         let meilisearch = setup_meilisearch(&options).unwrap();
@@ -77,9 +76,15 @@ impl Server {
         }
     }
 
-    pub async fn new_with_options(options: Opt) -> Self {
-        let meilisearch = setup_meilisearch(&options).unwrap();
-        let auth = AuthController::new(&options.db_path, &options.master_key).unwrap();
+    pub async fn new_auth() -> Self {
+        let dir = TempDir::new().unwrap();
+        let options = default_settings(dir.path());
+        Self::new_auth_with_options(options, dir).await
+    }
+
+    pub async fn new_with_options(options: Opt) -> Result<Self, anyhow::Error> {
+        let meilisearch = setup_meilisearch(&options)?;
+        let auth = AuthController::new(&options.db_path, &options.master_key)?;
         let service = Service {
             meilisearch,
             auth,
@@ -87,10 +92,10 @@ impl Server {
             api_key: None,
         };
 
-        Server {
+        Ok(Server {
             service,
             _dir: None,
-        }
+        })
     }
 
     /// Returns a view to an index. There is no guarantee that the index exists.
@@ -101,8 +106,27 @@ impl Server {
         }
     }
 
-    pub async fn list_indexes(&self) -> (Value, StatusCode) {
-        self.service.get("/indexes").await
+    pub async fn list_indexes(
+        &self,
+        offset: Option<usize>,
+        limit: Option<usize>,
+    ) -> (Value, StatusCode) {
+        let (offset, limit) = (
+            offset.map(|offset| format!("offset={offset}")),
+            limit.map(|limit| format!("limit={limit}")),
+        );
+        let query_parameter = offset
+            .as_ref()
+            .zip(limit.as_ref())
+            .map(|(offset, limit)| format!("{offset}&{limit}"))
+            .or_else(|| offset.xor(limit));
+        if let Some(query_parameter) = query_parameter {
+            self.service
+                .get(format!("/indexes?{query_parameter}"))
+                .await
+        } else {
+            self.service.get("/indexes").await
+        }
     }
 
     pub async fn version(&self) -> (Value, StatusCode) {
@@ -126,36 +150,18 @@ pub fn default_settings(dir: impl AsRef<Path>) -> Opt {
     Opt {
         db_path: dir.as_ref().join("db"),
         dumps_dir: dir.as_ref().join("dump"),
-        http_addr: "127.0.0.1:7700".to_owned(),
-        master_key: None,
         env: "development".to_owned(),
         #[cfg(all(not(debug_assertions), feature = "analytics"))]
         no_analytics: true,
-        max_index_size: Byte::from_unit(4.0, ByteUnit::GiB).unwrap(),
-        max_task_db_size: Byte::from_unit(4.0, ByteUnit::GiB).unwrap(),
+        max_index_size: Byte::from_unit(100.0, ByteUnit::MiB).unwrap(),
+        max_task_db_size: Byte::from_unit(1.0, ByteUnit::GiB).unwrap(),
         http_payload_size_limit: Byte::from_unit(10.0, ByteUnit::MiB).unwrap(),
-        ssl_cert_path: None,
-        ssl_key_path: None,
-        ssl_auth_path: None,
-        ssl_ocsp_path: None,
-        ssl_require_auth: false,
-        ssl_resumption: false,
-        ssl_tickets: false,
-        import_snapshot: None,
-        ignore_missing_snapshot: false,
-        ignore_snapshot_if_db_exists: false,
         snapshot_dir: ".".into(),
-        schedule_snapshot: false,
-        snapshot_interval_sec: 0,
-        import_dump: None,
-        ignore_missing_dump: false,
-        ignore_dump_if_db_exists: false,
         indexer_options: IndexerOpts {
             // memory has to be unlimited because several meilisearch are running in test context.
-            max_memory: MaxMemory::unlimited(),
-            ..Default::default()
+            max_indexing_memory: MaxMemory::unlimited(),
+            ..Parser::parse_from(None as Option<&str>)
         },
-        log_level: "off".into(),
-        scheduler_options: meilisearch_lib::options::SchedulerConfig::default(),
+        ..Parser::parse_from(None as Option<&str>)
     }
 }

meilisearch-http/tests/content_type.rs

@@ -7,23 +7,45 @@ use actix_web::test;
 use meilisearch_http::{analytics, create_app};
 use serde_json::{json, Value};
 
+enum HttpVerb {
+    Put,
+    Patch,
+    Post,
+    Get,
+    Delete,
+}
+
+impl HttpVerb {
+    fn test_request(&self) -> test::TestRequest {
+        match self {
+            HttpVerb::Put => test::TestRequest::put(),
+            HttpVerb::Patch => test::TestRequest::patch(),
+            HttpVerb::Post => test::TestRequest::post(),
+            HttpVerb::Get => test::TestRequest::get(),
+            HttpVerb::Delete => test::TestRequest::delete(),
+        }
+    }
+}
+
 #[actix_rt::test]
 async fn error_json_bad_content_type() {
+    use HttpVerb::{Patch, Post, Put};
+
     let routes = [
-        // all the POST routes except the dumps that can be created without any body or content-type
+        // all the routes except the dumps that can be created without any body or content-type
         // and the search that is not a strict json
-        "/indexes",
-        "/indexes/doggo/documents/delete-batch",
-        "/indexes/doggo/search",
-        "/indexes/doggo/settings",
-        "/indexes/doggo/settings/displayed-attributes",
-        "/indexes/doggo/settings/distinct-attribute",
-        "/indexes/doggo/settings/filterable-attributes",
-        "/indexes/doggo/settings/ranking-rules",
-        "/indexes/doggo/settings/searchable-attributes",
-        "/indexes/doggo/settings/sortable-attributes",
-        "/indexes/doggo/settings/stop-words",
-        "/indexes/doggo/settings/synonyms",
+        (Post, "/indexes"),
+        (Post, "/indexes/doggo/documents/delete-batch"),
+        (Post, "/indexes/doggo/search"),
+        (Patch, "/indexes/doggo/settings"),
+        (Put, "/indexes/doggo/settings/displayed-attributes"),
+        (Put, "/indexes/doggo/settings/distinct-attribute"),
+        (Put, "/indexes/doggo/settings/filterable-attributes"),
+        (Put, "/indexes/doggo/settings/ranking-rules"),
+        (Put, "/indexes/doggo/settings/searchable-attributes"),
+        (Put, "/indexes/doggo/settings/sortable-attributes"),
+        (Put, "/indexes/doggo/settings/stop-words"),
+        (Put, "/indexes/doggo/settings/synonyms"),
     ];
     let bad_content_types = [
         "application/csv",
@@ -45,10 +67,11 @@ async fn error_json_bad_content_type() {
         analytics::MockAnalytics::new(&server.service.options).0
     ))
     .await;
-    for route in routes {
+    for (verb, route) in routes {
         // Good content-type, we probably have an error since we didn't send anything in the json
         // so we only ensure we didn't get a bad media type error.
-        let req = test::TestRequest::post()
+        let req = verb
+            .test_request()
             .uri(route)
             .set_payload(document)
             .insert_header(("content-type", "application/json"))
@@ -59,7 +82,8 @@ async fn error_json_bad_content_type() {
         "calling the route `{}` with a content-type of json isn't supposed to throw a bad media type error", route);
 
         // No content-type.
-        let req = test::TestRequest::post()
+        let req = verb
+            .test_request()
             .uri(route)
             .set_payload(document)
             .to_request();
@@ -82,7 +106,8 @@ async fn error_json_bad_content_type() {
 
         for bad_content_type in bad_content_types {
             // Always bad content-type
-            let req = test::TestRequest::post()
+            let req = verb
+                .test_request()
                 .uri(route)
                 .set_payload(document.to_string())
                 .insert_header(("content-type", bad_content_type))

meilisearch-http/tests/documents/add_documents.rs

@@ -35,7 +35,7 @@ async fn add_documents_test_json_content_types() {
     let body = test::read_body(res).await;
     let response: Value = serde_json::from_slice(&body).unwrap_or_default();
     assert_eq!(status_code, 202);
-    assert_eq!(response["uid"], 0);
+    assert_eq!(response["taskUid"], 0);
 
     // put
     let req = test::TestRequest::put()
@@ -48,7 +48,7 @@ async fn add_documents_test_json_content_types() {
     let body = test::read_body(res).await;
     let response: Value = serde_json::from_slice(&body).unwrap_or_default();
     assert_eq!(status_code, 202);
-    assert_eq!(response["uid"], 1);
+    assert_eq!(response["taskUid"], 1);
 }
 
 /// any other content-type is must be refused
@@ -212,7 +212,7 @@ async fn error_add_malformed_csv_documents() {
     assert_eq!(
         response["message"],
         json!(
-            r#"The `csv` payload provided is malformed. `CSV error: record 1 (line: 2, byte: 12): found record with 3 fields, but the previous record has 2 fields`."#
+            r#"The `csv` payload provided is malformed: `CSV error: record 1 (line: 2, byte: 12): found record with 3 fields, but the previous record has 2 fields`."#
         )
     );
     assert_eq!(response["code"], json!("malformed_payload"));
@@ -236,7 +236,7 @@ async fn error_add_malformed_csv_documents() {
     assert_eq!(
         response["message"],
         json!(
-            r#"The `csv` payload provided is malformed. `CSV error: record 1 (line: 2, byte: 12): found record with 3 fields, but the previous record has 2 fields`."#
+            r#"The `csv` payload provided is malformed: `CSV error: record 1 (line: 2, byte: 12): found record with 3 fields, but the previous record has 2 fields`."#
         )
     );
     assert_eq!(response["code"], json!("malformed_payload"));
@@ -307,6 +307,58 @@ async fn error_add_malformed_json_documents() {
         response["link"],
         json!("https://docs.meilisearch.com/errors#malformed_payload")
     );
+
+    // truncate
+
+    // length = 100
+    let long = "0123456789".repeat(10);
+
+    let document = format!("\"{}\"", long);
+    let req = test::TestRequest::put()
+        .uri("/indexes/dog/documents")
+        .set_payload(document)
+        .insert_header(("content-type", "application/json"))
+        .to_request();
+    let res = test::call_service(&app, req).await;
+    let body = test::read_body(res).await;
+    let response: Value = serde_json::from_slice(&body).unwrap_or_default();
+    assert_eq!(status_code, 400);
+    assert_eq!(
+        response["message"],
+        json!(
+            r#"The `json` payload provided is malformed. `Couldn't serialize document value: invalid type: string "0123456789012345678901234567...890123456789", expected a documents, or a sequence of documents. at line 1 column 102`."#
+        )
+    );
+    assert_eq!(response["code"], json!("malformed_payload"));
+    assert_eq!(response["type"], json!("invalid_request"));
+    assert_eq!(
+        response["link"],
+        json!("https://docs.meilisearch.com/errors#malformed_payload")
+    );
+
+    // add one more char to the long string to test if the truncating works.
+    let document = format!("\"{}m\"", long);
+    let req = test::TestRequest::put()
+        .uri("/indexes/dog/documents")
+        .set_payload(document)
+        .insert_header(("content-type", "application/json"))
+        .to_request();
+    let res = test::call_service(&app, req).await;
+    let body = test::read_body(res).await;
+    let response: Value = serde_json::from_slice(&body).unwrap_or_default();
+    assert_eq!(status_code, 400);
+    assert_eq!(
+        response["message"],
+        json!(
+            r#"The `json` payload provided is malformed. `Couldn't serialize document value: invalid type: string "0123456789012345678901234567...90123456789m", expected a documents, or a sequence of documents. at line 1 column 103`."#
+        )
+    );
+    assert_eq!(response["code"], json!("malformed_payload"));
+    assert_eq!(response["type"], json!("invalid_request"));
+    assert_eq!(
+        response["link"],
+        json!("https://docs.meilisearch.com/errors#malformed_payload")
+    );
 }
 
 #[actix_rt::test]
@@ -547,7 +599,7 @@ async fn add_documents_no_index_creation() {
 
     let (response, code) = index.add_documents(documents, None).await;
     assert_eq!(code, 202);
-    assert_eq!(response["uid"], 0);
+    assert_eq!(response["taskUid"], 0);
     /*
      * currently we don’t check these field to stay ISO with meilisearch
      * assert_eq!(response["status"], "pending");
@@ -563,7 +615,7 @@ async fn add_documents_no_index_creation() {
     assert_eq!(code, 200);
     assert_eq!(response["status"], "succeeded");
     assert_eq!(response["uid"], 0);
-    assert_eq!(response["type"], "documentAddition");
+    assert_eq!(response["type"], "documentAdditionOrUpdate");
     assert_eq!(response["details"]["receivedDocuments"], 1);
     assert_eq!(response["details"]["indexedDocuments"], 1);
 
@@ -586,7 +638,7 @@ async fn error_document_add_create_index_bad_uid() {
     let (response, code) = index.add_documents(json!([{"id": 1}]), None).await;
 
     let expected_response = json!({
-        "message": "`883  fj!` is not a valid index uid. Index uid can be an integer or a string containing only alphanumeric characters, hyphens (-) and underscores (_).",
+        "message": "invalid index uid `883  fj!`, the uid must be an integer or a string containing only alphanumeric characters a-z A-Z 0-9, hyphens - and underscores _.",
         "code": "invalid_index_uid",
         "type": "invalid_request",
         "link": "https://docs.meilisearch.com/errors#invalid_index_uid"
@@ -603,7 +655,7 @@ async fn error_document_update_create_index_bad_uid() {
     let (response, code) = index.update_documents(json!([{"id": 1}]), None).await;
 
     let expected_response = json!({
-        "message": "`883  fj!` is not a valid index uid. Index uid can be an integer or a string containing only alphanumeric characters, hyphens (-) and underscores (_).",
+        "message": "invalid index uid `883  fj!`, the uid must be an integer or a string containing only alphanumeric characters a-z A-Z 0-9, hyphens - and underscores _.",
         "code": "invalid_index_uid",
         "type": "invalid_request",
         "link": "https://docs.meilisearch.com/errors#invalid_index_uid"
@@ -633,7 +685,7 @@ async fn document_addition_with_primary_key() {
     assert_eq!(code, 200);
     assert_eq!(response["status"], "succeeded");
     assert_eq!(response["uid"], 0);
-    assert_eq!(response["type"], "documentAddition");
+    assert_eq!(response["type"], "documentAdditionOrUpdate");
     assert_eq!(response["details"]["receivedDocuments"], 1);
     assert_eq!(response["details"]["indexedDocuments"], 1);
 
@@ -662,7 +714,7 @@ async fn document_update_with_primary_key() {
     assert_eq!(code, 200);
     assert_eq!(response["status"], "succeeded");
     assert_eq!(response["uid"], 0);
-    assert_eq!(response["type"], "documentPartial");
+    assert_eq!(response["type"], "documentAdditionOrUpdate");
     assert_eq!(response["details"]["indexedDocuments"], 1);
     assert_eq!(response["details"]["receivedDocuments"], 1);
 
@@ -766,7 +818,7 @@ async fn add_larger_dataset() {
     let (response, code) = index.get_task(update_id).await;
     assert_eq!(code, 200);
     assert_eq!(response["status"], "succeeded");
-    assert_eq!(response["type"], "documentAddition");
+    assert_eq!(response["type"], "documentAdditionOrUpdate");
     assert_eq!(response["details"]["indexedDocuments"], 77);
     assert_eq!(response["details"]["receivedDocuments"], 77);
     let (response, code) = index
@@ -775,8 +827,8 @@ async fn add_larger_dataset() {
             ..Default::default()
         })
         .await;
-    assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 77);
+    assert_eq!(code, 200, "failed with `{}`", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 77);
 }
 
 #[actix_rt::test]
@@ -788,7 +840,7 @@ async fn update_larger_dataset() {
     index.wait_task(0).await;
     let (response, code) = index.get_task(0).await;
     assert_eq!(code, 200);
-    assert_eq!(response["type"], "documentPartial");
+    assert_eq!(response["type"], "documentAdditionOrUpdate");
     assert_eq!(response["details"]["indexedDocuments"], 77);
     let (response, code) = index
         .get_all_documents(GetAllDocumentsOptions {
@@ -797,7 +849,7 @@ async fn update_larger_dataset() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 77);
+    assert_eq!(response["results"].as_array().unwrap().len(), 77);
 }
 
 #[actix_rt::test]
@@ -816,7 +868,12 @@ async fn error_add_documents_bad_document_id() {
     let (response, code) = index.get_task(1).await;
     assert_eq!(code, 200);
     assert_eq!(response["status"], json!("failed"));
-    assert_eq!(response["error"]["message"], json!("Document identifier `foo & bar` is invalid. A document identifier can be of type integer or string, only composed of alphanumeric characters (a-z A-Z 0-9), hyphens (-) and underscores (_)."));
+    assert_eq!(
+        response["error"]["message"],
+        json!(
+            r#"Document identifier `"foo & bar"` is invalid. A document identifier can be of type integer or string, only composed of alphanumeric characters (a-z A-Z 0-9), hyphens (-) and underscores (_)."#
+        )
+    );
     assert_eq!(response["error"]["code"], json!("invalid_document_id"));
     assert_eq!(response["error"]["type"], json!("invalid_request"));
     assert_eq!(
@@ -839,7 +896,12 @@ async fn error_update_documents_bad_document_id() {
     index.update_documents(documents, None).await;
     let response = index.wait_task(1).await;
     assert_eq!(response["status"], json!("failed"));
-    assert_eq!(response["error"]["message"], json!("Document identifier `foo & bar` is invalid. A document identifier can be of type integer or string, only composed of alphanumeric characters (a-z A-Z 0-9), hyphens (-) and underscores (_)."));
+    assert_eq!(
+        response["error"]["message"],
+        json!(
+            r#"Document identifier `"foo & bar"` is invalid. A document identifier can be of type integer or string, only composed of alphanumeric characters (a-z A-Z 0-9), hyphens (-) and underscores (_)."#
+        )
+    );
     assert_eq!(response["error"]["code"], json!("invalid_document_id"));
     assert_eq!(response["error"]["type"], json!("invalid_request"));
     assert_eq!(
@@ -939,7 +1001,7 @@ async fn error_document_field_limit_reached() {
 }
 
 #[actix_rt::test]
-async fn error_add_documents_invalid_geo_field() {
+async fn add_documents_invalid_geo_field() {
     let server = Server::new().await;
     let index = server.index("test");
     index.create(Some("id")).await;
@@ -958,16 +1020,7 @@ async fn error_add_documents_invalid_geo_field() {
     index.wait_task(2).await;
     let (response, code) = index.get_task(2).await;
     assert_eq!(code, 200);
-    assert_eq!(response["status"], "failed");
-
-    let expected_error = json!({
-        "message": r#"The document with the id: `11` contains an invalid _geo field: `foobar`."#,
-        "code": "invalid_geo_field",
-        "type": "invalid_request",
-        "link": "https://docs.meilisearch.com/errors#invalid_geo_field"
-    });
-
-    assert_eq!(response["error"], expected_error);
+    assert_eq!(response["status"], "succeeded");
 }
 
 #[actix_rt::test]

meilisearch-http/tests/documents/delete_documents.rs

@@ -72,7 +72,7 @@ async fn clear_all_documents() {
         .get_all_documents(GetAllDocumentsOptions::default())
         .await;
     assert_eq!(code, 200);
-    assert!(response.as_array().unwrap().is_empty());
+    assert!(response["results"].as_array().unwrap().is_empty());
 }
 
 #[actix_rt::test]
@@ -89,7 +89,7 @@ async fn clear_all_documents_empty_index() {
         .get_all_documents(GetAllDocumentsOptions::default())
         .await;
     assert_eq!(code, 200);
-    assert!(response.as_array().unwrap().is_empty());
+    assert!(response["results"].as_array().unwrap().is_empty());
 }
 
 #[actix_rt::test]
@@ -125,8 +125,8 @@ async fn delete_batch() {
         .get_all_documents(GetAllDocumentsOptions::default())
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 1);
-    assert_eq!(response.as_array().unwrap()[0]["id"], 3);
+    assert_eq!(response["results"].as_array().unwrap().len(), 1);
+    assert_eq!(response["results"][0]["id"], json!(3));
 }
 
 #[actix_rt::test]
@@ -143,5 +143,5 @@ async fn delete_no_document_batch() {
         .get_all_documents(GetAllDocumentsOptions::default())
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 3);
+    assert_eq!(response["results"].as_array().unwrap().len(), 3);
 }

meilisearch-http/tests/documents/get_documents.rs

@@ -1,5 +1,4 @@
-use crate::common::GetAllDocumentsOptions;
-use crate::common::Server;
+use crate::common::{GetAllDocumentsOptions, GetDocumentOptions, Server};
 
 use serde_json::json;
 
@@ -39,19 +38,51 @@ async fn get_document() {
     let documents = serde_json::json!([
         {
             "id": 0,
-            "content": "foobar",
+            "nested": { "content": "foobar" },
         }
     ]);
     let (_, code) = index.add_documents(documents, None).await;
     assert_eq!(code, 202);
-    index.wait_task(0).await;
+    index.wait_task(1).await;
     let (response, code) = index.get_document(0, None).await;
     assert_eq!(code, 200);
     assert_eq!(
         response,
-        serde_json::json!(        {
+        serde_json::json!({
             "id": 0,
-            "content": "foobar",
+            "nested": { "content": "foobar" },
+        })
+    );
+
+    let (response, code) = index
+        .get_document(
+            0,
+            Some(GetDocumentOptions {
+                fields: Some(vec!["id"]),
+            }),
+        )
+        .await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        response,
+        serde_json::json!({
+            "id": 0,
+        })
+    );
+
+    let (response, code) = index
+        .get_document(
+            0,
+            Some(GetDocumentOptions {
+                fields: Some(vec!["nested.content"]),
+            }),
+        )
+        .await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        response,
+        serde_json::json!({
+            "nested": { "content": "foobar" },
         })
     );
 }
@@ -88,7 +119,7 @@ async fn get_no_document() {
         .get_all_documents(GetAllDocumentsOptions::default())
         .await;
     assert_eq!(code, 200);
-    assert!(response.as_array().unwrap().is_empty());
+    assert!(response["results"].as_array().unwrap().is_empty());
 }
 
 #[actix_rt::test]
@@ -101,7 +132,7 @@ async fn get_all_documents_no_options() {
         .get_all_documents(GetAllDocumentsOptions::default())
         .await;
     assert_eq!(code, 200);
-    let arr = response.as_array().unwrap();
+    let arr = response["results"].as_array().unwrap();
     assert_eq!(arr.len(), 20);
     let first = serde_json::json!({
         "id":0,
@@ -137,8 +168,11 @@ async fn test_get_all_documents_limit() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 5);
-    assert_eq!(response.as_array().unwrap()[0]["id"], 0);
+    assert_eq!(response["results"].as_array().unwrap().len(), 5);
+    assert_eq!(response["results"][0]["id"], json!(0));
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["limit"], json!(5));
+    assert_eq!(response["total"], json!(77));
 }
 
 #[actix_rt::test]
@@ -154,8 +188,11 @@ async fn test_get_all_documents_offset() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 20);
-    assert_eq!(response.as_array().unwrap()[0]["id"], 13);
+    assert_eq!(response["results"].as_array().unwrap().len(), 20);
+    assert_eq!(response["results"][0]["id"], json!(5));
+    assert_eq!(response["offset"], json!(5));
+    assert_eq!(response["limit"], json!(20));
+    assert_eq!(response["total"], json!(77));
 }
 
 #[actix_rt::test]
@@ -171,20 +208,14 @@ async fn test_get_all_documents_attributes_to_retrieve() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 20);
-    assert_eq!(
-        response.as_array().unwrap()[0]
-            .as_object()
-            .unwrap()
-            .keys()
-            .count(),
-        1
-    );
-    assert!(response.as_array().unwrap()[0]
-        .as_object()
-        .unwrap()
-        .get("name")
-        .is_some());
+    assert_eq!(response["results"].as_array().unwrap().len(), 20);
+    for results in response["results"].as_array().unwrap() {
+        assert_eq!(results.as_object().unwrap().keys().count(), 1);
+        assert!(results["name"] != json!(null));
+    }
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["limit"], json!(20));
+    assert_eq!(response["total"], json!(77));
 
     let (response, code) = index
         .get_all_documents(GetAllDocumentsOptions {
@@ -193,15 +224,13 @@ async fn test_get_all_documents_attributes_to_retrieve() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 20);
-    assert_eq!(
-        response.as_array().unwrap()[0]
-            .as_object()
-            .unwrap()
-            .keys()
-            .count(),
-        0
-    );
+    assert_eq!(response["results"].as_array().unwrap().len(), 20);
+    for results in response["results"].as_array().unwrap() {
+        assert_eq!(results.as_object().unwrap().keys().count(), 0);
+    }
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["limit"], json!(20));
+    assert_eq!(response["total"], json!(77));
 
     let (response, code) = index
         .get_all_documents(GetAllDocumentsOptions {
@@ -210,15 +239,13 @@ async fn test_get_all_documents_attributes_to_retrieve() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 20);
-    assert_eq!(
-        response.as_array().unwrap()[0]
-            .as_object()
-            .unwrap()
-            .keys()
-            .count(),
-        0
-    );
+    assert_eq!(response["results"].as_array().unwrap().len(), 20);
+    for results in response["results"].as_array().unwrap() {
+        assert_eq!(results.as_object().unwrap().keys().count(), 0);
+    }
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["limit"], json!(20));
+    assert_eq!(response["total"], json!(77));
 
     let (response, code) = index
         .get_all_documents(GetAllDocumentsOptions {
@@ -227,15 +254,12 @@ async fn test_get_all_documents_attributes_to_retrieve() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 20);
-    assert_eq!(
-        response.as_array().unwrap()[0]
-            .as_object()
-            .unwrap()
-            .keys()
-            .count(),
-        2
-    );
+    assert_eq!(response["results"].as_array().unwrap().len(), 20);
+    for results in response["results"].as_array().unwrap() {
+        assert_eq!(results.as_object().unwrap().keys().count(), 2);
+        assert!(results["name"] != json!(null));
+        assert!(results["tags"] != json!(null));
+    }
 
     let (response, code) = index
         .get_all_documents(GetAllDocumentsOptions {
@@ -244,15 +268,10 @@ async fn test_get_all_documents_attributes_to_retrieve() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 20);
-    assert_eq!(
-        response.as_array().unwrap()[0]
-            .as_object()
-            .unwrap()
-            .keys()
-            .count(),
-        16
-    );
+    assert_eq!(response["results"].as_array().unwrap().len(), 20);
+    for results in response["results"].as_array().unwrap() {
+        assert_eq!(results.as_object().unwrap().keys().count(), 16);
+    }
 
     let (response, code) = index
         .get_all_documents(GetAllDocumentsOptions {
@@ -261,19 +280,99 @@ async fn test_get_all_documents_attributes_to_retrieve() {
         })
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 20);
+    assert_eq!(response["results"].as_array().unwrap().len(), 20);
+    for results in response["results"].as_array().unwrap() {
+        assert_eq!(results.as_object().unwrap().keys().count(), 16);
+    }
+}
+
+#[actix_rt::test]
+async fn get_document_s_nested_attributes_to_retrieve() {
+    let server = Server::new().await;
+    let index = server.index("test");
+    index.create(None).await;
+    let documents = json!([
+        {
+            "id": 0,
+            "content.truc": "foobar",
+        },
+        {
+            "id": 1,
+            "content": {
+                "truc": "foobar",
+                "machin": "bidule",
+            },
+        },
+    ]);
+    let (_, code) = index.add_documents(documents, None).await;
+    assert_eq!(code, 202);
+    index.wait_task(1).await;
+
+    let (response, code) = index
+        .get_document(
+            0,
+            Some(GetDocumentOptions {
+                fields: Some(vec!["content"]),
+            }),
+        )
+        .await;
+    assert_eq!(code, 200);
+    assert_eq!(response, json!({}));
+    let (response, code) = index
+        .get_document(
+            1,
+            Some(GetDocumentOptions {
+                fields: Some(vec!["content"]),
+            }),
+        )
+        .await;
+    assert_eq!(code, 200);
     assert_eq!(
-        response.as_array().unwrap()[0]
-            .as_object()
-            .unwrap()
-            .keys()
-            .count(),
-        16
+        response,
+        json!({
+            "content": {
+                "truc": "foobar",
+                "machin": "bidule",
+            },
+        })
+    );
+
+    let (response, code) = index
+        .get_document(
+            0,
+            Some(GetDocumentOptions {
+                fields: Some(vec!["content.truc"]),
+            }),
+        )
+        .await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        response,
+        json!({
+            "content.truc": "foobar",
+        })
+    );
+    let (response, code) = index
+        .get_document(
+            1,
+            Some(GetDocumentOptions {
+                fields: Some(vec!["content.truc"]),
+            }),
+        )
+        .await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        response,
+        json!({
+            "content": {
+                "truc": "foobar",
+            },
+        })
     );
 }
 
 #[actix_rt::test]
-async fn get_documents_displayed_attributes() {
+async fn get_documents_displayed_attributes_is_ignored() {
     let server = Server::new().await;
     let index = server.index("test");
     index
@@ -285,23 +384,19 @@ async fn get_documents_displayed_attributes() {
         .get_all_documents(GetAllDocumentsOptions::default())
         .await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_array().unwrap().len(), 20);
+    assert_eq!(response["results"].as_array().unwrap().len(), 20);
     assert_eq!(
-        response.as_array().unwrap()[0]
-            .as_object()
-            .unwrap()
-            .keys()
-            .count(),
-        1
+        response["results"][0].as_object().unwrap().keys().count(),
+        16
     );
-    assert!(response.as_array().unwrap()[0]
-        .as_object()
-        .unwrap()
-        .get("gender")
-        .is_some());
+    assert!(response["results"][0]["gender"] != json!(null));
+
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["limit"], json!(20));
+    assert_eq!(response["total"], json!(77));
 
     let (response, code) = index.get_document(0, None).await;
     assert_eq!(code, 200);
-    assert_eq!(response.as_object().unwrap().keys().count(), 1);
+    assert_eq!(response.as_object().unwrap().keys().count(), 16);
     assert!(response.as_object().unwrap().get("gender").is_some());
 }

meilisearch-http/tests/dumps.rs

@@ -1,22 +0,0 @@
-#![allow(dead_code)]
-mod common;
-
-use crate::common::Server;
-use serde_json::json;
-
-#[actix_rt::test]
-async fn get_unexisting_dump_status() {
-    let server = Server::new().await;
-
-    let (response, code) = server.get_dump_status("foobar").await;
-    assert_eq!(code, 404);
-
-    let expected_response = json!({
-    "message": "Dump `foobar` not found.",
-    "code": "dump_not_found",
-    "type": "invalid_request",
-    "link": "https://docs.meilisearch.com/errors#dump_not_found"
-    });
-
-    assert_eq!(response, expected_response);
-}

meilisearch-http/tests/dumps/data.rs

@@ -0,0 +1,73 @@
+use std::path::PathBuf;
+
+use manifest_dir_macros::exist_relative_path;
+
+pub enum GetDump {
+    MoviesRawV1,
+    MoviesWithSettingsV1,
+    RubyGemsWithSettingsV1,
+
+    MoviesRawV2,
+    MoviesWithSettingsV2,
+    RubyGemsWithSettingsV2,
+
+    MoviesRawV3,
+    MoviesWithSettingsV3,
+    RubyGemsWithSettingsV3,
+
+    MoviesRawV4,
+    MoviesWithSettingsV4,
+    RubyGemsWithSettingsV4,
+
+    TestV5,
+}
+
+impl GetDump {
+    pub fn path(&self) -> PathBuf {
+        match self {
+            GetDump::MoviesRawV1 => {
+                exist_relative_path!("tests/assets/v1_v0.20.0_movies.dump").into()
+            }
+            GetDump::MoviesWithSettingsV1 => {
+                exist_relative_path!("tests/assets/v1_v0.20.0_movies_with_settings.dump").into()
+            }
+            GetDump::RubyGemsWithSettingsV1 => {
+                exist_relative_path!("tests/assets/v1_v0.20.0_rubygems_with_settings.dump").into()
+            }
+
+            GetDump::MoviesRawV2 => {
+                exist_relative_path!("tests/assets/v2_v0.21.1_movies.dump").into()
+            }
+            GetDump::MoviesWithSettingsV2 => {
+                exist_relative_path!("tests/assets/v2_v0.21.1_movies_with_settings.dump").into()
+            }
+
+            GetDump::RubyGemsWithSettingsV2 => {
+                exist_relative_path!("tests/assets/v2_v0.21.1_rubygems_with_settings.dump").into()
+            }
+
+            GetDump::MoviesRawV3 => {
+                exist_relative_path!("tests/assets/v3_v0.24.0_movies.dump").into()
+            }
+            GetDump::MoviesWithSettingsV3 => {
+                exist_relative_path!("tests/assets/v3_v0.24.0_movies_with_settings.dump").into()
+            }
+            GetDump::RubyGemsWithSettingsV3 => {
+                exist_relative_path!("tests/assets/v3_v0.24.0_rubygems_with_settings.dump").into()
+            }
+
+            GetDump::MoviesRawV4 => {
+                exist_relative_path!("tests/assets/v4_v0.25.2_movies.dump").into()
+            }
+            GetDump::MoviesWithSettingsV4 => {
+                exist_relative_path!("tests/assets/v4_v0.25.2_movies_with_settings.dump").into()
+            }
+            GetDump::RubyGemsWithSettingsV4 => {
+                exist_relative_path!("tests/assets/v4_v0.25.2_rubygems_with_settings.dump").into()
+            }
+            GetDump::TestV5 => {
+                exist_relative_path!("tests/assets/v5_v0.28.0_test_dump.dump").into()
+            }
+        }
+    }
+}

meilisearch-http/tests/dumps/mod.rs

@@ -0,0 +1,677 @@
+mod data;
+
+use crate::common::{default_settings, GetAllDocumentsOptions, Server};
+use meilisearch_http::Opt;
+use serde_json::json;
+
+use self::data::GetDump;
+
+// all the following test are ignored on windows. See #2364
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v1() {
+    let temp = tempfile::tempdir().unwrap();
+
+    for path in [
+        GetDump::MoviesRawV1.path(),
+        GetDump::MoviesWithSettingsV1.path(),
+        GetDump::RubyGemsWithSettingsV1.path(),
+    ] {
+        let options = Opt {
+            import_dump: Some(path),
+            ..default_settings(temp.path())
+        };
+        let error = Server::new_with_options(options)
+            .await
+            .map(|_| ())
+            .unwrap_err();
+
+        assert_eq!(error.to_string(), "The version 1 of the dumps is not supported anymore. You can re-export your dump from a version between 0.21 and 0.24, or start fresh from a version 0.25 onwards.");
+    }
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v2_movie_raw() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::MoviesRawV2.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("indexUID"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("indexUID");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"genres": 53, "id": 53, "overview": 53, "poster": 53, "release_date": 53, "title": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({"displayedAttributes": ["*"], "searchableAttributes": ["*"], "filterableAttributes": [], "sortableAttributes": [], "rankingRules": ["words", "typo", "proximity", "attribute", "exactness"], "stopWords": [], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": {"oneTypo": 5, "twoTypos": 9}, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 } })
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks,
+        json!({ "results": [{"uid": 0, "indexUid": "indexUID", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": { "receivedDocuments": 0, "indexedDocuments": 31944 }, "duration": "PT41.751156S", "enqueuedAt": "2021-09-08T08:30:30.550282Z", "startedAt": "2021-09-08T08:30:30.553012Z", "finishedAt": "2021-09-08T08:31:12.304168Z" }], "limit": 20, "from": 0, "next": null })
+    );
+
+    // finally we're just going to check that we can still get a few documents by id
+    let (document, code) = index.get_document(100, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({"id": 100, "title": "Lock, Stock and Two Smoking Barrels", "overview": "A card shark and his unwillingly-enlisted friends need to make a lot of cash quick after losing a sketchy poker match. To do this they decide to pull a heist on a small-time gang who happen to be operating out of the flat next door.", "genres": ["Comedy", "Crime"], "poster": "https://image.tmdb.org/t/p/w500/8kSerJrhrJWKLk1LViesGcnrUPE.jpg", "release_date": 889056000})
+    );
+
+    let (document, code) = index.get_document(500, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({"id": 500, "title": "Reservoir Dogs", "overview": "A botched robbery indicates a police informant, and the pressure mounts in the aftermath at a warehouse. Crime begets violence as the survivors -- veteran Mr. White, newcomer Mr. Orange, psychopathic parolee Mr. Blonde, bickering weasel Mr. Pink and Nice Guy Eddie -- unravel.", "genres": ["Crime", "Thriller"], "poster": "https://image.tmdb.org/t/p/w500/AjTtJNumZyUDz33VtMlF1K8JPsE.jpg", "release_date": 715392000})
+    );
+
+    let (document, code) = index.get_document(10006, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({"id": 10006, "title": "Wild Seven", "overview": "In this darkly karmic vision of Arizona, a man who breathes nothing but ill will begins a noxious domino effect as quickly as an uncontrollable virus kills. As he exits Arizona State Penn after twenty-one long years, Wilson has only one thing on the brain, leveling the score with career criminal, Mackey Willis.", "genres": ["Action", "Crime", "Drama"], "poster": "https://image.tmdb.org/t/p/w500/y114dTPoqn8k2Txps4P2tI95YCS.jpg", "release_date": 1136073600})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v2_movie_with_settings() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::MoviesWithSettingsV2.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("indexUID"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("indexUID");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"genres": 53, "id": 53, "overview": 53, "poster": 53, "release_date": 53, "title": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({ "displayedAttributes": ["title", "genres", "overview", "poster", "release_date"], "searchableAttributes": ["title", "overview"], "filterableAttributes": ["genres"], "sortableAttributes": [], "rankingRules": ["words", "typo", "proximity", "attribute", "exactness"], "stopWords": ["of", "the"], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": { "oneTypo": 5, "twoTypos": 9 }, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 } })
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks,
+        json!({ "results": [{ "uid": 1, "indexUid": "indexUID", "status": "succeeded", "type": "settingsUpdate", "details": { "displayedAttributes": ["title", "genres", "overview", "poster", "release_date"], "searchableAttributes": ["title", "overview"], "filterableAttributes": ["genres"], "stopWords": ["of", "the"] }, "duration": "PT37.488777S", "enqueuedAt": "2021-09-08T08:24:02.323444Z", "startedAt": "2021-09-08T08:24:02.324145Z", "finishedAt": "2021-09-08T08:24:39.812922Z" }, { "uid": 0, "indexUid": "indexUID", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": { "receivedDocuments": 0, "indexedDocuments": 31944 }, "duration": "PT39.941318S", "enqueuedAt": "2021-09-08T08:21:14.742672Z", "startedAt": "2021-09-08T08:21:14.750166Z", "finishedAt": "2021-09-08T08:21:54.691484Z" }], "limit": 20, "from": 1, "next": null })
+    );
+
+    // finally we're just going to check that we can still get a few documents by id
+    let (document, code) = index.get_document(100, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 100, "title": "Lock, Stock and Two Smoking Barrels", "genres": ["Comedy", "Crime"], "overview": "A card shark and his unwillingly-enlisted friends need to make a lot of cash quick after losing a sketchy poker match. To do this they decide to pull a heist on a small-time gang who happen to be operating out of the flat next door.", "poster": "https://image.tmdb.org/t/p/w500/8kSerJrhrJWKLk1LViesGcnrUPE.jpg", "release_date": 889056000 })
+    );
+
+    let (document, code) = index.get_document(500, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 500, "title": "Reservoir Dogs", "genres": ["Crime", "Thriller"], "overview": "A botched robbery indicates a police informant, and the pressure mounts in the aftermath at a warehouse. Crime begets violence as the survivors -- veteran Mr. White, newcomer Mr. Orange, psychopathic parolee Mr. Blonde, bickering weasel Mr. Pink and Nice Guy Eddie -- unravel.", "poster": "https://image.tmdb.org/t/p/w500/AjTtJNumZyUDz33VtMlF1K8JPsE.jpg", "release_date": 715392000})
+    );
+
+    let (document, code) = index.get_document(10006, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 10006, "title": "Wild Seven", "genres": ["Action", "Crime", "Drama"], "overview": "In this darkly karmic vision of Arizona, a man who breathes nothing but ill will begins a noxious domino effect as quickly as an uncontrollable virus kills. As he exits Arizona State Penn after twenty-one long years, Wilson has only one thing on the brain, leveling the score with career criminal, Mackey Willis.", "poster": "https://image.tmdb.org/t/p/w500/y114dTPoqn8k2Txps4P2tI95YCS.jpg", "release_date": 1136073600})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v2_rubygems_with_settings() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::RubyGemsWithSettingsV2.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("rubygems"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("rubygems");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"description": 53, "id": 53, "name": 53, "summary": 53, "total_downloads": 53, "version": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({"displayedAttributes": ["name", "summary", "description", "version", "total_downloads"], "searchableAttributes": ["name", "summary"], "filterableAttributes": ["version"], "sortableAttributes": [], "rankingRules": ["typo", "words", "fame:desc", "proximity", "attribute", "exactness", "total_downloads:desc"], "stopWords": [], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": {"oneTypo": 5, "twoTypos": 9}, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 }})
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks["results"][0],
+        json!({"uid": 92, "indexUid": "rubygems", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": {"receivedDocuments": 0, "indexedDocuments": 1042}, "duration": "PT14.034672S", "enqueuedAt": "2021-09-08T08:40:31.390775Z", "startedAt": "2021-09-08T08:51:39.060642Z", "finishedAt": "2021-09-08T08:51:53.095314Z"})
+    );
+
+    // finally we're just going to check that we can still get a few documents by id
+    let (document, code) = index.get_document(188040, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "meilisearch", "summary": "An easy-to-use ruby client for Meilisearch API", "description": "An easy-to-use ruby client for Meilisearch API. See https://github.com/meilisearch/MeiliSearch", "id": "188040", "version": "0.15.2", "total_downloads": "7465"})
+    );
+
+    let (document, code) = index.get_document(191940, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "doggo", "summary": "RSpec 3 formatter - documentation, with progress indication", "description": "Similar to \"rspec -f d\", but also indicates progress by showing the current test number and total test count on each line.", "id": "191940", "version": "1.1.0", "total_downloads": "9394"})
+    );
+
+    let (document, code) = index.get_document(159227, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "vortex-of-agony", "summary": "You dont need to use nodejs or go, just install this plugin. It will crash your application at random", "description": "You dont need to use nodejs or go, just install this plugin. It will crash your application at random", "id": "159227", "version": "0.1.0", "total_downloads": "1007"})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v3_movie_raw() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::MoviesRawV3.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("indexUID"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("indexUID");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"genres": 53, "id": 53, "overview": 53, "poster": 53, "release_date": 53, "title": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({"displayedAttributes": ["*"], "searchableAttributes": ["*"], "filterableAttributes": [], "sortableAttributes": [], "rankingRules": ["words", "typo", "proximity", "attribute", "exactness"], "stopWords": [], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": {"oneTypo": 5, "twoTypos": 9}, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 } })
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks,
+        json!({ "results": [{"uid": 0, "indexUid": "indexUID", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": { "receivedDocuments": 0, "indexedDocuments": 31944 }, "duration": "PT41.751156S", "enqueuedAt": "2021-09-08T08:30:30.550282Z", "startedAt": "2021-09-08T08:30:30.553012Z", "finishedAt": "2021-09-08T08:31:12.304168Z" }], "limit": 20, "from": 0, "next": null })
+    );
+
+    // finally we're just going to check that we can still get a few documents by id
+    let (document, code) = index.get_document(100, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({"id": 100, "title": "Lock, Stock and Two Smoking Barrels", "overview": "A card shark and his unwillingly-enlisted friends need to make a lot of cash quick after losing a sketchy poker match. To do this they decide to pull a heist on a small-time gang who happen to be operating out of the flat next door.", "genres": ["Comedy", "Crime"], "poster": "https://image.tmdb.org/t/p/w500/8kSerJrhrJWKLk1LViesGcnrUPE.jpg", "release_date": 889056000})
+    );
+
+    let (document, code) = index.get_document(500, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({"id": 500, "title": "Reservoir Dogs", "overview": "A botched robbery indicates a police informant, and the pressure mounts in the aftermath at a warehouse. Crime begets violence as the survivors -- veteran Mr. White, newcomer Mr. Orange, psychopathic parolee Mr. Blonde, bickering weasel Mr. Pink and Nice Guy Eddie -- unravel.", "genres": ["Crime", "Thriller"], "poster": "https://image.tmdb.org/t/p/w500/AjTtJNumZyUDz33VtMlF1K8JPsE.jpg", "release_date": 715392000})
+    );
+
+    let (document, code) = index.get_document(10006, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({"id": 10006, "title": "Wild Seven", "overview": "In this darkly karmic vision of Arizona, a man who breathes nothing but ill will begins a noxious domino effect as quickly as an uncontrollable virus kills. As he exits Arizona State Penn after twenty-one long years, Wilson has only one thing on the brain, leveling the score with career criminal, Mackey Willis.", "genres": ["Action", "Crime", "Drama"], "poster": "https://image.tmdb.org/t/p/w500/y114dTPoqn8k2Txps4P2tI95YCS.jpg", "release_date": 1136073600})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v3_movie_with_settings() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::MoviesWithSettingsV3.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("indexUID"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("indexUID");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"genres": 53, "id": 53, "overview": 53, "poster": 53, "release_date": 53, "title": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({ "displayedAttributes": ["title", "genres", "overview", "poster", "release_date"], "searchableAttributes": ["title", "overview"], "filterableAttributes": ["genres"], "sortableAttributes": [], "rankingRules": ["words", "typo", "proximity", "attribute", "exactness"], "stopWords": ["of", "the"], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": { "oneTypo": 5, "twoTypos": 9 }, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 } })
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks,
+        json!({ "results": [{ "uid": 1, "indexUid": "indexUID", "status": "succeeded", "type": "settingsUpdate", "details": { "displayedAttributes": ["title", "genres", "overview", "poster", "release_date"], "searchableAttributes": ["title", "overview"], "filterableAttributes": ["genres"], "stopWords": ["of", "the"] }, "duration": "PT37.488777S", "enqueuedAt": "2021-09-08T08:24:02.323444Z", "startedAt": "2021-09-08T08:24:02.324145Z", "finishedAt": "2021-09-08T08:24:39.812922Z" }, { "uid": 0, "indexUid": "indexUID", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": { "receivedDocuments": 0, "indexedDocuments": 31944 }, "duration": "PT39.941318S", "enqueuedAt": "2021-09-08T08:21:14.742672Z", "startedAt": "2021-09-08T08:21:14.750166Z", "finishedAt": "2021-09-08T08:21:54.691484Z" }], "limit": 20, "from": 1, "next": null })
+    );
+
+    // finally we're just going to check that we can["results"] still get a few documents by id
+    let (document, code) = index.get_document(100, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 100, "title": "Lock, Stock and Two Smoking Barrels", "genres": ["Comedy", "Crime"], "overview": "A card shark and his unwillingly-enlisted friends need to make a lot of cash quick after losing a sketchy poker match. To do this they decide to pull a heist on a small-time gang who happen to be operating out of the flat next door.", "poster": "https://image.tmdb.org/t/p/w500/8kSerJrhrJWKLk1LViesGcnrUPE.jpg", "release_date": 889056000 })
+    );
+
+    let (document, code) = index.get_document(500, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 500, "title": "Reservoir Dogs", "genres": ["Crime", "Thriller"], "overview": "A botched robbery indicates a police informant, and the pressure mounts in the aftermath at a warehouse. Crime begets violence as the survivors -- veteran Mr. White, newcomer Mr. Orange, psychopathic parolee Mr. Blonde, bickering weasel Mr. Pink and Nice Guy Eddie -- unravel.", "poster": "https://image.tmdb.org/t/p/w500/AjTtJNumZyUDz33VtMlF1K8JPsE.jpg", "release_date": 715392000})
+    );
+
+    let (document, code) = index.get_document(10006, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 10006, "title": "Wild Seven", "genres": ["Action", "Crime", "Drama"], "overview": "In this darkly karmic vision of Arizona, a man who breathes nothing but ill will begins a noxious domino effect as quickly as an uncontrollable virus kills. As he exits Arizona State Penn after twenty-one long years, Wilson has only one thing on the brain, leveling the score with career criminal, Mackey Willis.", "poster": "https://image.tmdb.org/t/p/w500/y114dTPoqn8k2Txps4P2tI95YCS.jpg", "release_date": 1136073600})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v3_rubygems_with_settings() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::RubyGemsWithSettingsV3.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("rubygems"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("rubygems");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"description": 53, "id": 53, "name": 53, "summary": 53, "total_downloads": 53, "version": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({"displayedAttributes": ["name", "summary", "description", "version", "total_downloads"], "searchableAttributes": ["name", "summary"], "filterableAttributes": ["version"], "sortableAttributes": [], "rankingRules": ["typo", "words", "fame:desc", "proximity", "attribute", "exactness", "total_downloads:desc"], "stopWords": [], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": {"oneTypo": 5, "twoTypos": 9}, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 } })
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks["results"][0],
+        json!({"uid": 92, "indexUid": "rubygems", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": {"receivedDocuments": 0, "indexedDocuments": 1042}, "duration": "PT14.034672S", "enqueuedAt": "2021-09-08T08:40:31.390775Z", "startedAt": "2021-09-08T08:51:39.060642Z", "finishedAt": "2021-09-08T08:51:53.095314Z"})
+    );
+
+    // finally we're just going to check that we can still get a few documents by id
+    let (document, code) = index.get_document(188040, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "meilisearch", "summary": "An easy-to-use ruby client for Meilisearch API", "description": "An easy-to-use ruby client for Meilisearch API. See https://github.com/meilisearch/MeiliSearch", "id": "188040", "version": "0.15.2", "total_downloads": "7465"})
+    );
+
+    let (document, code) = index.get_document(191940, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "doggo", "summary": "RSpec 3 formatter - documentation, with progress indication", "description": "Similar to \"rspec -f d\", but also indicates progress by showing the current test number and total test count on each line.", "id": "191940", "version": "1.1.0", "total_downloads": "9394"})
+    );
+
+    let (document, code) = index.get_document(159227, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "vortex-of-agony", "summary": "You dont need to use nodejs or go, just install this plugin. It will crash your application at random", "description": "You dont need to use nodejs or go, just install this plugin. It will crash your application at random", "id": "159227", "version": "0.1.0", "total_downloads": "1007"})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v4_movie_raw() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::MoviesRawV4.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("indexUID"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("indexUID");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"genres": 53, "id": 53, "overview": 53, "poster": 53, "release_date": 53, "title": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({ "displayedAttributes": ["*"], "searchableAttributes": ["*"], "filterableAttributes": [], "sortableAttributes": [], "rankingRules": ["words", "typo", "proximity", "attribute", "exactness"], "stopWords": [], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": {"oneTypo": 5, "twoTypos": 9}, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 } })
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks,
+        json!({ "results": [{"uid": 0, "indexUid": "indexUID", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": { "receivedDocuments": 0, "indexedDocuments": 31944 }, "duration": "PT41.751156S", "enqueuedAt": "2021-09-08T08:30:30.550282Z", "startedAt": "2021-09-08T08:30:30.553012Z", "finishedAt": "2021-09-08T08:31:12.304168Z" }], "limit" : 20, "from": 0, "next": null })
+    );
+
+    // finally we're just going to check that we can still get a few documents by id
+    let (document, code) = index.get_document(100, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 100, "title": "Lock, Stock and Two Smoking Barrels", "overview": "A card shark and his unwillingly-enlisted friends need to make a lot of cash quick after losing a sketchy poker match. To do this they decide to pull a heist on a small-time gang who happen to be operating out of the flat next door.", "genres": ["Comedy", "Crime"], "poster": "https://image.tmdb.org/t/p/w500/8kSerJrhrJWKLk1LViesGcnrUPE.jpg", "release_date": 889056000})
+    );
+
+    let (document, code) = index.get_document(500, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 500, "title": "Reservoir Dogs", "overview": "A botched robbery indicates a police informant, and the pressure mounts in the aftermath at a warehouse. Crime begets violence as the survivors -- veteran Mr. White, newcomer Mr. Orange, psychopathic parolee Mr. Blonde, bickering weasel Mr. Pink and Nice Guy Eddie -- unravel.", "genres": ["Crime", "Thriller"], "poster": "https://image.tmdb.org/t/p/w500/AjTtJNumZyUDz33VtMlF1K8JPsE.jpg", "release_date": 715392000})
+    );
+
+    let (document, code) = index.get_document(10006, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 10006, "title": "Wild Seven", "overview": "In this darkly karmic vision of Arizona, a man who breathes nothing but ill will begins a noxious domino effect as quickly as an uncontrollable virus kills. As he exits Arizona State Penn after twenty-one long years, Wilson has only one thing on the brain, leveling the score with career criminal, Mackey Willis.", "genres": ["Action", "Crime", "Drama"], "poster": "https://image.tmdb.org/t/p/w500/y114dTPoqn8k2Txps4P2tI95YCS.jpg", "release_date": 1136073600})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v4_movie_with_settings() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::MoviesWithSettingsV4.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("indexUID"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("indexUID");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"genres": 53, "id": 53, "overview": 53, "poster": 53, "release_date": 53, "title": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({ "displayedAttributes": ["title", "genres", "overview", "poster", "release_date"], "searchableAttributes": ["title", "overview"], "filterableAttributes": ["genres"], "sortableAttributes": [], "rankingRules": ["words", "typo", "proximity", "attribute", "exactness"], "stopWords": ["of", "the"], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": { "oneTypo": 5, "twoTypos": 9 }, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 } })
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks,
+        json!({ "results": [{ "uid": 1, "indexUid": "indexUID", "status": "succeeded", "type": "settingsUpdate", "details": { "displayedAttributes": ["title", "genres", "overview", "poster", "release_date"], "searchableAttributes": ["title", "overview"], "filterableAttributes": ["genres"], "stopWords": ["of", "the"] }, "duration": "PT37.488777S", "enqueuedAt": "2021-09-08T08:24:02.323444Z", "startedAt": "2021-09-08T08:24:02.324145Z", "finishedAt": "2021-09-08T08:24:39.812922Z" }, { "uid": 0, "indexUid": "indexUID", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": { "receivedDocuments": 0, "indexedDocuments": 31944 }, "duration": "PT39.941318S", "enqueuedAt": "2021-09-08T08:21:14.742672Z", "startedAt": "2021-09-08T08:21:14.750166Z", "finishedAt": "2021-09-08T08:21:54.691484Z" }], "limit": 20, "from": 1, "next": null })
+    );
+
+    // finally we're just going to check that we can still get a few documents by id
+    let (document, code) = index.get_document(100, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 100, "title": "Lock, Stock and Two Smoking Barrels", "genres": ["Comedy", "Crime"], "overview": "A card shark and his unwillingly-enlisted friends need to make a lot of cash quick after losing a sketchy poker match. To do this they decide to pull a heist on a small-time gang who happen to be operating out of the flat next door.", "poster": "https://image.tmdb.org/t/p/w500/8kSerJrhrJWKLk1LViesGcnrUPE.jpg", "release_date": 889056000 })
+    );
+
+    let (document, code) = index.get_document(500, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 500, "title": "Reservoir Dogs", "genres": ["Crime", "Thriller"], "overview": "A botched robbery indicates a police informant, and the pressure mounts in the aftermath at a warehouse. Crime begets violence as the survivors -- veteran Mr. White, newcomer Mr. Orange, psychopathic parolee Mr. Blonde, bickering weasel Mr. Pink and Nice Guy Eddie -- unravel.", "poster": "https://image.tmdb.org/t/p/w500/AjTtJNumZyUDz33VtMlF1K8JPsE.jpg", "release_date": 715392000})
+    );
+
+    let (document, code) = index.get_document(10006, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "id": 10006, "title": "Wild Seven", "genres": ["Action", "Crime", "Drama"], "overview": "In this darkly karmic vision of Arizona, a man who breathes nothing but ill will begins a noxious domino effect as quickly as an uncontrollable virus kills. As he exits Arizona State Penn after twenty-one long years, Wilson has only one thing on the brain, leveling the score with career criminal, Mackey Willis.", "poster": "https://image.tmdb.org/t/p/w500/y114dTPoqn8k2Txps4P2tI95YCS.jpg", "release_date": 1136073600})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v4_rubygems_with_settings() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::RubyGemsWithSettingsV4.path()),
+        ..default_settings(temp.path())
+    };
+    let server = Server::new_with_options(options).await.unwrap();
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 1);
+    assert_eq!(indexes["results"][0]["uid"], json!("rubygems"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let index = server.index("rubygems");
+
+    let (stats, code) = index.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        stats,
+        json!({ "numberOfDocuments": 53, "isIndexing": false, "fieldDistribution": {"description": 53, "id": 53, "name": 53, "summary": 53, "total_downloads": 53, "version": 53 }})
+    );
+
+    let (settings, code) = index.settings().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        settings,
+        json!({ "displayedAttributes": ["name", "summary", "description", "version", "total_downloads"], "searchableAttributes": ["name", "summary"], "filterableAttributes": ["version"], "sortableAttributes": [], "rankingRules": ["typo", "words", "fame:desc", "proximity", "attribute", "exactness", "total_downloads:desc"], "stopWords": [], "synonyms": {}, "distinctAttribute": null, "typoTolerance": {"enabled": true, "minWordSizeForTypos": {"oneTypo": 5, "twoTypos": 9}, "disableOnWords": [], "disableOnAttributes": [] }, "faceting": { "maxValuesPerFacet": 100 }, "pagination": { "limitedTo": 1000 } })
+    );
+
+    let (tasks, code) = index.list_tasks().await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        tasks["results"][0],
+        json!({ "uid": 92, "indexUid": "rubygems", "status": "succeeded", "type": "documentAdditionOrUpdate", "details": {"receivedDocuments": 0, "indexedDocuments": 1042}, "duration": "PT14.034672S", "enqueuedAt": "2021-09-08T08:40:31.390775Z", "startedAt": "2021-09-08T08:51:39.060642Z", "finishedAt": "2021-09-08T08:51:53.095314Z"})
+    );
+
+    // finally we're just going to check that we can still get a few documents by id
+    let (document, code) = index.get_document(188040, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "meilisearch", "summary": "An easy-to-use ruby client for Meilisearch API", "description": "An easy-to-use ruby client for Meilisearch API. See https://github.com/meilisearch/MeiliSearch", "id": "188040", "version": "0.15.2", "total_downloads": "7465"})
+    );
+
+    let (document, code) = index.get_document(191940, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "doggo", "summary": "RSpec 3 formatter - documentation, with progress indication", "description": "Similar to \"rspec -f d\", but also indicates progress by showing the current test number and total test count on each line.", "id": "191940", "version": "1.1.0", "total_downloads": "9394"})
+    );
+
+    let (document, code) = index.get_document(159227, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(
+        document,
+        json!({ "name": "vortex-of-agony", "summary": "You dont need to use nodejs or go, just install this plugin. It will crash your application at random", "description": "You dont need to use nodejs or go, just install this plugin. It will crash your application at random", "id": "159227", "version": "0.1.0", "total_downloads": "1007"})
+    );
+}
+
+#[actix_rt::test]
+#[cfg_attr(target_os = "windows", ignore)]
+async fn import_dump_v5() {
+    let temp = tempfile::tempdir().unwrap();
+
+    let options = Opt {
+        import_dump: Some(GetDump::TestV5.path()),
+        ..default_settings(temp.path())
+    };
+    let mut server = Server::new_auth_with_options(options, temp).await;
+    server.use_api_key("MASTER_KEY");
+
+    let (indexes, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200, "{indexes}");
+
+    assert_eq!(indexes["results"].as_array().unwrap().len(), 2);
+    assert_eq!(indexes["results"][0]["uid"], json!("test"));
+    assert_eq!(indexes["results"][1]["uid"], json!("test2"));
+    assert_eq!(indexes["results"][0]["primaryKey"], json!("id"));
+
+    let expected_stats = json!({
+        "numberOfDocuments": 10,
+        "isIndexing": false,
+        "fieldDistribution": {
+            "cast": 10,
+            "director": 10,
+            "genres": 10,
+            "id": 10,
+            "overview": 10,
+            "popularity": 10,
+            "poster_path": 10,
+            "producer": 10,
+            "production_companies": 10,
+            "release_date": 10,
+            "tagline": 10,
+            "title": 10,
+            "vote_average": 10,
+            "vote_count": 10
+        }
+    });
+
+    let index1 = server.index("test");
+    let index2 = server.index("test2");
+
+    let (stats, code) = index1.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(stats, expected_stats);
+
+    let (docs, code) = index2
+        .get_all_documents(GetAllDocumentsOptions::default())
+        .await;
+    assert_eq!(code, 200);
+    assert_eq!(docs["results"].as_array().unwrap().len(), 10);
+    let (docs, code) = index1
+        .get_all_documents(GetAllDocumentsOptions::default())
+        .await;
+    assert_eq!(code, 200);
+    assert_eq!(docs["results"].as_array().unwrap().len(), 10);
+
+    let (stats, code) = index2.stats().await;
+    assert_eq!(code, 200);
+    assert_eq!(stats, expected_stats);
+
+    let (keys, code) = server.list_api_keys().await;
+    assert_eq!(code, 200);
+    let key = &keys["results"][0];
+
+    assert_eq!(key["name"], "my key");
+}

meilisearch-http/tests/index/create_index.rs

@@ -102,7 +102,7 @@ async fn error_create_with_invalid_index_uid() {
     let (response, code) = index.create(None).await;
 
     let expected_response = json!({
-        "message": "`test test#!` is not a valid index uid. Index uid can be an integer or a string containing only alphanumeric characters, hyphens (-) and underscores (_).",
+        "message": "invalid index uid `test test#!`, the uid must be an integer or a string containing only alphanumeric characters a-z A-Z 0-9, hyphens - and underscores _.",
         "code": "invalid_index_uid",
         "type": "invalid_request",
         "link": "https://docs.meilisearch.com/errors#invalid_index_uid"

meilisearch-http/tests/index/delete_index.rs

@@ -52,10 +52,10 @@ async fn loop_delete_add_documents() {
     let mut tasks = Vec::new();
     for _ in 0..50 {
         let (response, code) = index.add_documents(documents.clone(), None).await;
-        tasks.push(response["uid"].as_u64().unwrap());
+        tasks.push(response["taskUid"].as_u64().unwrap());
         assert_eq!(code, 202, "{}", response);
         let (response, code) = index.delete().await;
-        tasks.push(response["uid"].as_u64().unwrap());
+        tasks.push(response["taskUid"].as_u64().unwrap());
         assert_eq!(code, 202, "{}", response);
     }
 

meilisearch-http/tests/index/get_index.rs

@@ -16,12 +16,11 @@ async fn create_and_get_index() {
 
     assert_eq!(code, 200);
     assert_eq!(response["uid"], "test");
-    assert_eq!(response["name"], "test");
     assert!(response.get("createdAt").is_some());
     assert!(response.get("updatedAt").is_some());
     assert_eq!(response["createdAt"], response["updatedAt"]);
     assert_eq!(response["primaryKey"], Value::Null);
-    assert_eq!(response.as_object().unwrap().len(), 5);
+    assert_eq!(response.as_object().unwrap().len(), 4);
 }
 
 #[actix_rt::test]
@@ -45,10 +44,10 @@ async fn error_get_unexisting_index() {
 #[actix_rt::test]
 async fn no_index_return_empty_list() {
     let server = Server::new().await;
-    let (response, code) = server.list_indexes().await;
+    let (response, code) = server.list_indexes(None, None).await;
     assert_eq!(code, 200);
-    assert!(response.is_array());
-    assert!(response.as_array().unwrap().is_empty());
+    assert!(response["results"].is_array());
+    assert!(response["results"].as_array().unwrap().is_empty());
 }
 
 #[actix_rt::test]
@@ -59,10 +58,10 @@ async fn list_multiple_indexes() {
 
     server.index("test").wait_task(1).await;
 
-    let (response, code) = server.list_indexes().await;
+    let (response, code) = server.list_indexes(None, None).await;
     assert_eq!(code, 200);
-    assert!(response.is_array());
-    let arr = response.as_array().unwrap();
+    assert!(response["results"].is_array());
+    let arr = response["results"].as_array().unwrap();
     assert_eq!(arr.len(), 2);
     assert!(arr
         .iter()
@@ -72,6 +71,118 @@ async fn list_multiple_indexes() {
         .any(|entry| entry["uid"] == "test1" && entry["primaryKey"] == "key"));
 }
 
+#[actix_rt::test]
+async fn get_and_paginate_indexes() {
+    let server = Server::new().await;
+    const NB_INDEXES: usize = 50;
+    for i in 0..NB_INDEXES {
+        server.index(&format!("test_{i:02}")).create(None).await;
+        server
+            .index(&format!("test_{i:02}"))
+            .wait_task(i as u64)
+            .await;
+    }
+
+    // basic
+    let (response, code) = server.list_indexes(None, None).await;
+    assert_eq!(code, 200);
+    assert_eq!(response["limit"], json!(20));
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["total"], json!(NB_INDEXES));
+    assert!(response["results"].is_array());
+    let arr = response["results"].as_array().unwrap();
+    assert_eq!(arr.len(), 20);
+    // ensuring we get all the indexes in the alphabetical order
+    assert!((0..20)
+        .map(|idx| format!("test_{idx:02}"))
+        .zip(arr)
+        .all(|(expected, entry)| entry["uid"] == expected));
+
+    // with an offset
+    let (response, code) = server.list_indexes(Some(15), None).await;
+    assert_eq!(code, 200);
+    assert_eq!(response["limit"], json!(20));
+    assert_eq!(response["offset"], json!(15));
+    assert_eq!(response["total"], json!(NB_INDEXES));
+    assert!(response["results"].is_array());
+    let arr = response["results"].as_array().unwrap();
+    assert_eq!(arr.len(), 20);
+    assert!((15..35)
+        .map(|idx| format!("test_{idx:02}"))
+        .zip(arr)
+        .all(|(expected, entry)| entry["uid"] == expected));
+
+    // with an offset and not enough elements
+    let (response, code) = server.list_indexes(Some(45), None).await;
+    assert_eq!(code, 200);
+    assert_eq!(response["limit"], json!(20));
+    assert_eq!(response["offset"], json!(45));
+    assert_eq!(response["total"], json!(NB_INDEXES));
+    assert!(response["results"].is_array());
+    let arr = response["results"].as_array().unwrap();
+    assert_eq!(arr.len(), 5);
+    assert!((45..50)
+        .map(|idx| format!("test_{idx:02}"))
+        .zip(arr)
+        .all(|(expected, entry)| entry["uid"] == expected));
+
+    // with a limit lower than the default
+    let (response, code) = server.list_indexes(None, Some(5)).await;
+    assert_eq!(code, 200);
+    assert_eq!(response["limit"], json!(5));
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["total"], json!(NB_INDEXES));
+    assert!(response["results"].is_array());
+    let arr = response["results"].as_array().unwrap();
+    assert_eq!(arr.len(), 5);
+    assert!((0..5)
+        .map(|idx| format!("test_{idx:02}"))
+        .zip(arr)
+        .all(|(expected, entry)| entry["uid"] == expected));
+
+    // with a limit higher than the default
+    let (response, code) = server.list_indexes(None, Some(40)).await;
+    assert_eq!(code, 200);
+    assert_eq!(response["limit"], json!(40));
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["total"], json!(NB_INDEXES));
+    assert!(response["results"].is_array());
+    let arr = response["results"].as_array().unwrap();
+    assert_eq!(arr.len(), 40);
+    assert!((0..40)
+        .map(|idx| format!("test_{idx:02}"))
+        .zip(arr)
+        .all(|(expected, entry)| entry["uid"] == expected));
+
+    // with a limit higher than the default
+    let (response, code) = server.list_indexes(None, Some(80)).await;
+    assert_eq!(code, 200);
+    assert_eq!(response["limit"], json!(80));
+    assert_eq!(response["offset"], json!(0));
+    assert_eq!(response["total"], json!(NB_INDEXES));
+    assert!(response["results"].is_array());
+    let arr = response["results"].as_array().unwrap();
+    assert_eq!(arr.len(), 50);
+    assert!((0..50)
+        .map(|idx| format!("test_{idx:02}"))
+        .zip(arr)
+        .all(|(expected, entry)| entry["uid"] == expected));
+
+    // with a limit and an offset
+    let (response, code) = server.list_indexes(Some(20), Some(10)).await;
+    assert_eq!(code, 200);
+    assert_eq!(response["limit"], json!(10));
+    assert_eq!(response["offset"], json!(20));
+    assert_eq!(response["total"], json!(NB_INDEXES));
+    assert!(response["results"].is_array());
+    let arr = response["results"].as_array().unwrap();
+    assert_eq!(arr.len(), 10);
+    assert!((20..30)
+        .map(|idx| format!("test_{idx:02}"))
+        .zip(arr)
+        .all(|(expected, entry)| entry["uid"] == expected));
+}
+
 #[actix_rt::test]
 async fn get_invalid_index_uid() {
     let server = Server::new().await;

meilisearch-http/tests/index/stats.rs

@@ -35,7 +35,7 @@ async fn stats() {
 
     let (response, code) = index.add_documents(documents, None).await;
     assert_eq!(code, 202);
-    assert_eq!(response["uid"], 1);
+    assert_eq!(response["taskUid"], 1);
 
     index.wait_task(1).await;
 

meilisearch-http/tests/index/update_index.rs

@@ -21,7 +21,6 @@ async fn update_primary_key() {
     assert_eq!(code, 200);
 
     assert_eq!(response["uid"], "test");
-    assert_eq!(response["name"], "test");
     assert!(response.get("createdAt").is_some());
     assert!(response.get("updatedAt").is_some());
 
@@ -32,7 +31,7 @@ async fn update_primary_key() {
     assert!(created_at < updated_at);
 
     assert_eq!(response["primaryKey"], "primary");
-    assert_eq!(response.as_object().unwrap().len(), 5);
+    assert_eq!(response.as_object().unwrap().len(), 4);
 }
 
 #[actix_rt::test]

meilisearch-http/tests/integration.rs

@@ -2,6 +2,7 @@ mod auth;
 mod common;
 mod dashboard;
 mod documents;
+mod dumps;
 mod index;
 mod search;
 mod settings;

meilisearch-http/tests/search/errors.rs

@@ -36,6 +36,38 @@ async fn search_unexisting_parameter() {
         .await;
 }
 
+#[actix_rt::test]
+async fn search_invalid_highlight_and_crop_tags() {
+    let server = Server::new().await;
+    let index = server.index("test");
+
+    let fields = &["cropMarker", "highlightPreTag", "highlightPostTag"];
+
+    for field in fields {
+        // object
+        index
+            .search(
+                json!({field.to_string(): {"marker": "<crop>"}}),
+                |response, code| {
+                    assert_eq!(code, 400, "field {} passing object: {}", &field, response);
+                    assert_eq!(response["code"], "bad_request");
+                },
+            )
+            .await;
+
+        // array
+        index
+            .search(
+                json!({field.to_string(): ["marker", "<crop>"]}),
+                |response, code| {
+                    assert_eq!(code, 400, "field {} passing array: {}", &field, response);
+                    assert_eq!(response["code"], "bad_request");
+                },
+            )
+            .await;
+    }
+}
+
 #[actix_rt::test]
 async fn filter_invalid_syntax_object() {
     let server = Server::new().await;
@@ -50,7 +82,7 @@ async fn filter_invalid_syntax_object() {
     index.wait_task(1).await;
 
     let expected_response = json!({
-        "message": "Was expecting an operation `=`, `!=`, `>=`, `>`, `<=`, `<`, `TO` or `_geoRadius` at `title & Glass`.\n1:14 title & Glass",
+        "message": "Was expecting an operation `=`, `!=`, `>=`, `>`, `<=`, `<`, `TO`, `EXISTS`, `NOT EXISTS`, or `_geoRadius` at `title & Glass`.\n1:14 title & Glass",
         "code": "invalid_filter",
         "type": "invalid_request",
         "link": "https://docs.meilisearch.com/errors#invalid_filter"
@@ -77,7 +109,7 @@ async fn filter_invalid_syntax_array() {
     index.wait_task(1).await;
 
     let expected_response = json!({
-        "message": "Was expecting an operation `=`, `!=`, `>=`, `>`, `<=`, `<`, `TO` or `_geoRadius` at `title & Glass`.\n1:14 title & Glass",
+        "message": "Was expecting an operation `=`, `!=`, `>=`, `>`, `<=`, `<`, `TO`, `EXISTS`, `NOT EXISTS`, or `_geoRadius` at `title & Glass`.\n1:14 title & Glass",
         "code": "invalid_filter",
         "type": "invalid_request",
         "link": "https://docs.meilisearch.com/errors#invalid_filter"

meilisearch-http/tests/search/formatted.rs

@@ -0,0 +1,399 @@
+use super::*;
+use crate::common::Server;
+use serde_json::json;
+
+#[actix_rt::test]
+async fn formatted_contain_wildcard() {
+    let server = Server::new().await;
+    let index = server.index("test");
+
+    index
+        .update_settings(json!({ "displayedAttributes": ["id", "cattos"] }))
+        .await;
+
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(1).await;
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToRetrieve": ["father", "mother"], "attributesToHighlight": ["father", "mother", "*"], "attributesToCrop": ["doggos"], "showMatchesPosition": true }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "_formatted": {
+                "id": "852",
+                "cattos": "<em>pesti</em>",
+            },
+            "_matchesPosition": {"cattos": [{"start": 0, "length": 5}]},
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToRetrieve": ["*"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+            "cattos": "pesti",
+        })
+    );
+
+    let (response, code) = index
+        .search_post(
+            json!({ "q": "pesti", "attributesToRetrieve": ["*"], "attributesToHighlight": ["id"], "showMatchesPosition": true }),
+        )
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+            "cattos": "pesti",
+            "_formatted": {
+                "id": "852",
+                "cattos": "pesti",
+            },
+            "_matchesPosition": {"cattos": [{"start": 0, "length": 5}]},
+        })
+    );
+
+    let (response, code) = index
+        .search_post(
+            json!({ "q": "pesti", "attributesToRetrieve": ["*"], "attributesToCrop": ["*"] }),
+        )
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+            "cattos": "pesti",
+            "_formatted": {
+                "id": "852",
+                "cattos": "pesti",
+            }
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToCrop": ["*"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+            "cattos": "pesti",
+            "_formatted": {
+                "id": "852",
+                "cattos": "pesti",
+            }
+        })
+    );
+}
+
+#[actix_rt::test]
+async fn format_nested() {
+    let server = Server::new().await;
+    let index = server.index("test");
+
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(0).await;
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToRetrieve": ["doggos"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "doggos": [
+                {
+                    "name": "bobby",
+                    "age": 2,
+                },
+                {
+                    "name": "buddy",
+                    "age": 4,
+                },
+            ],
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToRetrieve": ["doggos.name"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "doggos": [
+                {
+                    "name": "bobby",
+                },
+                {
+                    "name": "buddy",
+                },
+            ],
+        })
+    );
+
+    let (response, code) = index
+        .search_post(
+            json!({ "q": "bobby", "attributesToRetrieve": ["doggos.name"], "showMatchesPosition": true }),
+        )
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "doggos": [
+                {
+                    "name": "bobby",
+                },
+                {
+                    "name": "buddy",
+                },
+            ],
+            "_matchesPosition": {"doggos.name": [{"start": 0, "length": 5}]},
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToRetrieve": [], "attributesToHighlight": ["doggos.name"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "_formatted": {
+                "doggos": [
+                    {
+                        "name": "bobby",
+                    },
+                    {
+                        "name": "buddy",
+                    },
+                ],
+            },
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToRetrieve": [], "attributesToCrop": ["doggos.name"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "_formatted": {
+                "doggos": [
+                    {
+                        "name": "bobby",
+                    },
+                    {
+                        "name": "buddy",
+                    },
+                ],
+            },
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToRetrieve": ["doggos.name"], "attributesToHighlight": ["doggos.age"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "doggos": [
+                {
+                    "name": "bobby",
+                },
+                {
+                    "name": "buddy",
+                },
+            ],
+            "_formatted": {
+                "doggos": [
+                    {
+                        "name": "bobby",
+                        "age": "2",
+                    },
+                    {
+                        "name": "buddy",
+                        "age": "4",
+                    },
+                ],
+            },
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "q": "pesti", "attributesToRetrieve": [], "attributesToHighlight": ["doggos.age"], "attributesToCrop": ["doggos.name"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "_formatted": {
+                "doggos": [
+                    {
+                        "name": "bobby",
+                        "age": "2",
+                    },
+                    {
+                        "name": "buddy",
+                        "age": "4",
+                    },
+                ],
+            },
+        })
+    );
+}
+
+#[actix_rt::test]
+async fn displayedattr_2_smol() {
+    let server = Server::new().await;
+    let index = server.index("test");
+
+    // not enough displayed for the other settings
+    index
+        .update_settings(json!({ "displayedAttributes": ["id"] }))
+        .await;
+
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(1).await;
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToRetrieve": ["father", "id"], "attributesToHighlight": ["mother"], "attributesToCrop": ["cattos"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToRetrieve": ["id"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToHighlight": ["id"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+            "_formatted": {
+                "id": "852",
+            }
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToCrop": ["id"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+            "_formatted": {
+                "id": "852",
+            }
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToHighlight": ["id"], "attributesToCrop": ["id"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+            "_formatted": {
+                "id": "852",
+            }
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToHighlight": ["cattos"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToCrop": ["cattos"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "id": 852,
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToRetrieve": ["cattos"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(response["hits"][0], json!({}));
+
+    let (response, code) = index
+        .search_post(
+            json!({ "attributesToRetrieve": ["cattos"], "attributesToHighlight": ["cattos"], "attributesToCrop": ["cattos"] }),
+        )
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(response["hits"][0], json!({}));
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToRetrieve": ["cattos"], "attributesToHighlight": ["id"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "_formatted": {
+                "id": "852",
+            }
+        })
+    );
+
+    let (response, code) = index
+        .search_post(json!({ "attributesToRetrieve": ["cattos"], "attributesToCrop": ["id"] }))
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(
+        response["hits"][0],
+        json!({
+            "_formatted": {
+                "id": "852",
+            }
+        })
+    );
+}

meilisearch-http/tests/search/mod.rs

@@ -2,38 +2,96 @@
 // should be tested in its own module to isolate tests and keep the tests readable.
 
 mod errors;
+mod formatted;
 
 use crate::common::Server;
 use once_cell::sync::Lazy;
 use serde_json::{json, Value};
 
-static DOCUMENTS: Lazy<Value> = Lazy::new(|| {
+pub(self) static DOCUMENTS: Lazy<Value> = Lazy::new(|| {
     json!([
         {
             "title": "Shazam!",
-            "id": "287947"
+            "id": "287947",
         },
         {
             "title": "Captain Marvel",
-            "id": "299537"
+            "id": "299537",
         },
         {
             "title": "Escape Room",
-            "id": "522681"
+            "id": "522681",
         },
-        { "title": "How to Train Your Dragon: The Hidden World", "id": "166428"
+        {
+            "title": "How to Train Your Dragon: The Hidden World",
+            "id": "166428",
         },
         {
             "title": "Glass",
-            "id": "450465"
+            "id": "450465",
         }
     ])
 });
 
+pub(self) static NESTED_DOCUMENTS: Lazy<Value> = Lazy::new(|| {
+    json!([
+        {
+            "id": 852,
+            "father": "jean",
+            "mother": "michelle",
+            "doggos": [
+                {
+                    "name": "bobby",
+                    "age": 2,
+                },
+                {
+                    "name": "buddy",
+                    "age": 4,
+                },
+            ],
+            "cattos": "pesti",
+        },
+        {
+            "id": 654,
+            "father": "pierre",
+            "mother": "sabine",
+            "doggos": [
+                {
+                    "name": "gros bill",
+                    "age": 8,
+                },
+            ],
+            "cattos": ["simba", "pestiféré"],
+        },
+        {
+            "id": 750,
+            "father": "romain",
+            "mother": "michelle",
+            "cattos": ["enigma"],
+        },
+        {
+            "id": 951,
+            "father": "jean-baptiste",
+            "mother": "sophie",
+            "doggos": [
+                {
+                    "name": "turbo",
+                    "age": 5,
+                },
+                {
+                    "name": "fast",
+                    "age": 6,
+                },
+            ],
+            "cattos": ["moumoute", "gomez"],
+        },
+    ])
+});
+
 #[actix_rt::test]
 async fn simple_placeholder_search() {
     let server = Server::new().await;
-    let index = server.index("test");
+    let index = server.index("basic");
 
     let documents = DOCUMENTS.clone();
     index.add_documents(documents, None).await;
@@ -45,6 +103,18 @@ async fn simple_placeholder_search() {
             assert_eq!(response["hits"].as_array().unwrap().len(), 5);
         })
         .await;
+
+    let index = server.index("nested");
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(1).await;
+
+    index
+        .search(json!({}), |response, code| {
+            assert_eq!(code, 200, "{}", response);
+            assert_eq!(response["hits"].as_array().unwrap().len(), 4);
+        })
+        .await;
 }
 
 #[actix_rt::test]
@@ -62,6 +132,18 @@ async fn simple_search() {
             assert_eq!(response["hits"].as_array().unwrap().len(), 1);
         })
         .await;
+
+    let index = server.index("nested");
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(1).await;
+
+    index
+        .search(json!({"q": "pesti"}), |response, code| {
+            assert_eq!(code, 200, "{}", response);
+            assert_eq!(response["hits"].as_array().unwrap().len(), 2);
+        })
+        .await;
 }
 
 #[actix_rt::test]
@@ -88,6 +170,27 @@ async fn search_multiple_params() {
             },
         )
         .await;
+
+    let index = server.index("nested");
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(1).await;
+
+    index
+        .search(
+            json!({
+                "q": "pesti",
+                "attributesToCrop": ["catto:2"],
+                "attributesToHighlight": ["catto"],
+                "limit": 2,
+                "offset": 0,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 2);
+            },
+        )
+        .await;
 }
 
 #[actix_rt::test]
@@ -114,6 +217,43 @@ async fn search_with_filter_string_notation() {
             },
         )
         .await;
+
+    let index = server.index("nested");
+
+    index
+        .update_settings(json!({"filterableAttributes": ["cattos", "doggos.age"]}))
+        .await;
+
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(3).await;
+
+    index
+        .search(
+            json!({
+                "filter": "cattos = pesti"
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 1);
+                assert_eq!(response["hits"][0]["id"], json!(852));
+            },
+        )
+        .await;
+
+    index
+        .search(
+            json!({
+                "filter": "doggos.age > 5"
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 2);
+                assert_eq!(response["hits"][0]["id"], json!(654));
+                assert_eq!(response["hits"][1]["id"], json!(951));
+            },
+        )
+        .await;
 }
 
 #[actix_rt::test]
@@ -170,6 +310,28 @@ async fn search_with_sort_on_numbers() {
             },
         )
         .await;
+
+    let index = server.index("nested");
+
+    index
+        .update_settings(json!({"sortableAttributes": ["doggos.age"]}))
+        .await;
+
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(3).await;
+
+    index
+        .search(
+            json!({
+                "sort": ["doggos.age:asc"]
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 4);
+            },
+        )
+        .await;
 }
 
 #[actix_rt::test]
@@ -196,6 +358,28 @@ async fn search_with_sort_on_strings() {
             },
         )
         .await;
+
+    let index = server.index("nested");
+
+    index
+        .update_settings(json!({"sortableAttributes": ["doggos.name"]}))
+        .await;
+
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(3).await;
+
+    index
+        .search(
+            json!({
+                "sort": ["doggos.name:asc"]
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 4);
+            },
+        )
+        .await;
 }
 
 #[actix_rt::test]
@@ -236,16 +420,94 @@ async fn search_facet_distribution() {
     index
         .search(
             json!({
-                "facetsDistribution": ["title"]
+                "facets": ["title"]
             }),
             |response, code| {
                 assert_eq!(code, 200, "{}", response);
-                let dist = response["facetsDistribution"].as_object().unwrap();
+                let dist = response["facetDistribution"].as_object().unwrap();
                 assert_eq!(dist.len(), 1);
                 assert!(dist.get("title").is_some());
             },
         )
         .await;
+
+    let index = server.index("nested");
+
+    index
+        .update_settings(json!({"filterableAttributes": ["father", "doggos.name"]}))
+        .await;
+
+    let documents = NESTED_DOCUMENTS.clone();
+    index.add_documents(documents, None).await;
+    index.wait_task(3).await;
+
+    // TODO: TAMO: fix the test
+    index
+        .search(
+            json!({
+                // "facets": ["father", "doggos.name"]
+                "facets": ["father"]
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                let dist = response["facetDistribution"].as_object().unwrap();
+                assert_eq!(dist.len(), 1);
+                assert_eq!(
+                    dist["father"],
+                    json!({ "jean": 1, "pierre": 1, "romain": 1, "jean-baptiste": 1})
+                );
+                /*
+                assert_eq!(
+                    dist["doggos.name"],
+                    json!({ "bobby": 1, "buddy": 1, "gros bill": 1, "turbo": 1, "fast": 1})
+                );
+                */
+            },
+        )
+        .await;
+
+    index
+        .update_settings(json!({"filterableAttributes": ["doggos"]}))
+        .await;
+    index.wait_task(4).await;
+
+    index
+        .search(
+            json!({
+                "facets": ["doggos.name"]
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                let dist = response["facetDistribution"].as_object().unwrap();
+                assert_eq!(dist.len(), 1);
+                assert_eq!(
+                    dist["doggos.name"],
+                    json!({ "bobby": 1, "buddy": 1, "gros bill": 1, "turbo": 1, "fast": 1})
+                );
+            },
+        )
+        .await;
+
+    index
+        .search(
+            json!({
+                "facets": ["doggos"]
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                let dist = response["facetDistribution"].as_object().unwrap();
+                assert_eq!(dist.len(), 3);
+                assert_eq!(
+                    dist["doggos.name"],
+                    json!({ "bobby": 1, "buddy": 1, "gros bill": 1, "turbo": 1, "fast": 1})
+                );
+                assert_eq!(
+                    dist["doggos.age"],
+                    json!({ "2": 1, "4": 1, "5": 1, "6": 1, "8": 1})
+                );
+            },
+        )
+        .await;
 }
 
 #[actix_rt::test]
@@ -265,5 +527,192 @@ async fn displayed_attributes() {
         .search_post(json!({ "attributesToRetrieve": ["title", "id"] }))
         .await;
     assert_eq!(code, 200, "{}", response);
-    assert!(response["hits"].get("title").is_none());
+    assert!(response["hits"][0].get("title").is_some());
+}
+
+#[actix_rt::test]
+async fn placeholder_search_is_hard_limited() {
+    let server = Server::new().await;
+    let index = server.index("test");
+
+    let documents: Vec<_> = (0..1200)
+        .map(|i| json!({ "id": i, "text": "I am unique!" }))
+        .collect();
+    index.add_documents(documents.into(), None).await;
+    index.wait_task(0).await;
+
+    index
+        .search(
+            json!({
+                "limit": 1500,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 1000);
+            },
+        )
+        .await;
+
+    index
+        .search(
+            json!({
+                "offset": 800,
+                "limit": 400,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 200);
+            },
+        )
+        .await;
+
+    index
+        .update_settings(json!({ "pagination": { "limitedTo": 10_000 } }))
+        .await;
+    index.wait_task(1).await;
+
+    index
+        .search(
+            json!({
+                "limit": 1500,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 1200);
+            },
+        )
+        .await;
+
+    index
+        .search(
+            json!({
+                "offset": 1000,
+                "limit": 400,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 200);
+            },
+        )
+        .await;
+}
+
+#[actix_rt::test]
+async fn search_is_hard_limited() {
+    let server = Server::new().await;
+    let index = server.index("test");
+
+    let documents: Vec<_> = (0..1200)
+        .map(|i| json!({ "id": i, "text": "I am unique!" }))
+        .collect();
+    index.add_documents(documents.into(), None).await;
+    index.wait_task(0).await;
+
+    index
+        .search(
+            json!({
+                "q": "unique",
+                "limit": 1500,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 1000);
+            },
+        )
+        .await;
+
+    index
+        .search(
+            json!({
+                "q": "unique",
+                "offset": 800,
+                "limit": 400,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 200);
+            },
+        )
+        .await;
+
+    index
+        .update_settings(json!({ "pagination": { "limitedTo": 10_000 } }))
+        .await;
+    index.wait_task(1).await;
+
+    index
+        .search(
+            json!({
+                "q": "unique",
+                "limit": 1500,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 1200);
+            },
+        )
+        .await;
+
+    index
+        .search(
+            json!({
+                "q": "unique",
+                "offset": 1000,
+                "limit": 400,
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                assert_eq!(response["hits"].as_array().unwrap().len(), 200);
+            },
+        )
+        .await;
+}
+
+#[actix_rt::test]
+async fn faceting_max_values_per_facet() {
+    let server = Server::new().await;
+    let index = server.index("test");
+
+    index
+        .update_settings(json!({ "filterableAttributes": ["number"] }))
+        .await;
+
+    let documents: Vec<_> = (0..10_000)
+        .map(|id| json!({ "id": id, "number": id * 10 }))
+        .collect();
+    index.add_documents(json!(documents), None).await;
+    index.wait_task(1).await;
+
+    index
+        .search(
+            json!({
+                "facets": ["number"]
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                let numbers = response["facetDistribution"]["number"].as_object().unwrap();
+                assert_eq!(numbers.len(), 100);
+            },
+        )
+        .await;
+
+    index
+        .update_settings(json!({ "faceting": { "maxValuesPerFacet": 10_000 } }))
+        .await;
+    index.wait_task(2).await;
+
+    index
+        .search(
+            json!({
+                "facets": ["number"]
+            }),
+            |response, code| {
+                assert_eq!(code, 200, "{}", response);
+                let numbers = dbg!(&response)["facetDistribution"]["number"]
+                    .as_object()
+                    .unwrap();
+                assert_eq!(numbers.len(), 10_000);
+            },
+        )
+        .await;
 }

meilisearch-http/tests/settings/get_settings.rs

@@ -24,6 +24,12 @@ static DEFAULT_SETTINGS_VALUES: Lazy<HashMap<&'static str, Value>> = Lazy::new(|
     );
     map.insert("stop_words", json!([]));
     map.insert("synonyms", json!({}));
+    map.insert(
+        "faceting",
+        json!({
+            "maxValuesByFacet": json!(100),
+        }),
+    );
     map
 });
 
@@ -43,7 +49,7 @@ async fn get_settings() {
     let (response, code) = index.settings().await;
     assert_eq!(code, 200);
     let settings = response.as_object().unwrap();
-    assert_eq!(settings.keys().len(), 8);
+    assert_eq!(settings.keys().len(), 11);
     assert_eq!(settings["displayedAttributes"], json!(["*"]));
     assert_eq!(settings["searchableAttributes"], json!(["*"]));
     assert_eq!(settings["filterableAttributes"], json!([]));
@@ -61,6 +67,18 @@ async fn get_settings() {
         ])
     );
     assert_eq!(settings["stopWords"], json!([]));
+    assert_eq!(
+        settings["faceting"],
+        json!({
+            "maxValuesPerFacet": 100,
+        })
+    );
+    assert_eq!(
+        settings["pagination"],
+        json!({
+            "limitedTo": 1000,
+        })
+    );
 }
 
 #[actix_rt::test]
@@ -122,7 +140,7 @@ async fn reset_all_settings() {
 
     let (response, code) = index.add_documents(documents, None).await;
     assert_eq!(code, 202);
-    assert_eq!(response["uid"], 0);
+    assert_eq!(response["taskUid"], 0);
     index.wait_task(0).await;
 
     index
@@ -179,7 +197,7 @@ async fn error_update_setting_unexisting_index_invalid_uid() {
     assert_eq!(code, 400);
 
     let expected = json!({
-        "message": "`test##!  ` is not a valid index uid. Index uid can be an integer or a string containing only alphanumeric characters, hyphens (-) and underscores (_).",
+        "message": "invalid index uid `test##!  `, the uid must be an integer or a string containing only alphanumeric characters a-z A-Z 0-9, hyphens - and underscores _.",
         "code": "invalid_index_uid",
         "type": "invalid_request",
         "link": "https://docs.meilisearch.com/errors#invalid_index_uid"});
@@ -214,7 +232,7 @@ macro_rules! test_setting_routes {
                         .chars()
                         .map(|c| if c == '_' { '-' } else { c })
                         .collect::<String>());
-                    let (response, code) = server.service.post(url, serde_json::Value::Null).await;
+                    let (response, code) = server.service.put(url, serde_json::Value::Null).await;
                     assert_eq!(code, 202, "{}", response);
                     server.index("").wait_task(0).await;
                     let (response, code) = server.index("test").get().await;
@@ -283,7 +301,7 @@ async fn error_set_invalid_ranking_rules() {
     assert_eq!(response["status"], "failed");
 
     let expected_error = json!({
-        "message": r#"`manyTheFish` ranking rule is invalid. Valid ranking rules are Words, Typo, Sort, Proximity, Attribute, Exactness and custom ranking rules."#,
+        "message": r#"`manyTheFish` ranking rule is invalid. Valid ranking rules are words, typo, sort, proximity, attribute, exactness and custom ranking rules."#,
         "code": "invalid_ranking_rule",
         "type": "invalid_request",
         "link": "https://docs.meilisearch.com/errors#invalid_ranking_rule"

meilisearch-http/tests/snapshot/mod.rs

@@ -41,7 +41,7 @@ async fn perform_snapshot() {
         ..default_settings(temp.path())
     };
 
-    let server = Server::new_with_options(options).await;
+    let server = Server::new_with_options(options).await.unwrap();
 
     let index = server.index("test");
     index
@@ -67,10 +67,10 @@ async fn perform_snapshot() {
         ..default_settings(temp.path())
     };
 
-    let snapshot_server = Server::new_with_options(options).await;
+    let snapshot_server = Server::new_with_options(options).await.unwrap();
 
     verify_snapshot!(server, snapshot_server, |server| =>
-        server.list_indexes(),
+        server.list_indexes(None, None),
         // for some reason the db sizes differ. this may be due to the compaction options we have
         // set when performing the snapshot
         //server.stats(),

meilisearch-http/tests/stats/mod.rs

@@ -54,7 +54,7 @@ async fn stats() {
 
     let (response, code) = index.add_documents(documents, None).await;
     assert_eq!(code, 202, "{}", response);
-    assert_eq!(response["uid"], 1);
+    assert_eq!(response["taskUid"], 1);
 
     index.wait_task(1).await;
 

meilisearch-http/tests/tasks/mod.rs

@@ -3,22 +3,6 @@ use serde_json::json;
 use time::format_description::well_known::Rfc3339;
 use time::OffsetDateTime;
 
-#[actix_rt::test]
-async fn error_get_task_unexisting_index() {
-    let server = Server::new().await;
-    let (response, code) = server.service.get("/indexes/test/tasks").await;
-
-    let expected_response = json!({
-        "message": "Index `test` not found.",
-        "code": "index_not_found",
-        "type": "invalid_request",
-        "link": "https://docs.meilisearch.com/errors#index_not_found"
-    });
-
-    assert_eq!(response, expected_response);
-    assert_eq!(code, 404);
-}
-
 #[actix_rt::test]
 async fn error_get_unexisting_task_status() {
     let server = Server::new().await;
@@ -58,22 +42,6 @@ async fn get_task_status() {
     // TODO check resonse format, as per #48
 }
 
-#[actix_rt::test]
-async fn error_list_tasks_unexisting_index() {
-    let server = Server::new().await;
-    let (response, code) = server.index("test").list_tasks().await;
-
-    let expected_response = json!({
-        "message": "Index `test` not found.",
-        "code": "index_not_found",
-        "type": "invalid_request",
-        "link": "https://docs.meilisearch.com/errors#index_not_found"
-    });
-
-    assert_eq!(response, expected_response);
-    assert_eq!(code, 404);
-}
-
 #[actix_rt::test]
 async fn list_tasks() {
     let server = Server::new().await;
@@ -91,10 +59,140 @@ async fn list_tasks() {
     assert_eq!(response["results"].as_array().unwrap().len(), 2);
 }
 
+#[actix_rt::test]
+async fn list_tasks_with_star_filters() {
+    let server = Server::new().await;
+    let index = server.index("test");
+    index.create(None).await;
+    index.wait_task(0).await;
+    index
+        .add_documents(
+            serde_json::from_str(include_str!("../assets/test_set.json")).unwrap(),
+            None,
+        )
+        .await;
+    let (response, code) = index.service.get("/tasks?indexUid=test").await;
+    assert_eq!(code, 200);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+
+    let (response, code) = index.service.get("/tasks?indexUid=*").await;
+    assert_eq!(code, 200);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+
+    let (response, code) = index.service.get("/tasks?indexUid=*,pasteque").await;
+    assert_eq!(code, 200);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+
+    let (response, code) = index.service.get("/tasks?type=*").await;
+    assert_eq!(code, 200);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+
+    let (response, code) = index
+        .service
+        .get("/tasks?type=*,documentAdditionOrUpdate&status=*")
+        .await;
+    assert_eq!(code, 200, "{:?}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+
+    let (response, code) = index
+        .service
+        .get("/tasks?type=*,documentAdditionOrUpdate&status=*,failed&indexUid=test")
+        .await;
+    assert_eq!(code, 200, "{:?}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+
+    let (response, code) = index
+        .service
+        .get("/tasks?type=*,documentAdditionOrUpdate&status=*,failed&indexUid=test,*")
+        .await;
+    assert_eq!(code, 200, "{:?}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+}
+
+#[actix_rt::test]
+async fn list_tasks_status_filtered() {
+    let server = Server::new().await;
+    let index = server.index("test");
+    index.create(None).await;
+    index.wait_task(0).await;
+    index
+        .add_documents(
+            serde_json::from_str(include_str!("../assets/test_set.json")).unwrap(),
+            None,
+        )
+        .await;
+
+    let (response, code) = index.filtered_tasks(&[], &["succeeded"]).await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 1);
+
+    // We can't be sure that the update isn't already processed so we can't test this
+    // let (response, code) = index.filtered_tasks(&[], &["processing"]).await;
+    // assert_eq!(code, 200, "{}", response);
+    // assert_eq!(response["results"].as_array().unwrap().len(), 1);
+
+    index.wait_task(1).await;
+
+    let (response, code) = index.filtered_tasks(&[], &["succeeded"]).await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+}
+
+#[actix_rt::test]
+async fn list_tasks_type_filtered() {
+    let server = Server::new().await;
+    let index = server.index("test");
+    index.create(None).await;
+    index.wait_task(0).await;
+    index
+        .add_documents(
+            serde_json::from_str(include_str!("../assets/test_set.json")).unwrap(),
+            None,
+        )
+        .await;
+
+    let (response, code) = index.filtered_tasks(&["indexCreation"], &[]).await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 1);
+
+    let (response, code) = index
+        .filtered_tasks(&["indexCreation", "documentAdditionOrUpdate"], &[])
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+}
+
+#[actix_rt::test]
+async fn list_tasks_status_and_type_filtered() {
+    let server = Server::new().await;
+    let index = server.index("test");
+    index.create(None).await;
+    index.wait_task(0).await;
+    index
+        .add_documents(
+            serde_json::from_str(include_str!("../assets/test_set.json")).unwrap(),
+            None,
+        )
+        .await;
+
+    let (response, code) = index.filtered_tasks(&["indexCreation"], &["failed"]).await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 0);
+
+    let (response, code) = index
+        .filtered_tasks(
+            &["indexCreation", "documentAdditionOrUpdate"],
+            &["succeeded", "processing", "enqueued"],
+        )
+        .await;
+    assert_eq!(code, 200, "{}", response);
+    assert_eq!(response["results"].as_array().unwrap().len(), 2);
+}
+
 macro_rules! assert_valid_summarized_task {
     ($response:expr, $task_type:literal, $index:literal) => {{
         assert_eq!($response.as_object().unwrap().len(), 5);
-        assert!($response["uid"].as_u64().is_some());
+        assert!($response["taskUid"].as_u64().is_some());
         assert_eq!($response["indexUid"], $index);
         assert_eq!($response["status"], "enqueued");
         assert_eq!($response["type"], $task_type);
@@ -119,16 +217,16 @@ async fn test_summarized_task_view() {
     assert_valid_summarized_task!(response, "settingsUpdate", "test");
 
     let (response, _) = index.update_documents(json!([{"id": 1}]), None).await;
-    assert_valid_summarized_task!(response, "documentPartial", "test");
+    assert_valid_summarized_task!(response, "documentAdditionOrUpdate", "test");
 
     let (response, _) = index.add_documents(json!([{"id": 1}]), None).await;
-    assert_valid_summarized_task!(response, "documentAddition", "test");
+    assert_valid_summarized_task!(response, "documentAdditionOrUpdate", "test");
 
     let (response, _) = index.delete_document(1).await;
     assert_valid_summarized_task!(response, "documentDeletion", "test");
 
     let (response, _) = index.clear_all_documents().await;
-    assert_valid_summarized_task!(response, "clearAll", "test");
+    assert_valid_summarized_task!(response, "documentDeletion", "test");
 
     let (response, _) = index.delete().await;
     assert_valid_summarized_task!(response, "indexDeletion", "test");

meilisearch-lib/Cargo.toml

@@ -1,67 +1,67 @@
 [package]
 name = "meilisearch-lib"
-version = "0.26.0"
+version = "0.28.0"
 edition = "2021"
 
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
 [dependencies]
-actix-web = { version = "4", default-features = false }
-anyhow = { version = "1.0.43", features = ["backtrace"] }
-async-stream = "0.3.2"
-async-trait = "0.1.51"
-byte-unit = { version = "4.0.12", default-features = false, features = ["std"] }
+actix-web = { version = "4.0.1", default-features = false }
+anyhow = { version = "1.0.56", features = ["backtrace"] }
+async-stream = "0.3.3"
+async-trait = "0.1.52"
+atomic_refcell = "0.1.8"
+byte-unit = { version = "4.0.14", default-features = false, features = ["std"] }
 bytes = "1.1.0"
+clap = { version = "3.1.6", features = ["derive", "env"] }
+crossbeam-channel = "0.5.2"
 csv = "1.1.6"
-crossbeam-channel = "0.5.1"
+derivative = "2.2.0"
 either = "1.6.1"
-flate2 = "1.0.21"
+flate2 = "1.0.22"
+fs_extra = "1.2.0"
 fst = "0.4.7"
-futures = "0.3.17"
-futures-util = "0.3.17"
-heed = { git = "https://github.com/Kerollmops/heed", tag = "v0.12.1" }
-http = "0.2.4"
-indexmap = { version = "1.7.0", features = ["serde-1"] }
-itertools = "0.10.1"
+futures = "0.3.21"
+futures-util = "0.3.21"
+http = "0.2.6"
+indexmap = { version = "1.8.0", features = ["serde-1"] }
+itertools = "0.10.3"
 lazy_static = "1.4.0"
 log = "0.4.14"
-meilisearch-error = { path = "../meilisearch-error" }
 meilisearch-auth = { path = "../meilisearch-auth" }
-milli = { git = "https://github.com/meilisearch/milli.git", tag = "v0.22.2" }
+meilisearch-types = { path = "../meilisearch-types" }
+milli = { git = "https://github.com/meilisearch/milli.git", tag = "v0.29.3-filter.beta.0" }
 mime = "0.3.16"
-num_cpus = "1.13.0"
-once_cell = "1.8.0"
-parking_lot = "0.11.2"
-rand = "0.8.4"
-rayon = "1.5.1"
-regex = "1.5.4"
-rustls = "0.19.1"
-serde = { version = "1.0.130", features = ["derive"] }
-serde_json = { version = "1.0.67", features = ["preserve_order"] }
-siphasher = "0.3.7"
-slice-group-by = "0.2.6"
-clap = { version = "3.0", features = ["derive", "env"] }
-tar = "0.4.37"
-tempfile = "3.2.0"
-thiserror = "1.0.28"
-time = { version = "0.3.7", features = ["serde-well-known", "formatting", "parsing", "macros"] }
-tokio = { version = "1.11.0", features = ["full"] }
-uuid = { version = "0.8.2", features = ["serde"] }
-walkdir = "2.3.2"
+num_cpus = "1.13.1"
 obkv = "0.2.0"
-pin-project = "1.0.8"
-whoami = { version = "1.1.3", optional = true }
-reqwest = { version = "0.11.4", features = ["json", "rustls-tls"], default-features = false, optional = true }
-sysinfo = "0.20.2"
-derivative = "2.2.0"
-fs_extra = "1.2.0"
-atomic_refcell = "0.1.8"
+once_cell = "1.10.0"
+parking_lot = "0.12.0"
+permissive-json-pointer = { path = "../permissive-json-pointer" }
+rand = "0.8.5"
+rayon = "1.5.1"
+regex = "1.5.5"
+reqwest = { version = "0.11.9", features = ["json", "rustls-tls"], default-features = false, optional = true }
+roaring = "0.9.0"
+rustls = "0.20.4"
+serde = { version = "1.0.136", features = ["derive"] }
+serde_json = { version = "1.0.79", features = ["preserve_order"] }
+siphasher = "0.3.10"
+slice-group-by = "0.3.0"
+sysinfo = "0.23.5"
+tar = "0.4.38"
+tempfile = "3.3.0"
+thiserror = "1.0.30"
+time = { version = "0.3.7", features = ["serde-well-known", "formatting", "parsing", "macros"] }
+tokio = { version = "1.17.0", features = ["full"] }
+uuid = { version = "0.8.2", features = ["serde", "v4"] }
+walkdir = "2.3.2"
+whoami = { version = "1.2.1", optional = true }
 
 [dev-dependencies]
-actix-rt = "2.2.0"
-mockall = "0.10.2"
-paste = "1.0.5"
-nelson = { git = "https://github.com/MarinPostma/nelson.git", rev = "675f13885548fb415ead8fbb447e9e6d9314000a"}
-meilisearch-error = { path = "../meilisearch-error", features = ["test-traits"] }
+actix-rt = "2.7.0"
+meilisearch-types = { path = "../meilisearch-types", features = ["test-traits"] }
+mockall = "0.11.0"
+nelson = { git = "https://github.com/meilisearch/nelson.git", rev = "675f13885548fb415ead8fbb447e9e6d9314000a"}
+paste = "1.0.6"
 proptest = "1.0.0"
 proptest-derive = "0.3.0"

meilisearch-lib/src/document_formats.rs

@@ -1,7 +1,9 @@
-use std::fmt;
+use std::borrow::Borrow;
+use std::fmt::{self, Debug, Display};
 use std::io::{self, BufRead, BufReader, BufWriter, Cursor, Read, Seek, Write};
 
-use meilisearch_error::{internal_error, Code, ErrorCode};
+use meilisearch_types::error::{Code, ErrorCode};
+use meilisearch_types::internal_error;
 use milli::documents::DocumentBatchBuilder;
 
 type Result<T> = std::result::Result<T, DocumentFormatError>;
@@ -23,17 +25,40 @@ impl fmt::Display for PayloadType {
     }
 }
 
-#[derive(thiserror::Error, Debug)]
+#[derive(Debug)]
 pub enum DocumentFormatError {
-    #[error("An internal error has occurred. `{0}`.")]
     Internal(Box<dyn std::error::Error + Send + Sync + 'static>),
-    #[error("The `{1}` payload provided is malformed. `{0}`.")]
-    MalformedPayload(
-        Box<dyn std::error::Error + Send + Sync + 'static>,
-        PayloadType,
-    ),
+    MalformedPayload(Box<milli::documents::Error>, PayloadType),
 }
 
+impl Display for DocumentFormatError {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            Self::Internal(e) => write!(f, "An internal error has occurred: `{}`.", e),
+            Self::MalformedPayload(me, b) => match me.borrow() {
+                milli::documents::Error::JsonError(se) => {
+                    // https://github.com/meilisearch/meilisearch/issues/2107
+                    // The user input maybe insanely long. We need to truncate it.
+                    let mut serde_msg = se.to_string();
+                    let ellipsis = "...";
+                    if serde_msg.len() > 100 + ellipsis.len() {
+                        serde_msg.replace_range(50..serde_msg.len() - 85, ellipsis);
+                    }
+
+                    write!(
+                        f,
+                        "The `{}` payload provided is malformed. `Couldn't serialize document value: {}`.",
+                        b, serde_msg
+                )
+                }
+                _ => write!(f, "The `{}` payload provided is malformed: `{}`.", b, me),
+            },
+        }
+    }
+}
+
+impl std::error::Error for DocumentFormatError {}
+
 impl From<(PayloadType, milli::documents::Error)> for DocumentFormatError {
     fn from((ty, error): (PayloadType, milli::documents::Error)) -> Self {
         match error {

meilisearch-lib/src/dump/compat/mod.rs

@@ -1,5 +1,6 @@
 pub mod v2;
 pub mod v3;
+pub mod v4;
 
 /// Parses the v1 version of the Asc ranking rules `asc(price)`and returns the field name.
 pub fn asc_ranking_rule(text: &str) -> Option<&str> {

meilisearch-lib/src/dump/compat/v2.rs

@@ -1,5 +1,5 @@
 use anyhow::bail;
-use meilisearch_error::Code;
+use meilisearch_types::error::Code;
 use milli::update::IndexDocumentsMethod;
 use serde::{Deserialize, Serialize};
 use time::OffsetDateTime;

meilisearch-lib/src/dump/compat/v3.rs

@@ -1,12 +1,13 @@
-use meilisearch_error::{Code, ResponseError};
+use meilisearch_types::error::{Code, ResponseError};
+use meilisearch_types::index_uid::IndexUid;
 use milli::update::IndexDocumentsMethod;
 use serde::{Deserialize, Serialize};
 use time::OffsetDateTime;
 use uuid::Uuid;
 
+use super::v4::{Task, TaskContent, TaskEvent};
 use crate::index::{Settings, Unchecked};
-use crate::index_resolver::IndexUid;
-use crate::tasks::task::{DocumentDeletion, Task, TaskContent, TaskEvent, TaskId, TaskResult};
+use crate::tasks::task::{DocumentDeletion, TaskId, TaskResult};
 
 use super::v2;
 
@@ -58,9 +59,9 @@ pub enum Update {
     ClearDocuments,
 }
 
-impl From<Update> for TaskContent {
-    fn from(other: Update) -> Self {
-        match other {
+impl From<Update> for super::v4::TaskContent {
+    fn from(update: Update) -> Self {
+        match update {
             Update::DeleteDocuments(ids) => {
                 TaskContent::DocumentDeletion(DocumentDeletion::Ids(ids))
             }
@@ -185,10 +186,10 @@ impl Failed {
 impl From<(UpdateStatus, String, TaskId)> for Task {
     fn from((update, uid, task_id): (UpdateStatus, String, TaskId)) -> Self {
         // Dummy task
-        let mut task = Task {
+        let mut task = super::v4::Task {
             id: task_id,
-            index_uid: IndexUid::new(uid).unwrap(),
-            content: TaskContent::IndexDeletion,
+            index_uid: IndexUid::new_unchecked(uid),
+            content: super::v4::TaskContent::IndexDeletion,
             events: Vec::new(),
         };
 

meilisearch-lib/src/dump/compat/v4.rs

@@ -0,0 +1,145 @@
+use meilisearch_types::error::ResponseError;
+use meilisearch_types::index_uid::IndexUid;
+use milli::update::IndexDocumentsMethod;
+use serde::{Deserialize, Serialize};
+use time::OffsetDateTime;
+use uuid::Uuid;
+
+use crate::index::{Settings, Unchecked};
+use crate::tasks::batch::BatchId;
+use crate::tasks::task::{
+    DocumentDeletion, TaskContent as NewTaskContent, TaskEvent as NewTaskEvent, TaskId, TaskResult,
+};
+
+#[derive(Debug, Serialize, Deserialize)]
+pub struct Task {
+    pub id: TaskId,
+    pub index_uid: IndexUid,
+    pub content: TaskContent,
+    pub events: Vec<TaskEvent>,
+}
+
+impl From<Task> for crate::tasks::task::Task {
+    fn from(other: Task) -> Self {
+        Self {
+            id: other.id,
+            content: NewTaskContent::from((other.index_uid, other.content)),
+            events: other.events.into_iter().map(Into::into).collect(),
+        }
+    }
+}
+
+#[derive(Debug, Serialize, Deserialize)]
+pub enum TaskEvent {
+    Created(#[serde(with = "time::serde::rfc3339")] OffsetDateTime),
+    Batched {
+        #[serde(with = "time::serde::rfc3339")]
+        timestamp: OffsetDateTime,
+        batch_id: BatchId,
+    },
+    Processing(#[serde(with = "time::serde::rfc3339")] OffsetDateTime),
+    Succeded {
+        result: TaskResult,
+        #[serde(with = "time::serde::rfc3339")]
+        timestamp: OffsetDateTime,
+    },
+    Failed {
+        error: ResponseError,
+        #[serde(with = "time::serde::rfc3339")]
+        timestamp: OffsetDateTime,
+    },
+}
+
+impl From<TaskEvent> for NewTaskEvent {
+    fn from(other: TaskEvent) -> Self {
+        match other {
+            TaskEvent::Created(x) => NewTaskEvent::Created(x),
+            TaskEvent::Batched {
+                timestamp,
+                batch_id,
+            } => NewTaskEvent::Batched {
+                timestamp,
+                batch_id,
+            },
+            TaskEvent::Processing(x) => NewTaskEvent::Processing(x),
+            TaskEvent::Succeded { result, timestamp } => {
+                NewTaskEvent::Succeeded { result, timestamp }
+            }
+            TaskEvent::Failed { error, timestamp } => NewTaskEvent::Failed { error, timestamp },
+        }
+    }
+}
+
+#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
+#[allow(clippy::large_enum_variant)]
+pub enum TaskContent {
+    DocumentAddition {
+        content_uuid: Uuid,
+        merge_strategy: IndexDocumentsMethod,
+        primary_key: Option<String>,
+        documents_count: usize,
+        allow_index_creation: bool,
+    },
+    DocumentDeletion(DocumentDeletion),
+    SettingsUpdate {
+        settings: Settings<Unchecked>,
+        /// Indicates whether the task was a deletion
+        is_deletion: bool,
+        allow_index_creation: bool,
+    },
+    IndexDeletion,
+    IndexCreation {
+        primary_key: Option<String>,
+    },
+    IndexUpdate {
+        primary_key: Option<String>,
+    },
+    Dump {
+        uid: String,
+    },
+}
+
+impl From<(IndexUid, TaskContent)> for NewTaskContent {
+    fn from((index_uid, content): (IndexUid, TaskContent)) -> Self {
+        match content {
+            TaskContent::DocumentAddition {
+                content_uuid,
+                merge_strategy,
+                primary_key,
+                documents_count,
+                allow_index_creation,
+            } => NewTaskContent::DocumentAddition {
+                index_uid,
+                content_uuid,
+                merge_strategy,
+                primary_key,
+                documents_count,
+                allow_index_creation,
+            },
+            TaskContent::DocumentDeletion(deletion) => NewTaskContent::DocumentDeletion {
+                index_uid,
+                deletion,
+            },
+            TaskContent::SettingsUpdate {
+                settings,
+                is_deletion,
+                allow_index_creation,
+            } => NewTaskContent::SettingsUpdate {
+                index_uid,
+                settings,
+                is_deletion,
+                allow_index_creation,
+            },
+            TaskContent::IndexDeletion => NewTaskContent::IndexDeletion { index_uid },
+            TaskContent::IndexCreation { primary_key } => NewTaskContent::IndexCreation {
+                index_uid,
+                primary_key,
+            },
+            TaskContent::IndexUpdate { primary_key } => NewTaskContent::IndexUpdate {
+                index_uid,
+                primary_key,
+            },
+            TaskContent::Dump { uid } => NewTaskContent::Dump { uid },
+        }
+    }
+}

meilisearch-lib/src/dump/error.rs

@@ -0,0 +1,36 @@
+use meilisearch_auth::error::AuthControllerError;
+use meilisearch_types::error::{Code, ErrorCode};
+use meilisearch_types::internal_error;
+
+use crate::{index_resolver::error::IndexResolverError, tasks::error::TaskError};
+
+pub type Result<T> = std::result::Result<T, DumpError>;
+
+#[derive(thiserror::Error, Debug)]
+pub enum DumpError {
+    #[error("An internal error has occurred. `{0}`.")]
+    Internal(Box<dyn std::error::Error + Send + Sync + 'static>),
+    #[error("{0}")]
+    IndexResolver(#[from] IndexResolverError),
+}
+
+internal_error!(
+    DumpError: milli::heed::Error,
+    std::io::Error,
+    tokio::task::JoinError,
+    tokio::sync::oneshot::error::RecvError,
+    serde_json::error::Error,
+    tempfile::PersistError,
+    fs_extra::error::Error,
+    AuthControllerError,
+    TaskError
+);
+
+impl ErrorCode for DumpError {
+    fn error_code(&self) -> Code {
+        match self {
+            DumpError::Internal(_) => Code::Internal,
+            DumpError::IndexResolver(e) => e.error_code(),
+        }
+    }
+}

meilisearch-lib/src/dump/handler.rs

@@ -0,0 +1,188 @@
+#[cfg(not(test))]
+pub use real::DumpHandler;
+
+#[cfg(test)]
+pub use test::MockDumpHandler as DumpHandler;
+
+use time::{macros::format_description, OffsetDateTime};
+
+/// Generate uid from creation date
+pub fn generate_uid() -> String {
+    OffsetDateTime::now_utc()
+        .format(format_description!(
+            "[year repr:full][month repr:numerical][day padding:zero]-[hour padding:zero][minute padding:zero][second padding:zero][subsecond digits:3]"
+        ))
+        .unwrap()
+}
+
+mod real {
+    use std::path::PathBuf;
+    use std::sync::Arc;
+
+    use log::{info, trace};
+    use meilisearch_auth::AuthController;
+    use milli::heed::Env;
+    use tokio::fs::create_dir_all;
+    use tokio::io::AsyncWriteExt;
+
+    use crate::analytics;
+    use crate::compression::to_tar_gz;
+    use crate::dump::error::{DumpError, Result};
+    use crate::dump::{MetadataVersion, META_FILE_NAME};
+    use crate::index_resolver::{
+        index_store::IndexStore, meta_store::IndexMetaStore, IndexResolver,
+    };
+    use crate::tasks::TaskStore;
+    use crate::update_file_store::UpdateFileStore;
+
+    pub struct DumpHandler<U, I> {
+        dump_path: PathBuf,
+        db_path: PathBuf,
+        update_file_store: UpdateFileStore,
+        task_store_size: usize,
+        index_db_size: usize,
+        env: Arc<Env>,
+        index_resolver: Arc<IndexResolver<U, I>>,
+    }
+
+    impl<U, I> DumpHandler<U, I>
+    where
+        U: IndexMetaStore + Sync + Send + 'static,
+        I: IndexStore + Sync + Send + 'static,
+    {
+        pub fn new(
+            dump_path: PathBuf,
+            db_path: PathBuf,
+            update_file_store: UpdateFileStore,
+            task_store_size: usize,
+            index_db_size: usize,
+            env: Arc<Env>,
+            index_resolver: Arc<IndexResolver<U, I>>,
+        ) -> Self {
+            Self {
+                dump_path,
+                db_path,
+                update_file_store,
+                task_store_size,
+                index_db_size,
+                env,
+                index_resolver,
+            }
+        }
+
+        pub async fn run(&self, uid: String) -> Result<()> {
+            trace!("Performing dump.");
+
+            create_dir_all(&self.dump_path).await?;
+
+            let temp_dump_dir = tokio::task::spawn_blocking(tempfile::TempDir::new).await??;
+            let temp_dump_path = temp_dump_dir.path().to_owned();
+
+            let meta = MetadataVersion::new_v5(self.index_db_size, self.task_store_size);
+            let meta_path = temp_dump_path.join(META_FILE_NAME);
+
+            let meta_bytes = serde_json::to_vec(&meta)?;
+            let mut meta_file = tokio::fs::File::create(&meta_path).await?;
+            meta_file.write_all(&meta_bytes).await?;
+
+            analytics::copy_user_id(&self.db_path, &temp_dump_path);
+
+            create_dir_all(&temp_dump_path.join("indexes")).await?;
+
+            let db_path = self.db_path.clone();
+            let temp_dump_path_clone = temp_dump_path.clone();
+            tokio::task::spawn_blocking(move || -> Result<()> {
+                AuthController::dump(db_path, temp_dump_path_clone)?;
+                Ok(())
+            })
+            .await??;
+            TaskStore::dump(
+                self.env.clone(),
+                &temp_dump_path,
+                self.update_file_store.clone(),
+            )
+            .await?;
+            self.index_resolver.dump(&temp_dump_path).await?;
+
+            let dump_path = self.dump_path.clone();
+            let dump_path = tokio::task::spawn_blocking(move || -> Result<PathBuf> {
+                // for now we simply copy the updates/updates_files
+                // FIXME: We may copy more files than necessary, if new files are added while we are
+                // performing the dump. We need a way to filter them out.
+
+                let temp_dump_file = tempfile::NamedTempFile::new_in(&dump_path)?;
+                to_tar_gz(temp_dump_path, temp_dump_file.path())
+                    .map_err(|e| DumpError::Internal(e.into()))?;
+
+                let dump_path = dump_path.join(uid).with_extension("dump");
+                temp_dump_file.persist(&dump_path)?;
+
+                Ok(dump_path)
+            })
+            .await??;
+
+            info!("Created dump in {:?}.", dump_path);
+
+            Ok(())
+        }
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use std::path::PathBuf;
+    use std::sync::Arc;
+
+    use milli::heed::Env;
+    use nelson::Mocker;
+
+    use crate::dump::error::Result;
+    use crate::index_resolver::IndexResolver;
+    use crate::index_resolver::{index_store::IndexStore, meta_store::IndexMetaStore};
+    use crate::update_file_store::UpdateFileStore;
+
+    use super::*;
+
+    pub enum MockDumpHandler<U, I> {
+        Real(super::real::DumpHandler<U, I>),
+        Mock(Mocker),
+    }
+
+    impl<U, I> MockDumpHandler<U, I> {
+        pub fn mock(mocker: Mocker) -> Self {
+            Self::Mock(mocker)
+        }
+    }
+
+    impl<U, I> MockDumpHandler<U, I>
+    where
+        U: IndexMetaStore + Sync + Send + 'static,
+        I: IndexStore + Sync + Send + 'static,
+    {
+        pub fn new(
+            dump_path: PathBuf,
+            db_path: PathBuf,
+            update_file_store: UpdateFileStore,
+            task_store_size: usize,
+            index_db_size: usize,
+            env: Arc<Env>,
+            index_resolver: Arc<IndexResolver<U, I>>,
+        ) -> Self {
+            Self::Real(super::real::DumpHandler::new(
+                dump_path,
+                db_path,
+                update_file_store,
+                task_store_size,
+                index_db_size,
+                env,
+                index_resolver,
+            ))
+        }
+        pub async fn run(&self, uid: String) -> Result<()> {
+            match self {
+                DumpHandler::Real(real) => real.run(uid).await,
+                DumpHandler::Mock(mocker) => unsafe { mocker.get("run").call(uid) },
+            }
+        }
+    }
+}

meilisearch-lib/src/dump/loaders/mod.rs

@@ -1,3 +1,4 @@
 pub mod v2;
 pub mod v3;
 pub mod v4;
+pub mod v5;

meilisearch-lib/src/dump/loaders/v2.rs

@@ -5,8 +5,8 @@ use std::path::{Path, PathBuf};
 use serde_json::{Deserializer, Value};
 use tempfile::NamedTempFile;
 
-use crate::index_controller::dump_actor::compat::{self, v2, v3};
-use crate::index_controller::dump_actor::Metadata;
+use crate::dump::compat::{self, v2, v3};
+use crate::dump::Metadata;
 use crate::options::IndexerOpts;
 
 /// The dump v2 reads the dump folder and patches all the needed file to make it compatible with a

meilisearch-lib/src/dump/loaders/v3.rs

@@ -9,11 +9,11 @@ use log::info;
 use tempfile::tempdir;
 use uuid::Uuid;
 
-use crate::index_controller::dump_actor::compat::v3;
-use crate::index_controller::dump_actor::Metadata;
+use crate::dump::compat::{self, v3};
+use crate::dump::Metadata;
 use crate::index_resolver::meta_store::{DumpEntry, IndexMeta};
 use crate::options::IndexerOpts;
-use crate::tasks::task::{Task, TaskId};
+use crate::tasks::task::TaskId;
 
 /// dump structure for V3:
 /// .
@@ -124,7 +124,7 @@ fn patch_updates(
                 .clone();
             serde_json::to_writer(
                 &mut dst_file,
-                &Task::from((entry.update, name, task_id as TaskId)),
+                &compat::v4::Task::from((entry.update, name, task_id as TaskId)),
             )?;
             dst_file.write_all(b"\n")?;
             Ok(())