Commit Graph

7187 Commits

Author SHA1 Message Date
cb8d5f2d4b Update Charabia to 0.7.1 2023-02-20 14:00:31 +01:00
0bc1a18f52 Use Languages list detected during indexing at search time 2023-02-01 18:57:43 +01:00
643d99e0f9 Add expectancy test 2023-02-01 18:39:54 +01:00
064158e4e2 Update test 2023-02-01 15:34:01 +01:00
77d32d0ee8 Fix codec deserialization 2023-02-01 15:26:26 +01:00
f4569b04ad Update Charabia version 2023-02-01 15:26:26 +01:00
2922c5c899 Fix code format 2023-01-31 11:28:05 +01:00
7681be5367 Format code 2023-01-31 11:28:05 +01:00
50bc156257 Fix tests 2023-01-31 11:28:05 +01:00
d8207356f4 Skip script,language insertion if language is undetected 2023-01-31 11:28:05 +01:00
2d58b28f43 Improve script language codec 2023-01-31 11:28:05 +01:00
fd60a39f1c Format code 2023-01-31 11:28:05 +01:00
369c05732e Add test checking if from script_language_docids database were removed
deleted docids
2023-01-31 11:28:05 +01:00
34d04f3d3f Filter from script_language_docids database soft deleted documents 2023-01-31 11:28:05 +01:00
a27f329e3a Add tests for checking that detected script and language associated with document(s) were stored during indexing 2023-01-31 11:28:05 +01:00
b216ddba63 Delete and clear data from the new database 2023-01-31 11:28:05 +01:00
d97fb6117e Extract and index data 2023-01-31 11:28:05 +01:00
c45d1e3610 Create a new database on index and add a specialized codec for it 2023-01-31 11:28:05 +01:00
bfb1f9279b Merge #3420 #3422
3420: Add image hyperlink in README.md r=curquiza a=gregsadetsky

# Pull Request

## What does this PR do?
- tiny README.md improvement: under "SDKs & integration tools", add a hyperlink to the image with all of the language logos so that clicking the image leads to the integrations page. Otherwise, right now, clicking this image leads to the image file in the repo, which is not really useful.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [X] Have you read the contributing guidelines?
- [X] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


3422: Remove cache from test CI r=dureuill a=curquiza

Comment the lines in CIs where we use the test CIs
We indeed have cache issues (lack of space on the machine) when running our test CIs
https://github.com/meilisearch/meilisearch/pull/3403

Co-authored-by: Greg Sadetsky <lepetitg@gmail.com>
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: curquiza <clementine@meilisearch.com>
2023-01-25 16:25:53 +00:00
48dabd27ea Update .github/workflows/rust.yml
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2023-01-25 16:58:15 +01:00
9c3830a19c Remove cache everywhere 2023-01-25 16:35:02 +01:00
ff6b8dfac4 Remove cache from Windows and macOs CIs 2023-01-25 16:24:04 +01:00
0544b60974 Merge #3409
3409: Bump libgit2-sys from 0.14.1+1.5.0 to 0.14.2+1.5.1 r=Kerollmops a=dependabot[bot]

Bumps [libgit2-sys](https://github.com/rust-lang/git2-rs) from 0.14.1+1.5.0 to 0.14.2+1.5.1.
<details>
<summary>Commits</summary>
<ul>
<li><a href="a233483a39"><code>a233483</code></a> Update to libgit2 1.5.1</li>
<li><a href="bce15556ef"><code>bce1555</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/git2-rs/issues/909">#909</a> from ehuss/ssh-keys</li>
<li><a href="222fbf3b9e"><code>222fbf3</code></a> Bump versions</li>
<li><a href="fa41943135"><code>fa41943</code></a> Change the certificate_check callback to support passthrough.</li>
<li><a href="84e21aad4e"><code>84e21aa</code></a> Add ability to get the SSH host key and its type.</li>
<li><a href="e6aa6666b9"><code>e6aa666</code></a> Bump git2-curl version. (<a href="https://github-redirect.dependabot.com/rust-lang/git2-rs/issues/861">#861</a>)</li>
<li><a href="46674cebd9"><code>46674ce</code></a> Fix warning about unused_must_use for Box::from_raw (<a href="https://github-redirect.dependabot.com/rust-lang/git2-rs/issues/860">#860</a>)</li>
<li><a href="951dce9dea"><code>951dce9</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/git2-rs/issues/858">#858</a> from davidkna/git2150</li>
<li><a href="8871f8e9b3"><code>8871f8e</code></a> bump libgit2 to 1.5.0</li>
<li><a href="04278a24ba"><code>04278a2</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/git2-rs/issues/839">#839</a> from davidkna/libgit2_143</li>
<li>Additional commits viewable in <a href="https://github.com/rust-lang/git2-rs/compare/0.14.1...libgit2-sys-0.14.2">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=libgit2-sys&package-manager=cargo&previous-version=0.14.1+1.5.0&new-version=0.14.2+1.5.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/meilisearch/meilisearch/network/alerts).

</details>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-25 09:34:34 +00:00
4223c51838 Add image hyperlink in README.md 2023-01-24 15:24:09 -05:00
6f71a2b38b Merge #3403
3403: Add `--all` to test CI r=curquiza a=curquiza

Discussed with `@irevoire` [here](https://meilisearch.slack.com/archives/G01A1F4KVGU/p1674144546920649?thread_ts=1674144456.561199&cid=G01A1F4KVGU) (internal link)

Co-authored-by: curquiza <clementine@meilisearch.com>
2023-01-24 16:23:08 +00:00
4c4baaf1ce Merge #3387
3387: Update create-issue-dependencies.yml r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2023-01-24 14:44:09 +00:00
5f4497935f Bump libgit2-sys from 0.14.1+1.5.0 to 0.14.2+1.5.1
Bumps [libgit2-sys](https://github.com/rust-lang/git2-rs) from 0.14.1+1.5.0 to 0.14.2+1.5.1.
- [Release notes](https://github.com/rust-lang/git2-rs/releases)
- [Commits](https://github.com/rust-lang/git2-rs/compare/0.14.1...libgit2-sys-0.14.2)

---
updated-dependencies:
- dependency-name: libgit2-sys
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-20 23:38:40 +00:00
2a1787ed22 Add --all in test CI 2023-01-19 17:26:47 +01:00
60018d0fe4 Merge #3343
3343: Extract creation and last updated timestamp for v3 dump r=curquiza a=FrancisMurillo

# Pull Request

## Related issue
Fixes #2988

## What does this PR do?

Inspired by the v4 dump implementation, this extracts the first `createdAt` and last `updatedAt` fields by parsing the task queue.

Questions:
- Should the parsing of the tasks be cached instead of being parsed for every index since it might add a performance penalty?
- I am not sure if the `created_at` and `processed_at` fields are correct 
- Should I assume the data is sorted in some order like with `uuid` or `updateId`? I assumed the list is unordered.
- I was planning to populate my dev instance with data and dump my data. Is there a way to dump with previous versions?

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Francis Murillo <evacuee.overlap.vs3op@aleeas.com>
2023-01-19 16:14:21 +00:00
798aa4ee92 Fix clippy issues 2023-01-19 19:38:20 +08:00
0ce1d6d487 Update create-issue-dependencies.yml 2023-01-18 23:43:33 +01:00
b3166df7ea Merge #3372
3372: Enhance facet string normalization r=ManyTheFish a=ManyTheFish

# Pull Request

Use compatibility decomposition normalizer in facet string extraction in order to have a more human friendly sort order.

Now, [é (U+00E9)](https://www.compart.com/fr/unicode/U+00E9) is converted to [e (U+0065)](https://www.compart.com/fr/unicode/U+0065) + [◌́ (U+0301)](https://www.compart.com/fr/unicode/U+0301). This way any word starting with an accented/diacritized version of a character is put just after the words starting with the unaccented version of the character.

## Related issue

Fixes #3260 


Co-authored-by: ManyTheFish <many@meilisearch.com>
2023-01-18 17:17:53 +00:00
1803998017 Merge #3335 #3353
3335: Remove test badge r=curquiza a=curquiza

Suggestion of removal: from my point of view, this badge does not provide any useful information, and most of all is often outdated. Currently ours is "no status" despite our tests passing.
Plus, sometimes our tests are not passing because we are still in development, but it does not mean our current provided binaries are not.

<img width="619" alt="Capture d’écran 2023-01-12 à 14 06 40" src="https://user-images.githubusercontent.com/20380692/212074200-f9e3ab3e-ad1d-4171-bd13-46584c3cd117.png">


3353: Bump svenstaro/upload-release-action from 2.3.0 to 2.4.0 r=curquiza a=dependabot[bot]

Bumps [svenstaro/upload-release-action](https://github.com/svenstaro/upload-release-action) from 2.3.0 to 2.4.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/svenstaro/upload-release-action/releases">svenstaro/upload-release-action's releases</a>.</em></p>
<blockquote>
<h2>2.4.0</h2>
<ul>
<li>Update to node 16</li>
<li>Bump most dependencies</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/svenstaro/upload-release-action/blob/master/CHANGELOG.md">svenstaro/upload-release-action's changelog</a>.</em></p>
<blockquote>
<h2>[2.4.0] - 2023-01-09</h2>
<ul>
<li>Update to node 16</li>
<li>Bump most dependencies</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="2728235f7d"><code>2728235</code></a> 2.4.0</li>
<li><a href="c2e0608dc4"><code>c2e0608</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/svenstaro/upload-release-action/issues/88">#88</a> from svenstaro/dependabot/npm_and_yarn/json5-1.0.2</li>
<li><a href="bd74772a1a"><code>bd74772</code></a> Don't shadow vars</li>
<li><a href="16e7903b2d"><code>16e7903</code></a> Bump json5 from 1.0.1 to 1.0.2</li>
<li><a href="f2c549b117"><code>f2c549b</code></a> Bump some more deps</li>
<li><a href="7a7d004438"><code>7a7d004</code></a> Bump some deps</li>
<li><a href="9c4a92ec0d"><code>9c4a92e</code></a> Use explicit any</li>
<li><a href="039214a996"><code>039214a</code></a> Bump jest and typescript versions</li>
<li><a href="2b373356cb"><code>2b37335</code></a> Update to node16</li>
<li><a href="fb1eb39e74"><code>fb1eb39</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/svenstaro/upload-release-action/issues/75">#75</a> from svenstaro/dependabot/npm_and_yarn/jsdom-16.7.0</li>
<li>Additional commits viewable in <a href="https://github.com/svenstaro/upload-release-action/compare/2.3.0...2.4.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=svenstaro/upload-release-action&package-manager=github_actions&previous-version=2.3.0&new-version=2.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 15:45:21 +00:00
1af3089456 Merge #3348
3348: fix cargo flaky r=irevoire a=irevoire

Partially fix https://github.com/meilisearch/meilisearch/issues/3273

Ideally, we should revert this commit and fix cargo-flaky directly to ensure we never forget to add a sub-crate to the CI.

-----

Here is an example of the CI running (and thus working); https://github.com/meilisearch/meilisearch/actions/runs/3932783699/jobs/6725755801

Co-authored-by: Tamo <tamo@meilisearch.com>
2023-01-18 14:46:50 +00:00
d1fc42b53a Use compatibility decomposition normalizer in facets 2023-01-18 15:02:13 +01:00
e64571a881 Add test sorting string with diacritics 2023-01-18 14:43:38 +01:00
32e2848a74 Bump svenstaro/upload-release-action from 2.3.0 to 2.4.0
Bumps [svenstaro/upload-release-action](https://github.com/svenstaro/upload-release-action) from 2.3.0 to 2.4.0.
- [Release notes](https://github.com/svenstaro/upload-release-action/releases)
- [Changelog](https://github.com/svenstaro/upload-release-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/svenstaro/upload-release-action/compare/2.3.0...2.4.0)

---
updated-dependencies:
- dependency-name: svenstaro/upload-release-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-18 11:02:44 +00:00
6b3da8a6de Merge #3346
3346: Import milli 🎉 r=Kerollmops a=Kerollmops

Fixes https://github.com/meilisearch/meilisearch/issues/2901

Main work
- integrate the milli repository as an internal crate into this repo  
- Update the Cargo.toml accordingly
- Ensure meilisearch-type now uses the internal milli crate and not the remote repository
- Update the milli's version to follow the meilisearch one

Also
- Removed the beta tests in test CI (will be re-integrated later if needed)
- Move and modify milli's README into the `milli` folder
- remove the script folder from `milli`
- Removed useless CI (release-drafter and enforce-label)

⚠️ Also, import all the `release-v1.0.0` until [a5c4fb](a5c4fbbcea) included (merged of the PR https://github.com/meilisearch/meilisearch/pull/3334)

Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: Samyak S Sarnayak <samyak201@gmail.com>
Co-authored-by: unvalley <kirohi.code@gmail.com>
Co-authored-by: Samyak Sarnayak <samyak201@gmail.com>
2023-01-18 10:19:42 +00:00
0769090dd6 Add a note in the README about the crates versionning 2023-01-18 10:08:12 +01:00
1d507c84b2 Fix the formatting 2023-01-17 18:25:55 +01:00
1b78231e18 Make clippy happy 2023-01-17 18:25:54 +01:00
2b1f6a7f11 Fix the CI to ignore a missing file 2023-01-17 16:26:03 +01:00
6993924f32 Use finished_at for v3 dumps instead 2023-01-17 23:11:49 +08:00
9dd01ff44b fix cargo flaky 2023-01-17 11:03:44 +01:00
507a7bad96 Use the local milli subcrate 2023-01-16 17:35:54 +01:00
cde62fcb5b Merge remote-tracking branch 'origin/release-v1.0.0' into import-milli 2023-01-16 17:35:18 +01:00
03a82136dc Remove the useless cli subcrate 2023-01-16 17:08:43 +01:00
e68758cec4 Refine the cargo workspace profile settings 2023-01-16 17:04:25 +01:00
4fb47492e5 Make clippy happy 2023-01-16 16:35:58 +01:00
5bab8cf7ec Remove useless CI configs 2023-01-16 16:31:46 +01:00