Commit Graph

3933 Commits

Author SHA1 Message Date
20589a41b5 Rename receivedDocumentIds into matchedDocuments
Changes DocumentDeletion task details response.
2022-10-01 21:59:20 +02:00
2827ff7957 Update README.md with Hacktoberfest section (#2794)
* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
2022-09-22 17:45:33 +02:00
d166a97d67 Update CONTRIBUTING.md for Hacktoberfest (#2793)
* Update CONTRIBUTING.md

* Update CONTRIBUTING.md

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>

* Update CONTRIBUTING.md

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
2022-09-22 17:27:42 +02:00
f4b81fa0a1 Merge #2772 #2773
2772: Improve issue template display to avoid support in Meilisearch issues r=curquiza a=curquiza

Move the support line higher to make it more visible
Like this: https://github.com/meilisearch/meilisearch/discussions/2780

2773: Allow building without specialized tokenizations r=curquiza a=jirutka

Fixes #2774

(Some of) these specialized tokenizations include huge dictionaries that currently account for 90% (!) of the meilisearch binary size.

This commit adds `chinese`, `hebrew`, `japanese`, and `thai` feature flags that are propagated via `milli` down to the `charabia` crate. To keep it backwards compatible, they are enabled by default.

Related to meilisearch/milli#632

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Jakub Jirutka <jakub@jirutka.cz>
2022-09-21 10:56:12 +00:00
0fdbf128e9 Merge #2790
2790: Improve Docker CI for cloud team r=curquiza a=curquiza

Work with `@eskombro` 
Update CI to send a signal to Cloud team CI when Docker image is pushed

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-21 09:55:50 +00:00
d3b984d862 Update CI to send a signal to Cloud team when Docker image is pushed
Co-authored-by: Samuel Jimenez <sjimenezre@gmail.com>
2022-09-21 11:39:58 +02:00
935f18efcf Allow building without specialized tokenizations
(Some of) these specialized tokenizations include huge dictionaries
that currently account for 90% (!) of the meilisearch binary size.

This commit adds chinese, hebrew, japanese, and thai feature flags
that are propagated via milli down to the charabia crate. To keep it
backward compatible, they are enabled by default.

Related to meilisearch/milli#632
2022-09-14 21:16:34 +02:00
5b57114771 Bump milli from 0.33.0 to 0.33.4 2022-09-14 20:52:11 +02:00
c2ab7a7939 Update config.yml 2022-09-14 14:40:36 +02:00
528a4721c1 Merge #2727
2727: Don't panic when the error length is slightly over 100 r=Kerollmops a=onyxcherry

# Pull Request

## What does this PR do?
Fixes PR #2207 as [the last commit](7ece7a9d9e) has changed number of the characters at the end to leave in place from `50` to `85` **but the lower limit of a string length wasn't changed**.
Therefore, any data (e.g. example string from issue #2680) was causing `meilisearch` to **panic**.

So I simply raised the minimum value from `100` to `135` (`50 + 85`) to ensure that `replace_range()` won't panic due to an inverted range.
At the same time I am in favor of the `85` value which was changed in the `@CNLHC's` last commit.


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing ~issue~ pull request?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Tomasz Wiśniewski <tomasz@wisniewski.app>
2022-09-07 16:16:43 +00:00
d1a30df23d Remove unneeded prints, format 2022-09-07 18:05:55 +02:00
3a62e97c9f Merge #2744
2744: Minor fixes in the just added update-version CI r=ManyTheFish a=curquiza

These fixes does not prevent us to use the current CI

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-06 13:14:06 +00:00
55aa83d75a Minor fixes in the just added update-version CI 2022-09-05 16:41:01 +02:00
0c962f6a76 Merge #2740
2740: Update checkout v2 to v3 in CI manifests and use a unique GitHub PAT r=Kerollmops a=curquiza

Upgrade the missing checkout v2 into v3
Probably a bad merge conflicts that make them removed when merging `stable` into `main` after v0.28.0 release.

Also, use `MEILI_BOT_GH_PAT` instead of `PUBLISH_TOKEN` and the default github token, which allow us to remove useless GitHub secrets (once this PR is merged and v0.29.0 is release because `PUBLISH_TOKEN` is still used on `release-v0.29.0`)

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 12:14:40 +00:00
2e0221cfc6 Merge #2739
2739: Add CI manifest to automate some steps when closing/creating a Milestone r=Kerollmops a=curquiza

For each Milestone created (not opened!), and if the release is NOT a patch release (only the patch changed)
- the roadmap issue is created, see the [template](https://github.com/meilisearch/core-team/blob/main/issue-templates/roadmap-issue.md)
- the changelog issue is created, see the [template](https://github.com/meilisearch/core-team/blob/main/issue-templates/changelog-issue.md)

For each Milestone closed
- the `vX.Y.Z` label is created
- this label is applied to all issues/PRs in the Milestone

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 11:42:10 +00:00
2d34b239ab Merge #2738
2738: Add missing env vars for dumps and snapshots features r=irevoire a=gmourier

# Pull Request

## What does this PR do?
Fixes #2721


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-09-05 11:12:09 +00:00
55853815a5 Merge #2741
2741: Add CI to update the Meilisearch version in Cargo.toml files r=ManyTheFish a=curquiza

Add a CI we can trigger manually to create a PR updating the Meilisearch version
The next step is to create a Slack bot that will trigger this CI
In the meantime, we can trigger this CI manually in the [Actions tab](https://github.com/meilisearch/milli/actions)

The `MEILI_BOT_GH_PAT` secrets has been added to the organization level, and is accessible for the following repositories (so far): Meilisearch, Milli and Charabia

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 08:32:43 +00:00
b897ce8dfa Add CI to update th Meilisearch version in Cargo.toml files 2022-09-04 11:57:23 +02:00
5f5b787483 Use meili-bot PAT everywhere 2022-09-04 11:32:22 +02:00
d0aa8042e2 Fix job names 2022-09-03 17:53:37 +02:00
9cb1e4af5c Rename workflow file 2022-09-03 17:50:24 +02:00
cc09aa8868 Refacto env var 2022-09-02 17:07:23 +02:00
2eca723a91 Update checkout v2 to v3 in CI manifests 2022-09-02 16:45:32 +02:00
ae14567f97 Add CI manifest to automate some step of the release management when creating/closing a Milestone 2022-09-02 16:28:58 +02:00
d0f1054f5c chore: cargo fmt 2022-09-01 22:37:07 +02:00
3878c289df feat: add missing env var for dumps and snapshots feature 2022-09-01 22:34:20 +02:00
d1b3642923 Extract input to trim lengths to variables 2022-09-01 20:50:11 +02:00
f45ddff312 Merge #2733
2733: Move if conditions to the steps and not to the whole job r=Kerollmops a=curquiza

Follows https://github.com/meilisearch/meilisearch/pull/2726
Fixes the CI. Moves the `if:` condition to the steps and not the whole `check-version` jobs, otherwise, the following jobs checking the compilation cannot run.
See: https://github.com/meilisearch/meilisearch/actions/runs/2968543407

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-01 11:49:40 +00:00
8f3b590436 Move if conditions to the steps and not to the whole job 2022-09-01 13:34:34 +02:00
60792eebcf Fix #2207 - do not panic when the error message length is between 100 and 135 2022-08-31 19:29:53 +02:00
4fbc8c705f Merge #2726
2726: Add dry run for publishing binaries: check the compilation works r=Kerollmops a=curquiza

To avoid realizing the compilation of one type of binary does not work during the release, I create a dry run for binary compilation every day at 2am 😇 

See the problem we had recently because missing this CI: https://github.com/meilisearch/meilisearch/issues/2718

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-08-31 15:35:05 +00:00
80b1f3e830 Add dry run for publishing binaries: check the compilation works 2022-08-31 17:28:42 +02:00
3efd06168f Merge #2719
2719: Update ubuntu-18.04 to 20.04 r=Kerollmops a=curquiza

ubuntu-18.04 is deprecated

<img width="1566" alt="Capture d’écran 2022-08-30 à 12 04 25" src="https://user-images.githubusercontent.com/20380692/187409661-5ac9c545-a737-47f5-ad2f-0369114be11a.png">

https://github.com/meilisearch/meilisearch/actions/runs/2949985452

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-08-30 13:42:09 +00:00
07bb32b622 Update ubuntu-18.04 to 20.04 2022-08-29 18:26:01 +02:00
0826aa35e2 Merge #2713
2713: Move prometheus behind a feature flag r=Kerollmops a=irevoire

We decided we wanted to continue working on this feature before making it public.

Co-authored-by: Tamo <tamo@meilisearch.com>
v0.29.0rc0
2022-08-29 12:38:42 +00:00
6aa3ad6b6c move prometheus behind a feature flag 2022-08-29 14:36:59 +02:00
47a1aa69f0 Merge #2702
2702: Add link to the main image r=curquiza a=brunoocasali

I have wrapped the image with a `<a>` link, and it seems to be working fine, WDYT?

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-08-24 16:54:08 +00:00
b3d89da74d Update README.md 2022-08-24 18:52:34 +02:00
0798170a64 Add link to the main image 2022-08-24 13:47:09 -03:00
446dfccc8c Merge #2504 #2697
2504: New README 🌟 r=curquiza a=curquiza

⚠️ Please do not only look at the Markdown but also how the GitHub renders the README 😇 

👉 👉 [Rendered](https://github.com/meilisearch/meilisearch/blob/new-readme/README.md) 👈 👈

2697: Accept an environment variable to enable the metrics route r=ManyTheFish a=Kerollmops

With the PR Meilisearch is able to accept the `MEILI_ENABLE_METRICS_ROUTE` environment variable to enable the newly introduces metrics route.

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-24 15:41:44 +00:00
43175cfb78 New README version
Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update according to guigui reuqest

Add demo link

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: CaroFG <48251481+CaroFG@users.noreply.github.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Put sentence in bold
2022-08-24 17:16:29 +02:00
a1bb49c351 Merge #2696
2696: Add the new `metrics.get` and `metrics.all` actions rights r=Kerollmops a=Kerollmops

Follow the specification and add the new `metrics.get` and `metrics.all` actions, making the `/metrics` route only accessible with those rights.

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-24 15:08:53 +00:00
bebd76064a Add test for the rights of /metrics route 2022-08-24 17:03:43 +02:00
f0b2ac6efb metrics.all must define metrics.get 2022-08-24 17:03:30 +02:00
08d86e33ca Accept an env variable to enable the metrics route 2022-08-24 16:39:56 +02:00
2c2efc7ab6 Remove the hand written numbers of the actions rights 2022-08-24 16:33:12 +02:00
381df43be4 Change the metrics route API access rights 2022-08-24 16:28:33 +02:00
f87ebfe477 Merge #2692
2692: Slight changes for prometheus metrics r=Kerollmops a=gmourier

# Pull Request

## What does this PR do?

- Replace "MeiliSearch" with "Meilisearch"
- Brings some consistency between rust identifier and exposed metrics names
- Add suffix describing unit, in plural form. e.g `MEILISEARCH_DB_SIZE_BYTES` (https://prometheus.io/docs/practices/naming/#metric-names)
- Update dashboard.json

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-08-24 10:12:24 +00:00
c445334070 Merge #2636
2636: Upgrade milli to v0.33.0 r=Kerollmops a=ManyTheFish

# Summary
- Update milli to v0.33.0
- Classify the new InvalidLmdbOpenOptions error as an Internal error
- Update filter error check in tests
- Introduce Terms Matching Policies

fixes #2479
fixes #2484
fixes #2486
fixes #2516
fixes #2578
fixes #2580
fixes #2583
fixes #2600
fixes #2640
fixes #2672
fixes #2679
fixes #2686

# Terms Matching Policies
This PR allows end users to customize matching term policies

## Todo

- [x] Update the API to return the number of pages and allow users to directly choose a page instead of computing an offset
- [x] Change generation of the query tree depending on the chosen settings https://github.com/meilisearch/milli/pull/598

## Small Documentation

### Default search query

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "doctor of tokio" }'
```

**result**:
```json
{
  "hits":[...],
  "estimatedTotalHits":32,
  "query":"doctor of tokio",
  "limit":20,
  "offset":0,
  "processingTimeMs":7
}
```

The default behavior doesn't change with the current Meilisearch behavior:
If we don't have enough documents to fit the requested limit, we remove the query words from the last to the first typed word.

## Search query with `optionalWords` parameter

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "doctor of tokio", "matchingStrategy": "all"}'
```

**result**:
```json
{
  "hits":[...],
  "estimatedTotalHits":1,
  "query":"doctor of tokio",
  "limit":20,
  "offset":0,
  "processingTimeMs":7
}
```

### allowed `matchingStrategy` values

#### `last`
The default behavior, If we don't have enough documents to fit the requested limit, we remove the query words from the last to the first typed word.

#### `all`
No word will be removed, If we don't have enough documents to fit the requested limit, we return the number of documents we found.

### In charge of the feature

Core: `@ManyTheFish` & `@curquiza`  
Docs: TBD
Integration: `@bidoubiwa` 


Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Many the fish <many@meilisearch.com>
2022-08-23 16:21:00 +00:00
651a22b1ed Enhance enum documentation
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-23 18:11:20 +02:00