8f5d9f501a
update the discussion link
2024-03-26 19:18:32 +01:00
8127c9a115
handle the case of a queue of zero elements
2024-03-26 19:04:39 +01:00
e7704f1fc1
add a test to ensure we effectively returns a retry-after when the search queue is full
2024-03-26 18:08:59 +01:00
e2a1bbae37
simplify and improve the http error
2024-03-26 17:53:37 +01:00
e433fd53e6
rename the method to get a permit and use it in all search requests
2024-03-26 17:28:03 +01:00
3f23fbb46d
create the experimental CLI argument
2024-03-26 16:43:40 +01:00
c41e1274dc
push and test the search queue datastructure
2024-03-26 15:56:43 +01:00
fc1c3f4a29
Merge #4466
...
4466: Implements the search cutoff r=irevoire a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4488
## What does this PR do?
- Adds a cutoff to the bucket sort after 150ms has been spent
- Adds a new setting to customize the default value of 150ms
- When the time is exceeded, we exit early with what we had the time to sort
- If the cutoff has been reached, the search details are updated with a new `Skip` ranking details for the ranking rules that were skipped
- Adds analytics to measure the total number of degraded search requests
- Adds the number of degraded search requests to the Prometheus metrics and Grafana dashboard
- The cutoff **must not** skip the filters; otherwise, we would leak documents to people who donât have the right to see them
Co-authored-by: Tamo <tamo@meilisearch.com >
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2024-03-20 13:06:53 +00:00
5046ffdf54
Merge #4512
...
4512: Revert "Revert "Merge remote-tracking branch 'origin/main' into release-v1.7.1"" r=Kerollmops a=irevoire
Reverts meilisearch/meilisearch#4510
This PR was supposed to be merged on `release-v1.7.1` not main đ€Š
Co-authored-by: Tamo <irevoire@protonmail.ch >
2024-03-20 09:14:43 +00:00
c5322df519
Revert "Revert "Merge remote-tracking branch 'origin/main' into release-v1.7.1""
2024-03-20 10:08:28 +01:00
6079141ea6
snapshot the scores side by side with the score details
2024-03-19 18:30:14 +01:00
2c3af8e513
query the detailed score detail in the test
2024-03-19 18:09:02 +01:00
098ab594eb
A score of 0.0 is now lesser than a sort result
...
handles the niche case đ© in the hybrid search where:
1. a sort ranking rule is the first rule.
2. the keyword search is skipped at the first rule.
3. the semantic search is not skipped at the first rule.
Previously, we would have the skipped search winning, whereas we want the non skipped one winning.
2024-03-19 17:32:32 +01:00
c495c8eb33
Merge #4510
...
4510: Revert "Merge remote-tracking branch 'origin/main' into release-v1.7.1" r=Kerollmops a=irevoire
In https://github.com/meilisearch/meilisearch/pull/4502 we merged main into release-v1.7.1 instead of a temporary branch thus we now need to revert this merge commit.
This reverts commit bd74cce86ad2f77e88bdtamo@meilisearch.com >
2024-03-19 16:02:24 +00:00
567194b925
Revert "Merge remote-tracking branch 'origin/main' into release-v1.7.1"
...
This reverts commit bd74cce86ad2f77e88bd
2024-03-19 16:56:21 +01:00
d8fe4fe49d
return the order in the score details
2024-03-19 15:45:04 +01:00
7b9e0d2944
forward the degraded parameter to the hybrid search
2024-03-19 15:11:21 +01:00
0ae39644f7
fix the facet search
2024-03-19 15:07:06 +01:00
bfec9468d4
Update milli/src/search/mod.rs
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2024-03-19 14:49:15 +01:00
5233534dc0
Merge #4477
...
4477: Add documentation for benchmarks r=dureuill a=dureuill
See [CONTRIBUTING.md](https://github.com/meilisearch/meilisearch/blob/benchmark-docs/CONTRIBUTING.md#logging )
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2024-03-19 13:23:48 +00:00
fced2ff9ab
Merge #4502
...
4502: Release v1.7.1 r=dureuill a=Kerollmops
Bring the v1.7.1 changes back to main.
Co-authored-by: Clément Renault <clement@meilisearch.com >
Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com >
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
2024-03-19 12:41:28 +00:00
bd74cce86a
Merge remote-tracking branch 'origin/main' into release-v1.7.1
2024-03-19 13:39:17 +01:00
f85c80d059
Merge #4503
...
4503: Add settings diff indexing benchmarks r=dureuill a=ManyTheFish
Add several benchmarks targetting settings diff-indexing enhancements
Co-authored-by: ManyTheFish <many@meilisearch.com >
2024-03-19 10:35:46 +00:00
2a92c04100
Adding new assets
2024-03-19 11:31:32 +01:00
4369e9e97c
add an error code test on the setting
2024-03-19 11:14:28 +01:00
e8516f00c4
move settings workload in root workload directory
2024-03-19 10:41:30 +01:00
7bd881b9bc
adds the degraded searches to the prometheus dashboard
2024-03-19 10:35:47 +01:00
6a0c399c2f
rename the search_cutoff parameter to search_cutoff_ms
2024-03-19 10:35:47 +01:00
038c26c118
stop returning the degraded boolean when a search was cutoff
2024-03-19 10:35:47 +01:00
ad9192fbbf
reduce the size of an integration test
2024-03-19 10:35:47 +01:00
b8cda6c300
fix the search cutoff and add a test
2024-03-19 10:35:47 +01:00
b72495eb58
fix the settings tests
2024-03-19 10:28:23 +01:00
d1db495119
add a settings for the search cutoff
2024-03-19 10:28:23 +01:00
4a467739cd
implements a first version of the cutoff without settings
2024-03-19 10:28:21 +01:00
29e71eedc7
Add benchmarks
2024-03-18 18:31:28 +01:00
10d053cd2f
Merge #4500
...
4500: Don't display dimensions as 0 when it is not set r=ManyTheFish a=dureuill
Fixes regression in embedders where `dimensions: 0` was displayed when it hadn't be set for the `openAi` source.
Was breaking a PHP SDK integration test: cbaecb8c55/tests/Settings/EmbeddersTest.php (L28)louis@meilisearch.com >
2024-03-18 15:21:24 +00:00
a302e258bd
Don't display dimensions as 0 when it is not set
2024-03-18 16:10:12 +01:00
29840473b4
Merge #4499
...
4499: Fix milli link in contributing doc r=curquiza a=mohsen-alizadeh
# Pull Request
## Related issue
Fixes #4498
## What does this PR do?
The milli link in CONTRIBUTING.md targeted the archived milli repository. it has to be changed to target to the milli crate in the main repo
## PR checklist
Please check if your PR fulfills the following requirements:
- [X] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [X] Have you read the contributing guidelines?
- [X] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Mohsen Alizadeh <mohsen@alizadeh.us >
Co-authored-by: Clémentine U. - curqui <clementine@meilisearch.com >
2024-03-18 14:39:26 +00:00
f4037c1a95
Update CONTRIBUTING.md
...
Co-authored-by: Clément Renault <renault.cle@gmail.com >
2024-03-18 15:39:01 +01:00
13cc62728b
Fix milli link in contributing doc
2024-03-17 19:29:42 -07:00
f84bcb09e1
Merge #4491
...
4491: chore: remove repetitive words r=curquiza a=shuangcui
# Pull Request
## Related issue
Fixes #<issue_number>
## What does this PR do?
- ...
## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [ ] Have you read the contributing guidelines?
- [ ] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: shuangcui <fliter@qq.com >
2024-03-14 17:44:01 +00:00
5c95b5c933
chore: remove repetitive words
...
Signed-off-by: shuangcui <fliter@qq.com >
2024-03-14 21:28:55 +08:00
0b7bebeeb6
Merge #4483
...
4483: Workflows: Fix reason param when benches are triggered from a comment. r=irevoire a=dureuill
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2024-03-13 17:05:30 +00:00
d2f77e88bd
Merge #4479
...
4479: Skip reindexing when modifying unknown faceted fields r=dureuill a=Kerollmops
This PR improves Meilisearch's decision to reindex when a faceted field is added to the settings, but not a single document contains this field. It is effectively a waste of time to reindex documents when the engine needs to know a field.
This is related to a conversation [we have with our biggest customer (internal link)](https://discord.com/channels/1006923006964154428/1101213808627830794/1217112918857089187 ). They have 170 million documents, so reindexing this amount would be problematic.
---
The image is available by using the following Docker command. You can see the advancement of the image's build [on the GitHub CI page](https://github.com/meilisearch/meilisearch/actions/runs/8251688778 ).
```
docker pull getmeili/meilisearch:prototype-no-reindex-unknown-fields-0
```
Here is the hand-made test that shows that when modifying unknown filterable attributes, here `lol`, it doesn't reindex. However, when modifying the known `genre` field, it does reindex. You can see all that by looking at the time spent processing the update.
```json
{
"uid": 3,
"indexUid": "movies",
"status": "succeeded",
"type": "settingsUpdate",
"canceledBy": null,
"details": {
"filterableAttributes": [
"genres"
]
},
"error": null,
"duration": "PT9.237703S",
"enqueuedAt": "2024-03-12T15:34:26.836083Z",
"startedAt": "2024-03-12T15:34:26.836374Z",
"finishedAt": "2024-03-12T15:34:36.074077Z"
},
{
"uid": 2,
"indexUid": "movies",
"status": "succeeded",
"type": "settingsUpdate",
"canceledBy": null,
"details": {
"filterableAttributes": [
"lol"
]
},
"error": null,
"duration": "PT0.000751S",
"enqueuedAt": "2024-03-12T15:33:53.563923Z",
"startedAt": "2024-03-12T15:33:53.565259Z",
"finishedAt": "2024-03-12T15:33:53.56601Z"
},
{
"uid": 0,
"indexUid": "movies",
"status": "succeeded",
"type": "documentAdditionOrUpdate",
"canceledBy": null,
"details": {
"receivedDocuments": 31944,
"indexedDocuments": 31944
},
"error": null,
"duration": "PT3.120723S",
"enqueuedAt": "2024-02-17T10:35:55.042864Z",
"startedAt": "2024-02-17T10:35:55.043505Z",
"finishedAt": "2024-02-17T10:35:58.164228Z"
}
```
Co-authored-by: Clément Renault <clement@meilisearch.com >
2024-03-13 16:23:32 +00:00
1d8c13f595
Merge #4487
...
4487: Update version for the next release (v1.7.1) in Cargo.toml r=Kerollmops a=meili-bot
â ïž This PR is automatically generated. Check the new version is the expected one and Cargo.lock has been updated before merging.
Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com >
2024-03-13 15:41:10 +00:00
7f3c495f5c
Update version for the next release (v1.7.1) in Cargo.toml
2024-03-13 14:49:21 +00:00
abd954755d
Merge #4476
...
4476: Make the `/facet-search` route use the `sortFacetValuesBy` setting r=irevoire a=Kerollmops
This PR fixes #4423 by ensuring that the `/facet-search` route uses the `sortFacetValuesBy` setting.
Note for the documentation team (to be moved in the tracking issue): Using the new `sortFacetValuesBy` setting can slow down the facet-search requests as Meilisearch iterates over the whole list of facet values and computes the count of documents on every entry. That is hardly or even impossible to optimize correctly.
### TODO
- [x] Create a custom HashMap wrapper for the facet `OrderBy` settings.
This wrapper will return the `OrderBy` setting of the facet, if not defined will use the default `*` one, and if not there either (strange) will fall back on the lexicographic one.
- [x] Create a `ValuesCollection` wrapper that implements the logic for the lexicographic and count order by.
- [x] Use it when there is no search query.
- [x] Use it when there is a search query with and without allowed typos.
- [x] Do not change the original logic, only use a wrapper.
- [x] Add tests
Co-authored-by: Clément Renault <clement@meilisearch.com >
2024-03-13 14:36:14 +00:00
f3fc2bd01f
Address some issues with preallocations
2024-03-13 15:22:14 +01:00
6fa3872268
Workflows: Fix reason param when benches are triggered from a comment.
2024-03-13 13:46:43 +01:00
6c9823d7bb
Add tests to sortFacetValuesBy count
2024-03-13 11:59:39 +01:00
