82647bcded
When retrieveVectors is true, retrieve _vectors.embedder even if there are no vector for that embedder
2024-07-16 13:39:00 +02:00
e83da00446
Milli changes to match to allow for more flexible lifetimes
2024-07-11 16:29:35 +02:00
7fb3e378ff
Do not fail sort comparisons when the field name or target point are different
2024-07-11 16:28:14 +02:00
29b44e5541
Merge #4626
...
4626: Edit Documents with Rhai r=ManyTheFish a=Kerollmops
This PR introduces a first version of [the _Update Documents with Function_ (internal)](https://www.notion.so/meilisearch/Update-Documents-by-Function-45f87b13e61c4435b73943768a490808 ). It uses [the Rhai programming language](https://rhai.rs/ ) to let users express the modifications they want apply.
You can read more about the way to use this functions on [the Usage PRD Page](https://meilisearch.notion.site/Edit-Documents-with-Rhai-0cff8fea7655436592e7c8a6de932062?pvs=25 ). The [prototype is available](https://github.com/meilisearch/meilisearch/actions/runs/9038384483 ) through Docker by using the following command:
```
docker run -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-edit-documents-with-rhai-3
```
## TODO
- [x] Support the `DocumentEdition` task in dumps.
- [x] Remove the unwraps and panics.
- [x] Improve error codes for the `function` parameter.
- [x] [Update Rhai to v1.19.0](https://github.com/rhaiscript/rhai/releases/tag/v1.19.0 ) đ
- [x] Make it an experimental feature (only restrict the HTTP calls).
- [x] It must be possible not to send a context.
- [x] Rebase on main.
- [x] Check that the script cannot do any io.
- [x] ~Introduce a `Documents.edit` action or~ require the `Documents.all` action.
- [x] Change the `editionCode` to the clearer `function` field name in the tasks.
- [x] Support a user provided context and maybe more (but keep function execution isolated for reproducibility).
- [x] Support deleting documents when the `doc` is `()` (nil, null).
- [x] Support canceling document edition.
- [x] Multithread document edition by using rayon (and [rayon-par-bridge](https://docs.rs/rayon-par-bridge/latest/rayon_par_bridge/ )).
- [x] Limit the number of instruction by function execution.
- [ ] ~Expose the limit of instructions in the settings.~ Not sure, in fact.
- [x] Ignore unmodified documents in the tasks count.
- [x] Make the `filter` field optional (not forced to be `null`).
Co-authored-by: Clément Renault <clement@meilisearch.com >
2024-07-11 09:02:55 +00:00
6e80364c50
Apply review comments
2024-07-11 11:00:27 +02:00
3bac22fd87
We do not do intersections with the universe when it is related to cache
2024-07-10 16:49:36 +02:00
ce61cb7fe6
Simplify and speedup an intersection pass
2024-07-10 16:49:36 +02:00
1693d1a311
Simplify the check to decide to stop a loop
2024-07-10 16:49:36 +02:00
febea735ca
Remove the unused universe parameter from resolve_negative_phrases
2024-07-10 16:49:36 +02:00
93ba051094
Remove the invalid get_phrases_docids universe parameter
2024-07-10 16:49:35 +02:00
cd7a20fa32
Make it work by avoid storing invalid stuff in the cache
2024-07-10 16:49:35 +02:00
41f51adbec
Do less useless intersections
2024-07-10 16:49:35 +02:00
0ca1a4e805
Always do the intersections with the universe
2024-07-10 16:49:34 +02:00
50a7393c55
Modify the compute_query_term_subset_docids function to accept the universe
2024-07-10 16:49:34 +02:00
837274f853
Restrict even more the Rhai engine
2024-07-10 16:30:18 +02:00
aace587dd1
Create errors for the internal processing ones
2024-07-10 16:29:18 +02:00
f35d6710f3
Update rhai to v1.19.0
2024-07-10 16:29:17 +02:00
81ec0abad1
Use the new rayon-par-bridge library
2024-07-10 16:29:04 +02:00
b67d385cf0
Parallelize the edition functions
2024-07-10 16:28:54 +02:00
dfecb25814
Disable the time package
2024-07-10 16:28:37 +02:00
2eae2015d7
Support aborting documents edition by function
2024-07-10 16:28:15 +02:00
33fa17bf12
Support deleting documents with functions
2024-07-10 16:28:15 +02:00
400e6b93ce
Support user-provided context for documents edition
2024-07-10 16:28:15 +02:00
f4add93043
Limit the number of script operations
2024-07-10 16:28:14 +02:00
2fae96ac14
Show the actual number of actually edited documents
2024-07-10 16:28:14 +02:00
45af18ae9c
Check the Rhai syntax before accepting the script
2024-07-10 16:28:13 +02:00
2d97164d9f
It works perfectly with some Rhai
2024-07-10 16:28:13 +02:00
efc156a4a4
Executing Lua works correctly
2024-07-10 16:27:36 +02:00
2099b4f0dd
Merge #4786
...
4786: Update dependencies r=Kerollmops a=irevoire
# Pull Request
## Related issue
Fixes #4753
## What does this PR do?
- Update all dependencies except rustls
- [x] Release charabia
- [x] Update charabia
- [x] Double check that the docker build works after updating charabia
Co-authored-by: Tamo <tamo@meilisearch.com >
Co-authored-by: Clément Renault <clement@meilisearch.com >
2024-07-10 13:23:54 +00:00
9d6885793e
Upgrade dependencies
2024-07-10 13:46:24 +02:00
5f4530ce57
Remove more unused dependencies
2024-07-10 13:36:34 +02:00
4d5005b01a
make clippy happy
2024-07-10 10:06:59 +02:00
952e742321
update charabia
2024-07-09 23:41:29 +02:00
0a40a98bb6
Make milli use edition 2021 ( #4770 )
...
* Make milli use edition 2021
* Add lifetime annotations to milli.
* Run cargo fmt
2024-07-09 17:25:39 +02:00
cd46ebd6b5
remove insta deprecating
2024-07-08 18:38:05 +02:00
6afa578688
update most incompatible dependencies
2024-07-08 18:31:15 +02:00
300bdfc2a7
update most dependencies
2024-07-08 18:09:12 +02:00
128e6c7502
Search: spans with a finer granularity
2024-07-02 16:13:53 +02:00
015d90a962
merge main
2024-07-01 11:50:36 +02:00
e53de15b8e
Fix behavior of limit and offset for hybrid search when keyword results are returned early
...
The test is fixed
2024-06-27 14:25:33 +02:00
ce08dc509b
add more tests and improve the location of the error
2024-06-27 11:51:45 +02:00
1daaed163a
Make _vectors.:embedding.regenerate mandatory + tests + error messages
2024-06-27 11:04:58 +02:00
7e3c306c54
Merge #4725
...
4725: Store primary key as String when Number exceeds i64 range r=irevoire a=JWSong
# Pull Request
## Related issue
Fixes #4696
## What does this PR do?
- When a Number value exceeding the range of i64 is received as a primary key, it will be stored as a String.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: JWSong <thdwjddn123@gmail.com >
2024-06-26 07:06:04 +00:00
dcdc83946f
accept large number as string
2024-06-25 21:41:47 +09:00
3c4c46377b
Merge #4665
...
4665: Add missing Korean support r=ManyTheFish a=junhochoi
Some configuration is missing `korean` features and add a test case in `milli/src/search/mod.rs`.
# Pull Request
## Related issue
#3443 #3882
## What does this PR do?
- Improvement on enabling Korean support
Inspired by the work (#3882 ) I tried to enable Korean features but have found some missing configurations.
This PR is add those missing configs (mostly Cargo.toml) and added one test case.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Junho Choi <jh.choi@catenoid.net >
2024-06-25 11:51:21 +00:00
d75e0098c7
Fixes for Rust v1.79
2024-06-25 11:16:06 +02:00
2e0ff56f3f
Add missing Korean support
...
Some configuration is missing `korean` features and
add a test case in `milli/src/search/mod.rs`.
2024-06-25 12:45:21 +09:00
1693332cab
Update arroy and always build the tree that need to be built
2024-06-24 10:14:03 +02:00
ddd564665b
Merge #4713
...
4713: Speed up facet distribution r=ManyTheFish a=Kerollmops
This PR is akin to #4682 , but this time, the same logic is applied to the facets. Bitmaps are not decoded, and we do an intersection on the bytes with the search candidates instead of materializing the RoaringBitmap to destroy it just after the operation.
A prospect raised some slow requests when performing facet searches, and I found out that the disk optimization intersection wasn't performed on the facets.
Co-authored-by: Clément Renault <clement@meilisearch.com >
2024-06-24 05:23:46 +00:00
9736e16a88
Make clippy happy
2024-06-20 13:02:44 +02:00
