0a40a98bb6
Make milli use edition 2021 ( #4770 )
...
* Make milli use edition 2021
* Add lifetime annotations to milli.
* Run cargo fmt
2024-07-09 17:25:39 +02:00
cd46ebd6b5
remove insta deprecating
2024-07-08 18:38:05 +02:00
1693332cab
Update arroy and always build the tree that need to be built
2024-06-24 10:14:03 +02:00
ddd564665b
Merge #4713
...
4713: Speed up facet distribution r=ManyTheFish a=Kerollmops
This PR is akin to #4682 , but this time, the same logic is applied to the facets. Bitmaps are not decoded, and we do an intersection on the bytes with the search candidates instead of materializing the RoaringBitmap to destroy it just after the operation.
A prospect raised some slow requests when performing facet searches, and I found out that the disk optimization intersection wasn't performed on the facets.
Co-authored-by: Clément Renault <clement@meilisearch.com >
2024-06-24 05:23:46 +00:00
9736e16a88
Make clippy happy
2024-06-20 13:02:44 +02:00
a04041c8f2
Only spawn the pool once
2024-06-19 16:25:33 +02:00
0a8f50695e
Fixes for Rust v1.79
2024-06-13 17:47:44 +02:00
e35ef31738
Small changes following review
2024-06-13 14:20:48 +02:00
3bc8f81abc
user_provided => regenerate
2024-06-12 18:12:20 +02:00
a89eea233b
Fix vectors injection
2024-06-12 17:10:19 +02:00
f5cf01e7d1
Rework extraction to use EmbedderAction
2024-06-12 14:50:55 +02:00
d1dd7e5d09
In transform for removed embedders, write back their user provided vectors in documents, and clear the writers
2024-06-12 14:50:55 +02:00
d18c1f77d7
Update embedder configs with a finer granularity
...
- no longer clear vector DB between any two embedder changes
2024-06-12 14:50:55 +02:00
7cef2299cf
Fix behavior when removing a document
2024-06-11 09:45:08 +02:00
2cdcb703d9
fix the deletion of vectors and add a test
2024-06-06 11:39:29 +02:00
d85ab23b82
rename all occurences of user_defined to user_provided for consistency
2024-06-06 11:39:29 +02:00
b7349910d9
implements mor review comments
2024-06-06 11:39:29 +02:00
376b3a19a7
makes clippy and fmt happy
2024-06-06 11:39:29 +02:00
5d50850e12
always push the user defined vectors in arroy
2024-06-06 11:39:29 +02:00
a73ccc78a6
forward the embedding config to the extractors
2024-06-06 11:39:28 +02:00
9eb6f522ea
wraps the index embedding config in a struct
2024-06-06 11:37:30 +02:00
84e498299b
Remove the vectors from the documents database
2024-06-06 11:36:11 +02:00
7a84697570
never store the _vectors as searchable or faceted fields
2024-06-06 11:36:11 +02:00
30293883e0
Fix condition mistake
2024-06-05 17:30:07 +02:00
b833be46b9
Avoid running proximity when only the exact attributes changes
2024-06-05 17:30:07 +02:00
0a4118329e
Put only_additional_fields to None if the difference gives an empty result.
2024-06-05 17:30:07 +02:00
261e92d7e6
Skip iterating over documents when the faceted field list doesn't change
2024-06-05 17:30:07 +02:00
5cd08979b1
iterate over the faceted fields instead of over the whole document
2024-06-05 17:30:07 +02:00
a998b881f6
Cache a lot of operations to know if a field must be indexed
2024-06-05 17:30:07 +02:00
b81953a65d
Add a span for the prepare_for_documents_reindexing
2024-06-05 17:30:07 +02:00
091bb157f1
Add a span for the settings diff creation
2024-06-05 17:30:07 +02:00
1b639ce44b
Reduce the number of complex calls to settings diff functions
2024-06-05 17:30:07 +02:00
87cf8a3c94
Introduce a new way to determine the operations to perform on the fields
2024-06-05 17:30:07 +02:00
0f578348f1
Introduce a dedicated function to write proximity entries in database
2024-06-05 17:30:07 +02:00
fad4675abe
Give the settings diff to the write_typed_chunk_into_index function
2024-06-05 17:30:07 +02:00
1ab03c4ede
Fix an issue with settings diff and * in the searchable attributes
2024-06-05 17:30:07 +02:00
0c6e4b2f00
Introducing a new into_del_add_obkv_conditional_operation function
2024-06-05 17:30:07 +02:00
42b3f52ef9
Introduce the SettingDiff only_additional_fields method
2024-06-05 17:30:07 +02:00
1ab88e10b9
Merge branch 'main' into merge-release-v1.8.1-in-main
2024-05-29 16:24:00 +02:00
e1fbfde6c4
Merge branch 'main' into merge-release-v1.8.1-in-main
2024-05-29 11:31:03 +02:00
27b75ec648
merge main into v1.8.1
2024-05-29 11:26:07 +02:00
d35278320e
Add support functions for accessing arroy writers and readers
2024-05-28 15:27:43 +02:00
dc949ab46a
Remove puffin usage
2024-05-27 15:59:14 +02:00
19acc65ad2
Merge #4646
...
4646: Reduce `Transform`'s disk usage r=Kerollmops a=Kerollmops
This PR implements what is described in #4485 . It reduces the number of disk writes and disk usage.
Co-authored-by: Clément Renault <clement@meilisearch.com >
2024-05-23 16:06:50 +00:00
fe17c0f52e
Construct the minimal OBKVs according to the settings diff
2024-05-23 11:23:57 +02:00
bc5663e673
FieldIdsMap no longer useful thanks to #4631
2024-05-22 16:06:15 +02:00
8a941c0241
Smaller review changes
2024-05-22 14:44:42 +02:00
16037e2169
Don't remove embedders that are not in the config from the document DB
2024-05-22 12:24:51 +02:00
500ddc76b5
Make the flattened sorter optional
2024-05-21 16:16:36 +02:00
1aa8ed9ef7
Make the original sorter optional
2024-05-21 14:53:26 +02:00
