a04041c8f2
Only spawn the pool once
2024-06-19 16:25:33 +02:00
0a8f50695e
Fixes for Rust v1.79
2024-06-13 17:47:44 +02:00
e35ef31738
Small changes following review
2024-06-13 14:20:48 +02:00
3bc8f81abc
user_provided => regenerate
2024-06-12 18:12:20 +02:00
a89eea233b
Fix vectors injection
2024-06-12 17:10:19 +02:00
f5cf01e7d1
Rework extraction to use EmbedderAction
2024-06-12 14:50:55 +02:00
d1dd7e5d09
In transform for removed embedders, write back their user provided vectors in documents, and clear the writers
2024-06-12 14:50:55 +02:00
d18c1f77d7
Update embedder configs with a finer granularity
...
- no longer clear vector DB between any two embedder changes
2024-06-12 14:50:55 +02:00
7cef2299cf
Fix behavior when removing a document
2024-06-11 09:45:08 +02:00
2cdcb703d9
fix the deletion of vectors and add a test
2024-06-06 11:39:29 +02:00
d85ab23b82
rename all occurences of user_defined to user_provided for consistency
2024-06-06 11:39:29 +02:00
b7349910d9
implements mor review comments
2024-06-06 11:39:29 +02:00
376b3a19a7
makes clippy and fmt happy
2024-06-06 11:39:29 +02:00
5d50850e12
always push the user defined vectors in arroy
2024-06-06 11:39:29 +02:00
a73ccc78a6
forward the embedding config to the extractors
2024-06-06 11:39:28 +02:00
9eb6f522ea
wraps the index embedding config in a struct
2024-06-06 11:37:30 +02:00
84e498299b
Remove the vectors from the documents database
2024-06-06 11:36:11 +02:00
7a84697570
never store the _vectors as searchable or faceted fields
2024-06-06 11:36:11 +02:00
30293883e0
Fix condition mistake
2024-06-05 17:30:07 +02:00
b833be46b9
Avoid running proximity when only the exact attributes changes
2024-06-05 17:30:07 +02:00
0a4118329e
Put only_additional_fields to None if the difference gives an empty result.
2024-06-05 17:30:07 +02:00
261e92d7e6
Skip iterating over documents when the faceted field list doesn't change
2024-06-05 17:30:07 +02:00
5cd08979b1
iterate over the faceted fields instead of over the whole document
2024-06-05 17:30:07 +02:00
a998b881f6
Cache a lot of operations to know if a field must be indexed
2024-06-05 17:30:07 +02:00
b81953a65d
Add a span for the prepare_for_documents_reindexing
2024-06-05 17:30:07 +02:00
091bb157f1
Add a span for the settings diff creation
2024-06-05 17:30:07 +02:00
1b639ce44b
Reduce the number of complex calls to settings diff functions
2024-06-05 17:30:07 +02:00
87cf8a3c94
Introduce a new way to determine the operations to perform on the fields
2024-06-05 17:30:07 +02:00
0f578348f1
Introduce a dedicated function to write proximity entries in database
2024-06-05 17:30:07 +02:00
fad4675abe
Give the settings diff to the write_typed_chunk_into_index function
2024-06-05 17:30:07 +02:00
1ab03c4ede
Fix an issue with settings diff and * in the searchable attributes
2024-06-05 17:30:07 +02:00
0c6e4b2f00
Introducing a new into_del_add_obkv_conditional_operation function
2024-06-05 17:30:07 +02:00
42b3f52ef9
Introduce the SettingDiff only_additional_fields method
2024-06-05 17:30:07 +02:00
1ab88e10b9
Merge branch 'main' into merge-release-v1.8.1-in-main
2024-05-29 16:24:00 +02:00
e1fbfde6c4
Merge branch 'main' into merge-release-v1.8.1-in-main
2024-05-29 11:31:03 +02:00
27b75ec648
merge main into v1.8.1
2024-05-29 11:26:07 +02:00
d35278320e
Add support functions for accessing arroy writers and readers
2024-05-28 15:27:43 +02:00
dc949ab46a
Remove puffin usage
2024-05-27 15:59:14 +02:00
19acc65ad2
Merge #4646
...
4646: Reduce `Transform`'s disk usage r=Kerollmops a=Kerollmops
This PR implements what is described in #4485 . It reduces the number of disk writes and disk usage.
Co-authored-by: Clément Renault <clement@meilisearch.com >
2024-05-23 16:06:50 +00:00
fe17c0f52e
Construct the minimal OBKVs according to the settings diff
2024-05-23 11:23:57 +02:00
bc5663e673
FieldIdsMap no longer useful thanks to #4631
2024-05-22 16:06:15 +02:00
8a941c0241
Smaller review changes
2024-05-22 14:44:42 +02:00
16037e2169
Don't remove embedders that are not in the config from the document DB
2024-05-22 12:24:51 +02:00
500ddc76b5
Make the flattened sorter optional
2024-05-21 16:16:36 +02:00
1aa8ed9ef7
Make the original sorter optional
2024-05-21 14:53:26 +02:00
f762307838
Fix clippy
2024-05-21 13:44:20 +02:00
3e94a90722
Fixes
2024-05-21 13:39:46 +02:00
fc7e817221
Index geo points based on the settings differences
2024-05-20 12:27:26 +02:00
d05d49ffd8
Fix tests
2024-05-20 10:36:18 +02:00
0462ebbe58
Don't write an empty _vectors field
2024-05-20 10:36:18 +02:00
