Louis Dureuil
f5cf01e7d1
Rework extraction to use EmbedderAction
2024-06-12 14:50:55 +02:00
Louis Dureuil
d1dd7e5d09
In transform for removed embedders, write back their user provided vectors in documents, and clear the writers
2024-06-12 14:50:55 +02:00
Louis Dureuil
d18c1f77d7
Update embedder configs with a finer granularity
...
- no longer clear vector DB between any two embedder changes
2024-06-12 14:50:55 +02:00
Louis Dureuil
d0b05ae691
Add EmbedderAction to settings
2024-06-12 14:50:54 +02:00
Louis Dureuil
e9bf4eb100
Reformulate ParsedVectorsDiff in terms of VectorState
2024-06-12 14:11:44 +02:00
Louis Dureuil
b368105272
Add EmbedderConfigs::into_inner
2024-06-12 14:11:44 +02:00
Tamo
3493093c4f
add a batch of tests
2024-06-11 16:03:54 +02:00
Louis Dureuil
7cef2299cf
Fix behavior when removing a document
2024-06-11 09:45:08 +02:00
Tamo
600e97d9dc
gate the retrieveVectors parameter behind the vectors feature flag
2024-06-10 18:26:12 +02:00
Tamo
0502b17501
log the state of the index-scheduler in all failed tests
2024-06-10 10:52:49 +02:00
Tamo
734d1c53ad
fix a panic in yaup
2024-06-06 16:31:07 +02:00
Tamo
63dded3961
implements the new analytics for the get documents routes
2024-06-06 11:39:29 +02:00
Tamo
2cdcb703d9
fix the deletion of vectors and add a test
2024-06-06 11:39:29 +02:00
Tamo
6607875f49
add the retrieveVectors parameter to the get and fetch documents route
2024-06-06 11:39:29 +02:00
Tamo
ea61e5cbec
makes clippy happy x2
2024-06-06 11:39:29 +02:00
Tamo
31a793d226
fix the regeneration of the embeddings in the search
2024-06-06 11:39:29 +02:00
Tamo
d85ab23b82
rename all occurences of user_defined to user_provided for consistency
2024-06-06 11:39:29 +02:00
Tamo
b7349910d9
implements mor review comments
2024-06-06 11:39:29 +02:00
Tamo
49fa41ce65
apply first round of review comments
2024-06-06 11:39:29 +02:00
Tamo
400cf3eb92
add api error test on the new retrieveVectors parameter
2024-06-06 11:39:29 +02:00
Tamo
376b3a19a7
makes clippy and fmt happy
2024-06-06 11:39:29 +02:00
Tamo
d92c173fdc
update the new similar tests
2024-06-06 11:39:29 +02:00
Tamo
b867829ef1
remove useless dbg
2024-06-06 11:39:29 +02:00
Tamo
6b29676e7e
update snapshots
2024-06-06 11:39:29 +02:00
Tamo
caad40964a
implements the analytics
2024-06-06 11:39:29 +02:00
Tamo
cc5dca8321
fix two bug and add a dump test
2024-06-06 11:39:29 +02:00
Tamo
5d50850e12
always push the user defined vectors in arroy
2024-06-06 11:39:29 +02:00
Tamo
a73ccc78a6
forward the embedding config to the extractors
2024-06-06 11:39:28 +02:00
Tamo
9eb6f522ea
wraps the index embedding config in a struct
2024-06-06 11:37:30 +02:00
Tamo
04f6523f3c
expose a new parameter to retrieve the embedders at search time
2024-06-06 11:36:11 +02:00
Tamo
30d66abf8d
fix the test
2024-06-06 11:36:11 +02:00
Tamo
84e498299b
Remove the vectors from the documents database
2024-06-06 11:36:11 +02:00
Tamo
7a84697570
never store the _vectors as searchable or faceted fields
2024-06-06 11:36:11 +02:00
Tamo
4148fbbe85
provide a method to get all the nested fields ids from a name
2024-06-06 11:36:11 +02:00
meili-bors[bot]
cb765ad249
Merge #4684
...
4684: Update Charabia v0.8.11 r=irevoire a=ManyTheFish
# Update Charabia v0.8.11
### Adds a new normalizer to normalize œ to oe and æ to ae
Now search words containing `œ` or `æ` will be retrieved using `oe` or `ae`, like `Daemon` <=> `Dæmon`
### Fix: make `chinese-normalization-pinyin` feature flag compile
Fixes #4629
Co-authored-by: ManyTheFish <many@meilisearch.com >
2024-06-06 08:59:49 +00:00
ManyTheFish
2e50c6ec81
Update Charabia
2024-06-06 10:18:43 +02:00
meili-bors[bot]
40b2345394
Merge #4680
...
4680: Speedup additional searchables r=Kerollmops a=ManyTheFish
Fixes #4492 .
## To Do
- [x] Do not call the `InnerSettingsDiff::only_additional_fields` function too many times
Co-authored-by: Clément Renault <clement@meilisearch.com >
Co-authored-by: ManyTheFish <many@meilisearch.com >
2024-06-05 15:39:28 +00:00
ManyTheFish
30293883e0
Fix condition mistake
2024-06-05 17:30:07 +02:00
ManyTheFish
b833be46b9
Avoid running proximity when only the exact attributes changes
2024-06-05 17:30:07 +02:00
ManyTheFish
0a4118329e
Put only_additional_fields to None if the difference gives an empty result.
2024-06-05 17:30:07 +02:00
ManyTheFish
261e92d7e6
Skip iterating over documents when the faceted field list doesn't change
2024-06-05 17:30:07 +02:00
ManyTheFish
5cd08979b1
iterate over the faceted fields instead of over the whole document
2024-06-05 17:30:07 +02:00
Clément Renault
2af7e4dbe9
Rename the embeddings workloads
2024-06-05 17:30:07 +02:00
Clément Renault
a998b881f6
Cache a lot of operations to know if a field must be indexed
2024-06-05 17:30:07 +02:00
Clément Renault
b81953a65d
Add a span for the prepare_for_documents_reindexing
2024-06-05 17:30:07 +02:00
Clément Renault
091bb157f1
Add a span for the settings diff creation
2024-06-05 17:30:07 +02:00
Clément Renault
1b639ce44b
Reduce the number of complex calls to settings diff functions
2024-06-05 17:30:07 +02:00
Clément Renault
87cf8a3c94
Introduce a new way to determine the operations to perform on the fields
2024-06-05 17:30:07 +02:00
Clément Renault
0f578348f1
Introduce a dedicated function to write proximity entries in database
2024-06-05 17:30:07 +02:00
Clément Renault
fad4675abe
Give the settings diff to the write_typed_chunk_into_index function
2024-06-05 17:30:07 +02:00
