b33ec9ba3f
Introduce the FieldIdFacetIsNullDocidsExtractor
2024-09-04 17:50:08 +02:00
9c0a1cd9fd
Introduce the FieldIdFacetExistsDocidsExtractor
2024-09-04 17:48:49 +02:00
0b061f1e70
Introduce the FieldIdFacetIsEmptyDocidsExtractor
2024-09-04 17:40:24 +02:00
19d937ab21
Introduce the facet extractors
2024-09-04 17:03:54 +02:00
1d59c19cd2
Send the WordsFst by using an Mmap
2024-09-04 14:30:09 +02:00
98e48371c3
Factorize some stuff
2024-09-04 12:17:13 +02:00
6d74fb0229
Introduce the WordFidWordDocids database
2024-09-04 11:40:55 +02:00
1eb75a1040
remove milli/src/update/new/extract/tokenize_document.rs
2024-09-04 11:40:26 +02:00
3b82d8b5b9
Fix the cache to serialize entries correctly
2024-09-04 10:55:36 +02:00
781a186f75
remove milli/src/update/new/extract/extract_word_docids.rs
2024-09-04 10:28:31 +02:00
6a399556b5
Implement more searchable extractor
2024-09-04 10:20:18 +02:00
27b4cab857
Extract and write the documents and words fst in the database
2024-09-04 09:59:19 +02:00
52d32b4ee9
Move the channel sender in the closure to stop the merger thread
2024-09-03 16:08:33 +02:00
da61408e52
Remove unimplemented from document changes
2024-09-03 15:14:16 +02:00
fe69385bd7
Fix tokenizer test
2024-09-03 14:24:37 +02:00
ed19b7c3c3
Only reindex if the size increased
2024-09-03 12:07:59 +02:00
1ac008926b
Add maxBytes parameter
2024-09-03 12:07:15 +02:00
c49d892c82
Changes to prompt
2024-09-03 12:07:10 +02:00
de962a26f3
New error type when maxBytes is null
2024-09-03 12:01:04 +02:00
c1557734dc
Use the GlobalFieldsIdsMap everywhere and write it to disk
...
Co-authored-by: Dureuill <louis@meilisearch.com >
Co-authored-by: ManyTheFish <many@meilisearch.com >
2024-09-03 12:01:01 +02:00
c50d3edc4a
Integrate first searchable exctrator
2024-09-03 11:02:39 +02:00
5369bf4a62
Change some lifetimes
2024-09-02 19:51:22 +02:00
bcb1aa3d22
Find a temporary solution to par into iter on an HashMap
...
Spoiler: Do not use an HashMap but drain it into a Vec
2024-09-02 19:39:48 +02:00
9b7858fb90
Expose the new indexer
2024-09-02 15:21:59 +02:00
ab01679a8f
Remove the useless option from the document changes
2024-09-02 15:21:00 +02:00
521775f788
I push for Many
2024-09-02 15:10:21 +02:00
72e7b7846e
Renaming the indexers
2024-09-02 14:42:27 +02:00
6526ce1208
Fix the merging of documents
2024-09-02 14:41:20 +02:00
21296190a3
Reindex embedders
2024-09-02 13:00:53 +02:00
4464d319af
Change default template to use the new facility
2024-09-02 11:30:59 +02:00
580ea2f450
Pass the fields <-> ids map with metadata to render
2024-09-02 11:30:10 +02:00
915cf4bae5
Add field.is_searchable property to fields
2024-09-02 11:28:53 +02:00
e639ec79d1
Move the indexers into their own modules
2024-09-02 10:42:19 +02:00
bb885a5810
Fix the merge for roaring bitmap
2024-09-01 23:20:19 +02:00
b625d31c7d
Introduce the PartialDumpIndexer indexer that generates document ids in parallel
2024-08-30 15:07:21 +02:00
6487a67f2b
Introduce the ConcurrentAvailableIds struct and rename the other to AvailableIds
2024-08-30 15:06:50 +02:00
271ce91b3b
Add the rayon Threadpool to the index function parameter
2024-08-30 14:34:24 +02:00
54f2eb4507
Remove duplication of grenad merger
2024-08-30 14:34:05 +02:00
794ebcd582
Replace grenad with the new grenad various-improvement branch
2024-08-30 11:53:59 +02:00
b7c77c7a39
Use the latest version of the obkv crate
2024-08-30 11:53:59 +02:00
0c57cf7565
Replace obkv with the temporary new version of it
2024-08-30 11:53:58 +02:00
27df9e6c73
Introduce the indexer::index function that runs the indexation
2024-08-30 11:53:58 +02:00
45c060831e
Introduce typed channels and the merger loop
2024-08-30 11:53:58 +02:00
874c1ac538
First channels types
2024-08-30 11:53:58 +02:00
e6ffa4d454
Implement the document merge function for the replace method
2024-08-30 11:53:58 +02:00
637a9c8bdd
Implement the document merge function for the update method
2024-08-30 11:53:58 +02:00
c683fa98e6
WIP
...
Co-authored-by: Kerollmops <clement@meilisearch.com >
Co-authored-by: ManyTheFish <many@meilisearch.com >
2024-08-30 11:53:57 +02:00
9a756cf2c5
Merge #4888
...
4888: bring back v1.10.0 into main r=Kerollmops a=ManyTheFish
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com >
Co-authored-by: ManyTheFish <many@meilisearch.com >
2024-08-27 14:02:08 +00:00
b12e997c8a
Add pinyin flag
2024-08-21 14:38:04 +02:00
8bf89ec394
Infer locales from index settings
2024-08-21 10:47:40 +02:00
