f3356ddaa4
Fix the errors when using the try_map_try_init method
2024-10-01 11:11:10 +02:00
31de5c747e
WIP using try_map_try_init
2024-10-01 11:10:53 +02:00
3843240940
Prefer using Ars instead of Options
2024-10-01 11:10:53 +02:00
8cb5e7437d
try using try_map_try_init
2024-10-01 11:10:53 +02:00
5b776556fe
Add ParallelIteratorExt
2024-10-01 11:10:53 +02:00
bb7a503e5d
Compute prefix databases
...
We are now computing the prefix FST and a prefix delta in the Merger thread,
after all the databases are written, the main thread will recompute the prefix databases based on the prefix delta without needing any grenad temporary file anymore
2024-10-01 09:57:06 +02:00
64589278ac
Appease *some* of clippy warnings
2024-09-30 16:08:29 +02:00
8df6daf308
Remove fid_wordcount_docids.rs
2024-09-30 11:52:31 +02:00
5b552caf42
Fix position in insertions
2024-09-30 11:46:32 +02:00
2b51a63418
Remove dead code
2024-09-30 11:42:36 +02:00
3d8024fb2b
write the weighted fields ids map
2024-09-30 11:35:03 +02:00
4b0da0ff24
Fix inversion of field_id and position
2024-09-30 11:34:50 +02:00
960060ebdf
Fix fst builder when their is no previous FST
2024-09-25 16:53:00 +02:00
3d244451df
Reduce the lru key size from 8 to 12 bytes
2024-09-25 16:14:13 +02:00
5f53935c8a
Fix a bug in the Lru
2024-09-25 16:09:34 +02:00
29a7623c3f
Fxi some logs
2024-09-25 15:57:50 +02:00
e97041f7d0
Replace the Lru free list by a simple increment
2024-09-25 15:55:52 +02:00
52d7f3ed1c
Reduce the lru key size from 20 to 8 bytes
2024-09-25 15:37:13 +02:00
86d5e6d9ff
Use the new Lru
2024-09-25 14:54:56 +02:00
759b9b1546
Introduce a new custom Lru
2024-09-25 14:49:12 +02:00
3f7a500f3b
Build prefix fst
2024-09-25 14:36:06 +02:00
974272f2e9
Merge branch 'main' into indexer-edition-2024
2024-09-25 07:41:16 +02:00
7ad037841f
Move the tracing info to eprintln
2024-09-24 18:21:58 +02:00
e0c7067355
Expose an IndexedParallelIterator to the index function
2024-09-24 17:24:59 +02:00
6e87332410
Change the way the FST is built
2024-09-24 16:28:31 +02:00
2d1caf27df
Use eprintln to log
2024-09-24 15:59:50 +02:00
7f148c127c
Measure the SmallVec efficacity
2024-09-24 15:32:15 +02:00
4ce5d3d66d
Do not check before pushing in bitmaps
2024-09-24 09:43:16 +02:00
42b093687d
Introduce the new PushOptimizedBitmap
2024-09-23 16:38:21 +02:00
f00664247d
Add more stats about the channel message sent
2024-09-23 15:13:52 +02:00
013acb3d93
Measure merger writer channel contention
2024-09-23 11:07:59 +02:00
1113c42de0
fix broken comments
2024-09-19 16:18:36 +02:00
b6b73fe41c
Update milli/src/update/settings.rs
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2024-09-19 15:41:14 +02:00
163f8023a1
remove debug println
2024-09-19 12:13:25 +02:00
633537ccd7
fix updating documents without updating the settings
2024-09-19 12:00:58 +02:00
3f6301dbc9
fix the missing embedder name in the error message when trying to disable the binary quantization
2024-09-19 12:00:58 +02:00
2b6952eda1
rename the ArroyReader to an ArroyWrapper since it can read and write
2024-09-19 12:00:58 +02:00
79f29eed3c
fix the tests and the arroy_readers method
2024-09-19 12:00:58 +02:00
cc45e264ca
implement the binary quantization in meilisearch
2024-09-19 12:00:56 +02:00
f4ab1f168e
Prefer using Rc<str> than String when cloning a lot
2024-09-16 15:41:29 +02:00
1a0e962299
Replace hashmap by vectors in wpp
2024-09-16 15:01:20 +02:00
f13e076b8a
Use hashmap instead of Btree in wpp extractor
2024-09-16 14:40:40 +02:00
7ba49b849e
Extract and write facet databases
2024-09-16 09:35:16 +02:00
f7652186e1
WIP geo fields
2024-09-12 18:01:02 +02:00
b2f4e67c9a
Do not store useless updates
2024-09-12 15:38:31 +02:00
ff5d3b59f5
Move the document id extraction to the primary key code
2024-09-12 12:01:42 +02:00
aa69308e45
Use a bufWriter to build word FSTs
2024-09-12 11:48:00 +02:00
eb9a20ff0b
Fix fid_word_docids extraction
2024-09-12 11:08:18 +02:00
3e9198ebaa
Support guessing primary key again
2024-09-11 17:25:40 +02:00
2a0ad0982f
Fix the document counter
2024-09-11 15:59:36 +02:00
