Commit Graph

288 Commits

Author SHA1 Message Date
d1a4da9812 Generate a real UUIDv4 when ids are auto-generated 2022-07-12 15:14:06 +02:00
c8ebf0de47 Rename the validate function as an enriching function 2022-07-12 15:14:06 +02:00
905af2a2e9 Use the primary key and external id in the transform 2022-07-12 15:14:05 +02:00
742543091e Constify the default primary key name 2022-07-12 14:55:52 +02:00
5f1bfb73ee Extract the primary key name and make it accessible 2022-07-12 14:55:52 +02:00
6a0a0ae94f Make the Transform read from an EnrichedDocumentsBatchReader 2022-07-12 14:55:52 +02:00
8ebf5eed0d Make the nested primary key work 2022-07-12 14:55:52 +02:00
19eb3b4708 Make sur that we do not accept floats as documents ids 2022-07-12 14:55:52 +02:00
2ceeb51c37 Support the auto-generated ids when validating documents 2022-07-12 14:55:51 +02:00
399eec5c01 Fix the indexation tests 2022-07-12 14:55:51 +02:00
fcfc4caf8c Move the Object type in the lib.rs file and use it everywhere 2022-07-12 14:55:51 +02:00
0146175fe6 Introduce the validate_documents_batch function 2022-07-12 14:55:51 +02:00
bdc4263883 Introduce the validate_documents_batch function 2022-07-12 14:55:51 +02:00
e8297ad27e Fix the tests for the new DocumentsBatchBuilder/Reader 2022-07-12 14:52:56 +02:00
ebddfdb9a3 Merge #578
578: Bump uuid to 1.1.2 r=ManyTheFish a=Kerollmops

Just to [align the version with Meilisearch](https://github.com/meilisearch/meilisearch/pull/2584).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-07-05 14:56:08 +00:00
1bfdcfc84f Bump uuid to 1.1.2 2022-07-05 16:23:36 +02:00
eaf28b0628 Apply review suggestions
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-07-05 15:30:33 +02:00
3b309f654a Fasten the document deletion
When a document deletion occurs, instead of deleting the document we mark it as deleted
in the new “soft deleted” bitmap. It is then removed from the search, and all the other
endpoints.
2022-07-05 15:30:33 +02:00
d0aaa7ff00 Fix wrong internal ids assignments 2022-06-07 15:49:33 +02:00
31776fdc3f add failing test 2022-06-07 15:49:33 +02:00
86ac8568e6 Use Charabia in milli 2022-06-02 16:59:11 +02:00
08c6d50cd1 Merge #531
531: fix the mixed dataset geosearch indexing bug r=Kerollmops a=irevoire

port #529 to main

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-16 16:06:36 +00:00
0af399a6d7 fix the mixed dataset geosearch indexing bug 2022-05-16 17:37:45 +02:00
f586028f9a fix the searchable fields bug when a field is nested
Update milli/src/index.rs

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-05-16 17:24:36 +02:00
65e6aa0de2 Merge #523
523: Improve geosearch error messages r=irevoire a=irevoire

Improve the geosearch error messages (#488).
And try to parse the string as specified in https://github.com/meilisearch/meilisearch/issues/2354

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-04 13:36:11 +00:00
c55368ddd4 apply code suggestion
Co-authored-by: Kerollmops <kero@meilisearch.com>
2022-05-04 14:11:03 +02:00
211c8763b9 Make sure that we do not generate too long keys 2022-05-03 10:03:15 +02:00
7e47031bdc Add a test for long keys in LMDB 2022-05-03 10:03:13 +02:00
3cb1f6d0a1 improve geosearch error messages 2022-05-02 19:20:47 +02:00
f19d2dc548 Only flatten the required fields
apply review comments

Co-authored-by: Kerollmops <kero@meilisearch.com>
2022-04-26 12:33:46 +02:00
eb5830aa40 Add a test to make sure that long words are handled 2022-04-21 13:45:28 +02:00
00f78d6b5a Apply code suggestions
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-04-14 11:14:08 +02:00
399fba16bb only flatten an object if it's nested 2022-04-14 11:14:08 +02:00
ee64f4a936 Use smartstring to store the external id in our hashmap
We need to store all the external id (primary key) in a hashmap
associated to their internal id during.
The smartstring remove heap allocation / memory usage and should
improve the cache locality.
2022-04-13 21:22:07 +02:00
4f3ce6d9cd nested fields 2022-04-07 16:58:46 +02:00
b799f3326b rename merge_nothing to merge_ignore_values 2022-04-05 18:44:35 +02:00
201fea0fda limit extract_word_docids memory usage 2022-04-05 14:14:15 +02:00
b85cd4983e remove field_id_from_position 2022-04-05 09:50:34 +02:00
b7694c34f5 remove println 2022-04-04 21:00:07 +02:00
6cabd47c32 fix typo in comment 2022-04-04 20:59:20 +02:00
6b2c2509b2 fix bug in exact search 2022-04-04 20:54:03 +02:00
e8f06f6c06 extract exact_word_prefix_docids 2022-04-04 20:54:03 +02:00
ba0bb29cd8 refactor WordPrefixDocids to take dbs instead of indexes 2022-04-04 20:54:02 +02:00
c4c6e35352 query exact_word_docids in resolve_query_tree 2022-04-04 20:54:02 +02:00
8d46a5b0b5 extract exact word docids 2022-04-04 20:54:02 +02:00
0a77be4ec0 introduce exact_word_docids db 2022-04-04 20:54:02 +02:00
5f9f82757d refactor spawn_extraction_task 2022-04-04 20:54:02 +02:00
d5b8b5a2f8 Replace the ugly unwraps by clean if let Somes 2022-02-28 16:31:33 +01:00
8d26f3040c Remove a useless grenad file merging 2022-02-28 16:31:33 +01:00
04b1bbf932 Reintroduce appending sorted entries when possible 2022-02-24 14:50:45 +01:00