c0f2724c2d
get rids of the new introduced error code in favor of an io::Error
2023-10-10 15:12:23 +02:00
d772073dfa
use a bufreader everytime there is a grenad<file>
2023-10-10 15:00:30 +02:00
eef95de30e
First iteration on exposing puffin profiling
2023-07-18 17:38:13 +02:00
602ad98cb8
improve the way we handle the fsts
2023-05-22 11:15:14 +02:00
4391cba6ca
fix the addition + deletion bug
2023-05-17 18:28:57 +02:00
895ab2906c
apply review suggestions
2023-02-16 18:42:47 +01:00
74dcfe9676
Fix a bug when you update a document that was already present in the db, deleted and then inserted again in the same transform
2023-02-14 19:09:40 +01:00
1b1703a609
make a small optimization to merge obkvs a little bit faster
2023-02-14 18:32:41 +01:00
fb5e4957a6
fix and test the early exit in case a grenad ends with a deletion
2023-02-14 18:23:57 +01:00
8de3c9f737
Update milli/src/update/index_documents/transform.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com >
2023-02-14 17:57:14 +01:00
43a19d0709
document the operation enum + the grenads
2023-02-14 17:55:26 +01:00
746b31c1ce
makes clippy happy
2023-02-09 12:23:01 +01:00
93f130a400
fix all warnings
2023-02-08 20:57:35 +01:00
421a9cf05e
provide a new method on the transform to remove documents
2023-02-08 16:06:09 +01:00
8f64fba1ce
rewrite the current transform to handle a new byte specifying the kind of operation it's merging
2023-02-08 12:53:38 +01:00
89675e5f15
clippy: Replace seek 0 by rewind
2023-01-31 09:32:40 +01:00
13c95d25aa
Remove uses of UserError::MissingPrimaryKey not related to inference
2022-12-21 15:13:36 +01:00
67d8cec209
Fix bug in handling of soft deleted documents when updating settings
2022-12-06 15:09:19 +01:00
37b3c5c323
Fix transform to use all_documents and ignore soft_deleted documents
2022-11-08 14:23:16 +01:00
3009981d31
Fix clippy errors
...
Add clippy job
Add clippy job to CI
2022-11-04 08:58:14 +09:00
c8f16530d5
Merge #616
...
616: Introduce an indexation abortion function when indexing documents r=Kerollmops a=Kerollmops
Co-authored-by: Kerollmops <clement@meilisearch.com >
Co-authored-by: Clément Renault <clement@meilisearch.com >
2022-10-26 11:41:18 +00:00
6603437cb1
Introduce an indexation abortion function when indexing documents
2022-10-17 17:28:03 +02:00
beb987d3d1
Fixing piles of clippy errors.
...
Most of these are calling clone when the struct supports Copy.
Many are using & and &mut on `self` when the function they are called
from already has an immutable or mutable borrow so this isn't needed.
I tried to stay away from actual changes or places where I'd have to
name fresh variables.
2022-10-13 22:02:54 +02:00
3794962330
Use an unstable algorithm for grenad::Sorter when possible
2022-09-13 14:49:53 +02:00
2668f841d1
Fix update indexing
2022-08-17 15:03:37 +02:00
7fc35c5586
remove the useless prints
2022-08-02 10:31:22 +02:00
f156d7dd3b
Stop reindexing already indexed documents
2022-08-02 10:31:20 +02:00
fc9f3f31e7
Change DocumentsBatchReader to access cursor and index at same time
...
Otherwise it is not possible to iterate over all documents while
using the fields index at the same time.
2022-07-18 16:08:14 +02:00
ab1571cdec
Simplify Transform::read_documents, enabled by enriched documents reader
2022-07-18 12:45:47 +02:00
5d149d631f
Remove tests for a function that no more exists
2022-07-12 15:14:06 +02:00
0bbcc7b180
Expose the DocumentId struct to be sure to inject the generated ids
2022-07-12 15:14:06 +02:00
905af2a2e9
Use the primary key and external id in the transform
2022-07-12 15:14:05 +02:00
6a0a0ae94f
Make the Transform read from an EnrichedDocumentsBatchReader
2022-07-12 14:55:52 +02:00
8ebf5eed0d
Make the nested primary key work
2022-07-12 14:55:52 +02:00
fcfc4caf8c
Move the Object type in the lib.rs file and use it everywhere
2022-07-12 14:55:51 +02:00
0146175fe6
Introduce the validate_documents_batch function
2022-07-12 14:55:51 +02:00
bdc4263883
Introduce the validate_documents_batch function
2022-07-12 14:55:51 +02:00
e8297ad27e
Fix the tests for the new DocumentsBatchBuilder/Reader
2022-07-12 14:52:56 +02:00
ebddfdb9a3
Merge #578
...
578: Bump uuid to 1.1.2 r=ManyTheFish a=Kerollmops
Just to [align the version with Meilisearch](https://github.com/meilisearch/meilisearch/pull/2584 ).
Co-authored-by: Kerollmops <clement@meilisearch.com >
2022-07-05 14:56:08 +00:00
1bfdcfc84f
Bump uuid to 1.1.2
2022-07-05 16:23:36 +02:00
3b309f654a
Fasten the document deletion
...
When a document deletion occurs, instead of deleting the document we mark it as deleted
in the new “soft deleted” bitmap. It is then removed from the search, and all the other
endpoints.
2022-07-05 15:30:33 +02:00
d0aaa7ff00
Fix wrong internal ids assignments
2022-06-07 15:49:33 +02:00
f19d2dc548
Only flatten the required fields
...
apply review comments
Co-authored-by: Kerollmops <kero@meilisearch.com >
2022-04-26 12:33:46 +02:00
00f78d6b5a
Apply code suggestions
...
Co-authored-by: Clément Renault <clement@meilisearch.com >
2022-04-14 11:14:08 +02:00
399fba16bb
only flatten an object if it's nested
2022-04-14 11:14:08 +02:00
ee64f4a936
Use smartstring to store the external id in our hashmap
...
We need to store all the external id (primary key) in a hashmap
associated to their internal id during.
The smartstring remove heap allocation / memory usage and should
improve the cache locality.
2022-04-13 21:22:07 +02:00
4f3ce6d9cd
nested fields
2022-04-07 16:58:46 +02:00
f367cc2e75
Finally bump grenad to v0.4.1
2022-02-16 15:28:48 +01:00
0c84a40298
document batch support
...
reusable transform
rework update api
add indexer config
fix tests
review changes
Co-authored-by: Clément Renault <clement@meilisearch.com >
fmt
2022-01-19 12:40:20 +01:00
3599df77f0
Change some error messages
2021-10-27 19:33:01 +02:00
