Commit Graph

90 Commits

Author SHA1 Message Date
dd7d758a6f Start testing unexpected errors and panics in index scheduler 2022-10-25 10:47:36 +02:00
3e45fdf7c5 rename the dumpExport to dumpCreation 2022-10-24 19:08:15 +02:00
3283bb0454 fix clippy 2022-10-22 17:04:47 +02:00
8bda5e0787 fix the autobatched document deletion 2022-10-22 16:07:55 +02:00
2387db8ab7 fix the index creation in case an index already exists 2022-10-22 16:07:55 +02:00
9219bec053 fix the failed document addition with a primary key 2022-10-22 16:07:55 +02:00
d065a3e4f0 fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting 2022-10-22 16:07:55 +02:00
3f6bd7fb11 Introduce a rustfmt file 2022-10-20 18:00:07 +02:00
2a51161258 Fix allow_index_creation useless field 2022-10-20 16:39:25 +02:00
f1acafcf1c fix the index deletion when the index doesn’t exists but would be created by one of the autobatched tasks 2022-10-20 16:39:24 +02:00
3357c439e3 it probably works but it's also horrendous 2022-10-20 16:39:23 +02:00
8514c1c12b Apply review suggestions and stop using rtxn.commit 2022-10-20 16:39:23 +02:00
14a44776f6 Implement POST /indexes-swap 2022-10-20 16:39:23 +02:00
835745ac22 Add some documentation to the index scheduler 2022-10-20 16:39:22 +02:00
1eeef1c1c8 Implement task date filters
before/after enqueued/started/finished at
2022-10-20 16:39:21 +02:00
f3c2be1eb5 fix the auto-generated details 2022-10-20 16:37:29 +02:00
aa13fc448d remove the unused variants from the autobatcher 2022-10-20 16:37:29 +02:00
188e95d00f Delete the tasks content file once the transaction has been successfully committed 2022-10-20 16:37:28 +02:00
fd73306793 Extract the must_stop flag out of the RwLock 2022-10-20 16:37:27 +02:00
07db4cfab0 Prefer using an u64 instead of a usize in some places 2022-10-20 16:37:27 +02:00
cc6e2fd1e8 Delete the persisted data when we cancel a task 2022-10-20 16:37:27 +02:00
afafe8d207 Put the original URL query in the tasks details 2022-10-20 16:37:26 +02:00
01ed1fb128 Update the canceledBy and finishedAt fields 2022-10-20 16:37:26 +02:00
74657bba2c Introduce the core algorithm of task cancelation 2022-10-20 16:37:26 +02:00
bcb7d1744a Introduce the task cancelation task type 2022-10-20 16:35:06 +02:00
b8803781dc Introduce the ProcessingTasks struct 2022-10-20 16:35:05 +02:00
baeb168b6e Delete a task's persisted data when appropriate 2022-10-20 16:34:32 +02:00
0bde0abd74 share the rtxn between the access to the tasks and to the indexes 2022-10-20 16:14:37 +02:00
8e469d8d1d flush the dump-writer only once everything has been inserted 2022-10-20 16:14:36 +02:00
b87b071718 apply most style comments of the review 2022-10-20 16:14:36 +02:00
8c9b758fa8 stop dumping the current dumping task as enqueued so it's not looping for ever 2022-10-20 16:13:17 +02:00
cac924b663 first mostly working version 2022-10-20 16:13:16 +02:00
6fae317277 update the API a little bit 2022-10-20 16:12:26 +02:00
1a3fea4d8c fix the tests 2022-10-20 16:12:26 +02:00
9323f9f1c4 write the dump export 2022-10-20 16:12:25 +02:00
7ce336306d Fix number of deleted tasks details after duplicate task deletion 2022-10-20 16:09:33 +02:00
d49d7e9c2d Add task deletion tests where the same task is deleted twice 2022-10-20 16:09:33 +02:00
9067148270 Apply suggested changes from PR review 2022-10-20 16:09:32 +02:00
012c3e986c Finish first draft of the DELETE /tasks route 2022-10-20 16:09:32 +02:00
ef3e9e87f5 Continue implementation of task deletion
1. Matched tasks are a roaring bitmap
2. Start implementation in meilisearch-http
3. Snapshots use meili-snap
4. Rename to TaskDeletion
2022-10-20 16:08:16 +02:00
7d4527728e Make sure that we do not batch tasks from different indexes 2022-10-20 16:00:58 +02:00
e533e740d4 refactor the Task a little bit 2022-10-20 15:59:52 +02:00
e6c033bd6b start moving a lot of task types to meilisearch_types 2022-10-20 15:59:52 +02:00
f456fb5e0b get rids of the index crate + the document_types crate 2022-10-20 15:57:52 +02:00
5914889a29 Apply suggestions from code review
Co-authored-by: Tamo <tamo@meilisearch.com>
2022-10-20 15:45:57 +02:00
fd73e65165 Add a DetailsView type and improve index scheduler snapshots
The DetailsView type is necessary because serde incorrectly
deserialises the `Details` type, so the database fails to correctly
decode Tasks
2022-10-20 15:45:57 +02:00
05753c663f Implement TaskDeletion in the index scheduler 2022-10-20 15:44:58 +02:00
ee352b6c7c Allow a user to disable the auto batching system 2022-10-20 15:44:58 +02:00
a358eaa816 Create the index only if the task has the rights to do so 2022-10-20 15:44:35 +02:00
134634d99a Correctly batch tasks with different index creation rights 2022-10-20 15:44:35 +02:00