Commit Graph

446 Commits

Author SHA1 Message Date
87cac158c4 Update index-scheduler/src/batch.rs 2022-10-27 18:08:21 +02:00
c9f89d38e3 Merge branch 'main' into index-swap-error-handling 2022-10-27 18:06:45 +02:00
01687c87a2 Get rids of the unecessary tasks when an index_uid is specified 2022-10-27 18:00:04 +02:00
313f204f39 merge the settings and do one indexation at the end 2022-10-27 16:38:21 +02:00
d16ea755d8 Merge #2982
2982: Adapt task queries to account for special index swap rules r=irevoire a=loiclec

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/2970 

## What does this PR do?
- Replace the `get_tasks` method with a `get_tasks_from_authorized_indexes` which returns the list of tasks matched by the query **from the point of view of the user**. That is, it takes into consideration the list of authorised indexes as well as the special case of `IndexSwap` which should not be returned if an index_uid is specified or if any of its associated indexes are not authorised.
- Adapt the code in other places following this change
- Add some tests
- Also the method `get_task_ids_from_authorized_indexes` now takes a read transaction as argument. This is because we want to make sure that the implementation of `get_tasks_from_authorized_indexes` only uses one read transaction. Otherwise, we could (1) get a list of task ids matching the query, then (2) one of these task ids is deleted by a taskDeletion task, and finally (3) we try to get the `Task`s associated with each returned task ids, and get a `CorruptedTaskQueue` error.



Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-10-27 14:28:04 +00:00
8152ab5dfc Revert change in initialisation of TempDir for index scheduler tests 2022-10-27 16:26:17 +02:00
2c31d7c50a Apply review suggestions 2022-10-27 16:24:08 +02:00
68f80dbacf Update version for the next release (v0.30.0) in Cargo.toml files 2022-10-27 11:35:44 +00:00
4f4fc20acf Make clippy happy 2022-10-27 13:00:30 +02:00
78ffa00f98 Move index swap error handling from meilisearch-http to index-scheduler
And make index_not_found error asynchronous, since we can't know
whether the index will exist by the time the index swap task is
processed.

Improve the index-swap test to verify that future tasks are not swapped
and to test the new error messages that were introduced.
2022-10-27 11:45:38 +02:00
7b93ba40bd Reimplement task queries to account for special index swap rules 2022-10-27 11:44:51 +02:00
7307c4dacd fix clippy 2022-10-27 11:35:22 +02:00
33996071ea fix clippy from the CI 2022-10-27 11:35:21 +02:00
7c908fadcf Remove a useless clippy silence 2022-10-27 11:35:21 +02:00
07d39776f9 fix clippy _once again_ 2022-10-27 11:35:21 +02:00
8ec3681cf8 fix clippy part1 2022-10-27 11:35:20 +02:00
2ba5e3b519 Clean up some code 2022-10-27 11:35:20 +02:00
4f955e68b3 Apply suggestions from code review 2022-10-27 11:35:19 +02:00
6c98752922 move the commit before the insertion in the map 2022-10-27 11:35:19 +02:00
4e1b6b514e update reviewer change 2022-10-27 11:35:19 +02:00
64e55b4db9 fix the index creation. When an index is being created we insert it in the index_map straight away to avoid someone else from trying to re-open it. The definitive fix should be made on milli's side 2022-10-27 11:35:18 +02:00
1f75caae88 Fix a few index swap bugs.
1. Details of the indexSwap task
2. Query tasks with type=indexUid
3. Synchronous error message for multiple index not found
2022-10-27 11:35:17 +02:00
29bdcb880c update the snapshot 2022-10-27 11:35:17 +02:00
a3fc0d3bd9 Fix the last regression 2022-10-27 11:35:17 +02:00
2de8a0711a Cargo insta test/review 2022-10-27 11:35:16 +02:00
2f577b6fcd Patch the IndexScheduler in meilisearch-http to use the options struct 2022-10-27 11:35:16 +02:00
71b50853dc Introduce an options struct to create the IndexScheduler 2022-10-27 11:35:16 +02:00
7074872a78 cargo insta accept 2022-10-27 11:35:15 +02:00
035e8eeff5 Clean-up some TODOs 2022-10-27 11:35:15 +02:00
e35fe33712 Fix some bugs with files 2022-10-27 11:35:15 +02:00
942b7c338b Compress the snapshot in a tarball 2022-10-27 11:35:15 +02:00
4cafc63561 Reintroduce the versioning functions 2022-10-27 11:35:14 +02:00
89e127e4f4 Declare the auth path in the index scheduler 2022-10-27 11:35:14 +02:00
eec43ec953 Implement a first version of the snapshots 2022-10-27 11:35:14 +02:00
c063f154fb Add the snapshots directory path to the IndexScheduler 2022-10-27 11:35:14 +02:00
e0548e42e7 Rename the Snapshot task into SnapshotCreation 2022-10-27 11:35:14 +02:00
4d43a9f5b1 Rename the index-scheduler module into insta_snapshot 2022-10-27 11:35:14 +02:00
901c405919 Fix the inta-snapshot typos in the tests 2022-10-27 11:35:13 +02:00
6db90ba6cc Make sure that we don't delete or cancel future tasks
This should already have been the case before, but there is no harm
in adding another check.
2022-10-27 11:35:13 +02:00
e0821ad4b0 remove an useless dbg 2022-10-27 11:35:13 +02:00
61f0940f8c fix an issue with the dates 2022-10-27 11:35:13 +02:00
241300d2d8 add more naive tests around the document addition + remove the old unused snapshot files 2022-10-27 11:35:13 +02:00
570b2d1167 add some naive document addition tests 2022-10-27 11:35:12 +02:00
d92425658e Add index scheduler tests for task cancelation 2022-10-27 11:35:12 +02:00
16fac10074 Fix crash when batching an index swap task containing 0 swaps 2022-10-27 11:35:12 +02:00
0aca5e84b9 rename received_document_ids to matched_documents in the DocumentDeletion task type (reimplementation of #2826) 2022-10-27 11:35:12 +02:00
7ed3f00b1e reformat 2022-10-27 11:35:12 +02:00
9c00b159ba fix clippy 2022-10-27 11:35:11 +02:00
7e52f1effb remove a lot of unecessary clone and ref 2022-10-27 11:35:11 +02:00
4d25c159e6 Apply code review suggestions 2022-10-27 11:35:11 +02:00