d16ea755d8
Merge #2982
...
2982: Adapt task queries to account for special index swap rules r=irevoire a=loiclec
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/2970
## What does this PR do?
- Replace the `get_tasks` method with a `get_tasks_from_authorized_indexes` which returns the list of tasks matched by the query **from the point of view of the user**. That is, it takes into consideration the list of authorised indexes as well as the special case of `IndexSwap` which should not be returned if an index_uid is specified or if any of its associated indexes are not authorised.
- Adapt the code in other places following this change
- Add some tests
- Also the method `get_task_ids_from_authorized_indexes` now takes a read transaction as argument. This is because we want to make sure that the implementation of `get_tasks_from_authorized_indexes` only uses one read transaction. Otherwise, we could (1) get a list of task ids matching the query, then (2) one of these task ids is deleted by a taskDeletion task, and finally (3) we try to get the `Task`s associated with each returned task ids, and get a `CorruptedTaskQueue` error.
Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com >
2022-10-27 14:28:04 +00:00
2c31d7c50a
Apply review suggestions
2022-10-27 16:24:08 +02:00
68f80dbacf
Update version for the next release (v0.30.0) in Cargo.toml files
2022-10-27 11:35:44 +00:00
4f4fc20acf
Make clippy happy
2022-10-27 13:00:30 +02:00
7b93ba40bd
Reimplement task queries to account for special index swap rules
2022-10-27 11:44:51 +02:00
7307c4dacd
fix clippy
2022-10-27 11:35:22 +02:00
33996071ea
fix clippy from the CI
2022-10-27 11:35:21 +02:00
7c908fadcf
Remove a useless clippy silence
2022-10-27 11:35:21 +02:00
07d39776f9
fix clippy _once again_
2022-10-27 11:35:21 +02:00
8ec3681cf8
fix clippy part1
2022-10-27 11:35:20 +02:00
2ba5e3b519
Clean up some code
2022-10-27 11:35:20 +02:00
4f955e68b3
Apply suggestions from code review
2022-10-27 11:35:19 +02:00
6c98752922
move the commit before the insertion in the map
2022-10-27 11:35:19 +02:00
4e1b6b514e
update reviewer change
2022-10-27 11:35:19 +02:00
64e55b4db9
fix the index creation. When an index is being created we insert it in the index_map straight away to avoid someone else from trying to re-open it. The definitive fix should be made on milli's side
2022-10-27 11:35:18 +02:00
1f75caae88
Fix a few index swap bugs.
...
1. Details of the indexSwap task
2. Query tasks with type=indexUid
3. Synchronous error message for multiple index not found
2022-10-27 11:35:17 +02:00
29bdcb880c
update the snapshot
2022-10-27 11:35:17 +02:00
a3fc0d3bd9
Fix the last regression
2022-10-27 11:35:17 +02:00
2de8a0711a
Cargo insta test/review
2022-10-27 11:35:16 +02:00
2f577b6fcd
Patch the IndexScheduler in meilisearch-http to use the options struct
2022-10-27 11:35:16 +02:00
71b50853dc
Introduce an options struct to create the IndexScheduler
2022-10-27 11:35:16 +02:00
7074872a78
cargo insta accept
2022-10-27 11:35:15 +02:00
035e8eeff5
Clean-up some TODOs
2022-10-27 11:35:15 +02:00
e35fe33712
Fix some bugs with files
2022-10-27 11:35:15 +02:00
942b7c338b
Compress the snapshot in a tarball
2022-10-27 11:35:15 +02:00
4cafc63561
Reintroduce the versioning functions
2022-10-27 11:35:14 +02:00
89e127e4f4
Declare the auth path in the index scheduler
2022-10-27 11:35:14 +02:00
eec43ec953
Implement a first version of the snapshots
2022-10-27 11:35:14 +02:00
c063f154fb
Add the snapshots directory path to the IndexScheduler
2022-10-27 11:35:14 +02:00
e0548e42e7
Rename the Snapshot task into SnapshotCreation
2022-10-27 11:35:14 +02:00
4d43a9f5b1
Rename the index-scheduler module into insta_snapshot
2022-10-27 11:35:14 +02:00
901c405919
Fix the inta-snapshot typos in the tests
2022-10-27 11:35:13 +02:00
6db90ba6cc
Make sure that we don't delete or cancel future tasks
...
This should already have been the case before, but there is no harm
in adding another check.
2022-10-27 11:35:13 +02:00
e0821ad4b0
remove an useless dbg
2022-10-27 11:35:13 +02:00
61f0940f8c
fix an issue with the dates
2022-10-27 11:35:13 +02:00
241300d2d8
add more naive tests around the document addition + remove the old unused snapshot files
2022-10-27 11:35:13 +02:00
570b2d1167
add some naive document addition tests
2022-10-27 11:35:12 +02:00
d92425658e
Add index scheduler tests for task cancelation
2022-10-27 11:35:12 +02:00
16fac10074
Fix crash when batching an index swap task containing 0 swaps
2022-10-27 11:35:12 +02:00
0aca5e84b9
rename received_document_ids to matched_documents in the DocumentDeletion task type (reimplementation of #2826 )
2022-10-27 11:35:12 +02:00
7ed3f00b1e
reformat
2022-10-27 11:35:12 +02:00
9c00b159ba
fix clippy
2022-10-27 11:35:11 +02:00
7e52f1effb
remove a lot of unecessary clone and ref
2022-10-27 11:35:11 +02:00
4d25c159e6
Apply code review suggestions
2022-10-27 11:35:11 +02:00
e9cd6cbbee
Revert implementation of get_status to query only the database
2022-10-27 11:35:11 +02:00
424202d773
Pause the index scheduler for one second when a fatal error occurs
2022-10-27 11:35:11 +02:00
4a35eb9849
Fix (hopefully) queries that include processing tasks
2022-10-27 11:35:11 +02:00
493a8cff31
Adjust task details correctly following index swap
2022-10-27 11:35:10 +02:00
4de445d386
Start testing unexpected errors and panics in index scheduler
2022-10-27 11:35:10 +02:00
e3848b5f28
Add assert method to verify validity of index scheduler state
2022-10-27 11:35:10 +02:00
