880 Commits

Author SHA1 Message Date
Tamo
0e6394fafc
add analytics on the task route
* Add all the missing fields of the new task query type
* Create a new analytics for the task deletion
* Create a new analytics for the task creation
2022-11-16 18:45:19 +01:00
bors[bot]
637ca7b9fa
Merge #3067
3067: Fix task details serialization r=Kerollmops a=ManyTheFish

# Pull Request

- document addition task details always contain the field `indexedDocuments`
  - value is set to `null` when the task is enqueued or processing
  - value is set to `0` when the task is canceled or failed
- the field `deletedDocuments` of the document deletion task details is set to `0` when the task is canceled or failed
- the field `deletedDocuments` of the document clearAll task details is set to `0` when the task is canceled or failed
- the field `deletedTasks` of the task deletion task details is set to `0` when the task is canceled or failed
- the field `canceledTasks` of the task cancelation task details is set to `0` when the task is canceled or failed

## Related issue
Fixes #3057
Fixes #3058


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-11-16 16:15:03 +00:00
bors[bot]
f908ae2ef4
Merge #3068
3068: Prepend question mark to the `originalFilters` of the task deletion and cancelation r=irevoire a=Kerollmops

This pull request fixes #3064 by prepending [the HTTP query question mark](https://en.wikipedia.org/wiki/Question_mark#Computing) to the `originalFilters` of the task deletion and cancelation.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-11-16 15:02:04 +00:00
bors[bot]
8ddec58430
Merge #3061
3061: Name spawned threads r=irevoire a=dureuill

# Pull Request

## Related issue
None, this is to improve debuggability

## What does this PR do?
- This PR replaces the raw `thread::spawn(...)` calls by `thread::Builder::new().name(...).spawn(...).unwrap()` calls so that we can give meaningful names to threads. 
- This PR also setup the `rayon` thread pool to give a name to its threads.
- This improves debuggability, as the thread names are reported by debuggers:

<img width="411" alt="Capture d’écran 2022-11-16 à 10 26 27" src="https://user-images.githubusercontent.com/41078892/202141870-a88663aa-d2f8-494f-b4da-709fdbd072ba.png">

(screen showing vscode's debugger and its main/scheduler/indexing threads)

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-11-16 14:25:32 +00:00
Kerollmops
3525c964a7
Add the question mark to the task cancelation query filter 2022-11-16 14:30:35 +01:00
Kerollmops
ed51df41e5
Add the question mark to the task deletion query filter 2022-11-16 14:28:30 +01:00
ManyTheFish
07b28ea8cf Fix task details serialization 2022-11-16 13:47:08 +01:00
Tamo
10ab5f6a58
implements the analytics on the health and version routes 2022-11-16 13:06:10 +01:00
Tamo
684b90066d
update the analytics on the search route 2022-11-16 12:30:49 +01:00
Tamo
93ab019304
update the distinct attributes to the spec update 2022-11-16 12:25:33 +01:00
Louis Dureuil
1a1ede96de
Spawn rayon threads with names 2022-11-16 10:28:25 +01:00
Louis Dureuil
93afeedcea
Spawn threads with names 2022-11-16 09:50:47 +01:00
Tamo
25d057b75e
Add analytics on all the settings 2022-11-15 19:09:02 +01:00
Tamo
b44c381c2a
Store analytics for the documents deletions 2022-11-15 19:08:45 +01:00
Tamo
4953b62712
reformat, sorry @kero 2022-11-15 17:57:27 +01:00
Tamo
9473cccc27
add a comment over the new infos structure 2022-11-15 17:56:07 +01:00
Tamo
9327db3e91
Apply suggestions from code review
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-11-15 17:53:23 +01:00
Tamo
0fced6f270
refactor the way we send the cli informations + add the analytics for the config file and ssl usage 2022-11-15 16:32:31 +01:00
Loïc Lecrenier
52b38bee9d Make rustfmt happy
>:-(
2022-11-08 15:45:53 +01:00
Loïc Lecrenier
f5454dfa60 Make clippy happy
They're a happy clip now.
2022-11-08 15:31:08 +01:00
Loïc Lecrenier
6126fc8d98 Rename original_query to original_filters everywhere 2022-11-08 13:18:18 +01:00
Loïc Lecrenier
20fa103992 Add canceledBy task filter 2022-11-08 13:18:18 +01:00
Loïc Lecrenier
d5638d2c27 Use more precise error codes/message for the task routes
+ Allow star operator in delete/cancel tasks
+ rename originalQuery to originalFilters
+ Display error/canceled_by in task view even when they are = null
+ Rename task filter fields by using their plural forms
+ Prepare an error code for canceledBy filter
+ Only return global tasks if the API key action `index.*` is there
2022-11-08 13:18:17 +01:00
Clément Renault
932414bf72 WIP Introduce the invalid_task_uid error code 2022-11-08 13:17:56 +01:00
Tamo
c9f89d38e3
Merge branch 'main' into index-swap-error-handling 2022-10-27 18:06:45 +02:00
bors[bot]
d16ea755d8
Merge #2982
2982: Adapt task queries to account for special index swap rules r=irevoire a=loiclec

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/2970 

## What does this PR do?
- Replace the `get_tasks` method with a `get_tasks_from_authorized_indexes` which returns the list of tasks matched by the query **from the point of view of the user**. That is, it takes into consideration the list of authorised indexes as well as the special case of `IndexSwap` which should not be returned if an index_uid is specified or if any of its associated indexes are not authorised.
- Adapt the code in other places following this change
- Add some tests
- Also the method `get_task_ids_from_authorized_indexes` now takes a read transaction as argument. This is because we want to make sure that the implementation of `get_tasks_from_authorized_indexes` only uses one read transaction. Otherwise, we could (1) get a list of task ids matching the query, then (2) one of these task ids is deleted by a taskDeletion task, and finally (3) we try to get the `Task`s associated with each returned task ids, and get a `CorruptedTaskQueue` error.



Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-10-27 14:28:04 +00:00
Loïc Lecrenier
8dd7942656 Cargo fmt 2022-10-27 16:24:09 +02:00
Irevoire
4d9e9f4a9d
isolate the search in another task
In case there is a failure on milli's side that should avoid blocking the tokio main thread
2022-10-27 13:12:42 +02:00
Loïc Lecrenier
4f4fc20acf Make clippy happy 2022-10-27 13:00:30 +02:00
Loïc Lecrenier
78ffa00f98 Move index swap error handling from meilisearch-http to index-scheduler
And make index_not_found error asynchronous, since we can't know
whether the index will exist by the time the index swap task is
processed.

Improve the index-swap test to verify that future tasks are not swapped
and to test the new error messages that were introduced.
2022-10-27 11:45:38 +02:00
Loïc Lecrenier
7b93ba40bd Reimplement task queries to account for special index swap rules 2022-10-27 11:44:51 +02:00
Irevoire
6c2ecec4d0
fix the return of the task cancelation and task deletion 2022-10-27 11:35:23 +02:00
Irevoire
6280bd51a9
actually fix the test and the swap_indexes name resolution 2022-10-27 11:35:23 +02:00
Irevoire
3cf8aaa4d0
reformat 2022-10-27 11:35:23 +02:00
Kerollmops
0dd8e00929
Reapply #2601 2022-10-27 11:35:22 +02:00
Irevoire
a99ddf85f7
fix clippy once again 2022-10-27 11:35:22 +02:00
Irevoire
8ec3681cf8
fix clippy part1 2022-10-27 11:35:20 +02:00
Kerollmops
861a07792e
Remove useless task module 2022-10-27 11:35:20 +02:00
Kerollmops
ee6597da60
Fix all the tests 2022-10-27 11:35:20 +02:00
Loïc Lecrenier
e641d08846
Cargo fmt 2022-10-27 11:35:18 +02:00
Loïc Lecrenier
36c9f05998
Revert "Display more than one indexUid in a task view if necessary"
This reverts commit 1f2e253bb6b100a6c000218697dfdb4b4bf45532.
2022-10-27 11:35:18 +02:00
Loïc Lecrenier
3b158bb966
Return invalid API key error in /swap-indexes 2022-10-27 11:35:18 +02:00
Loïc Lecrenier
08b5123380
Display more than one indexUid in a task view if necessary 2022-10-27 11:35:17 +02:00
Loïc Lecrenier
1f75caae88
Fix a few index swap bugs.
1. Details of the indexSwap task
2. Query tasks with type=indexUid
3. Synchronous error message for multiple index not found
2022-10-27 11:35:17 +02:00
Irevoire
a16604af80
fix all the tests 2022-10-27 11:35:17 +02:00
Irevoire
1d014a538e
comment out a test that makes the CI crash 2022-10-27 11:35:17 +02:00
Kerollmops
2f577b6fcd
Patch the IndexScheduler in meilisearch-http to use the options struct 2022-10-27 11:35:16 +02:00
Tamo
eccbdb74cf
remove useless print
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-10-27 11:35:16 +02:00
Irevoire
a85d5b4981
test the details of all tasks type 2022-10-27 11:35:16 +02:00
Kerollmops
4736e00253
Handle the CLI options related to snapshots 2022-10-27 11:35:15 +02:00