Commit Graph

900 Commits

Author SHA1 Message Date
bb9b3e0bbb rename the two new functions 2022-12-13 17:25:49 +01:00
79656585dd Fix typos
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-12-13 17:02:07 +01:00
bbe4a84ddc Fix the import of dumps and snapshot.
Some flags were badly applied + the database wrongly deleted when they shouldn't
2022-12-13 16:39:05 +01:00
ee7a4be95c Fix the instance-uid in the data.ms
We were writing the instance-uid as bytes instead of string in the data.ms and thus we were unable to parse it later.
Also it was less practical for our user to retrieve it and send it to us.
2022-12-07 18:22:36 +01:00
1eba5d45ea Check if the master key is missing before returning an error 2022-11-24 16:02:14 +01:00
f724f8adfe Merge #3122
3122: Display the `dumpUid` as `null` until we create it r=irevoire a=Kerollmops

This PR fixes #3117 by displaying the `DumpCreation` `dumpUid` details field as `null` until we compute the dump and the task is finished.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-11-23 14:37:50 +00:00
cde2a96486 Display a null dumpUid until we computed the dump itself on disk 2022-11-23 15:16:58 +01:00
370a45a58b send the ranking rules as a string because amplitude is too dumb to process an array as a single value 2022-11-23 14:56:22 +01:00
fb785dc5ac Add more analytics on the ranking rules positions 2022-11-23 12:51:34 +01:00
aaea5f87db Don't multiply total memory returned by sysinfo anymore
sysinfo now returns bytes rather than KB
2022-11-21 15:28:05 +01:00
dfaf845382 Merge #3080
3080: Rename `originalFilters` into `originalFilter` on the cancelation and deletion routes r=irevoire a=Kerollmops

This PR fixes https://github.com/meilisearch/meilisearch/issues/3079.

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-11-17 13:49:50 +00:00
9fe32e1e3b Rename matchedDocuments into providedIds 2022-11-17 13:48:23 +01:00
388305fcb6 Rename originalFilters into originalFilters 2022-11-17 13:44:00 +01:00
49bc45e0d4 Merge #3074
3074: add the analytics of the swap-indexes route r=irevoire a=irevoire

implements https://github.com/meilisearch/specifications/pull/192/files#diff-dbac052211a8ea4b2c5d068a6264380740b022efead552b4dfc9e4e8d961f0f4R276-R284

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-11-17 11:29:35 +00:00
b478b18218 Merge #3071
3071: Analytics on the tasks route r=Kerollmops a=irevoire

Implement the missing analytics on the delete and cancel task routes.
+ Batch the analytics on the `GET tasks` route to avoid flooding ourselves while polling meilisearch.

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-11-17 11:05:56 +00:00
877d1735b1 Merge #3077
3077: Don't remove DB if unreadable r=irevoire a=dureuill

# Pull Request

## Related issue
Related to #3069

Will fix it after merging into `main`

## What does this PR do?

### User standpoint

- When the DB cannot be read after opening it, Meilisearch exits with an error message like previously, but it now leaves the DB untouched
- When the DB cannot be read after importing it from a snapshot or dump, it is still removed, like previously.

### Dev standpoint

- Add new local enum `OnFailure` that is used as a parameter to the `meilisearch_builder` closure to control when to keep or remove the DB in case of failure.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-11-17 09:38:27 +00:00
a1a29e92fd Stop removing the DB when failing to read it 2022-11-17 09:33:48 +01:00
fcca7475fa add the analytics of the swap-indexes route 2022-11-16 19:10:01 +01:00
3fc1d7e67b Update the finite pagination analytics 2022-11-16 19:01:21 +01:00
f1884d6910 batch the tasks seen events 2022-11-16 18:45:19 +01:00
0e6394fafc add analytics on the task route
* Add all the missing fields of the new task query type
* Create a new analytics for the task deletion
* Create a new analytics for the task creation
2022-11-16 18:45:19 +01:00
637ca7b9fa Merge #3067
3067: Fix task details serialization r=Kerollmops a=ManyTheFish

# Pull Request

- document addition task details always contain the field `indexedDocuments`
  - value is set to `null` when the task is enqueued or processing
  - value is set to `0` when the task is canceled or failed
- the field `deletedDocuments` of the document deletion task details is set to `0` when the task is canceled or failed
- the field `deletedDocuments` of the document clearAll task details is set to `0` when the task is canceled or failed
- the field `deletedTasks` of the task deletion task details is set to `0` when the task is canceled or failed
- the field `canceledTasks` of the task cancelation task details is set to `0` when the task is canceled or failed

## Related issue
Fixes #3057
Fixes #3058


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-11-16 16:15:03 +00:00
f908ae2ef4 Merge #3068
3068: Prepend question mark to the `originalFilters` of the task deletion and cancelation r=irevoire a=Kerollmops

This pull request fixes #3064 by prepending [the HTTP query question mark](https://en.wikipedia.org/wiki/Question_mark#Computing) to the `originalFilters` of the task deletion and cancelation.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-11-16 15:02:04 +00:00
8ddec58430 Merge #3061
3061: Name spawned threads r=irevoire a=dureuill

# Pull Request

## Related issue
None, this is to improve debuggability

## What does this PR do?
- This PR replaces the raw `thread::spawn(...)` calls by `thread::Builder::new().name(...).spawn(...).unwrap()` calls so that we can give meaningful names to threads. 
- This PR also setup the `rayon` thread pool to give a name to its threads.
- This improves debuggability, as the thread names are reported by debuggers:

<img width="411" alt="Capture d’écran 2022-11-16 à 10 26 27" src="https://user-images.githubusercontent.com/41078892/202141870-a88663aa-d2f8-494f-b4da-709fdbd072ba.png">

(screen showing vscode's debugger and its main/scheduler/indexing threads)

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-11-16 14:25:32 +00:00
3525c964a7 Add the question mark to the task cancelation query filter 2022-11-16 14:30:35 +01:00
ed51df41e5 Add the question mark to the task deletion query filter 2022-11-16 14:28:30 +01:00
07b28ea8cf Fix task details serialization 2022-11-16 13:47:08 +01:00
10ab5f6a58 implements the analytics on the health and version routes 2022-11-16 13:06:10 +01:00
684b90066d update the analytics on the search route 2022-11-16 12:30:49 +01:00
93ab019304 update the distinct attributes to the spec update 2022-11-16 12:25:33 +01:00
1a1ede96de Spawn rayon threads with names 2022-11-16 10:28:25 +01:00
93afeedcea Spawn threads with names 2022-11-16 09:50:47 +01:00
25d057b75e Add analytics on all the settings 2022-11-15 19:09:02 +01:00
b44c381c2a Store analytics for the documents deletions 2022-11-15 19:08:45 +01:00
4953b62712 reformat, sorry @kero 2022-11-15 17:57:27 +01:00
9473cccc27 add a comment over the new infos structure 2022-11-15 17:56:07 +01:00
9327db3e91 Apply suggestions from code review
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-11-15 17:53:23 +01:00
0fced6f270 refactor the way we send the cli informations + add the analytics for the config file and ssl usage 2022-11-15 16:32:31 +01:00
52b38bee9d Make rustfmt happy
>:-(
2022-11-08 15:45:53 +01:00
f5454dfa60 Make clippy happy
They're a happy clip now.
2022-11-08 15:31:08 +01:00
6126fc8d98 Rename original_query to original_filters everywhere 2022-11-08 13:18:18 +01:00
20fa103992 Add canceledBy task filter 2022-11-08 13:18:18 +01:00
d5638d2c27 Use more precise error codes/message for the task routes
+ Allow star operator in delete/cancel tasks
+ rename originalQuery to originalFilters
+ Display error/canceled_by in task view even when they are = null
+ Rename task filter fields by using their plural forms
+ Prepare an error code for canceledBy filter
+ Only return global tasks if the API key action `index.*` is there
2022-11-08 13:18:17 +01:00
932414bf72 WIP Introduce the invalid_task_uid error code 2022-11-08 13:17:56 +01:00
c9f89d38e3 Merge branch 'main' into index-swap-error-handling 2022-10-27 18:06:45 +02:00
d16ea755d8 Merge #2982
2982: Adapt task queries to account for special index swap rules r=irevoire a=loiclec

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/2970 

## What does this PR do?
- Replace the `get_tasks` method with a `get_tasks_from_authorized_indexes` which returns the list of tasks matched by the query **from the point of view of the user**. That is, it takes into consideration the list of authorised indexes as well as the special case of `IndexSwap` which should not be returned if an index_uid is specified or if any of its associated indexes are not authorised.
- Adapt the code in other places following this change
- Add some tests
- Also the method `get_task_ids_from_authorized_indexes` now takes a read transaction as argument. This is because we want to make sure that the implementation of `get_tasks_from_authorized_indexes` only uses one read transaction. Otherwise, we could (1) get a list of task ids matching the query, then (2) one of these task ids is deleted by a taskDeletion task, and finally (3) we try to get the `Task`s associated with each returned task ids, and get a `CorruptedTaskQueue` error.



Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2022-10-27 14:28:04 +00:00
8dd7942656 Cargo fmt 2022-10-27 16:24:09 +02:00
4d9e9f4a9d isolate the search in another task
In case there is a failure on milli's side that should avoid blocking the tokio main thread
2022-10-27 13:12:42 +02:00
4f4fc20acf Make clippy happy 2022-10-27 13:00:30 +02:00
78ffa00f98 Move index swap error handling from meilisearch-http to index-scheduler
And make index_not_found error asynchronous, since we can't know
whether the index will exist by the time the index swap task is
processed.

Improve the index-swap test to verify that future tasks are not swapped
and to test the new error messages that were introduced.
2022-10-27 11:45:38 +02:00