Commit Graph

4772 Commits

Author SHA1 Message Date
12c3d432f9 Merge #3293
3293: Explicitly restrict log level options to those that are documented r=loiclec a=loiclec

Fixes https://github.com/meilisearch/meilisearch/issues/3292





Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2023-01-04 10:30:35 +00:00
d082ded7ad Explicitly restrict log level options to those that are documented
Fixes https://github.com/meilisearch/meilisearch/issues/3292
2023-01-04 09:40:24 +01:00
947f08793a Merge #3296
3296: Remove `--disable-auto-batching` CLI option r=gmourier a=loiclec

Fixes #3294 

The `index-scheduler` code is not modified, only the CLI options have changed.

Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2023-01-03 16:57:14 +00:00
17dac72464 Characters -> bytes 2023-01-03 15:31:02 +01:00
b821c72459 Remove --disable-auto-batching CLI option 2023-01-03 15:01:04 +01:00
7b2575c646 Master Key -> master key 2023-01-03 14:45:23 +01:00
ab655a85e8 Merge #3279
3279: Clarify error message when the db and engine versions are incompatible r=irevoire a=dureuill

# Pull Request

## Related issue

Related to https://github.com/meilisearch/meilisearch/issues/2752

## What does this PR do?
- Implements https://github.com/meilisearch/product/discussions/572#discussioncomment-4390616

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2023-01-02 17:18:11 +00:00
6425e06cf2 Merge #3274
3274: Reject master keys that are less than 16 bytes and add `--generate-master-key` CLI option r=irevoire a=dureuill

# Pull Request

## Related issue
Fix #3272 
Fix #3287

## What does this PR do?

### User standpoint

---

- Adds a `--generate-master-key` CLI flag to generate a fresh Master Key and exit.

<img width="1351" alt="Capture d’écran 2022-12-22 à 14 18 58" src="https://user-images.githubusercontent.com/41078892/209142778-eab52eeb-eaa8-409b-897a-c0d5728c8aaa.png">

---

(relevant fragment of the `--help` message)

<img width="1351" alt="Capture d’écran 2022-12-22 à 14 19 40" src="https://user-images.githubusercontent.com/41078892/209142891-ebfa2ed6-f231-4f76-a3ae-b7542c7aef04.png">

---

- When `meilisearch` is started in the `development` environment and no Master Key has been provided, then the binary prints a warning before starting.

<img width="1351" alt="Capture d’écran 2022-12-22 à 14 14 49" src="https://user-images.githubusercontent.com/41078892/209142158-54eba3b7-bf71-4f3f-8840-0600b13a1a9f.png">

---

- When `meilisearch` is started in the `development` environment and the provided Master Key is shorter than 16 bytes, then the binary prints a warning before starting.

<img width="1351" alt="Capture d’écran 2022-12-22 à 14 15 58" src="https://user-images.githubusercontent.com/41078892/209142295-0209fe47-c03b-424f-a73f-cee9b633137a.png">

---

- When `meilisearch` is started in the `production` environment, and no Master Key is provided, the error message is altered to generate a fresh Master Key.

<img width="1351" alt="Capture d’écran 2022-12-22 à 17 29 02" src="https://user-images.githubusercontent.com/41078892/209180540-0def5798-15db-47f0-a6ec-8cfa081dea77.png">


---

- When `meilisearch` is started in the `production` environment, and the provided Master Key is shorter than 16 bytes, then the binary exits with an error.

<img width="1351" alt="Capture d’écran 2022-12-22 à 17 28 47" src="https://user-images.githubusercontent.com/41078892/209180567-fa54fe33-fbc4-4b9f-b281-7dfb7b33af85.png">


---

This implements the solution B described here: https://github.com/meilisearch/product/discussions/538#discussioncomment-4391346 

### Implementation standpoint

- Add a new `meilisearch-auth::generate_master_key` function that uses a Cryptographic Random Number Generator (CRNG) to fill a vector of 32 bytes before encoding these bytes as base64

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2023-01-02 16:00:40 +00:00
1692f58b83 slightly update the message associated with the cli parameter + accept an env variable 2023-01-02 16:49:35 +01:00
9ba4d0f921 update the error messages according to the spec 2023-01-02 16:43:23 +01:00
4b6ffe0cd1 Update meilisearch-auth/src/lib.rs 2023-01-02 16:33:02 +01:00
336c77aa45 Merge #3245
3245: Enable create_raw_index(...) to specify time r=irevoire a=amab8901

# Pull Request

## Related issue
Partially fixes #2983 

## What does this PR do?
- Enables [`create_raw_index`](660be071b5/index-scheduler/src/lib.rs (L868)) to specify time

## PR checklist
Please check if your PR fulfills the following requirements:
- [ X ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [ X ] Have you read the contributing guidelines?
- [ X ] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: amab8901 <amab8901@protonmail.com>
2023-01-02 14:11:39 +00:00
776acb5ed3 Merge #3276
3276: README: Replace Slack link with Discord r=dureuill a=shivaylamba

# Pull Request

## Related issue
Fixes #3275 

## What does this PR do?

Update Slack link with Discord link in the README

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Shivay Lamba <shivaylamba@gmail.com>
2022-12-29 10:32:42 +00:00
3e9834abff Change error message when the db version is incompatible with engine version. 2022-12-26 17:34:36 +01:00
3cba476a9f Add --generate-master-key CLI option 2022-12-26 10:36:45 +01:00
57e851d8a9 Check for key length 2022-12-26 10:36:45 +01:00
9c45850bd2 README: Replace Slack link with Discord 2022-12-26 00:19:13 +05:30
3e0e8164a3 fixup! Adjust + Cleanup changes. 2022-12-22 18:01:54 +01:00
0bc4572905 Adjust + Cleanup changes.
Indeed, I missed some of the changed that were introduced by #3190.
2022-12-22 17:53:33 +01:00
4e6c663a2e Release unecessary ownership. 2022-12-22 17:47:58 +01:00
e2775c6f49 Remove unused object. 2022-12-22 17:47:58 +01:00
c07a5932cb Apply fmt. 2022-12-22 17:47:58 +01:00
528a944997 Reimplement v5 date extraction.
Indeed, before this patch the implementation wasn't correct.
2022-12-22 17:47:58 +01:00
13fb5ce974 Re-Open tasks list when needed.
Indeed, before this patch we were using the reference instead of
"reopening" the task list each time we needed to access it.
Without this patch, all other usage of the task attribute will
break.
2022-12-22 17:47:57 +01:00
a43a0712fa Add reader.v5.tasks.Task.updated_at.
There was no way to "quickly" get the update date.
2022-12-22 17:47:57 +01:00
1be4619b91 Add reader.v5.tasks.Task.created_at.
There was no way to "quickly" get the creation date.
2022-12-22 17:47:57 +01:00
cf50f85986 Add reader.v5.tasks.Task.processed_at.
There was no way to "quickly" get the processed date.
2022-12-22 17:47:57 +01:00
61b3a29ff3 Extract the dates out of the dumpv5.
This patch possibly fixes #2986.

This patch introduces a way to fill the IndexMetadata.created_at
and IndexMetadata.updated_at keys from the tasks events.
This is done by reading the creation date of the first event
(created_at) and the creation date of the last event (updated_at).
2022-12-22 17:47:57 +01:00
66e18eae79 auth: add generate_master_key function 2022-12-22 11:55:27 +01:00
9a39c4e40d Get date from IndexMetaData 2022-12-22 11:46:17 +01:00
df176aaf01 Insert dump_reader.date() into create_raw_index(_) argument 2022-12-21 15:16:31 +01:00
0893b175dc Merge branch 'main' into 2983-forward-date-to-milli 2022-12-21 14:31:19 +01:00
d5978d11e1 Refactor 2022-12-21 14:28:00 +01:00
2780e365e2 test update and ndjson serde use from_slice 2022-12-21 14:31:45 +08:00
bf2a401a05 serde ndjson fix 2022-12-21 11:27:15 +08:00
9925309492 Merge #3263
3263: Handle most io error instead of tagging everything as an internal r=dureuill a=irevoire

Fix https://github.com/meilisearch/meilisearch/issues/2255
Fix https://github.com/meilisearch/meilisearch/issues/2785
Close https://github.com/meilisearch/milli/pull/580

- [x] Find a way to catch the `io::Error` contained in `serde_json::Error`: We can't: https://docs.rs/serde_json/latest/serde_json/struct.Error.html
- [x] Check the `grenad::Error` as well => the `grenad::Error::Io` error are correctly converted to a `milli::Error::Io` error 
- [x] Ensure the error code mean the same thing under windows

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-12-20 17:15:53 +00:00
9e0cce5ca4 Update dump/src/error.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-12-20 18:08:51 +01:00
336ea57384 Update dump/src/error.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-12-20 18:08:44 +01:00
c637bfba37 convert all the document format error due to io to io::Error 2022-12-20 17:49:38 +01:00
3040172562 update the error message as well 2022-12-20 17:31:13 +01:00
52aa34d984 remove an unused error handling file 2022-12-20 16:32:51 +01:00
2c86d42a44 Merge #3264
3264: Remove macos-latest and windows-latest usages r=curquiza a=curquiza

Related to https://github.com/meilisearch/meilisearch/issues/3109#issuecomment-1359151297

Remove the `macos-latest` and `windows-latest` to replace them with the specific version: this will avoid "surprises" in the future when GitHub changes the `latest` version.
This way, it will also allow us to let the documentation team know about the changes, since we will control the macOS/Windows version we support

Co-authored-by: curquiza <clementine@meilisearch.com>
2022-12-20 10:53:37 +00:00
8ce3a34ffa Remove macos-latest and windows-latest usages 2022-12-20 11:10:09 +01:00
259c04eb28 Merge #3261
3261: Use ubuntu-18.04 container instead of GitHub hosted actions r=curquiza a=curquiza

Related to (but does not fix totally) https://github.com/meilisearch/meilisearch/issues/3109 and https://github.com/meilisearch/product/discussions/547#discussioncomment-4109143

## For reviewers, what's the PR changes:
- Use ubuntu-latest where compiling with ubuntu-18.04 is not needed (`update-version-cargo-toml`, `fmt`, `clippy` jobs)
- Where ubuntu-18.04 is required
  - Use `ubuntu-latest` as runner
  - Use `ubuntu:18.04` as Docker container
  - Install the required dependencies (curl and cc)
  - Use `actions-rs/toolchain@v1` instead of `hecrj/setup-rust-action@master`. It's more stable and followed alternative. Plus it was easy to make it work with our container contrary to the old one. Change applied in all our CIs to be more consistent
- Remove some useless space to increase readability.

Co-authored-by: curquiza <clementine@meilisearch.com>
2022-12-20 09:28:09 +00:00
d8fb506c92 handle most io error instead of tagging everything as an internal 2022-12-19 20:50:40 +01:00
aa03e02fdc Apply Rustfmt 2022-12-19 19:24:56 +01:00
7ef23addb6 Add comment to bring more context 2022-12-19 18:46:27 +01:00
b3fce7c366 Remove useless continue-on-error 2022-12-19 18:39:35 +01:00
5099a40484 Use ubuntu-18.04 container in publish CIs 2022-12-19 18:35:33 +01:00
19ee9a828f Merge #3262
3262: Clippy fixes after updating Rust to v1.66 r=curquiza a=dureuill

Ran `cargo clippy --fix`

Fixes the CI.


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-12-19 14:05:59 +00:00