7e47cea0c4
Add tracing to Meilisearch
2024-02-08 15:03:31 +01:00
5d7061682e
Add tracing to milli
2024-02-08 15:03:31 +01:00
02e6c8a440
Add tracing to index-scheduler
2024-02-08 15:03:31 +01:00
89401d097b
Add tracing-trace
2024-02-08 15:03:30 +01:00
698ea5139d
Update Cargo.lock
2024-02-01 10:40:23 +01:00
c57f7f7379
Update version for the next release (v1.6.1) in Cargo.toml
2024-02-01 10:33:26 +01:00
b6fc181993
Merge #4304
...
4304: Add CUDA GPU support for Hugging Face embedders r=Kerollmops a=dureuill
Adds a "cuda" feature to `milli`.
Compiling with this feature requires that the CUDA support library be installed (see "with CUDA support" paragraph in https://huggingface.github.io/candle/guide/installation.html ), and adds CUDA support to the `huggingFace` embedder.
To enable GPU support, users will need to:
1. Have a compatible NVidia GPU under Linux
2. Follow [the guide](https://huggingface.github.io/candle/guide/installation.html ) to install the CUDA dependencies
3. Compile Meilisearch with the `cuda` feature: `cargo build --release --features cuda`
# Impact
Enabling the CUDA feature allows to use an available GPU to compute embeddings with a `huggingFace` embedder.
On an AWS Graviton 2, this yields a x3 - x5 improvement on indexing time.
# Technical details
- I had to change the CI so that the cuda feature is not included in the `Tests all features` workflow
- To achieve that, I had to add a binary following the `cargo xtask` design pattern, to list all features excepted the cuda one.
- I then changed the workflow accordingly (renamed to "Tests almost all features" đ )
- A test run of the new feature was done on a temporary version of this PR that had it enabled for PRs: [See the results here](https://github.com/meilisearch/meilisearch/actions/runs/7461331929/job/20301216732 )
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2024-01-22 13:55:04 +00:00
d35fe43fd5
Update lock file
2024-01-22 10:49:17 +01:00
4aa4a15dc9
Add to Cargo.lock
2024-01-22 10:25:54 +01:00
84f49d76cd
Add cuda feature
2024-01-22 10:25:16 +01:00
b5b2333a05
Bump h2 from 0.3.20 to 0.3.24
...
Bumps [h2](https://github.com/hyperium/h2 ) from 0.3.20 to 0.3.24.
- [Release notes](https://github.com/hyperium/h2/releases )
- [Changelog](https://github.com/hyperium/h2/blob/v0.3.24/CHANGELOG.md )
- [Commits](https://github.com/hyperium/h2/compare/v0.3.20...v0.3.24 )
---
updated-dependencies:
- dependency-name: h2
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
2024-01-19 16:20:22 +00:00
50e1d34c66
Rollback http to 0.2.11
2024-01-16 16:57:33 +01:00
01e2c3d6bb
Bump arroy to v0.2.0
2024-01-16 16:45:55 +01:00
0c8d1644a6
Rollback rustls to 0.20.9
2024-01-16 15:55:16 +01:00
7f125bfb12
Update incompatible dependencies
2024-01-16 15:15:54 +01:00
5869ca7716
Upgrade all compatible dependencies
2024-01-16 15:05:03 +01:00
d9d0419845
Update the dependencies
2024-01-16 14:38:48 +01:00
12edc2c20a
Update arroy to a fixed version
2024-01-03 15:59:37 +01:00
43e822e802
Merge #4238
...
4238: Task queue webhook r=dureuill a=irevoire
# Prototype `prototype-task-queue-webhook-1`
The prototype is available through Docker by using the following command:
```bash
docker run -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-task-queue-webhook-1
```
# Pull Request
Implements the task queue webhook.
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4236
## What does this PR do?
- Provide a new cli and env var for the webhook, respectively called `--task-webhook-url` and `MEILI_TASK_WEBHOOK_URL`
- Also supports sending the requests with a custom `Authorization` header by specifying the optional `--task-webhook-authorization-header` CLI parameter or `MEILI_TASK_WEBHOOK_AUTHORIZATION_HEADER` env variable.
- Throw an error if the specified URL is invalid
- Every time a batch is processed, send all the finished tasks into the webhook with our public `TaskView` type as a JSON Line GZIPed body.
- Add one test.
## PR checklist
### Before becoming ready to review
- [x] Add a test
- [x] Compress the data we send
- [x] Chunk and stream the data we send
- [x] Remove the unwrap in the index-scheduler when sending the data fails
- [x] The analytics are missing
### Before merging
- [x] Release a prototype
Co-authored-by: Tamo <tamo@meilisearch.com >
Co-authored-by: Clément Renault <clement@meilisearch.com >
2023-12-21 14:43:46 +00:00
be72326c0a
gzip the tasks
2023-12-19 10:35:51 +01:00
547379abb0
parse the url correctly
2023-12-19 10:35:51 +01:00
d78ad51082
Implement the webhook
2023-12-19 10:35:50 +01:00
942d49314c
Remove dependency that requires libstdc++
2023-12-18 22:17:18 +01:00
50d6317ec0
Update version for the next release (v1.6.0) in Cargo.toml
2023-12-18 13:57:46 +00:00
61bd2fb7a9
Update arroy
2023-12-14 16:08:41 +01:00
65e49b7092
Remove stuff, add distribution shift (WIP)
2023-12-14 16:08:38 +01:00
cb4ebe163e
WIP
2023-12-14 16:07:49 +01:00
dde3a04679
WIP arroy integration
2023-12-14 16:07:49 +01:00
13c2c6c16b
Small commit to add hybrid search and autoembedding
2023-12-14 16:07:48 +01:00
21bcf32109
Add candle and hg_hub, updating a lot of deps in the process
2023-12-14 16:07:48 +01:00
56571f762a
Merge remote-tracking branch 'origin/main' into tmp-release-v1.5.1
2023-12-13 11:57:01 +01:00
4b644f6bc0
Update version for the next release (v1.5.1) in Cargo.toml
2023-12-11 17:15:11 +00:00
d32eb11329
Move to the v0.20.0-alpha.9 of heed
2023-11-27 11:52:22 +01:00
0d4482625a
Make the changes to use heed v0.20-alpha.6
2023-11-23 11:43:58 +01:00
56a0d91ecd
Update the heed dependency and lock file
2023-11-22 15:11:09 +01:00
7cb7e37ba8
Merge branch 'main' into tmp-release-v1.5.0
2023-11-21 16:30:46 +01:00
b10c060bf7
Cleanup TOML
2023-11-01 14:03:04 +01:00
c71b1d33ae
Sort entries using rayon in the transform sorters
2023-11-01 11:07:16 +01:00
b57b818b67
Don't use the last version of clap
2023-10-30 16:57:31 +01:00
f7ea94e5f4
Modify the Dockerfile to compile meilisearch and meilitool
2023-10-30 16:32:17 +01:00
13416ccbf7
Introduce a new meilitool to help the cloud team
2023-10-30 14:30:20 +01:00
5be569e3e2
Update obkv
2023-10-30 11:40:20 +01:00
17b647dfe5
Wip
2023-10-30 11:13:08 +01:00
4c6fddb1cb
update charabia
2023-10-26 17:01:10 +02:00
ee6f79d60b
Update version for the next release (v1.5.0) in Cargo.toml
2023-10-23 11:49:07 +00:00
2042229927
Update version for the next release (v1.4.2) in Cargo.toml
2023-10-23 12:02:45 +02:00
e761db582f
Bump rustix from 0.36.15 to 0.36.16
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.15 to 0.36.16.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.15...v0.36.16 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
2023-10-18 18:42:12 +00:00
c5f7893fbb
Remove the puffin http dependency
2023-10-13 13:11:08 +02:00
0913373a5e
Merge #4122
...
4122: Bring back changes from `release-v1.4.1` into `main` r=Kerollmops a=curquiza
Co-authored-by: curquiza <curquiza@users.noreply.github.com >
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com >
Co-authored-by: Vivek Kumar <vivek.26@outlook.com >
Co-authored-by: Clément Renault <clement@meilisearch.com >
2023-10-12 15:57:47 +00:00
8a95bf28e5
Update version for the next release (v1.4.1) in Cargo.toml
2023-10-10 09:01:45 +00:00
