Commit Graph

12805 Commits

Author SHA1 Message Date
Kerollmops
3ef1afc0f1 Introduce some backoff retries 2025-11-05 15:16:43 +01:00
Kerollmops
dbb5abebb6 Support cancelation 2025-11-05 15:16:42 +01:00
Kerollmops
700f33bd39 Clean up the code 2025-11-05 15:16:42 +01:00
Kerollmops
d01bbbccde Support clean CLI options 2025-11-05 15:16:42 +01:00
Kerollmops
4fc506f267 Remove unused imports/code on Windows 2025-11-05 15:16:42 +01:00
Kerollmops
dc456276e5 Make clippy happy 2025-11-05 15:16:42 +01:00
Kerollmops
b2ea50cb10 Make the compression level configurable 2025-11-05 15:16:42 +01:00
Kerollmops
5074cf92ab Disable compression entirely to avoid being CPU bound 2025-11-05 15:16:42 +01:00
Clément Renault
a92bc8d192 Improve the way we create the snapshot path 2025-11-05 15:16:42 +01:00
Clément Renault
ee538cf045 Remove useless dependencies 2025-11-05 15:16:42 +01:00
Kerollmops
2b05d63a0f Make it finaly work but without async on the write side 2025-11-05 15:16:42 +01:00
Kerollmops
104e8918ce Seeking the tasks/data.mdb file to the begining made the trick 2025-11-05 15:16:42 +01:00
Kerollmops
d6ec4d4f4a Improve understanding of S3-related errors 2025-11-05 15:16:42 +01:00
Kerollmops
f0e7326b7a Retrieve the bytesMut only when released 2025-11-05 15:16:42 +01:00
Kerollmops
c8106a0006 Fix minimum part size 2025-11-05 15:16:42 +01:00
Kerollmops
c9ab5bc0b6 Improve error messaging when missing env var 2025-11-05 15:16:42 +01:00
Clément Renault
5e0f15fd43 WIP 2025-11-05 15:16:42 +01:00
Kerollmops
4c30f090c7 WIP Do more tests 2025-11-05 15:16:42 +01:00
Clément Renault
63f247cdda WIP sending multiparts of 250MiB 2025-11-05 15:16:42 +01:00
Clément Renault
e109fa9529 Rename the update_path function 2025-11-05 15:16:42 +01:00
Clément Renault
76e4ec2168 Geenrate an async tarball 2025-11-05 15:16:42 +01:00
Kerollmops
982babdb74 WIP 2025-11-05 15:16:42 +01:00
Kerollmops
7ae2ae33d9 Make max in flights parts fro upload configurable 2025-11-05 15:16:42 +01:00
Kerollmops
cb0788ae07 Use a good mem advice for uploads 2025-11-05 15:16:42 +01:00
Kerollmops
cb3e5dc234 Move the S3 snapshots to disk into a dedicated method 2025-11-05 15:16:41 +01:00
Clément Renault
59d40a2821 Upload ten parts at a time 2025-11-05 15:16:41 +01:00
Clément Renault
98a678e73d Use the Bytes crate to send the parts 2025-11-05 15:16:41 +01:00
Clément Renault
70292aae3c Upload indexes under their uuids 2025-11-05 15:16:41 +01:00
Clément Renault
73521f0069 Initial working S3 uploads to RustFS 2025-11-05 15:16:41 +01:00
Louis Dureuil
4533179604 Pass tokio handle to index-scheduler 2025-11-05 15:16:41 +01:00
Clément Renault
1a21cc1a17 Merge pull request #5959 from meilisearch/parallelize-word-prefix-docids
Parallelize the word prefix docids
2025-11-05 10:10:07 +00:00
Clément Renault
d08042f8a7 Merge pull request #5967 from meilisearch/bump-lmdb-version
Fix the LMDB fork memory leak
2025-11-05 09:33:59 +00:00
Louis Dureuil
77aadb5f22 Merge pull request #5968 from meilisearch/engprod-2116-privilege-escalation-from-webhook-api-key-to-master-key
Redact Authorization header in webhooks
2025-11-05 08:51:07 +00:00
Kerollmops
4fd913f7eb Use the latest version of heed 2025-11-05 09:42:03 +01:00
Louis Dureuil
8618a4d2ba document hide_secret 2025-11-04 17:03:12 +01:00
Louis Dureuil
e9c5df7993 happy clippy 2025-11-03 17:23:27 +01:00
Louis Dureuil
8a28b3aa77 Update snap 2025-11-03 15:52:35 +01:00
Louis Dureuil
1a0b100ad9 rename webhook to highlight redaction 2025-11-03 15:52:22 +01:00
Louis Dureuil
ff93563f41 Redact webhook authorize header on display 2025-11-03 15:51:56 +01:00
Louis Dureuil
2f25258191 Extract crate::settings::hide_secret as a public function 2025-11-03 15:50:37 +01:00
Clément Renault
2859079c32 Merge pull request #5961 from meilisearch/add-flickr-demo
Add Flickr example to README
2025-11-03 14:44:54 +00:00
Clément Renault
74b83d305f Add Flickr example to README
This PR adds the new Flickr demo to the README.
2025-10-29 18:27:36 +01:00
Clément Renault
70f6e4b828 Parallelize the word prefix docids 2025-10-27 17:20:12 +01:00
Clément Renault
6df196034e Merge pull request #5950 from meilisearch/update-version-v1.24.0
Update version to v1.24.0
v1.24.0
2025-10-20 11:17:15 +00:00
Clément Renault
a63762737c Upgrade index scheduler 2025-10-20 12:22:27 +02:00
Clément Renault
77394bd4b9 Update insta tests 2025-10-20 10:54:16 +02:00
Clément Renault
cb87201c8b Fix dumpless upgrade and do nothing 2025-10-20 10:42:35 +02:00
Clément Renault
1a9c38794f Bump version to v1.24.0 2025-10-20 10:38:48 +02:00
Clément Renault
34233efb63 Merge pull request #5946 from meilisearch/fix-compaction-issues
Improve compaction behaviors
2025-10-16 15:42:38 +00:00
Clément Renault
af0608ebd6 Continue to the next index if index doesn't exists 2025-10-16 16:39:51 +02:00