2169 Commits

Author	SHA1	Message	Date
Kerollmops	dfb4860578	Make some parameters experimental and link the dedicated issue	2025-11-06 12:43:12 +01:00
Kerollmops	ce62713f02	Always create the update_files directory	2025-11-06 12:43:12 +01:00
Kerollmops	8b5d04d60f	Move the code to append a file to the tarball in a dedicated function	2025-11-06 12:07:08 +01:00
Kerollmops	1b74709b91	Extract more logic into dedicated functions	2025-11-06 12:00:25 +01:00
Kerollmops	a5c0a282c5	Better document safety problems and unwraps	2025-11-06 11:26:45 +01:00
Kerollmops	4fc048ff20	Better handle errors when trying to clone the handles	2025-11-06 11:15:34 +01:00
Kerollmops	375b5600cd	Move the tarball streaming into a dedicated function	2025-11-06 11:11:26 +01:00
Kerollmops	32b997d817	Extracts the streaming closure into an async function	2025-11-06 11:04:16 +01:00
Kerollmops	ff3090e3cc	Remove the slash-path dependency	2025-11-06 10:53:07 +01:00
Kerollmops	6c6645f945	Better seeking-to-the-start explanation	2025-11-06 10:45:47 +01:00
Clément Renault	af6473d999	Improve the comments ... Co-authored-by: Louis Dureuil <louis@meilisearch.com>	2025-11-06 10:43:14 +01:00
Kerollmops	11851f9701	Fix warnings on Windows	2025-11-05 16:40:33 +01:00
Kerollmops	cc4654eabd	Remove useless clippy flag	2025-11-05 16:38:05 +01:00
Kerollmops	0bb91f4a77	Immediately panic when using S3 snapshot options on Windows	2025-11-05 15:16:43 +01:00
Kerollmops	f9d57f54df	Keep the smae naming behavior as before	2025-11-05 15:16:43 +01:00
Kerollmops	3ef1afc0f1	Introduce some backoff retries	2025-11-05 15:16:43 +01:00
Kerollmops	dbb5abebb6	Support cancelation	2025-11-05 15:16:42 +01:00
Kerollmops	700f33bd39	Clean up the code	2025-11-05 15:16:42 +01:00
Kerollmops	d01bbbccde	Support clean CLI options	2025-11-05 15:16:42 +01:00
Kerollmops	4fc506f267	Remove unused imports/code on Windows	2025-11-05 15:16:42 +01:00
Kerollmops	dc456276e5	Make clippy happy	2025-11-05 15:16:42 +01:00
Kerollmops	b2ea50cb10	Make the compression level configurable	2025-11-05 15:16:42 +01:00
Kerollmops	5074cf92ab	Disable compression entirely to avoid being CPU bound	2025-11-05 15:16:42 +01:00
Clément Renault	a92bc8d192	Improve the way we create the snapshot path	2025-11-05 15:16:42 +01:00
Clément Renault	ee538cf045	Remove useless dependencies	2025-11-05 15:16:42 +01:00
Kerollmops	2b05d63a0f	Make it finaly work but without async on the write side	2025-11-05 15:16:42 +01:00
Kerollmops	104e8918ce	Seeking the tasks/data.mdb file to the begining made the trick	2025-11-05 15:16:42 +01:00
Kerollmops	d6ec4d4f4a	Improve understanding of S3-related errors	2025-11-05 15:16:42 +01:00
Kerollmops	f0e7326b7a	Retrieve the bytesMut only when released	2025-11-05 15:16:42 +01:00
Kerollmops	c8106a0006	Fix minimum part size	2025-11-05 15:16:42 +01:00
Kerollmops	c9ab5bc0b6	Improve error messaging when missing env var	2025-11-05 15:16:42 +01:00
Clément Renault	5e0f15fd43	WIP	2025-11-05 15:16:42 +01:00
Kerollmops	4c30f090c7	WIP Do more tests	2025-11-05 15:16:42 +01:00
Clément Renault	63f247cdda	WIP sending multiparts of 250MiB	2025-11-05 15:16:42 +01:00
Clément Renault	e109fa9529	Rename the update_path function	2025-11-05 15:16:42 +01:00
Clément Renault	76e4ec2168	Geenrate an async tarball	2025-11-05 15:16:42 +01:00
Kerollmops	982babdb74	WIP	2025-11-05 15:16:42 +01:00
Kerollmops	7ae2ae33d9	Make max in flights parts fro upload configurable	2025-11-05 15:16:42 +01:00
Kerollmops	cb0788ae07	Use a good mem advice for uploads	2025-11-05 15:16:42 +01:00
Kerollmops	cb3e5dc234	Move the S3 snapshots to disk into a dedicated method	2025-11-05 15:16:41 +01:00
Clément Renault	59d40a2821	Upload ten parts at a time	2025-11-05 15:16:41 +01:00
Clément Renault	98a678e73d	Use the Bytes crate to send the parts	2025-11-05 15:16:41 +01:00
Clément Renault	70292aae3c	Upload indexes under their uuids	2025-11-05 15:16:41 +01:00
Clément Renault	73521f0069	Initial working S3 uploads to RustFS	2025-11-05 15:16:41 +01:00
Louis Dureuil	4533179604	Pass tokio handle to index-scheduler	2025-11-05 15:16:41 +01:00
Clément Renault	1a21cc1a17	Merge pull request #5959 from meilisearch/parallelize-word-prefix-docids ... Parallelize the word prefix docids	2025-11-05 10:10:07 +00:00
Clément Renault	d08042f8a7	Merge pull request #5967 from meilisearch/bump-lmdb-version ... Fix the LMDB fork memory leak	2025-11-05 09:33:59 +00:00
Kerollmops	4fd913f7eb	Use the latest version of heed	2025-11-05 09:42:03 +01:00
Louis Dureuil	8618a4d2ba	document hide_secret	2025-11-04 17:03:12 +01:00
Louis Dureuil	e9c5df7993	happy clippy	2025-11-03 17:23:27 +01:00