Commit Graph

44 Commits

Author SHA1 Message Date
60a1188a58 Optim L: Delete datetimes by range 2025-08-12 09:33:38 +02:00
62ad7b345a Make code more readable 2025-08-11 18:09:11 +02:00
16c57414ad Split progress step in half 2025-08-11 17:52:59 +02:00
e23bf92502 Optim K: Make iterations sequential 2025-08-11 17:36:41 +02:00
3fc09107ec Clean code 2025-08-11 15:48:58 +02:00
05c0b6ee6e Fix mistake 2025-08-08 16:04:10 +02:00
1cb9816f44 Optim J: Use iterator on batch reader 2025-08-08 15:43:34 +02:00
6f8d788aa8 Optim I: read batches before writing 2025-08-08 15:30:15 +02:00
6a99c5b2f3 Optim H: Remove batches in iterator 2025-08-08 15:07:48 +02:00
cb36257537 Rename variable 2025-08-08 15:07:35 +02:00
ba0f0c3c30 Optim G: Use the datetime delete function on batches 2025-08-08 14:55:24 +02:00
39eebac7e5 Optim F - Delay wtxn 2025-08-08 14:36:07 +02:00
4c61d3a939 Optim E 2025-08-08 13:06:48 +02:00
eb8ff31513 Make it more idiomatic 2025-08-08 12:56:49 +02:00
418730ef73 Optim D 2025-08-08 12:24:47 +02:00
14cb1bbbfb Optimization B
Speeds up the subtask "deleting batches" by 9%
2025-08-08 11:32:17 +02:00
31c4215ad2 Add debug assertion 2025-08-08 11:25:06 +02:00
ff68802ffc Optimization A
Speeds up the "deleting tasks" subtask by 23%
2025-08-08 11:24:04 +02:00
8fef48f8ca Merge pull request #5670 from meilisearch/export-and-transfer-route
Introduce a new route to export indexes
2025-07-01 14:37:02 +00:00
ad03c86c44 Display an accurate number of uploaded documents 2025-06-30 18:46:47 +02:00
6db5939f84 Re-integrate embedder stats 2025-06-30 09:52:06 +02:00
82fe80b360 Replace the legacy Settings::execute by the new one 2025-06-26 18:14:14 +02:00
63031219c5 Add the payload size to the parameters 2025-06-26 13:57:32 +02:00
e023ee4b6b Working first implementation 2025-06-25 15:26:47 +02:00
e74c3b692a Introduce a new route to export documents and enqueue the export task 2025-06-25 15:26:46 +02:00
d7721fe607 Format 2025-06-24 12:20:22 +02:00
2f82d94502 Fix the test and simplify types 2025-06-23 18:55:23 +02:00
4925b30196 Move embedder stats out of progress 2025-06-23 15:24:14 +02:00
4cadc8113b Add embedder stats in batches 2025-06-20 12:42:22 +02:00
20d0aa499a Apply suggestions from code review
Co-authored-by: Clément Renault <clement@meilisearch.com>
2025-04-29 16:03:30 +02:00
c5360bcdbf When canceling an upgrade task, execute the rollback code 2025-04-24 16:59:03 +02:00
5820d822c8 Add more details about the finalizing progress step 2025-03-26 09:49:43 +01:00
637bea0370 Compute and store the database sizes 2025-03-26 09:49:42 +01:00
05cc8c650c Expose the write channel congestion in the batches 2025-02-19 15:47:54 +01:00
8439aeb7cf improve error message in case of unexpected panic while processing tasks 2025-01-29 11:51:06 +01:00
ef47a0d820 apply review comment 2025-01-28 16:53:50 +01:00
58f90b70c7 store the enqueued at to eases the batch deletion 2025-01-28 16:53:50 +01:00
f21ae1f5d1 Remove the batch id from the date time databases 2025-01-28 16:53:50 +01:00
c27c923439 introduce a trait to upgrade the indexes 2025-01-23 16:51:23 +01:00
7740997ea8 reintroduce the unrecoverable error and use it where its supposed to be used 2025-01-23 16:51:22 +01:00
5458850d21 write a test ensuring the index-scheduler is effectively down when the upgrade task fail and try to process it when it restarts. There is a bug when deleting this task 2025-01-23 16:51:19 +01:00
3ef7a478cd move the version check to the task queue 2025-01-23 16:48:32 +01:00
d3654906bf Add the new tasks with most of the job done 2025-01-23 16:48:32 +01:00
cb82b0798a Split the index-scheduler in ~500 loc modules 2025-01-06 14:08:26 +01:00