48812229a9
Remove a log that would log too much
2025-02-03 10:24:52 +01:00
915cc377fb
Refine the env variable and the max readers
2025-02-03 10:24:52 +01:00
aaefbfae1f
Do not create too many rayon tasks
2025-01-30 16:36:12 +01:00
97e17f52a1
Add more logs to see calls to the embedders
2025-01-30 16:36:12 +01:00
8ed39f5de0
Merge #5300
...
5300: Improve unexpected panic message r=irevoire a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5273
## What does this PR do?
- When an unexpected panic happens in the index-scheduler we catch it and rebuild an error message from the join_error
- Same when the upgrade index-scheduler fails
Co-authored-by: Tamo <tamo@meilisearch.com >
2025-01-30 09:23:17 +00:00
bdd3005d10
Log the progress when a batch fails
2025-01-29 16:36:23 +01:00
cb1b7513af
Log the memory metrics only once
2025-01-29 15:21:52 +01:00
a9d0f4a002
Improve english comments
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2025-01-29 15:16:40 +01:00
db032079d8
Show indexation allocated memory
2025-01-29 14:21:02 +01:00
a00796c46a
Improve the naming in the log message
2025-01-29 14:21:02 +01:00
6112bd8caa
Display the channel congestion
2025-01-29 14:21:02 +01:00
cec88cfc29
Measure the bbqueue congestion
2025-01-29 14:21:02 +01:00
8439aeb7cf
improve error message in case of unexpected panic while processing tasks
2025-01-29 11:51:06 +01:00
1beda3b9af
fix another flaky test
2025-01-28 16:53:50 +01:00
8676e94f5c
fix the flaky tests
2025-01-28 16:53:50 +01:00
ef47a0d820
apply review comment
2025-01-28 16:53:50 +01:00
e0f0da57e2
make sure the batches we snapshots actually all contains an enqueued_at
2025-01-28 16:53:50 +01:00
485e3127c7
use the remove_n_tasks_datetime_earlier_than function when updating batches
2025-01-28 16:53:50 +01:00
58f90b70c7
store the enqueued at to eases the batch deletion
2025-01-28 16:53:50 +01:00
508db9020d
update the snapshots
2025-01-28 16:53:50 +01:00
6ff37c6fc4
Fix the insta snapshots
2025-01-28 16:53:50 +01:00
f21ae1f5d1
Remove the batch id from the date time databases
2025-01-28 16:53:50 +01:00
528d9d6d8b
Removed CouldNotUpgrade from error file
2025-01-26 21:04:57 +05:30
50280bf02b
Support offline upgrade up to v1.12.7
2025-01-24 12:25:33 +01:00
9b579069df
Comment the max grant of the bbqueue
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2025-01-24 12:18:32 +01:00
f5a4a1c8b2
Give more RAM to bbqueue.
...
- bbqueue buffers used to have (5% * 2%) / num_threads
- they now have 5% / num_threads
2025-01-24 12:18:32 +01:00
5ab4cdb1f3
Reduce the maximum grant possible we can store in the BBQueue
2025-01-24 12:18:32 +01:00
73d8a4eace
Remove db.snapshot
2025-01-23 17:21:42 +01:00
c1e5897076
Do not assume v1.12 when there is no index-scheduler version
2025-01-23 17:16:53 +01:00
718a98fbbf
remove : char from filenames
2025-01-23 17:08:35 +01:00
86bf231d29
Change to meilitool after rebase
2025-01-23 16:59:32 +01:00
182c3f4b80
Write assumed version to the index-scheduler version db when it is missing
2025-01-23 16:51:25 +01:00
c1eba66443
introduce a corruption in the v1.12 data.ms field distribution
2025-01-23 16:51:24 +01:00
7197ced673
fix the bad index version on opening
2025-01-23 16:51:24 +01:00
4f21ee6c66
update the data.ms snapshot
2025-01-23 16:51:24 +01:00
787472453d
write the version of the index while upgrading it
2025-01-23 16:51:24 +01:00
8f65f35de9
rewrite part of the index-scheduler upgrade test
2025-01-23 16:51:23 +01:00
c27c923439
introduce a trait to upgrade the indexes
2025-01-23 16:51:23 +01:00
fd5649091d
add the upgradeTo field in the details
2025-01-23 16:51:23 +01:00
9a57736773
fix the early exit when rewriting a batch
2025-01-23 16:51:23 +01:00
7740997ea8
reintroduce the unrecoverable error and use it where its supposed to be used
2025-01-23 16:51:22 +01:00
7eb23f73ba
add the version to the index-scheduler snapshots + fix a bug when opening an index scheduler for the first time
2025-01-23 16:51:22 +01:00
b9e9fc376a
add the version in the index-scheduler
2025-01-23 16:51:22 +01:00
27bf2f1298
remove the empty progress made for the upgrade database
2025-01-23 16:51:22 +01:00
d4d82fbd0c
commit the index wtxn before the index-scheduler wtxn
2025-01-23 16:51:21 +01:00
eda09a54da
improve the index-scheduler tests
2025-01-23 16:51:21 +01:00
b132d70413
fix the details in all cases
2025-01-23 16:51:21 +01:00
e41ebd3047
expose the number of database in the index-scheduler and rewrite the lib.rs to use the value provided in the options instead of a magic number
2025-01-23 16:51:21 +01:00
705d31e8bd
apply all the comments changes
2025-01-23 16:51:21 +01:00
7d95950ce6
fix warning
2025-01-23 16:51:21 +01:00
