0625d08e4e
adding a function toextract batch_uid from json and modifying the get_batch interface for easier call - did not work, so falling back to hard coded batch id for now.
2025-01-07 12:07:33 +11:00
9269086fda
fixing a rebase issue
2025-01-07 11:48:09 +11:00
98e3ecb86b
Format fixes after running: cargo +nightly fmt
2025-01-07 11:16:37 +11:00
4b107b17cb
test: improve performance of get_index.rs
2025-01-06 17:38:44 +01:00
d7cb319217
#4840 - Partial fix - Confirm task success after waiting for it - continued, few missing cases - batch 2
2024-12-24 23:07:43 +11:00
15062e7dba
#4840 - Partial fix - Confirm task success after waiting for it - continued, few missing cases.
2024-12-24 23:06:07 +11:00
bf19f86e38
#4840 - Partial fix - Confirm task success after waiting for it.
2024-12-24 23:06:07 +11:00
91c7ef8723
#4840 - Partial fix - Remove hard coded task ids to prevent flaky tests.
...
# Conflicts:
# crates/meilisearch/tests/documents/add_documents.rs
# crates/meilisearch/tests/search/facet_search.rs
# crates/meilisearch/tests/settings/get_settings.rs
# crates/meilisearch/tests/snapshot/mod.rs
2024-12-24 23:05:59 +11:00
d3491851bc
Merge #5187
...
Indexing bench (push) / Run and upload benchmarks (push) Waiting to run
Benchmarks of indexing (push) / Run and upload benchmarks (push) Waiting to run
Benchmarks of search for geo (push) / Run and upload benchmarks (push) Waiting to run
Benchmarks of search for songs (push) / Run and upload benchmarks (push) Waiting to run
Benchmarks of search for Wikipedia articles (push) / Run and upload benchmarks (push) Waiting to run
Test suite / Tests on ${{ matrix.os }} (macos-13) (push) Waiting to run
Test suite / Tests on ${{ matrix.os }} (windows-2022) (push) Failing after 12s
Test suite / Tests on ubuntu-20.04 (push) Failing after 20s
Test suite / Tests almost all features (push) Has been skipped
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Run tests in debug (push) Failing after 16s
Test suite / Run Clippy (push) Successful in 33m58s
Test suite / Run Rustfmt (push) Successful in 11m45s
Run the indexing fuzzer / Setup the action (push) Successful in 1h10m33s
5187: Bring back v1.12.0 of pre-release changes into `main` r=irevoire a=curquiza
Co-authored-by: ManyTheFish <many@meilisearch.com >
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
Co-authored-by: Clément Renault <clement@meilisearch.com >
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Many the fish <many@meilisearch.com >
2024-12-23 10:59:33 +00:00
4bcdd7a9f9
fix the flaky batches test
2024-12-18 11:51:12 +01:00
18ce95dcbf
Add test reproducing the bug
2024-12-12 14:56:45 +01:00
1fc90fbacb
Merge #5147
...
5147: Batch progress r=dureuill a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5068
## What does this PR do?
- ...
## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [ ] Have you read the contributing guidelines?
- [ ] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Tamo <tamo@meilisearch.com >
2024-12-12 09:15:54 +00:00
ab75f53efd
update all snapshots
2024-12-11 16:25:02 +01:00
d075be798a
Fix tests
2024-12-10 13:39:07 +01:00
7cf6707ed3
Extend test to add the ==512 bytes case
2024-12-10 11:05:42 +01:00
b75f1f4c17
fix tests
...
# Conflicts:
# crates/index-scheduler/src/batch.rs
# crates/index-scheduler/src/snapshots/lib.rs/fail_in_process_batch_for_document_deletion/after_removing_the_documents.snap
# crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_bad_primary_key/fifth_task_succeeds.snap
# crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_bad_primary_key/fourth_task_fails.snap
# crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key/second_task_fails.snap
# crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key/third_task_fails.snap
# crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key_batch_wrong_key/second_and_third_tasks_fails.snap
# crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/all_other_tasks_succeeds.snap
# crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/second_task_fails.snap
# crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/third_task_succeeds.snap
# Conflicts:
# crates/index-scheduler/src/batch.rs
# crates/meilisearch/src/search/mod.rs
# crates/meilisearch/tests/vector/mod.rs
# Conflicts:
# crates/index-scheduler/src/batch.rs
2024-12-06 02:03:02 +08:00
26be5e0733
Merge #5123
...
5123: Fix batch details r=dureuill a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5079
Fixes https://github.com/meilisearch/meilisearch/issues/5112
## What does this PR do?
- Make the processing tasks actually processing in the stats of the batch instead of enqueued
- Stop counting one extra task for all non-prioritized batches in the stats
- Add a test
Co-authored-by: Tamo <tamo@meilisearch.com >
2024-12-05 15:21:55 +00:00
a0a3b55700
Change error code
2024-12-05 14:48:29 +01:00
214b51de87
try to fix the snapshot on demand flaky test
2024-12-05 14:45:54 +01:00
fc1df5793c
fix tests
2024-12-04 14:35:20 +01:00
953a82ca04
Add new error message
2024-12-04 11:15:29 +01:00
a439fa3e1a
While spamming the batches route we could see a processing batch becoming missing and then finished, this commit ensures the batches goes from processing to finished directly
2024-12-02 12:02:16 +01:00
90b428a8c3
Apply change requests
2024-11-28 15:16:13 +01:00
5383f41bba
Polish test_setting_routes!
2024-11-28 12:04:21 +01:00
68c4717e21
Change the settings tests and macros to avoid oversights
2024-11-28 11:34:35 +01:00
d0b2c0a523
Merge #5091
...
Test suite / Tests on ${{ matrix.os }} (macos-13) (push) Waiting to run
Test suite / Tests almost all features (push) Has been skipped
Test suite / Test disabled tokenization (push) Has been skipped
Test suite / Tests on ubuntu-20.04 (push) Failing after 11s
Test suite / Run tests in debug (push) Failing after 10s
Test suite / Tests on ${{ matrix.os }} (windows-2022) (push) Failing after 39s
Test suite / Run Rustfmt (push) Successful in 1m38s
Test suite / Run Clippy (push) Successful in 23m11s
5091: Settings opt out r=Kerollmops a=ManyTheFish
# Pull Request
Related PRD: https://www.notion.so/meilisearch/API-usage-Settings-to-opt-out-indexing-features-fff4b06b651f8108ade3f858aeb16b14?pvs=4
## Related issue
Fixes #4979
- [x] Add setting opt-out
- [x] Add analytics
- [x] Add tests
Co-authored-by: ManyTheFish <many@meilisearch.com >
Co-authored-by: Many the fish <many@meilisearch.com >
2024-11-26 15:50:28 +00:00
2e896f30a5
Fix PR comments
2024-11-26 16:06:33 +01:00
e241f91285
Merge #5062
...
5062: Fix bugs for v1.12 r=Kerollmops a=ManyTheFish
# Pull Request
## Related issue
Fixes #4984
Fixes https://github.com/meilisearch/meilisearch/issues/4974
Fixes [SDK test](https://github.com/meilisearch/meilisearch/actions/runs/11886701996/job/33118278794 )
## What does this PR do?
- add 3 tests
- fix bugs
Co-authored-by: ManyTheFish <many@meilisearch.com >
2024-11-26 08:10:50 +00:00
d66dc363ed
Test and implement settings opt-out
2024-11-25 18:23:22 +01:00
19e6f675b3
Merge #4900
...
4900: Indexer edition 2024 r=Kerollmops a=dureuill
This PR is implementing the indexer edition 2024, largely inspired by [the ideas from this blog post](https://blog.kerollmops.com/meilisearch-is-too-slow ).
Fixes https://github.com/meilisearch/meilisearch/issues/4985
## Features
- Stream-first approach to reading documents.
- Minimum disk write operations.
- RAM usage-first approach to avoid modifying common bitmaps on disk but in memory.
- Reduced LMDB fragmentation by writing entries only once...
- ...computing the final version of the entries in parallel...
- ...and storing them in write-optimized data structures before sending them to the BTree (LMDB).
- Indexing in multiple transactions to improve large dataset support (dumps).
Co-authored-by: ManyTheFish <many@meilisearch.com >
Co-authored-by: Clément Renault <clement@meilisearch.com >
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2024-11-21 16:19:10 +00:00
36962b943b
First batch of PR comment
2024-11-21 16:38:11 +01:00
1e694ae432
improve the count of the number of tasks in a batch
2024-11-20 17:48:26 +01:00
bda2b41d11
update snaps after merge
2024-11-20 17:08:30 +01:00
6e6acfcf1b
Merge branch 'main' into indexer-edition-2024
2024-11-20 16:59:58 +01:00
4d616f8794
Parse every attributes and filter before tokenization
2024-11-20 15:15:25 +01:00
35bbe1c2a2
Add failing test on settings changes
2024-11-20 14:48:12 +01:00
a7ac590e9e
implements the reverse query parameter for the batches
2024-11-20 13:29:52 +01:00
7e379b3d14
remove useless prints
2024-11-20 12:27:12 +01:00
56eacd221f
update the tests after the rebase
2024-11-20 10:54:38 +01:00
b906e3ed70
improve the way we access the mutex
2024-11-20 10:51:06 +01:00
4abcd9c04e
add some stats on the batches
2024-11-20 10:51:06 +01:00
5d10c2312b
remove unused file
2024-11-20 10:51:06 +01:00
f1d38581e5
add the front end tests on the batches routes
2024-11-20 10:51:06 +01:00
d489f5635f
add the mapping between the task and batches
2024-11-20 10:49:23 +01:00
6062914654
add the batch_id to the tasks
2024-11-20 10:42:54 +01:00
057fcb3993
Add indices field to _matchesPosition to specify where in an array a match comes from ( #5005 )
...
Indexing bench (push) / Run and upload benchmarks (push) Waiting to run
Benchmarks of indexing (push) / Run and upload benchmarks (push) Waiting to run
Benchmarks of search for geo (push) / Run and upload benchmarks (push) Waiting to run
Benchmarks of search for songs (push) / Run and upload benchmarks (push) Waiting to run
Benchmarks of search for Wikipedia articles (push) / Run and upload benchmarks (push) Waiting to run
Run the indexing fuzzer / Setup the action (push) Successful in 1h4m31s
* Remove unreachable code
* Add `indices` field to `MatchBounds`
For matches inside arrays, this field holds the indices of the array
elements that matched. For example, searching for `cat` inside
`{ "a": ["dog", "cat", "fox"] }` would return `indices: [1]`. For nested
arrays, this contains multiple indices, starting with the one for the
top-most array. For matches in fields without arrays, `indices` is not
serialized (does not exist) to save space.
2024-11-20 01:00:43 +01:00
8924d486db
Add a test reproducing the bug
2024-11-18 16:08:55 +01:00
e0c3f3d560
Fix #4984
2024-11-18 16:08:53 +01:00
a05e448cf8
Add test
2024-11-18 12:15:11 +01:00
9150c8f052
Accept changes to vector format
2024-11-18 11:04:57 +01:00
