Commit Graph

10890 Commits

Author SHA1 Message Date
30f3c30389 Merge #4962
4962: test: improve performance of create_index.rs r=irevoire a=DerTimonius

# Pull Request

## Related issue
related to #4840 

## What does this PR do?
This PR follows the instructions in #4840 and improves the performance of `meilisearch/tests/index/create_index.rs`. The tests run locally, if they fail in the CI I'll try to fix them

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Timon Jurschitsch <timon.jurschitsch@gmail.com>
2024-10-08 13:00:56 +00:00
d907d1b22d Merge #4990
4990: Add image source label to dockerfiles r=curquiza a=wuast94

To get changelogs shown with Renovate a docker container has to add the source label described in the OCI Image Format Specification.

For reference: https://github.com/renovatebot/renovate/blob/main/lib/modules/datasource/docker/readme.md

Co-authored-by: Marc <github@wuast24.de>
Co-authored-by: Clémentine <clementine@meilisearch.com>
2024-10-08 12:19:38 +00:00
ed267fa063 Apply suggestions from code review 2024-10-08 14:14:16 +02:00
6af55b1a80 Update Dockerfile 2024-10-08 11:59:43 +02:00
2230674c0a Merge branch 'fix-append-only-vec' into indexer-edition-2024 2024-10-08 10:32:45 +02:00
5b04189f7a remove flaky assert 2024-10-07 16:50:57 +02:00
eb09dfed04 Avoid reallocation with the ThreadLocal pool 2024-10-07 16:41:17 +02:00
83c09d0db0 Remove the now, useless AppendOnlyVec library 2024-10-07 16:38:45 +02:00
c0912aa685 add missing shared servers 2024-10-07 16:29:47 +02:00
af38f46621 Merge branch 'main' of https://github.com/meilisearch/meilisearch into test/improve-create-index 2024-10-07 16:27:57 +02:00
c11b7e5c0f Reduce number of cache created by using thread_local 2024-10-07 15:58:16 +02:00
03579aba13 Adjust test 2024-10-04 11:38:47 +03:00
c3de3a9ab7 Refactor 2024-10-04 11:30:31 +03:00
386ca86297 Merge #4963
4963: test: improve performance of delete_index.rs r=curquiza a=DerTimonius

# Pull Request

## Related issue
related to #4840

## What does this PR do?
This PR follows the instructions in #4840 and improves the performance of `meilisearch/tests/index/delete_index.rs`. The tests run locally, if they fail in the CI I'll try to fix them

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Timon Jurschitsch <timon.jurschitsch@gmail.com>
2024-10-03 15:40:07 +00:00
dff2d54784 Merge pull request #4976 from meilisearch/fix-append-only-vec
Fix append only `Vec` by using a `LinkedList`
2024-10-03 17:26:00 +02:00
58d96fbea3 Rename Node parent to next 2024-10-03 16:15:05 +02:00
4665bfcb19 Move the parent assignation before the exchange operation 2024-10-03 16:14:23 +02:00
a7a01646cf Remove the useless Manually drop 2024-10-03 15:57:31 +02:00
0409a26cd8 Replace the concurrent vec by a linked list 2024-10-03 15:15:29 +02:00
8221c94e7f Split into multiple files, refactor 2024-10-03 15:37:51 +03:00
35f78b5423 TO REMOVE: usefull debug prints 2024-10-03 11:13:01 +02:00
14261f8f04 Integrate facet level bulk update
Only the facet bulk update has been added so far, the incremental must be completely rewritten

Factorize facet merging

Fix facet level extraction
2024-10-03 11:13:00 +02:00
774ed28539 Fix Prefix FST when a document is modified 2024-10-03 11:12:26 +02:00
d79f75f630 Compute and Write external-documents-ids database 2024-10-03 11:11:56 +02:00
c427d9e2ad Merge branch 'main' into change-matches-position-phrase-search 2024-10-03 10:42:34 +03:00
40336ce87d Fix and refactor crop_bounds 2024-10-03 10:40:14 +03:00
2a18917af3 add delete_index_fail function 2024-10-02 16:23:21 +02:00
ccf01c2471 Merge pull request #4969 from meilisearch/indexer-edition-2024-try-map
Indexer edition 2024 try map
2024-10-02 11:25:05 +02:00
0566f2549d Merge #4972
4972: Add binary quantized to error messages r=irevoire a=dureuill

was missing in error messages

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
v1.11.0-rc.1
2024-10-02 09:23:55 +00:00
0c2661ea90 Fix tests 2024-10-02 11:20:29 +02:00
62dfbd6255 Add binary quantized to allowed fields for source adds its sources 2024-10-02 11:20:02 +02:00
cc669f90d5 Merge #4971
4971: update arroy r=dureuill a=irevoire

# Pull Request

Fix part of https://github.com/meilisearch/meilisearch/issues/3715


## What does this PR do?
- Update arroy to the latest version, most change are maintenance changes
- The performances of adding vectors to arroy should slightly improve
- Forward the build cancellation function to arroy so it can stop building trees when we have to stop an indexing process


Co-authored-by: Tamo <tamo@meilisearch.com>
2024-10-02 05:53:51 +00:00
37a9d64c44 Fix failing test, refactor 2024-10-01 22:52:01 +03:00
b1dc10e771 uses the new cancellation method in arroy 2024-10-01 17:45:49 +02:00
4b598fa648 update arroy 2024-10-01 17:31:12 +02:00
17571805b4 use shared servers 2024-10-01 17:27:27 +02:00
2654ce6e6c use shared servers 2024-10-01 17:01:47 +02:00
d9e4db9983 Refactor 2024-10-01 17:50:59 +03:00
6d16230f17 Refactor 2024-10-01 17:19:15 +03:00
b7a5ba100e Move the ParallelIteratorExt into the parallel_iterator_ext module 2024-10-01 11:11:52 +02:00
dead7a56a3 Keep the caches in the AppendOnlyVec 2024-10-01 11:11:39 +02:00
0a8cb471df Introduce the AppendOnlyVec struct for the parallel computing 2024-10-01 11:11:25 +02:00
00e045b249 Rename and use the try_arc_for_each_try_init method 2024-10-01 11:11:25 +02:00
d83c9a4074 Introduce the try_for_each_try_init method to be used with Arced Errors 2024-10-01 11:11:25 +02:00
f3356ddaa4 Fix the errors when using the try_map_try_init method 2024-10-01 11:11:10 +02:00
31de5c747e WIP using try_map_try_init 2024-10-01 11:10:53 +02:00
3843240940 Prefer using Ars instead of Options 2024-10-01 11:10:53 +02:00
8cb5e7437d try using try_map_try_init 2024-10-01 11:10:53 +02:00
5b776556fe Add ParallelIteratorExt 2024-10-01 11:10:53 +02:00
bb7a503e5d Compute prefix databases
We are now computing the prefix FST and a prefix delta in the Merger thread,
after all the databases are written, the main thread will recompute the prefix databases based on the prefix delta without needing any grenad temporary file anymore
2024-10-01 09:57:06 +02:00