Commit Graph

7405 Commits

Author SHA1 Message Date
11fc9059cf Change base index size to 2TiB prototype-unlimited-indexes-1 2023-02-16 14:31:08 +01:00
be01a33cea Skip computing index budget in tests 2023-02-16 14:31:08 +01:00
d93a60c005 Compute budget 2023-02-16 10:53:57 +01:00
babfbd9c87 Add dichotomic search to utils 2023-02-16 10:53:57 +01:00
5daaf7cd5f Fix doc comment
Co-authored-by: Clément Renault <clement@meilisearch.com>
2023-02-16 10:53:57 +01:00
82ae5df706 Document generation and let it wrap-around as is it safe to do. 2023-02-16 10:53:57 +01:00
d3920a7e8a Retry in case of timeout while reopening 2023-02-16 10:53:57 +01:00
b07614932e Default to 100 concurrent indexes in unixes, 10 in Windows 2023-02-16 10:53:57 +01:00
89e74dcc31 Change default index map size to 10GiB 2023-02-16 10:53:57 +01:00
a4d0a56fc4 Add basic tests for index eviction and resize 2023-02-16 10:53:57 +01:00
06fa22203b Fix TODO 2023-02-16 10:53:57 +01:00
52bf260f73 Rewrite where evicted indexes are added to the set 2023-02-16 10:53:57 +01:00
f3c3ccc4b3 WIP: evict indexes in unavailable 2023-02-16 10:53:57 +01:00
e6cd7a68cc Parameterize growth factor and index count 2023-02-16 10:53:57 +01:00
de771a8bd7 Use LRU cache 2023-02-16 10:53:56 +01:00
9107ac86f1 Add LruMap 2023-02-16 10:53:56 +01:00
3908fdec29 Make sure we don't leave the in memory hashmap in an inconsistent state 2023-02-16 10:53:41 +01:00
081dfb82ce Resize indexes when they're full 2023-02-16 10:53:41 +01:00
52bdccee77 Add IndexMapper::resize_index fn 2023-02-16 10:53:41 +01:00
380a2bec04 Add IndexStatus::BeingResized 2023-02-16 10:53:41 +01:00
7b30f3e4de IndexScheduler::tick returns a TickOutcome 2023-02-16 10:53:41 +01:00
4a792e6e98 create_or_open_index takes a map_size argument 2023-02-16 10:53:41 +01:00
ad70f461b5 Add Batch::index_uid 2023-02-16 10:53:41 +01:00
49e18da23e Do not escape tag name
$() syntax is not interpreted by the Dockerfile
2023-02-16 10:53:14 +01:00
54240db495 Add note in code so one does not forget next time 2023-02-16 10:53:14 +01:00
e1ed4bc750 Change Dockerfile to also pass the VERGEN_GIT_SEMVER_LIGHTWEIGHT when building 2023-02-16 10:53:14 +01:00
9bd1cfb3a3 Ignore -dirty flag 2023-02-16 10:53:14 +01:00
a341c94871 Update contributing.md 2023-02-16 10:53:14 +01:00
f46cf46b8c Add prototype to analytics if any 2023-02-16 10:53:14 +01:00
c3a30a5a91 If using a prototype, display its name at Meilisearch startup 2023-02-16 10:53:14 +01:00
143e3cf948 Merge #3490
3490: Fix attributes set candidates r=curquiza a=ManyTheFish

# Pull Request

Fix attributes set candidates for v1.1.0

## details

The attribute criterion was not returning the remaining candidates when its internal algorithm was been exhausted.
We had a loss of candidates by the attribute criterion leading to the bug reported in the issue linked below.
After some investigation, it seems that it was the only criterion that had this behavior.

We are now returning the remaining candidates instead of an empty bitmap.

## Related issue

Fixes #3483
PR on milli for v1.0.1: https://github.com/meilisearch/milli/pull/777


Co-authored-by: ManyTheFish <many@meilisearch.com>
2023-02-15 17:38:07 +00:00
91ce8a5e67 Merge #3492
3492: Bump deserr r=Kerollmops a=irevoire

Bump deserr to the latest version;
- We now use the default actix-web extractors that deserr provides (which were copy/pasted from meilisearch)
- We also use the default `JsonError` message provided by deserr instead of defining our own in meilisearch
- Finally, we get the new `did you mean?` error message. Fix #3493

Co-authored-by: Tamo <tamo@meilisearch.com>
2023-02-15 10:05:05 +00:00
fd7ae1883b Merge #3495
3495: Add tests with rust nightly in CI r=curquiza a=ztkmkoo

# Pull Request

## Related issue
Fixes #3402 

## What does this PR do?
- add ci test with rust nightly
- make test with rust stable not run on schedule event

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Kebron <ztkmkoo@gmail.com>
2023-02-15 07:53:17 +00:00
42a3cdca66 get rids of the unwrap_any function in favor of take_cf_content 2023-02-14 20:06:31 +01:00
a43765d454 use the pre-defined deserr extractors 2023-02-14 20:05:30 +01:00
769576fd94 get rids of the whole error_message module since it has been integrated into the last version of deserr 2023-02-14 20:05:27 +01:00
8fb7b1d10f bump deserr 2023-02-14 20:04:30 +01:00
d494c29768 Merge #3479
3479: Unify "Bad latitude" & "Bad longitude" errors r=irevoire a=cymruu

# Pull Request

## Related issue
Fix part of #3006

## What does this PR do?
- Moved out `BadGeoLat`, `BadGeoLng`, `BadGeoBoundingBoxTopIsBelowBottom` from `FilterError` into newly introduced error type `ParseGeoError`. 
- Renamed `BadGeo` error  to `ReservedGeo`
- Used new `ParseGeoError` type in `FilterError` and `AscDescError`

Screenshot: 
![image](https://user-images.githubusercontent.com/2981598/217927231-fe23b6a3-2ea8-4145-98af-38eb61c4ff16.png)

I ran `cargo test --package milli -- --test-threads 1` and tests passed.
`--test-threads` was set to 1 because my OS complained about too many opened files.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [ ] Have you made sure that the title is accurate and descriptive of the changes?


Co-authored-by: Filip Bachul <filipbachul@gmail.com>
Co-authored-by: filip <filipbachul@gmail.com>
2023-02-14 18:35:51 +00:00
f3b54337f9 Merge #3174
3174: Allow wildcards at the end of index names for API Keys and Tenant tokens r=irevoire a=Kerollmops

This PR introduces the wildcards at the end of the index names when identifying indexes in the API Keys and tenant tokens. It fixes #2788 and fixes #2908. This PR is based on `@akhildevelops'` work.

Note that when a tenant token filter is chosen to restrict a search, it is always the most restrictive pattern that is chosen. If we have an index pattern _prod*_ that defines _filter1_ and _p*_ that defines _filter2_, the engine will choose _filter1_ over _filter2_ as it is defined for a most restrictive pattern, _prod*_. This restrictiveness is defined by 1. is it exact, without _*_ 2. the length of the pattern.

It is a continuation of work that has already started and should close #2869.

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
2023-02-14 16:12:01 +00:00
7f3ae40204 Remove a useless comment regarding the index pattern error code 2023-02-14 17:09:20 +01:00
a53536836b fmt 2023-02-14 17:04:22 +01:00
b095325bf8 Add tests with rust nightly in CI 2023-02-14 15:33:12 +00:00
d7ad39ad77 fix: clippy error 2023-02-14 00:15:35 +01:00
849de089d2 add thiserror for AscDescError 2023-02-14 00:15:35 +01:00
7f25007d31 Update milli/src/asc_desc.rs
Co-authored-by: Tamo <irevoire@protonmail.ch>
2023-02-14 00:15:35 +01:00
c810af3ebf implement From<ParseGeoError> for AscDescError 2023-02-14 00:15:35 +01:00
c0b77773ba fmt asc_desc 2023-02-14 00:15:35 +01:00
7481559e8b move BadGeo to FilterError 2023-02-14 00:15:35 +01:00
83c765ce6c implement From<ParseGeoError> for FilterError 2023-02-14 00:15:35 +01:00
4c91037602 use ParseGeoError in sort parser 2023-02-14 00:15:35 +01:00