Commit Graph

7170 Commits

Author SHA1 Message Date
dbd91e7151 chore(ci): use smarter dependencies cache 2021-09-07 18:16:33 +03:00
720becb5e8 Merge #341
341: Throw a query time error when a sort parameter is used but the sort ranking rule is missing r=Kerollmops a=Kerollmops

This PR makes the engine throw an error for when the ranking rules don't contain the `sort` rule, the `sortable_fields` are correctly set but the user tries to use the `sort` query parameter. Doing so will have no effect on the returned documents so we preferred returning an error to help debug this.

That's breaking on the MeiliSearch side as we added a new variant to the `UserError` enum.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-09-07 14:45:05 +00:00
e2cefc9b4f Move the sort ranking rule before the exactness ranking rule 2021-09-07 16:41:33 +02:00
a0b3620b05 Merge #346
346: remove unused grenad default features r=Kerollmops a=MarinPostma

Milli is not using any of grenad default features, and it's zstd feature creates conflict with meilisearch. This pr simply remove the unused features.


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-09-07 14:30:20 +00:00
cd043d4461 remove unused grenad default features 2021-09-07 16:21:46 +02:00
efa69875d9 refactor(http): use reset_sortable_fields 2021-09-07 15:04:44 +03:00
5989528833 Add a test to make sure we throw the right error message 2021-09-07 11:02:00 +02:00
fd3daa4423 Throw a query time error when a sort param is used but sort ranking rule is missing 2021-09-07 11:02:00 +02:00
8dca36433c Introduce the new SortRankingRuleMissing user error variant 2021-09-07 11:01:59 +02:00
446ed17589 Merge #338
338: Fix string fields sorting r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/milli/issues/333

<details>
  <summary>curl checks</summary>
  
 ```console
➜  ~ curl -s 'localhost:7700/indexes/movies/search' -d '{"sort": ["title:asc"], "limit": 30}' | jq -r '.hits | map(.title)[]'
#1 Cheerleader Camp
#Horror
#RealityHigh
#Roxy
#SquadGoals
$5 a Day
$9.99
'71
(2)
(500) Days of Summer
(Girl)Friend
*batteries not included
...And God Created Woman
...And Justice for All
...E fuori nevica!
.45
1
1 Mile To You
1 Night
10
10 Cloverfield Lane
10 giorni senza mamma
10 Items or Less
10 Rillington Place
10 Rules for Sleeping Around
10 Things I Hate About You
10 to Midnight
10 Years
10,000 BC
10,000 Saints

➜  ~ curl -s 'localhost:7700/indexes/movies/search' -d '{"sort": ["title:desc"], "limit": 30}' | jq -r '.hits | map(.title)[]'
크게 될 놈
왓칭
뷰티플 마인드
노무현과 바보들
ハニー
Счастье – это… Часть 2
СОТКА
Смотри мою любовь
Позивний 'Бандерас'
Лошо момиче
Күлүк Хомус
Куда течет море
Каникулы президента
Ακίνητο Ποτάμι
Üç Harfliler: Beddua
È nata una Star?
Æon Flux
Ága
À propos de Nice
À Nos Amours
À l'aventure
¡Three Amigos!
Zulu Dawn
Zulu
Zulu
Zu: Warriors from the Magic Mountain
Zu Warriors
Zorro
Zorba the Greek
Zootopia
```
</details>

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-09-07 08:28:23 +00:00
59797bfc7b meilisearch/MeiliSearch#1673 Add curl binary to final stage image 2021-09-06 19:35:23 +02:00
c0f9c891f5 Set max_memory value to unlimited during tests
because tests run several meilisearch in parallel,
we over estimate the value for max_memory making the tests on widows crash
2021-09-06 14:38:10 +02:00
33514b28be Merge pull request #1588 from meilisearch/test-new-indexer
Integrate the new indexer
v0.22.0rc0
2021-09-06 10:21:42 +02:00
0be09555f1 test(search): asc/desc criteria for large datasets 2021-09-03 18:00:08 +03:00
c2517e7d5f fix(facet): string fields sorting 2021-09-03 11:58:26 +03:00
e3a913e03f Merge #1660
1660: Update version for the next release (v0.22.0) r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-09-02 16:43:32 +00:00
7e80337e5b Bump milli to v0.12.0 2021-09-02 18:19:12 +02:00
8d4723d91b Update lock file 2021-09-02 18:19:12 +02:00
4cdf680a81 Make the MaxMemory use the default value when undefined 2021-09-02 18:19:11 +02:00
63e67f72e3 Update tokenizer and new milli version 2021-09-02 18:19:00 +02:00
0cd66c3a89 Bump the milli version 2021-09-02 18:19:00 +02:00
b092a624ed Introduce the MaxMemory struct that defaults to 2/3 of the available memory 2021-09-02 18:18:59 +02:00
24e84d7ca1 Test new indexer 2021-09-02 18:11:20 +02:00
5cbe879325 Merge #308
308: Implement a better parallel indexer r=Kerollmops a=ManyTheFish

Rewrite the indexer:
- enhance memory consumption control
- optimize parallelism using rayon and crossbeam channel
- factorize the different parts and make new DB implementation easier
- optimize and fix prefix databases


Co-authored-by: many <maxime@meilisearch.com>
2021-09-02 15:03:52 +00:00
741a4444a9 Remove log in chunk generator 2021-09-02 16:57:46 +02:00
7f7fafb857 Make document_chunk_size settable from update builder 2021-09-02 15:25:39 +02:00
db0c681bae Fix Pr comments 2021-09-02 15:17:52 +02:00
14f9056349 Merge #1662
1662: Fix link in download script r=irevoire a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-09-02 11:27:14 +00:00
46f7df232a Merge #337
337: Update version for the next release (v0.12.0) r=Kerollmops a=curquiza

Breaking because of the new indexer that implies DB changes #308 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-09-02 10:13:31 +00:00
285849e3a6 Update version for the next release (v0.12.0) 2021-09-02 10:08:41 +02:00
b444e2e074 Proposed fix for ARM binary on RHEL
https://github.com/meilisearch/MeiliSearch/issues/1410
2021-09-02 01:01:46 +00:00
a589f6c60b Merge #335
335: Get sortable_fields from index only if criteria present in query r=Kerollmops a=shekhirin

Seems like we don't need to retrieve `sortable_fields` from the index if there's no any `sort_criteria` in the query.

Small 🤏  optimization opportunity out there.

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-09-01 16:01:00 +00:00
3e0a78acf3 Merge #329
329: Run all benchmarks once every friday r=irevoire a=irevoire

All the benchmarks run every Friday on the `main` branch.
To avoid having pending benchmarks everywhere, we execute one benchmark every 8 hours.
Then the results are uploaded as if it was a normal user-run benchmark.

This PR closes #314 and #321 

Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2021-09-01 15:20:49 +00:00
4860fd4529 Ignore empty facet values 2021-09-01 16:48:40 +02:00
b3a22f31f6 Fix memory consuption in word pair proximity extractor 2021-09-01 16:48:40 +02:00
9452fabfb2 Optimize cbo roaring bitmaps merge 2021-09-01 16:48:40 +02:00
8f702828ca Ignore errors comming from crossbeam channel senders 2021-09-01 16:48:40 +02:00
e09eec37bc Handle distance addition with hard separators 2021-09-01 16:48:40 +02:00
fc7cc770d4 Add logging timers 2021-09-01 16:48:40 +02:00
a2f59a28f7 Remove unwrap sending errors in channel 2021-09-01 16:48:40 +02:00
5c962c03dd Fix and optimize word_prefix_pair_proximity_docids database 2021-09-01 16:48:40 +02:00
2d1727697d Take stop word in account 2021-09-01 16:48:40 +02:00
823da19745 Fix test and use progress callback 2021-09-01 16:48:39 +02:00
1d314328f0 Plug new indexer 2021-09-01 16:48:36 +02:00
3aaf1d62f3 Publish grenad CompressionType type in milli 2021-09-01 16:42:08 +02:00
723cb4d520 Fix link in download script 2021-09-01 15:57:11 +02:00
90116155b4 Update version for the next release (v0.22.0) 2021-09-01 12:33:30 +02:00
0e379558a1 fix(search): get sortable_fields only if criteria present 2021-08-31 21:35:41 +03:00
d6bba0663a Merge #334
334: Wrap long values into BStr for warn logs r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/milli/issues/263

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-08-31 17:38:54 +00:00
0b02eb456c chore(update): wrap long values into BStr for warn logs 2021-08-31 20:28:16 +03:00