Commit Graph

207 Commits

Author SHA1 Message Date
ab2cf69e8d Update milli/src/update/delete_documents.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-01 17:04:10 +02:00
8e6d1ff0dc Update milli/src/update/index_documents/store.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-01 17:04:02 +02:00
c701f8bf36 Use field id word count database in exactness criterion 2021-05-31 16:27:28 +02:00
4ddf008be2 add field id word count database 2021-05-31 16:27:28 +02:00
2f5e61bacb Merge #184
184: Transfer numbers and strings facets into the appropriate facet databases r=Kerollmops a=Kerollmops

This pull request is related to https://github.com/meilisearch/milli/issues/152 and changes the layout of the facets values, numbers and strings are now in dedicated databases and the user no more needs to define the type of the fields. No more conversion between the two types is done, numbers (floats and integers converted to f64) go to the facet float database and strings go to the strings facet database.

There is one related issue that I found regarding CSVs, the values in a CSV are always considered to be strings, [meilisearch/specifications#28](d916b57d74/text/0028-indexing-csv.md) fixes this issue by allowing the user to define the fields types using `:` in the "CSV Formatting Rules" section.

All previous tests on facets have been modified to pass again and I have also done hand-driven tests with the 115m songs dataset. Everything seems to be good!

Fixes #192.

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-05-31 13:32:58 +00:00
1c0a5cd136 Resolve code modification suggestions 2021-05-31 15:22:50 +02:00
a5e98cf46d Fix plane sweep algorithm 2021-05-25 18:21:55 +02:00
3a4a150ef0 Fix the tests and remaining warnings 2021-05-25 11:31:06 +02:00
02c655ff1a Refine the facet distribution to use both databases 2021-05-25 11:30:00 +02:00
79efded841 Refine the FacetCondition from_array constructor 2021-05-25 11:30:00 +02:00
f7efde11d9 Refine the facet condition to use both facet databases 2021-05-25 11:30:00 +02:00
e62b89a2ed Make the facet distinct work with the new split facets 2021-05-25 11:30:00 +02:00
bd7b285bae Split the update side to use the number and the strings facet databases 2021-05-25 11:30:00 +02:00
038e03a4e4 Use both facet databases in the FacetIter type 2021-05-25 11:30:00 +02:00
597144b0b9 Use both number and string facet databases in the distinct system 2021-05-25 11:29:59 +02:00
837c1041c7 Clear and delete the documents from the facet database 2021-05-25 11:28:36 +02:00
a56c46b6f1 Explode the string and f64 facet databases into two 2021-05-25 11:28:36 +02:00
df7a32e3d0 Move the creation date initialization into a function 2021-05-25 11:28:35 +02:00
a3944a7083 Introduce a filtered_candidates field 2021-05-11 11:37:40 +02:00
efba662ca6 Fix clippy warnings in cirteria 2021-05-10 10:27:18 +02:00
e923d51b8f Make bucket candidates optionals 2021-05-10 10:27:04 +02:00
44b6843de7 Fix pull request reviews
Update milli/src/fields_ids_map.rs
Update milli/src/search/criteria/exactness.rs
Update milli/src/search/criteria/mod.rs
2021-05-06 14:31:03 +02:00
c1ce4e4ca9 Introduce mocked ExactAttribute step in exactness criterion 2021-05-06 14:28:31 +02:00
a3f8686fbf Introduce exactness criterion 2021-05-06 14:28:30 +02:00
25f75d4d03 Merge #189
189: Update version for the next release (v0.2.1) r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-05-05 15:28:56 +00:00
1e11578ef0 Update version for the next release (v0.2.1) 2021-05-05 14:57:34 +02:00
f8d0f5265f fix(update): fields distribution after documents merge 2021-05-04 22:12:20 +03:00
d61566787e provide an iterator over all the documents in a milli index 2021-05-04 11:23:51 +02:00
a8680887d8 Upgrade Milli version (v0.2.0) 2021-05-03 14:50:47 +02:00
34e02aba42 Upgrade Tokenizer version (v0.2.2) 2021-05-03 10:55:55 +02:00
d81c0e8bba feat(update): disable autogenerate_docids by default 2021-04-30 21:41:34 +03:00
e8e32e0ba1 make document addition number visible 2021-04-29 20:05:07 +02:00
ee09e50e7f Remove excluded document in criteria iterations
- pass excluded document to criteria to remove them in higher levels of the bucket-sort
- merge already returned document with excluded documents to avoid duplicas

Related to #125 and #112
Fix #170
2021-04-29 12:09:38 +02:00
31607bf9cd Add a threshold on proximity when choosing between linear/set algorithm 2021-04-28 14:57:22 +02:00
3b7e6afb55 Make some refacto and add documentation 2021-04-28 13:53:27 +02:00
0add4d735c Update milli/src/search/criteria/attribute.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:40:34 +02:00
3794ffc952 Update milli/src/search/criteria/attribute.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:39:23 +02:00
329bd4a1bb Update milli/src/search/criteria/attribute.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:39:03 +02:00
3b1358b62f Update milli/src/search/criteria/attribute.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:32:19 +02:00
c862b1bc6b Update milli/src/search/criteria/attribute.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:32:10 +02:00
e92d137676 Update milli/src/search/criteria/attribute.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:31:42 +02:00
b3d6c6a9a0 Update milli/src/search/criteria/attribute.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:31:13 +02:00
498c2b298c Update milli/src/search/criteria/attribute.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:30:02 +02:00
0e4e6dfada Update milli/src/search/criteria/proximity.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 17:29:52 +02:00
47d780b8ce Update milli/src/search/criteria/mod.rs
Co-authored-by: Irevoire <tamo@meilisearch.com>
2021-04-27 14:39:53 +02:00
0daa0e170a Fix PR comments
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-04-27 14:39:53 +02:00
0d7d3ce802 Update roaring package 2021-04-27 14:39:53 +02:00
71740805a7 Fix forgotten typo tests 2021-04-27 14:39:53 +02:00
e77291a6f3 Optimize Atrribute criterion on big requests 2021-04-27 14:39:53 +02:00
716c8e22b0 Add style and comments 2021-04-27 14:39:52 +02:00