Commit Graph

356 Commits

Author SHA1 Message Date
9992c36ced Merge branch 'stable'
fix conflict with master
2021-03-16 16:59:39 +01:00
d1f34f926e [ci] Add all target to check 2021-03-02 20:48:57 +01:00
1df0fdf3e2 fix synonyms normalization
Synonyms needs to be indexed in ascendant order,
and the new normalization step for synonyms potentially changes this order
which break the indexation process
because "Harry Potter" > "HP"  but "harry potter" < "hp"
2021-02-03 15:21:06 +01:00
8760beed1c bump meilisearch 2021-02-02 14:23:33 +01:00
c984fa1071 Merge #1176
1176: fix race condition in  document addition r=Kerollmops a=MarinPostma

As described in #1160, there was a race condition when updating settings and adding documents simultaneously. This was due to the schema being updated and document addition being processed in two different transactions. This PR moves the schema update logic for the primary key in the same transaction as the document addition, while maintaining the input checks for the validity of the primary key in the http route, in order not to break the error reporting for the document addition route.

close #1160.

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: marin <postma.marin@protonmail.com>
2021-02-02 09:26:32 +00:00
81e9fd8933 Merge #1184
1184: normalize synonyms during indexation r=MarinPostma a=LegendreM

fix #1135 #964

Normalizes the synonyms before indexing them, so they are not case sensitive anymore. Then normalization also involves deunicoding is some cases, such as accents, so `été` and `ete` are considered equivalent in a search for synonyms.

Co-authored-by: many <maxime@meilisearch.com>
Co-authored-by: Many <legendre.maxime.isn@gmail.com>
2021-02-01 14:12:57 +00:00
940f83698c Update meilisearch-core/src/update/settings_update.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-02-01 12:06:48 +01:00
f37a420a04 Merge #1174
1174: Limit query words number r=MarinPostma a=MarinPostma

This pr adds a limit to the number of words taken into account in a search query. Using query string that are too long leads to huge performance hits and ressources consumtion, that occasionally crashes the machine. The limit has been hard set to 10, and tests have been added to make sure that it is taken into account.

close #941

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-28 17:38:34 +00:00
eeccdce33a update tokenizer to v0.1.3 2021-01-28 10:33:44 +01:00
1d910dbb42 Update meilisearch-core/src/update/documents_addition.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-01-15 00:55:31 +01:00
7a7cb9bcbf update dependencies 2021-01-13 15:48:53 +01:00
9b47bbc1ac bump meilisearch 2021-01-13 15:37:15 +01:00
430a5f902b fix race condition in document addition 2021-01-13 13:17:52 +01:00
bc0d53e819 Update meilisearch-core/src/update/settings_update.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-01-13 13:17:19 +01:00
06b2a587af normalize synonyms during indexation 2021-01-12 13:53:32 +01:00
81f343a46a add word limit to search queries 2021-01-08 16:23:23 +01:00
948c89c26f bump meilisearch 2021-01-06 11:41:44 +01:00
677627586c fix test set
fix dump tests
2021-01-05 21:37:05 +01:00
0731971300 fix style 2021-01-05 15:21:06 +01:00
c290719984 remove byte offset in index_seq 2021-01-05 15:21:06 +01:00
2a145e288c fix style 2021-01-05 15:21:06 +01:00
aeb676e757 skip indexation while token is not a word 2021-01-05 15:21:06 +01:00
2852349e68 update tokenizer version 2021-01-05 15:21:06 +01:00
748a8240dd fix highlight shifting bug 2021-01-05 15:21:05 +01:00
808be4678a fix style 2021-01-05 15:21:05 +01:00
398577f116 bump tokenizer 2021-01-05 15:21:05 +01:00
8e64a24d19 fix suggestions 2021-01-05 15:21:05 +01:00
8b149c9aa3 update tokenizer dep to release 2021-01-05 15:21:05 +01:00
a7c88c7951 restore synonyms tests 2021-01-05 15:21:05 +01:00
db64e19b8d all tests pass 2021-01-05 15:21:05 +01:00
b574960755 fix split_query_string 2021-01-05 15:21:05 +01:00
c6434f609c fix indexing length 2021-01-05 15:21:05 +01:00
206308c1aa replace hashset with fst::Set 2021-01-05 15:21:05 +01:00
6527d3e492 better separator handling 2021-01-05 15:21:05 +01:00
e616b1e356 hard separator offset 2021-01-05 15:21:05 +01:00
8843062604 fix indexer tests 2021-01-05 15:21:05 +01:00
5e00842087 integration with new tokenizer wip 2021-01-05 15:21:05 +01:00
8a4d05b7bb remove meilisearch tokenizer 2021-01-05 15:21:05 +01:00
061832af7f Merge #1163
1163: remove benches r=LegendreM a=MarinPostma

remove unused benches, that did not compile either


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-05 13:27:42 +00:00
0e04c90abe remove benches 2021-01-05 10:54:19 +01:00
48eb78b14d bump deps 2021-01-04 16:56:28 +01:00
5fe0e06342 fix clippy warnings 2020-12-15 12:42:19 +01:00
2904ca7f57 update codebase with shcema refactor 2020-12-15 12:04:51 +01:00
d45c794a9e bump rustyline 2020-12-09 12:41:36 +01:00
c9dd7e10b9 bump ordered floats 2020-12-09 12:40:24 +01:00
56ad400c49 update heed 2020-12-09 11:27:38 +01:00
e2b0402cf5 bump regex 2020-12-09 10:28:22 +01:00
0c7fffeaf6 update env-logger 2020-12-09 10:25:17 +01:00
5f8dc21dd2 bump once-cell 2020-12-09 10:22:14 +01:00
3ec76ac33d bump meilisearch 2020-11-30 16:35:56 +01:00