Commit Graph

12013 Commits

Author SHA1 Message Date
dcf29e1081 fix the error handling in case there is a panic while creating a dump 2021-05-24 17:33:42 +02:00
529f7962f4 handle parallel requests for the dump actor 2021-05-24 15:42:12 +02:00
49bee2ebc5 Merge #190
190: Make bucket candidates optionals r=Kerollmops a=LegendreM

Before the bucket candidates were the result of the facet filters or result of the query tree.
They will now be only the result of the query tree, making the number of candidates more consistent between the same request with or without facet filters.

Fix some clippy warnings.

Fix #186 

Co-authored-by: many <maxime@meilisearch.com>
2021-05-24 11:19:32 +00:00
8a11c6c429 Implements the legacy behaviour of the dump
When asked if a dump exists we check if it's the current dump, and if
it's not then we check on the filesystem for any file matching our
`uid.dump`
2021-05-24 12:35:46 +02:00
4cbf866821 merge with main 2021-05-12 18:12:37 +02:00
e0e23636c6 fix the serializer + reformat the file 2021-05-12 17:04:24 +02:00
295f496e8a atomic index dump load 2021-05-12 16:21:37 +02:00
47a1bc34de Merge #189
189: Fix snapshots r=irevoire a=MarinPostma



Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-12 09:28:50 +00:00
6d837e3e07 the route to create a dump must return a 202 2021-05-11 17:34:34 +02:00
1b671d4302 fix-snapshot 2021-05-11 13:57:18 +02:00
c30b32e173 add the criterion attribute when importing dumps from the v1 2021-05-11 13:21:36 +02:00
9e798fea75 fix the import of dump without unprocessing updates 2021-05-11 13:03:47 +02:00
384afb3455 fix the way we return the settings 2021-05-11 11:47:04 +02:00
a3944a7083 Introduce a filtered_candidates field 2021-05-11 11:37:40 +02:00
92a7c8cd17 make clippy happy 2021-05-11 00:27:22 +02:00
8b7735c20a move the import of the updates in the v2 and ignore the v1 for now 2021-05-11 00:20:55 +02:00
7d748fa384 integrate the new Settings in the dumps 2021-05-10 20:48:06 +02:00
d767990424 fix the import of the updates in the dump 2021-05-10 20:25:12 +02:00
ef438852cd fix the v1 2021-05-10 20:25:12 +02:00
40ced3ff8d first working version 2021-05-10 20:25:12 +02:00
5f5402a3ab provide a way to access the internal content path of all processing State 2021-05-10 20:25:12 +02:00
26dcb9e66d bump milli version and fix a performance issue for large dumps 2021-05-10 20:25:12 +02:00
956012da95 fix dump lock 2021-05-10 20:25:12 +02:00
24192fc550 fix tests 2021-05-10 20:25:12 +02:00
efca63f9ce [WIP] rebase on main 2021-05-10 20:25:09 +02:00
c3552cecdf WIP rebase on main 2021-05-10 20:24:18 +02:00
0f94ef8abc WIP: dump 2021-05-10 20:24:18 +02:00
0275b36fb0 [WIP] rebase on main 2021-05-10 20:24:14 +02:00
1b5fc61eb6 [WIP] rebase on main 2021-05-10 20:23:12 +02:00
0fee81678e [WIP] rebase on main 2021-05-10 20:22:18 +02:00
c4d898a265 split the dumps between v1 and v2 2021-05-10 20:20:57 +02:00
e389c088eb WIP: rebasing on master 2021-05-10 20:20:57 +02:00
ceb8d6e1c9 Merge #186
186: settings fix r=MarinPostma a=MarinPostma

add type checked settigns validation. For now it only transform the settings accepting wildcard


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-10 16:42:12 +00:00
0cc79d414f add test 2021-05-10 18:34:25 +02:00
8d11b368d1 implement check 2021-05-10 18:22:41 +02:00
706643dfed type setting struct 2021-05-10 17:30:09 +02:00
efba662ca6 Fix clippy warnings in cirteria 2021-05-10 10:27:18 +02:00
e923d51b8f Make bucket candidates optionals 2021-05-10 10:27:04 +02:00
eeb0c70ea2 meilisearch compatible primary key inference 2021-05-06 22:42:32 +02:00
313c362461 early return on empty document addition 2021-05-06 18:14:16 +02:00
c620626515 Merge pull request #188 from meilisearch/exactness-criterion
Exactness criterion
2021-05-06 17:56:21 +02:00
44b6843de7 Fix pull request reviews
Update milli/src/fields_ids_map.rs
Update milli/src/search/criteria/exactness.rs
Update milli/src/search/criteria/mod.rs
2021-05-06 14:31:03 +02:00
c1ce4e4ca9 Introduce mocked ExactAttribute step in exactness criterion 2021-05-06 14:28:31 +02:00
a3f8686fbf Introduce exactness criterion 2021-05-06 14:28:30 +02:00
b192cb9c1f enable string syntax for the filters 2021-05-06 12:48:31 +02:00
25f75d4d03 Merge #189
189: Update version for the next release (v0.2.1) r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-05-05 15:28:56 +00:00
7e63e32960 Merge #187
187: Fix fields distribution after documents merge r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/milli/issues/174

The problem was with calculation of fields distribution before the merge in `output_from_sorter()`. So if you'd import two documents with the same primary key value, fields distribution will count it as two documents, while `output_from_sorter()` will merge these documents into one.

---

```console
➜ Downloads cat short_movies.json
[
{"id":"47474","title":"The Serpent's Egg","poster":"https://image.tmdb.org/t/p/w500/n7z0doFkXHcvo8QQWHLFnkEPXRU.jpg","overview":"The Serpent's Egg follows a week in the life of Abel Rosenberg, an out-of-work American circus acrobat living in poverty-stricken Berlin following Germany's defeat in World War I.","release_date":246844800,"genres":["Thriller","Drama","Mystery"]},
{"id":"47474","title":"The Serpent's Egg","poster":"https://image.tmdb.org/t/p/w500/n7z0doFkXHcvo8QQWHLFnkEPXRU.jpg","overview":"The Serpent's Egg follows a week in the life of Abel Rosenberg, an out-of-work American circus acrobat living in poverty-stricken Berlin following Germany's defeat in World War I.","release_date":246844800,"genres":["Thriller","Drama","Mystery"]}
]
➜ Downloads curl -X POST -H "Content-Type: text/json" --data-binary @short_movies.json 127.0.0.1:7700/indexes/movies/documents
{"updateId":0}
```

## Before
```console
➜ Downloads curl -s 127.0.0.1:7700/indexes/movies/stats | jq
{
  "numberOfDocuments": 1,
  "isIndexing": false,
  "fieldsDistribution": {
    "release_date": 2,
    "poster": 2,
    "title": 2,
    "overview": 2,
    "genres": 2,
    "id": 2
  }
}
```

## After
```console
➜ Downloads curl -s 127.0.0.1:7700/indexes/movies/stats | jq
{
  "numberOfDocuments": 1,
  "isIndexing": false,
  "fieldsDistribution": {
    "poster": 1,
    "release_date": 1,
    "title": 1,
    "genres": 1,
    "id": 1,
    "overview": 1
  }
}
```

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-05-05 14:45:08 +00:00
1e11578ef0 Update version for the next release (v0.2.1) 2021-05-05 14:57:34 +02:00
998d5ead34 Merge #182
182: remove facet setting r=MarinPostma a=MarinPostma

remove useless code


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-05 11:22:12 +00:00
ec7eb7798f remove facet setting 2021-05-04 22:36:31 +02:00