Commit Graph

83 Commits

Author SHA1 Message Date
Kerollmops
21ec334dcc Fix the compilation error of the dependency versions 2022-03-15 11:17:45 +01:00
Kerollmops
288a879411 Remove three useless dependencies 2022-03-15 11:17:44 +01:00
Kerollmops
9142ba9dd4 Fix the parsing of ndjson requests to index more than the first line 2022-02-02 17:55:13 +01:00
Marin Postma
0c84a40298 document batch support
reusable transform

rework update api

add indexer config

fix tests

review changes

Co-authored-by: Clément Renault <clement@meilisearch.com>

fmt
2022-01-19 12:40:20 +01:00
Samyak S Sarnayak
c0313f3026 Use chars for highlight instead of graphemes
Tokenizer v0.2.7 uses chars instead of graphemes for matching bytes.
`unicode-segmentation` dependency isn't needed anymore.

Also, oxidised the highlight code :)

Co-authored-by: many <maxime@meilisearch.com>
2022-01-17 13:15:31 +05:30
Samyak S Sarnayak
30247d70cd Fix search highlight for non-unicode chars
The `matching_bytes` function takes a `&Token` now and:
- gets the number of bytes to highlight (unchanged).
- uses `Token.num_graphemes_from_bytes` to get the number of grapheme
  clusters to highlight.

In essence, the `matching_bytes` function returns the number of matching
grapheme clusters instead of bytes. Should this function be renamed
then?

Added proper highlighting in the HTTP UI:
- requires dependency on `unicode-segmentation` to extract grapheme
  clusters from tokens
- `<mark>` tag is put around only the matched part
    - before this change, the entire word was highlighted even if only a
      part of it matched
2022-01-17 11:37:44 +05:30
Clément Renault
1c6c89f345 Fix the binaries that use the new optional filters 2021-12-09 11:57:53 +01:00
Marin Postma
6eb47ab792 remove update_id in UpdateBuilder 2021-11-16 13:07:04 +01:00
Tamo
6831c23449 merge with main 2021-11-06 16:34:30 +01:00
Tamo
07a5ffb04c update http-ui 2021-11-04 15:52:22 +01:00
marin postma
baddd80069 implement review suggestions 2021-10-25 18:29:12 +02:00
Tamo
e25ca9776f start updating the exposed function to makes other modules happy 2021-10-22 17:23:22 +02:00
bors[bot]
c7db4176f3 Merge #384
384: Replace memmap with memmap2 r=Kerollmops a=palfrey

[memmap is unmaintained](https://rustsec.org/advisories/RUSTSEC-2020-0077.html) and needs replacing. memmap2 is a drop-in replacement fork that's well maintained. Note that the version numbers got reset on fork, hence the lower values.

Co-authored-by: Tom Parker-Shemilt <palfrey@tevp.net>
2021-10-13 13:47:23 +00:00
many
c5a6075484 Make max_position_per_attributes changable 2021-10-12 10:10:50 +02:00
Tom Parker-Shemilt
2dfe24f067 memmap -> memmap2 2021-10-10 22:47:12 +01:00
many
1bd15d849b Reduce candidates threshold 2021-10-05 18:52:14 +02:00
many
d2427f18e5 Enhance CSV document parsing 2021-09-29 10:25:33 +02:00
Tamo
cc732fe95e update http-ui to use the sort-error 2021-09-28 11:15:24 +02:00
mpostma
aa6c5df0bc Implement documents format
document reader transform

remove update format

support document sequences

fix document transform

clean transform

improve error handling

add documents! macro

fix transform bug

fix tests

remove csv dependency

Add comments on the transform process

replace search cli

fmt

review edits

fix http ui

fix clippy warnings

Revert "fix clippy warnings"

This reverts commit a1ce3cd96e603633dbf43e9e0b12b2453c9c5620.

fix review comments

remove smallvec in transform loop

review edits
2021-09-21 16:58:33 +02:00
Irevoire
a84f3a8b31 Apply suggestions from code review
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-09-09 15:09:35 +02:00
Tamo
7483614b75 [HTTP-UI] add the sorters 2021-09-08 18:24:09 +02:00
Kerollmops
68856e5e2f Disable the default snappy compression for the http-ui crate 2021-09-08 14:17:32 +02:00
many
1d314328f0 Plug new indexer 2021-09-01 16:48:36 +02:00
Tamo
d106eb5b90 add the sortable attributes to http-ui and fix the tests 2021-08-30 16:25:10 +02:00
Kerollmops
af65485ba7 Reexport the grenad CompressionType from milli 2021-08-24 18:15:31 +02:00
Clément Renault
89d0758713 Revert "Revert "Sort at query time"" 2021-08-24 11:55:16 +02:00
Clémentine Urquizar
922f9fd4d5 Revert "Sort at query time" 2021-08-20 18:09:17 +02:00
Kerollmops
5b88df508e Use the new Asc/Desc syntax everywhere 2021-08-17 14:15:22 +02:00
Irevoire
4562b278a8 remove a warning and add a log
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-07-05 17:46:02 +02:00
Tamo
a57e522a67 introduce a die route let the program exit itself alone 2021-07-05 17:38:10 +02:00
bors[bot]
634201244c Merge #250 #251
250: Add the limit field to http-ui r=Kerollmops a=irevoire



251: Fix the limit r=Kerollmops a=irevoire

There was no check on the limit and thus if a user specified a very large number this line could cause a panic.

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-22 13:00:52 +00:00
Tamo
81643e6d70 add the limit field to http-ui 2021-06-22 14:47:23 +02:00
Tamo
77eb37934f add jemalloc to http-ui and the benchmarks 2021-06-22 14:17:56 +02:00
Tamo
9716fb3b36 format the whole project 2021-06-16 18:33:33 +02:00
Kerollmops
78fe4259a9 Fix the http-ui crate 2021-06-14 18:06:23 +02:00
Kerollmops
103dddba2f Move the UpdateStore into the http-ui crate 2021-06-08 17:59:51 +02:00
Kerollmops
3b1cd4c4b4 Rename the FacetCondition into FilterCondition 2021-06-02 16:24:58 +02:00
Kerollmops
c10469ddb6 Patch the http-ui crate to support filterable fields 2021-06-02 16:24:58 +02:00
Tamo
608c5bad24 fix http-ui 2021-06-01 16:24:46 +02:00
Kerollmops
5012cc3a32 Fix the http-ui crate to support split facet databases 2021-05-25 11:31:06 +02:00
Kerollmops
e65bad16cc Compute the words prefixes at the end of an update 2021-04-27 14:39:52 +02:00
Clément Renault
7aa5753ed2 Make the attribute positions range bounds to be fixed 2021-04-27 14:35:43 +02:00
Kerollmops
e8cc7f9cee Expose a route in the http-ui to update the WordsLevelPositions 2021-04-27 14:25:34 +02:00
tamo
7fa3a1d23e makes clippy happy http-ui 2021-04-21 12:36:45 +02:00
Alexey Shekhirin
33860bc3b7 test(update, settings): set & reset synonyms
fixes after review

more fixes after review
2021-04-18 11:24:17 +03:00
Alexey Shekhirin
e39aabbfe6 feat(search, update): synonyms 2021-04-18 11:24:17 +03:00
Alexey Shekhirin
3af8fa194c test(http): combine settings assert_(ser|de)_tokens into 1 test 2021-04-10 12:13:59 +03:00
Alexey Shekhirin
84c1dda39d test(http): setting enum serialize/deserialize 2021-04-08 17:03:40 +03:00
Alexey Shekhirin
dc636d190d refactor(http, update): introduce setting enum 2021-04-08 17:03:40 +03:00
tamo
a2f46029c7 implement a first version of the stop_words
The front must provide a BTreeSet containing the stop words
The stop_words are set at None if an empty Set is provided
add the stop-words in the http-ui interface

Use maplit in the test
and remove all the useless drop(rtxn) at the end of all tests
2021-04-01 13:57:55 +02:00