Commit Graph

76 Commits

Author SHA1 Message Date
Irevoire
0defeb268c bump milli 2022-02-16 13:27:41 +01:00
Clémentine Urquizar
d03b3ceb58 Update version for the next release (v0.22.1) 2022-02-07 18:39:29 +01:00
Tamo
367f403693 bump milli 2022-01-17 16:41:34 +01:00
Samyak S Sarnayak
c0313f3026 Use chars for highlight instead of graphemes
Tokenizer v0.2.7 uses chars instead of graphemes for matching bytes.
`unicode-segmentation` dependency isn't needed anymore.

Also, oxidised the highlight code :)

Co-authored-by: many <maxime@meilisearch.com>
2022-01-17 13:15:31 +05:30
Samyak S Sarnayak
c10f58b7bd Update tokenizer to v0.2.7 2022-01-17 13:02:00 +05:30
Samyak S Sarnayak
30247d70cd Fix search highlight for non-unicode chars
The `matching_bytes` function takes a `&Token` now and:
- gets the number of bytes to highlight (unchanged).
- uses `Token.num_graphemes_from_bytes` to get the number of grapheme
  clusters to highlight.

In essence, the `matching_bytes` function returns the number of matching
grapheme clusters instead of bytes. Should this function be renamed
then?

Added proper highlighting in the HTTP UI:
- requires dependency on `unicode-segmentation` to extract grapheme
  clusters from tokens
- `<mark>` tag is put around only the matched part
    - before this change, the entire word was highlighted even if only a
      part of it matched
2022-01-17 11:37:44 +05:30
many
1b3923b5ce Update all packages to 0.21.0 2021-11-29 12:17:59 +01:00
many
64ef5869d7 Update tokenizer v0.2.6 2021-11-18 16:56:05 +01:00
many
743ed9f57f Bump milli version 2021-11-04 14:04:21 +01:00
many
702589104d Update version for the next release (v0.20.1) 2021-11-03 14:20:01 +01:00
Clémentine Urquizar
056ff13c4d Update version for the next release (v0.20.0) 2021-10-28 14:52:57 +02:00
Clémentine Urquizar
679fe18b17 Update version for the next release (v0.19.0) 2021-10-25 11:52:17 +02:00
Clémentine Urquizar
f8fe9316c0 Update version for the next release (v0.18.1) 2021-10-21 11:56:14 +02:00
Clémentine Urquizar
2209acbfe2 Update version for the next release (v0.18.2) 2021-10-18 13:45:48 +02:00
bors[bot]
c7db4176f3 Merge #384
384: Replace memmap with memmap2 r=Kerollmops a=palfrey

[memmap is unmaintained](https://rustsec.org/advisories/RUSTSEC-2020-0077.html) and needs replacing. memmap2 is a drop-in replacement fork that's well maintained. Note that the version numbers got reset on fork, hence the lower values.

Co-authored-by: Tom Parker-Shemilt <palfrey@tevp.net>
2021-10-13 13:47:23 +00:00
Clémentine Urquizar
dd56e82dba Update version for the next release (v0.17.2) 2021-10-11 15:20:35 +02:00
Tom Parker-Shemilt
2dfe24f067 memmap -> memmap2 2021-10-10 22:47:12 +01:00
Clémentine Urquizar
05d8a33a28 Update version for the next release (v0.17.1) 2021-10-02 16:21:31 +02:00
Clémentine Urquizar
0e8665bf18 Update version for the next release (v0.17.0) 2021-09-28 19:38:12 +02:00
Clémentine Urquizar
1eacab2169 Update version for the next release (v0.15.1) 2021-09-22 17:18:54 +02:00
Clémentine Urquizar
f8ecbc28e2 Update version for the next release (v0.15.0) 2021-09-21 18:09:14 +02:00
mpostma
aa6c5df0bc Implement documents format
document reader transform

remove update format

support document sequences

fix document transform

clean transform

improve error handling

add documents! macro

fix transform bug

fix tests

remove csv dependency

Add comments on the transform process

replace search cli

fmt

review edits

fix http ui

fix clippy warnings

Revert "fix clippy warnings"

This reverts commit a1ce3cd96e603633dbf43e9e0b12b2453c9c5620.

fix review comments

remove smallvec in transform loop

review edits
2021-09-21 16:58:33 +02:00
Clémentine Urquizar
3f1453f470 Update version for the next release (v0.14.0) 2021-09-20 18:12:23 +02:00
Clémentine Urquizar
f167f7b412 Update version for the next release (v0.13.1) 2021-09-10 09:48:17 +02:00
Clémentine Urquizar
eb7b9d9dbf Update version for the next release (v0.13.0) 2021-09-08 10:59:30 +02:00
Clémentine Urquizar
285849e3a6 Update version for the next release (v0.12.0) 2021-09-02 10:08:41 +02:00
Kerollmops
af65485ba7 Reexport the grenad CompressionType from milli 2021-08-24 18:15:31 +02:00
Kerollmops
2f20257070 Update milli to the v0.11.0 2021-08-24 18:10:11 +02:00
Clémentine Urquizar
88f6c18665 Update version for the next release (v0.10.2) 2021-08-23 11:33:30 +02:00
bors[bot]
198c416bd8 Merge #312
312: Update milli version to v0.10.1 r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-08-18 12:08:04 +00:00
Clémentine Urquizar
6cb9c3b81f Update milli version to v0.10.1 2021-08-18 13:46:27 +02:00
Clémentine Urquizar
42cf847a63 Update tokenizer version to v0.2.5 2021-08-18 13:37:41 +02:00
Clémentine Urquizar
fcc520e49a Update version for the next release (v0.10.0) 2021-08-16 12:00:28 +02:00
Clémentine Urquizar
7f26c75610 Update milli to v0.9.0 2021-08-04 16:04:55 +02:00
Kerollmops
341c244965 Bump milli to v0.8.1 2021-07-29 15:56:36 +02:00
Clémentine Urquizar
6a141694da Update version for the next release (v0.8.0) 2021-07-27 16:38:42 +02:00
Kerollmops
91c5d0c042 Use the AlwaysFreePages flag when opening an index 2021-07-05 16:36:13 +02:00
Kerollmops
a6b4069172 Bump to v0.7.2 2021-07-05 10:54:53 +02:00
Clémentine Urquizar
b489515f4d Update milli version to v0.7.1 2021-06-30 13:52:46 +02:00
Clément Renault
80c6aaf1fd Bump milli to 0.7.0 2021-06-28 18:31:56 +02:00
Clément Renault
bdc5599b73 Bump heed to use the git repo with v0.12.0 2021-06-28 18:26:20 +02:00
Kerollmops
98285b4b18 Bump milli to 0.6.0 2021-06-23 17:30:26 +02:00
Clémentine Urquizar
9885fb4159 Update version for the next release (v0.5.1) 2021-06-23 14:05:20 +02:00
Tamo
77eb37934f add jemalloc to http-ui and the benchmarks 2021-06-22 14:17:56 +02:00
Clémentine Urquizar
320670f8fe Update version for the next release (v0.5.0) 2021-06-21 15:59:17 +02:00
Clémentine Urquizar
35fcc351a0 Update version for the next release (v0.4.2) 2021-06-20 17:37:24 +02:00
Kerollmops
ccd6f13793 Update version to the next release (0.4.1) 2021-06-17 15:01:20 +02:00
Clémentine Urquizar
f5ff3e8e19 Update version for the next release (v0.4.0) 2021-06-16 14:01:05 +02:00
Clémentine Urquizar
7d5395c12b Update Tokenizer version to v0.2.3 2021-06-10 17:00:04 +02:00
Clémentine Urquizar
dc64e139b9 Update version for the next release (v0.3.1) 2021-06-09 14:39:21 +02:00