|
|
cdd69290c3
|
test: Make the tests work with new separator limits
|
2019-09-24 20:49:42 +02:00 |
|
|
|
175b3dcb75
|
fix: Do not consider underscores and middle dash hard
|
2019-09-24 20:14:20 +02:00 |
|
|
|
6b9426a051
|
feat: Support underscore as a split character
|
2019-09-24 13:56:32 +02:00 |
|
|
|
abf7191eec
|
feat: Make the Tokenizer able to support tokenizing sequences
|
2019-04-29 14:32:00 +02:00 |
|
|
|
a745819ddf
|
feat: Simplify the Tokenizer to use the LinearStrGroupBy type
|
2019-04-29 14:31:37 +02:00 |
|
|
|
19e67dcf0b
|
feat: Move query splitting into the tokenizer workspace
|
2019-04-29 14:31:37 +02:00 |
|
|
|
1897da5348
|
feat: Move tokenizer things into the meilidb-tokenizer workspace
|
2019-04-29 14:31:37 +02:00 |
|
