snowykami/meilisearch
1
0
Fork 0
mirror of https://github.com/meilisearch/meilisearch.git synced 2025-10-25 13:06:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,987 Commits 252 Branches 544 Tags
aa7a34ffe8b9572c44b4bd36c30f7cf3805a9ed7
Commit Graph

1023 Commits

Author SHA1 Message Date
Tamo
e1f27de51a remove the Stats Seen analytic 2024-04-16 18:49:41 +02:00
Tamo
abae31aee0 remove the Task Seen analytic 2024-04-16 18:48:10 +02:00
Tamo
70ce0095ea remove the Version Seen analytic 2024-04-16 18:48:03 +02:00
ManyTheFish
a1ea224da9 Fix tests 2024-04-16 17:29:34 +02:00
ManyTheFish
a489b406b4 fix test 2024-04-16 14:39:06 +02:00
Louis Dureuil
a9013ed683 Fix comment mistake 
Co-authored-by: Tamo <tamo@meilisearch.com>
 2024-04-04 17:21:47 +02:00
Louis Dureuil
ca499a0302 Fix test after rebase 2024-04-04 16:04:07 +02:00
Louis Dureuil
355e5282b2 Remove _semanticScore 2024-04-04 16:04:07 +02:00
Louis Dureuil
7c27417a5d Add tests 2024-04-04 16:04:07 +02:00
Louis Dureuil
1ff2a2d6fb Add semanticHitCount 2024-04-04 16:04:06 +02:00
Louis Dureuil
4564a38ae7 Bail earlier when the experimental feature is not enabled 2024-04-04 15:58:19 +02:00
Louis Dureuil
6ebb6b55a6 Lazily embed, don't fail hybrid search on embedding failure 2024-04-04 15:58:17 +02:00
Louis Dureuil
190933f6e1 Breaking: Remove vector from SearchResult 2024-04-04 15:57:29 +02:00
Louis Dureuil
928e6e4c05 Breaking change: remove vector for score details 2024-04-04 15:57:29 +02:00
meili-bors[bot]
5509bafff8 Merge #4535 
4535: Support Negative Keywords r=ManyTheFish a=Kerollmops

This PR fixes #4422 by supporting `-` before any word in the query.

The minus symbol `-`, from the ASCII table, is not the only character that can be considered the negative operator. You can see the two other matching characters under the `Based on "-" (U+002D)` section on [this unicode reference website](https://www.compart.com/en/unicode/U+002D).

It's important to notice the strange behavior when a query includes and excludes the same word; only the derivative ( synonyms and split) will be kept:
 - If you input `progamer -progamer`, the engine will still search for `pro gamer`.
 - If you have the synonym `like = love` and you input `like -like`, it will still search for `love`.

## TODO
 - [x] Add analytics
 - [x] Add support to the `-` operator
 - [x] Make sure to support spaces around `-` well
 - [x] Support phrase negation
 - [x] Add tests


Co-authored-by: Clément Renault <clement@meilisearch.com>
 2024-04-04 13:10:27 +00:00
Clément Renault
90e812fc0b Add some tests 2024-04-04 15:08:37 +02:00
meili-bors[bot]
56bf8503db Merge #4537 
4537: Expose distribution shift in settings r=ManyTheFish a=dureuill

See [usage page](https://meilisearch.notion.site/v1-8-AI-search-API-usage-135552d6e85a4a52bc7109be82aeca42#d652adc0890445658aaf36352dbc8802)

# Changes

- Distribution shift added to all embedders.
- Exposed in settings
- Changed the reindexing logic to not trigger a reindex operation when only the distribution shift or API key change

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
 2024-04-03 09:08:58 +00:00
meili-bors[bot]
78668584cd Merge #4533 
4533: Hide api key in settings and task queue r=dureuill a=dureuill

# Pull Request

See [Usage page](https://meilisearch.notion.site/v1-8-AI-search-API-usage-135552d6e85a4a52bc7109be82aeca42#117f5ff7b19f4d95bb3ae0005f6c6633)

## Motivation

See [slack discussion (internal link)](https://meilisearch.slack.com/archives/C06GQP7FQ6P/p1709804022298749)


## Changes

- The value of the `apiKey` parameter is now hidden in the settings and the details of the task queue.

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
 2024-03-28 16:02:53 +00:00
meili-bors[bot]
fa9748cc99 Merge #4536 
4536: Limit concurrent search requests r=ManyTheFish a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4489

## What does this PR do?
- Adds a « search queue » that limits the number of search requests we can process at the same time and stores search requests to be processed
- Process only one search request per core/thread (we use available_parallelism)
- When the search queue is full, new search requests replace old ones **randomly**. The reason is that:
  - If we serve the oldest one first, like Typesense, we give the worst performances to everyone
  - If we serve the latest one, it gets too easy to DoS us (you just need to fill the queue with as many search requests as we can process simultaneously to ensure no other request will ever be processed)
  - By picking the search request randomly, we give a chance to recent search requests to be processed while ensuring that we can't be owned unless they fill our queue entirely and we start returning errors 5xx
- Adds an experimental parameter to control the size of the queue
- Adds a bunch of tests to ensure the search queue works correctly
- Ensure the loop consuming the search queue is running in the health route and crashes if it’s not the case

Co-authored-by: Tamo <tamo@meilisearch.com>
 2024-03-28 15:01:52 +00:00
Tamo
06a11b5b21 Improve error message 2024-03-27 17:34:49 +01:00
Tamo
b7c582e4f3 connect the search queue with the health route 2024-03-27 15:49:43 +01:00
Tamo
03c886ac1b adds a bit of documentation 2024-03-27 15:38:36 +01:00
Louis Dureuil
cde7ce4f44 Add test 2024-03-27 14:02:09 +01:00
Tamo
087a96d22e fix flaky test 2024-03-27 11:05:37 +01:00
meili-bors[bot]
34dfea72cc Merge #4509 
4509: Rest embedder r=ManyTheFish a=dureuill

Fixes #4531 

See [Usage page](https://meilisearch.notion.site/v1-8-AI-search-API-usage-135552d6e85a4a52bc7109be82aeca42?pvs=25#e6f58c3b742c4effb4ddc625ce12ee16)

### Implementation changes

- Remove tokio, futures, reqwests
- Add a new `milli::vector::rest::Embedder` embedder
- Update OpenAI and Ollama embedders to use the REST embedder internally
- Make Embedder::embed a sync method
- Add the new embedder source as described in the usage


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
 2024-03-27 09:27:46 +00:00
Tamo
3a1f458139 fix a flaky test 2024-03-26 21:06:55 +01:00
Tamo
55df9daaa0 adds a comment about the safety of an operation 2024-03-26 19:34:55 +01:00
Tamo
2e36f069c2 fmt imports 2024-03-26 19:23:55 +01:00
Tamo
8f5d9f501a update the discussion link 2024-03-26 19:18:32 +01:00
Tamo
8127c9a115 handle the case of a queue of zero elements 2024-03-26 19:04:39 +01:00
Tamo
e7704f1fc1 add a test to ensure we effectively returns a retry-after when the search queue is full 2024-03-26 18:08:59 +01:00
Clément Renault
34262c7a0d Add analytics for the negative operator 2024-03-26 18:01:27 +01:00
Tamo
e2a1bbae37 simplify and improve the http error 2024-03-26 17:53:37 +01:00
Tamo
e433fd53e6 rename the method to get a permit and use it in all search requests 2024-03-26 17:28:03 +01:00
Tamo
3f23fbb46d create the experimental CLI argument 2024-03-26 16:43:40 +01:00
Tamo
c41e1274dc push and test the search queue datastructure 2024-03-26 15:56:43 +01:00
Louis Dureuil
9a95ed619d Add tests 2024-03-26 10:36:56 +01:00
Louis Dureuil
f82d056072 Hide secrets in settings and task queue 2024-03-26 10:36:24 +01:00
meili-bors[bot]
5ea017b922 Merge #4530 
4530: fix: set the histogram bucket boundaries to follow the otel spec r=curquiza a=rohankmr414

# Pull Request

## What does this PR do?
- Fixes the http request duration histogram bucket boundaries to follow the opentelemetry spec, currently the bucket boundaries are too granular and only track latencies below 1s.

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Rohan Kumar <rohankmr414@gmail.com>
 2024-03-25 12:23:31 +00:00
Louis Dureuil
dfa5e41ea6 Check validity of the URL setting 2024-03-25 11:23:16 +01:00
Louis Dureuil
f649f58013 embed no longer async 2024-03-25 11:23:03 +01:00
Rohan Kumar
13a84ae557 fix: set the histogram bucket boundaries to follow the otel spec 2024-03-25 11:20:30 +05:30
Rohan Kumar
5833070358 feat: add status code label to prometheus http request counter 2024-03-25 10:49:40 +05:30
Tamo
d8fe4fe49d return the order in the score details 2024-03-19 15:45:04 +01:00
Tamo
0ae39644f7 fix the facet search 2024-03-19 15:07:06 +01:00
Tamo
4369e9e97c add an error code test on the setting 2024-03-19 11:14:28 +01:00
Tamo
7bd881b9bc adds the degraded searches to the prometheus dashboard 2024-03-19 10:35:47 +01:00
Tamo
6a0c399c2f rename the search_cutoff parameter to search_cutoff_ms 2024-03-19 10:35:47 +01:00
Tamo
038c26c118 stop returning the degraded boolean when a search was cutoff 2024-03-19 10:35:47 +01:00
Tamo
ad9192fbbf reduce the size of an integration test 2024-03-19 10:35:47 +01:00