5422: Add more progress levels to measure merging r=Kerollmops a=Kerollmops
I found out that Meilisearch was not correctly reporting the long indexing times in the progress and that a lot of time was spent on extracting words with all documents already extracted. The reason was that there was no step to report merging the cache and sending the entries to write to the writer thread. This PR adds these entries to the progress.
Co-authored-by: Kerollmops <clement@meilisearch.com>
5421: Accept total batch size in human size r=irevoire a=Kerollmops
This PR fixes the new `experimental-limit-batched-tasks-total-size` to accept human-defined sizes in bytes.
Co-authored-by: Kerollmops <clement@meilisearch.com>
5369: exhaustive facet search r=ManyTheFish a=ManyTheFish
Fixes#5403
This PR adds an `exhaustiveFacetCount` field to the `/facet-search` API allowing the end-user to have a better facet count when having a distinct attribute set in the index settings.
# Usage
`POST /index/:index_uid/facet-search`
**Body:**
```json
{
"facetQuery": "blob",
"facetName": "genres",
"q": "",
"exhaustiveFacetCount": true
}
```
# Prototype Docker images
```sh
$ docker pull getmeili/meilisearch:prototype-exhaustive-facet-search-00
```
Co-authored-by: ManyTheFish <many@meilisearch.com>
5254: Granular Filterable attribute settings r=ManyTheFish a=ManyTheFish
# Related
**Issue:** https://github.com/meilisearch/meilisearch/issues/5163
**PRD:** https://meilisearch.notion.site/API-usage-Settings-to-opt-out-indexing-features-filterableAttributes-1764b06b651f80aba8bdf359b2df3ca8
# Summary
Change the `filterableAttributes` settings to let the user choose which facet feature he wants to activate or not.
Deactivating a feature will avoid some database computation in the indexing process and save time and disk size.
# Example
`PATCH /indexes/:index_uid/settings`
```json
{
"filterableAttributes": [
{
"patterns": [
"cattos",
"doggos.age"
],
"features": {
"facetSearch": false,
"filter": {
"equality": true,
"comparison": false
}
}
}
]
}
```
# Impact on the codebase
- Settings API:
- `/settings`
- `/settings/filterable-attributes`
- OpenAPI
- may impact the LocalizedAttributesRules due to the AttributePatterns factorization
- Database:
- Filterable attributes format changed
- Faceted field_ids are no more stored in the database
- FieldIdsMap has no more unexisting fields
- Search:
- Search using filters
- Facet search
- `Attributes` ranking rule
- Distinct attribute
- Facet distribution
- Settings reindexing:
- searchable
- facet
- vector
- geo
- Document indexing:
- searchable
- facet
- vector
- geo
- Dump import
# Note for the reviewers
The changes are huge and have been split in different commits with a dedicated explanation, I suggest reviewing the commit 1by1
Co-authored-by: ManyTheFish <many@meilisearch.com>
