Commit Graph

12801 Commits

Author SHA1 Message Date
ManyTheFish
ce87d5a89e Update Charabia v0.9.8 2025-11-10 09:33:31 +01:00
Louis Dureuil
a9d6e86077 Merge pull request #5775 from meilisearch/experimental-search-personnalization
Experimental search personalization
2025-11-06 18:45:18 +00:00
Clément Renault
346f9efe3a Merge pull request #5977 from meilisearch/fix-rust-analyzer-false-positive
Fix error that rust-analyzer reports because it is compiling all code with the `test` cfg
2025-11-06 18:42:43 +00:00
Louis Dureuil
a987d698c1 Fix error that rust-analyzer reports because it is compiling all code with the test cfg 2025-11-06 18:10:59 +01:00
ManyTheFish
dbb45dec1a ignore flaky test 2025-11-06 17:55:27 +01:00
ManyTheFish
5f69a43846 Early return if time budget is already exceeded 2025-11-06 17:55:27 +01:00
ManyTheFish
fe1e4814fa Return an error when personalize is used in federated queries 2025-11-06 17:55:27 +01:00
ManyTheFish
c29749741b Use the time budget instead of defining a deadline outside the scope 2025-11-06 17:55:27 +01:00
ManyTheFish
3e47201365 Fix too many argument clippy warning 2025-11-06 17:55:27 +01:00
ManyTheFish
ec9719f3b1 Fix simple PR comments 2025-11-06 17:55:27 +01:00
ManyTheFish
b2cc9e4db8 remove useless clones 2025-11-06 17:55:27 +01:00
ManyTheFish
56198bae48 Initialize personalization API once 2025-11-06 17:55:27 +01:00
ManyTheFish
888059b2d0 Fix PR comments 2025-11-06 17:55:27 +01:00
ManyTheFish
410f2fc8c3 add some failure tests for personalization 2025-11-06 17:55:01 +01:00
ManyTheFish
54e244d2f3 Return an error when the feature is disabled 2025-11-06 17:55:01 +01:00
ManyTheFish
e0c36972fb remove deadcode 2025-11-06 17:55:01 +01:00
ManyTheFish
daadcddb5e Reduce personalization footprint on the codebase 2025-11-06 17:55:01 +01:00
ManyTheFish
7f92dafa02 User context is no more optional 2025-11-06 17:55:01 +01:00
ManyTheFish
cc5d12a368 FIx padding 2025-11-06 17:55:01 +01:00
ManyTheFish
0f98b996b5 Fix PR comments 2025-11-06 17:55:01 +01:00
ManyTheFish
d005ca5bf7 remove irrelevant tests 2025-11-06 17:55:01 +01:00
ManyTheFish
7e65fb1d3e feat(metrics): add personalization count to metrics endpoint
- Add MEILISEARCH_PERSONALIZED_SEARCH_REQUESTS metric to track personalized searches
- Increment metric directly in search analytics when personalization is used
- Metric automatically exposed in /metrics endpoint for monitoring
2025-11-06 17:55:01 +01:00
ManyTheFish
cdefb3f665 feat(analytics): add personalization tracking to segment analytics
- Add total_personalized field to SearchAggregator to track personalization usage
- Track when search requests include personalization parameters
- Include personalization data in analytics JSON output
- Maintain clean personalization service interface
2025-11-06 17:55:01 +01:00
ManyTheFish
a91887221a refactor(personalization): improve Cohere reranking logic and error handling
- Replace and_then() with early return for missing personalization
- Simplify reranking by building new hits vector instead of swapping
- Add debug logging for reranked indices
- Fix potential index out-of-bounds issues in reranking
2025-11-06 17:55:01 +01:00
ManyTheFish
9c66b20a97 refactor: split PersonalizationService into enum with CohereService
- Refactor PersonalizationService as enum with Cohere and Uninitialized variants
- Create dedicated CohereService struct with rerank_search_results method
- Split constructor into cohere() and uninitialized() methods
- Move all Cohere logic into CohereService for better separation of concerns
- Update tests and lib.rs to use new API
- Improve code organization and maintainability
2025-11-06 17:55:01 +01:00
ManyTheFish
a48283527e feat: refine personalization query by merging with user context
- Merge initial query with user context to create a comprehensive prompt
- Only skip reranking if both query and user_context are None
- Support reranking with query-only, user_context-only, or both
- Use 'let else' pattern for cleaner error handling
- Add comprehensive tests for different parameter combinations
- Improve prompt format for better reranking effectiveness
2025-11-06 17:55:01 +01:00
ManyTheFish
73f78c19b0 refactor: rename personalization API fields and move checks inside service
- Rename 'personalization' field to 'personalize' in API
- Rename 'userProfile' to 'userContext' in personalization object
- Remove 'personalized' boolean field (activation now based on non-null 'personalize')
- Move personalization checks inside rerank_search_results function
- Use 'let else' pattern for better error handling
- Update error types and messages to reflect new field names
- Update all search routes and analytics to use new field names
2025-11-06 17:55:01 +01:00
ManyTheFish
34639e346e feat: add personalization service with EnglishV3-only reranking
- Add new personalization module with Cohere integration
- Implement rerank_search_results method using EnglishV3 model
- Remove fallback logic to EnglishV2 for simplified behavior
- Add comprehensive error handling and logging
- Include unit tests for service behavior
- Update search route to support personalization feature
2025-11-06 17:55:01 +01:00
ManyTheFish
7af2a254d6 feat: add personalization parameters to /search route
- Add Personalization struct with personalized boolean and user_profile string
- Add personalizationPersonalized and personalizationUserProfile query parameters to SearchQueryGet
- Follow same pattern as hybrid parameters (hybridEmbedder, hybridSemanticRatio)
- Add validation: personalizationUserProfile requires personalizationPersonalized
- Add error codes for personalization parameters
- Update analytics and facet search to handle new personalization field
- Remove serde dependencies from Personalization struct, use Deserr only
2025-11-06 17:55:01 +01:00
ManyTheFish
0f9d262a1c feat: add experimental_personalization_api_key feature to RoFeatures
- Add MEILI_EXPERIMENTAL_PERSONALIZATION_API_KEY environment variable
- Add experimental_personalization_api_key field to Opt struct with CLI and env support
- Add experimental_personalization_api_key field to InstanceTogglableFeatures
- Store personalization API key in FeatureData for access through IndexScheduler
- Add experimental_personalization_api_key() method to IndexScheduler
- Update analytics destructuring to include new field
- Maintain RoFeatures Copy trait while properly handling Option<String>
2025-11-06 17:55:01 +01:00
Clément Renault
34765b556b Merge pull request #5948 from meilisearch/new-s3-snapshots
Upload snapshot tarballs to S3
2025-11-06 15:10:37 +00:00
Kerollmops
dfb4860578 Make some parameters experimental and link the dedicated issue prototype-v1.24.0.s3-snapshots-5 2025-11-06 12:43:12 +01:00
Kerollmops
ce62713f02 Always create the update_files directory 2025-11-06 12:43:12 +01:00
Kerollmops
8b5d04d60f Move the code to append a file to the tarball in a dedicated function 2025-11-06 12:07:08 +01:00
Kerollmops
1b74709b91 Extract more logic into dedicated functions 2025-11-06 12:00:25 +01:00
Kerollmops
a5c0a282c5 Better document safety problems and unwraps 2025-11-06 11:26:45 +01:00
Kerollmops
4fc048ff20 Better handle errors when trying to clone the handles 2025-11-06 11:15:34 +01:00
Kerollmops
375b5600cd Move the tarball streaming into a dedicated function 2025-11-06 11:11:26 +01:00
Kerollmops
32b997d817 Extracts the streaming closure into an async function 2025-11-06 11:04:16 +01:00
Kerollmops
ff3090e3cc Remove the slash-path dependency 2025-11-06 10:53:07 +01:00
Kerollmops
6c6645f945 Better seeking-to-the-start explanation 2025-11-06 10:45:47 +01:00
Clément Renault
af6473d999 Improve the comments
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-11-06 10:43:14 +01:00
Kerollmops
11851f9701 Fix warnings on Windows 2025-11-05 16:40:33 +01:00
Kerollmops
cc4654eabd Remove useless clippy flag 2025-11-05 16:38:05 +01:00
Kerollmops
0bb91f4a77 Immediately panic when using S3 snapshot options on Windows 2025-11-05 15:16:43 +01:00
Kerollmops
f9d57f54df Keep the smae naming behavior as before 2025-11-05 15:16:43 +01:00
Kerollmops
3ef1afc0f1 Introduce some backoff retries 2025-11-05 15:16:43 +01:00
Kerollmops
dbb5abebb6 Support cancelation 2025-11-05 15:16:42 +01:00
Kerollmops
700f33bd39 Clean up the code 2025-11-05 15:16:42 +01:00
Kerollmops
d01bbbccde Support clean CLI options 2025-11-05 15:16:42 +01:00