Commit Graph

2185 Commits

Author SHA1 Message Date
ManyTheFish
b4cf88859f remove irrelevant tests 2025-11-19 17:27:00 +01:00
ManyTheFish
79f87a622e feat(metrics): add personalization count to metrics endpoint
- Add MEILISEARCH_PERSONALIZED_SEARCH_REQUESTS metric to track personalized searches
- Increment metric directly in search analytics when personalization is used
- Metric automatically exposed in /metrics endpoint for monitoring
2025-11-19 17:27:00 +01:00
ManyTheFish
9856a449ed feat(analytics): add personalization tracking to segment analytics
- Add total_personalized field to SearchAggregator to track personalization usage
- Track when search requests include personalization parameters
- Include personalization data in analytics JSON output
- Maintain clean personalization service interface
2025-11-19 17:26:59 +01:00
ManyTheFish
0549d30a93 refactor(personalization): improve Cohere reranking logic and error handling
- Replace and_then() with early return for missing personalization
- Simplify reranking by building new hits vector instead of swapping
- Add debug logging for reranked indices
- Fix potential index out-of-bounds issues in reranking
2025-11-19 17:26:59 +01:00
ManyTheFish
e1a7b0ee3a refactor: split PersonalizationService into enum with CohereService
- Refactor PersonalizationService as enum with Cohere and Uninitialized variants
- Create dedicated CohereService struct with rerank_search_results method
- Split constructor into cohere() and uninitialized() methods
- Move all Cohere logic into CohereService for better separation of concerns
- Update tests and lib.rs to use new API
- Improve code organization and maintainability
2025-11-19 17:26:59 +01:00
ManyTheFish
1376ba5abe feat: refine personalization query by merging with user context
- Merge initial query with user context to create a comprehensive prompt
- Only skip reranking if both query and user_context are None
- Support reranking with query-only, user_context-only, or both
- Use 'let else' pattern for cleaner error handling
- Add comprehensive tests for different parameter combinations
- Improve prompt format for better reranking effectiveness
2025-11-19 17:26:59 +01:00
ManyTheFish
e89436f126 refactor: rename personalization API fields and move checks inside service
- Rename 'personalization' field to 'personalize' in API
- Rename 'userProfile' to 'userContext' in personalization object
- Remove 'personalized' boolean field (activation now based on non-null 'personalize')
- Move personalization checks inside rerank_search_results function
- Use 'let else' pattern for better error handling
- Update error types and messages to reflect new field names
- Update all search routes and analytics to use new field names
2025-11-19 17:26:59 +01:00
ManyTheFish
af8c08afba feat: add personalization service with EnglishV3-only reranking
- Add new personalization module with Cohere integration
- Implement rerank_search_results method using EnglishV3 model
- Remove fallback logic to EnglishV2 for simplified behavior
- Add comprehensive error handling and logging
- Include unit tests for service behavior
- Update search route to support personalization feature
2025-11-19 17:26:59 +01:00
ManyTheFish
af0944d52d feat: add personalization parameters to /search route
- Add Personalization struct with personalized boolean and user_profile string
- Add personalizationPersonalized and personalizationUserProfile query parameters to SearchQueryGet
- Follow same pattern as hybrid parameters (hybridEmbedder, hybridSemanticRatio)
- Add validation: personalizationUserProfile requires personalizationPersonalized
- Add error codes for personalization parameters
- Update analytics and facet search to handle new personalization field
- Remove serde dependencies from Personalization struct, use Deserr only
2025-11-19 17:26:59 +01:00
ManyTheFish
81cadba03d feat: add experimental_personalization_api_key feature to RoFeatures
- Add MEILI_EXPERIMENTAL_PERSONALIZATION_API_KEY environment variable
- Add experimental_personalization_api_key field to Opt struct with CLI and env support
- Add experimental_personalization_api_key field to InstanceTogglableFeatures
- Store personalization API key in FeatureData for access through IndexScheduler
- Add experimental_personalization_api_key() method to IndexScheduler
- Update analytics destructuring to include new field
- Maintain RoFeatures Copy trait while properly handling Option<String>
2025-11-19 17:26:59 +01:00
Louis Dureuil
1dc125aa56 Fix error that rust-analyzer reports because it is compiling all code with the test cfg 2025-11-19 17:26:59 +01:00
Kerollmops
af48592b2f Make some parameters experimental and link the dedicated issue 2025-11-19 17:26:59 +01:00
Kerollmops
361f9ff12b Always create the update_files directory 2025-11-19 17:26:59 +01:00
Kerollmops
9767bc4a3a Move the code to append a file to the tarball in a dedicated function 2025-11-19 17:26:59 +01:00
Kerollmops
ecb7ca79d1 Extract more logic into dedicated functions 2025-11-19 17:26:59 +01:00
Kerollmops
d92ee598d9 Better document safety problems and unwraps 2025-11-19 17:26:59 +01:00
Kerollmops
e57298d87a Better handle errors when trying to clone the handles 2025-11-19 17:26:58 +01:00
Kerollmops
626eb22f6d Move the tarball streaming into a dedicated function 2025-11-19 17:26:58 +01:00
Kerollmops
3722156aab Extracts the streaming closure into an async function 2025-11-19 17:26:58 +01:00
Kerollmops
5a956e470d Remove the slash-path dependency 2025-11-19 17:26:58 +01:00
Kerollmops
ec2e2577ec Better seeking-to-the-start explanation 2025-11-19 17:26:58 +01:00
Clément Renault
f3489200e1 Improve the comments
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-11-19 17:26:58 +01:00
Kerollmops
11c45966e3 Fix warnings on Windows 2025-11-19 17:26:58 +01:00
Kerollmops
474c3a7d42 Remove useless clippy flag 2025-11-19 17:26:58 +01:00
Kerollmops
4a15141a22 Immediately panic when using S3 snapshot options on Windows 2025-11-19 17:26:58 +01:00
Kerollmops
fb9cc727ee Keep the smae naming behavior as before 2025-11-19 17:26:58 +01:00
Kerollmops
1f68ac194b Introduce some backoff retries 2025-11-19 17:26:58 +01:00
Kerollmops
fb6f87c9fb Support cancelation 2025-11-19 17:26:58 +01:00
Kerollmops
f83de18ff3 Clean up the code 2025-11-19 17:26:58 +01:00
Kerollmops
3d52a2914a Support clean CLI options 2025-11-19 17:26:58 +01:00
Kerollmops
65f3be192e Remove unused imports/code on Windows 2025-11-19 17:26:58 +01:00
Kerollmops
3e8ee1dfab Make clippy happy 2025-11-19 17:26:58 +01:00
Kerollmops
701743655b Make the compression level configurable 2025-11-19 17:26:57 +01:00
Kerollmops
71d049a8d2 Disable compression entirely to avoid being CPU bound 2025-11-19 17:26:57 +01:00
Clément Renault
4eac7f2b48 Improve the way we create the snapshot path 2025-11-19 17:26:57 +01:00
Clément Renault
fc9811c9e8 Remove useless dependencies 2025-11-19 17:26:57 +01:00
Kerollmops
48b29a1da9 Make it finaly work but without async on the write side 2025-11-19 17:26:57 +01:00
Kerollmops
1fb821bab9 Seeking the tasks/data.mdb file to the begining made the trick 2025-11-19 17:26:57 +01:00
Kerollmops
2a1652ce02 Improve understanding of S3-related errors 2025-11-19 17:26:57 +01:00
Kerollmops
5415051af1 Retrieve the bytesMut only when released 2025-11-19 17:26:57 +01:00
Kerollmops
f66e2a3a3c Fix minimum part size 2025-11-19 17:26:57 +01:00
Kerollmops
e8193c0cb4 Improve error messaging when missing env var 2025-11-19 17:26:57 +01:00
Clément Renault
a9657e356b WIP 2025-11-19 17:26:57 +01:00
Kerollmops
1b69d270b8 WIP Do more tests 2025-11-19 17:26:57 +01:00
Clément Renault
5a254452fc WIP sending multiparts of 250MiB 2025-11-19 17:26:57 +01:00
Clément Renault
4d554e2811 Rename the update_path function 2025-11-19 17:26:57 +01:00
Clément Renault
4fe5730fb9 Geenrate an async tarball 2025-11-19 17:26:56 +01:00
Kerollmops
425e6a73cb WIP 2025-11-19 17:26:56 +01:00
Kerollmops
9b9e60aa5d Make max in flights parts fro upload configurable 2025-11-19 17:26:56 +01:00
Kerollmops
81bdf6f217 Use a good mem advice for uploads 2025-11-19 17:26:56 +01:00