ManyTheFish
34639e346e
feat: add personalization service with EnglishV3-only reranking
...
- Add new personalization module with Cohere integration
- Implement rerank_search_results method using EnglishV3 model
- Remove fallback logic to EnglishV2 for simplified behavior
- Add comprehensive error handling and logging
- Include unit tests for service behavior
- Update search route to support personalization feature
2025-11-06 17:55:01 +01:00
ManyTheFish
7af2a254d6
feat: add personalization parameters to /search route
...
- Add Personalization struct with personalized boolean and user_profile string
- Add personalizationPersonalized and personalizationUserProfile query parameters to SearchQueryGet
- Follow same pattern as hybrid parameters (hybridEmbedder, hybridSemanticRatio)
- Add validation: personalizationUserProfile requires personalizationPersonalized
- Add error codes for personalization parameters
- Update analytics and facet search to handle new personalization field
- Remove serde dependencies from Personalization struct, use Deserr only
2025-11-06 17:55:01 +01:00
ManyTheFish
0f9d262a1c
feat: add experimental_personalization_api_key feature to RoFeatures
...
- Add MEILI_EXPERIMENTAL_PERSONALIZATION_API_KEY environment variable
- Add experimental_personalization_api_key field to Opt struct with CLI and env support
- Add experimental_personalization_api_key field to InstanceTogglableFeatures
- Store personalization API key in FeatureData for access through IndexScheduler
- Add experimental_personalization_api_key() method to IndexScheduler
- Update analytics destructuring to include new field
- Maintain RoFeatures Copy trait while properly handling Option<String>
2025-11-06 17:55:01 +01:00
Louis Dureuil
747476a225
Add customMetadata to all documents routes
2025-11-06 17:07:18 +01:00
Clément Renault
34765b556b
Merge pull request #5948 from meilisearch/new-s3-snapshots
...
Upload snapshot tarballs to S3
2025-11-06 15:10:37 +00:00
Kerollmops
dfb4860578
Make some parameters experimental and link the dedicated issue
2025-11-06 12:43:12 +01:00
Kerollmops
ce62713f02
Always create the update_files directory
2025-11-06 12:43:12 +01:00
Kerollmops
8b5d04d60f
Move the code to append a file to the tarball in a dedicated function
2025-11-06 12:07:08 +01:00
Kerollmops
1b74709b91
Extract more logic into dedicated functions
2025-11-06 12:00:25 +01:00
Kerollmops
a5c0a282c5
Better document safety problems and unwraps
2025-11-06 11:26:45 +01:00
Kerollmops
4fc048ff20
Better handle errors when trying to clone the handles
2025-11-06 11:15:34 +01:00
Kerollmops
375b5600cd
Move the tarball streaming into a dedicated function
2025-11-06 11:11:26 +01:00
Kerollmops
32b997d817
Extracts the streaming closure into an async function
2025-11-06 11:04:16 +01:00
Kerollmops
ff3090e3cc
Remove the slash-path dependency
2025-11-06 10:53:07 +01:00
Kerollmops
6c6645f945
Better seeking-to-the-start explanation
2025-11-06 10:45:47 +01:00
Clément Renault
af6473d999
Improve the comments
...
Co-authored-by: Louis Dureuil <louis@meilisearch.com >
2025-11-06 10:43:14 +01:00
Kerollmops
11851f9701
Fix warnings on Windows
2025-11-05 16:40:33 +01:00
Kerollmops
cc4654eabd
Remove useless clippy flag
2025-11-05 16:38:05 +01:00
Kerollmops
0bb91f4a77
Immediately panic when using S3 snapshot options on Windows
2025-11-05 15:16:43 +01:00
Kerollmops
f9d57f54df
Keep the smae naming behavior as before
2025-11-05 15:16:43 +01:00
Kerollmops
3ef1afc0f1
Introduce some backoff retries
2025-11-05 15:16:43 +01:00
Kerollmops
dbb5abebb6
Support cancelation
2025-11-05 15:16:42 +01:00
Kerollmops
700f33bd39
Clean up the code
2025-11-05 15:16:42 +01:00
Kerollmops
d01bbbccde
Support clean CLI options
2025-11-05 15:16:42 +01:00
Kerollmops
4fc506f267
Remove unused imports/code on Windows
2025-11-05 15:16:42 +01:00
Kerollmops
dc456276e5
Make clippy happy
2025-11-05 15:16:42 +01:00
Kerollmops
b2ea50cb10
Make the compression level configurable
2025-11-05 15:16:42 +01:00
Kerollmops
5074cf92ab
Disable compression entirely to avoid being CPU bound
2025-11-05 15:16:42 +01:00
Clément Renault
a92bc8d192
Improve the way we create the snapshot path
2025-11-05 15:16:42 +01:00
Clément Renault
ee538cf045
Remove useless dependencies
2025-11-05 15:16:42 +01:00
Kerollmops
2b05d63a0f
Make it finaly work but without async on the write side
2025-11-05 15:16:42 +01:00
Kerollmops
104e8918ce
Seeking the tasks/data.mdb file to the begining made the trick
2025-11-05 15:16:42 +01:00
Kerollmops
d6ec4d4f4a
Improve understanding of S3-related errors
2025-11-05 15:16:42 +01:00
Kerollmops
f0e7326b7a
Retrieve the bytesMut only when released
2025-11-05 15:16:42 +01:00
Kerollmops
c8106a0006
Fix minimum part size
2025-11-05 15:16:42 +01:00
Kerollmops
c9ab5bc0b6
Improve error messaging when missing env var
2025-11-05 15:16:42 +01:00
Clément Renault
5e0f15fd43
WIP
2025-11-05 15:16:42 +01:00
Kerollmops
4c30f090c7
WIP Do more tests
2025-11-05 15:16:42 +01:00
Clément Renault
63f247cdda
WIP sending multiparts of 250MiB
2025-11-05 15:16:42 +01:00
Clément Renault
e109fa9529
Rename the update_path function
2025-11-05 15:16:42 +01:00
Clément Renault
76e4ec2168
Geenrate an async tarball
2025-11-05 15:16:42 +01:00
Kerollmops
982babdb74
WIP
2025-11-05 15:16:42 +01:00
Kerollmops
7ae2ae33d9
Make max in flights parts fro upload configurable
2025-11-05 15:16:42 +01:00
Kerollmops
cb0788ae07
Use a good mem advice for uploads
2025-11-05 15:16:42 +01:00
Kerollmops
cb3e5dc234
Move the S3 snapshots to disk into a dedicated method
2025-11-05 15:16:41 +01:00
Clément Renault
59d40a2821
Upload ten parts at a time
2025-11-05 15:16:41 +01:00
Clément Renault
98a678e73d
Use the Bytes crate to send the parts
2025-11-05 15:16:41 +01:00
Clément Renault
70292aae3c
Upload indexes under their uuids
2025-11-05 15:16:41 +01:00
Clément Renault
73521f0069
Initial working S3 uploads to RustFS
2025-11-05 15:16:41 +01:00
Louis Dureuil
4533179604
Pass tokio handle to index-scheduler
2025-11-05 15:16:41 +01:00
