Commit Graph

1983 Commits

Author SHA1 Message Date
ManyTheFish
ad804b60fc Add header-based metadata opt-in for search responses
- Add Meili-Include-Metadata header constant
- Modify perform_search to conditionally include metadata based on header
- Modify perform_federated_search to conditionally include metadata based on header
- Update all search routes to check for header and pass include_metadata parameter
- Forward Meili-Include-Metadata header to remote requests for federated search
- Ensure remote queries include primaryKey metadata when header is present
2025-10-02 11:04:19 +02:00
ManyTheFish
9fddbff916 WIP: Add metadata field with queryUid, indexUid, primaryKey, and remote
- Add SearchMetadata struct with queryUid, indexUid, primaryKey, and remote fields
- Update SearchResult to include metadata field
- Update FederatedSearchResult to include metadata array
- Refactor federated search metadata building to maintain query order
- Support primary key extraction from both local and remote results
- Add remote field to identify remote instance for federated queries
- Ensure metadata array matches query order in federated search

Features:
- queryUid: UUID v7 for each query
- indexUid: Index identifier
- primaryKey: Primary key field name (null if not available)
- remote: Remote instance name (null for local queries)

This provides complete traceability for search operations across local and remote instances.
2025-10-01 17:55:15 +02:00
ManyTheFish
7fe2b0a177 feat: add indexUid to SearchMetadata
- Add indexUid field to SearchMetadata struct
- Update perform_search to include indexUid in metadata
- Update federated search to include indexUid for each query

The metadata field now contains both queryUid and indexUid:
- For /search: single object with queryUid and indexUid
- For /multi-search: each result has metadata with both fields
- For federated search: array of objects, each with queryUid and indexUid
2025-09-30 17:34:29 +02:00
ManyTheFish
060562d156 feat: add metadata field with queryUid to search responses
- Add SearchMetadata struct with queryUid field (UUID v7)
- Add metadata field to SearchResult for /search route
- Add metadata field to FederatedSearchResult for /multi-search route
- Update perform_search to generate queryUid and set metadata
- Update federated search to generate queryUid for each query
- Update multi-search non-federated path to include metadata
- Fix pattern matching in analytics and other code

The metadata field contains:
- For /search: single object with queryUid
- For /multi-search: array of objects, one per query
- For federated search: array of objects, one per query

All queryUid values are generated using Uuid::now_v7() for time-ordered uniqueness.
2025-09-30 15:22:35 +02:00
ManyTheFish
44fdc8b350 Add uuid v7 feature 2025-09-23 11:44:14 +02:00
ManyTheFish
a1a7643554 fix tests 2025-09-23 11:44:14 +02:00
ManyTheFish
1d78637872 Add requestUid field in search response and add debug logs with requestUid 2025-09-23 11:44:14 +02:00
Louis Dureuil
4fe073cc1a Merge pull request #5896 from meilisearch/fix-doc-template
Document template: Correctly render when indexing first item in array
2025-09-22 07:20:38 +00:00
Tamo
d7ad76ea1e improve the prometheus content type we return 2025-09-18 17:04:13 +02:00
Louis Dureuil
e82bb93221 Fix indexing bug 2025-09-18 16:57:20 +02:00
Kerollmops
d7de819d11 Bump Hannoy to fix unreachable documents 2025-09-18 14:26:13 +02:00
Tamo
26d9070aa7 increase rust version from 1.85 to 1.89 2025-09-16 17:21:33 +02:00
Louis Dureuil
06b3ca9eb5 Merge pull request #5890 from meilisearch/upgrade-dumpless-for-v1.21
Update dumpless upgrade for v1.21
2025-09-15 09:47:52 +00:00
Louis Dureuil
7dc1c03a36 Update dumpless upgrade for v1.21 2025-09-15 10:46:40 +02:00
ManyTheFish
0f80249b70 Update Charabia v0.9.7 2025-09-15 09:33:21 +02:00
Louis Dureuil
fd795c513b add documentation warnings 2025-09-10 09:44:41 +02:00
Louis Dureuil
ce136ec0c1 Support missing search_fragments and indexing_fragments 2025-09-10 09:43:39 +02:00
Tamo
4d4f6d2c20 Merge pull request #5767 from meilisearch/arroy-becomes-hannoy
Add index setting to switch from arroy to hannoy
2025-09-09 17:59:45 +00:00
Louis Dureuil
4cc8fb2c5c Add comment about upgrade procedure
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-09-09 17:42:33 +02:00
Louis Dureuil
16461a9145 add unit test 2025-09-09 14:58:14 +02:00
Tamo
17810394b8 fix the quantic progress trace 2025-09-09 11:04:54 +02:00
Louis Dureuil
15690b9e22 Merge branch 'main' into arroy-becomes-hannoy 2025-09-08 17:05:05 +02:00
Louis Dureuil
a8cd81c7f4 get_vector_store returns an option, handles it in Index::settings 2025-09-08 16:53:57 +02:00
Louis Dureuil
6376571df0 Add VectorStoreBackend to the list of components 2025-09-08 16:44:16 +02:00
Louis Dureuil
cfb040e647 remove extraneous space 2025-09-08 16:41:48 +02:00
Louis Dureuil
f54773781a Revert the fake 1.22 in index-scheduler as well 2025-09-08 15:00:02 +02:00
Louis Dureuil
226c102bab Update snapshot and upgrade proc 2025-09-08 10:00:44 +02:00
Louis Dureuil
13df964564 Adopt neutral terminology where arroy/hannoy would be confusing 2025-09-03 16:11:40 +02:00
Louis Dureuil
0faf495173 cargo fmt 2025-09-03 14:49:24 +02:00
Louis Dureuil
c32c74671d Rename HannoyStats to VectorStoreStats
The stats can be provided by any backend
2025-09-03 14:45:31 +02:00
Louis Dureuil
b05bcf2c13 happy clippy 2025-09-03 14:13:08 +02:00
Louis Dureuil
90cc5263f6 Remove MEILI_EMBEDDINGS_CHUNK_SIZE 2025-09-03 13:57:58 +02:00
Louis Dureuil
424d0e277e Merge branch 'main' into arroy-becomes-hannoy-with-sharding 2025-09-03 13:46:35 +02:00
Louis Dureuil
34eba61c0d Add new tests 2025-09-03 13:42:56 +02:00
Louis Dureuil
687260bc13 Change approach to arroy <-> migration after encountering multiple issues 2025-09-02 17:49:22 +02:00
Louis Dureuil
6b6e69b07a rename Arroy to "stable" and Hannoy to "experimental" in setting values 2025-09-02 14:52:43 +02:00
Louis Dureuil
a25111f32e get old backend before it mutates 2025-09-02 14:52:18 +02:00
Tamo
b144d9ab2b fix warnings 2025-09-02 14:31:24 +02:00
Tamo
c3cefbc170 send the version when returning prometheus metrics 2025-09-02 12:40:18 +02:00
Louis Dureuil
c4848e6cc0 Set back snapshot to what it was 2025-09-01 17:56:20 +02:00
Louis Dureuil
454581dbc9 Support progress 2025-09-01 17:48:50 +02:00
Louis Dureuil
bc5100dddd Update snap 2025-09-01 17:01:01 +02:00
Louis Dureuil
118c6da64d Update hannoy to v0.0.5 2025-09-01 16:42:08 +02:00
Louis Dureuil
a989f52657 Fix signature of backend change function 2025-09-01 16:38:39 +02:00
Louis Dureuil
a8cc66899c Derive ToSchema for VectorStoreBackend 2025-09-01 16:38:18 +02:00
Louis Dureuil
c9cc748f42 Mark get_vector_store as public 2025-09-01 16:37:52 +02:00
Louis Dureuil
4ccce18d7b Add settings route 2025-09-01 16:36:24 +02:00
Louis Dureuil
00d1006cd9 add experimental feature 2025-09-01 16:35:48 +02:00
Clémentine
5cad65cca5 Merge pull request #5869 from meilisearch/dependabot/cargo/tracing-subscriber-0.3.20
Bump tracing-subscriber from 0.3.19 to 0.3.20
2025-09-01 14:23:26 +00:00
Tamo
7fe9d07247 Merge pull request #5858 from shreeup/5835DispProgressTrace
Display the progressTrace in real time
2025-09-01 10:21:36 +00:00