Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						18a2c13e4e
					 | 
					
						
						
							
							add analytics
						
						
						
						
						
						
					 | 
					
						2024-09-03 12:07:59 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						ed19b7c3c3
					 | 
					
						
						
							
							Only reindex if the size increased
						
						
						
						
						
						
					 | 
					
						2024-09-03 12:07:59 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						66bda2ce8a
					 | 
					
						
						
							
							fix tests
						
						
						
						
						
						
					 | 
					
						2024-09-03 12:07:58 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						1ac008926b
					 | 
					
						
						
							
							Add maxBytes parameter
						
						
						
						
						
						
					 | 
					
						2024-09-03 12:07:15 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						c49d892c82
					 | 
					
						
						
							
							Changes to prompt
						
						
						
						
						
						
					 | 
					
						2024-09-03 12:07:10 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						de962a26f3
					 | 
					
						
						
							
							New error type when maxBytes is null
						
						
						
						
						
						
					 | 
					
						2024-09-03 12:01:04 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						c1557734dc
					 | 
					
						
						
							
							Use the GlobalFieldsIdsMap everywhere and write it to disk
						
						
						
						
						
						
						
						Co-authored-by: Dureuill <louis@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com> 
						
						
					 | 
					
						2024-09-03 12:01:01 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						005204e9e5
					 | 
					
						
						
							
							make the code of init_web_app in common between most tests
						
						
						
						
						
						
					 | 
					
						2024-09-03 11:40:05 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						1040e5e2b4
					 | 
					
						
						
							
							spawn on search queue per test
						
						
						
						
						
						
					 | 
					
						2024-09-03 11:20:25 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								ManyTheFish
							
						 
					 | 
					
						
						
							
						
						c50d3edc4a
					 | 
					
						
						
							
							Integrate first searchable exctrator
						
						
						
						
						
						
					 | 
					
						2024-09-03 11:02:39 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								meili-bors[bot]
							
						 
					 | 
					
						
						
							
						
						80408c92dc
					 | 
					
						
						
							
							Merge #4906
						
						
						
						
						
						
						
						4906: Add searchable fields to template r=dureuill a=dureuill
# Pull Request
## Related issue
Fixes #4886 
See [public usage](https://meilisearch.notion.site/v1-11-AI-search-changes-0e37727193884a70999f254fa953ce6e#1dd6f0eee5a1422888e1c5d48e107cd1)
## What does this PR do?
- `Prompt::render` now requires and uses metadata to indicate if the fields are searchable or not
- Changes default template
- Updated tests
- Correctly reindex vectors when the list of searchable fields changes in a settings update.
Co-authored-by: Louis Dureuil <louis@meilisearch.com> 
						
						
					 | 
					
						2024-09-03 07:14:58 +00:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						5369bf4a62
					 | 
					
						
						
							
							Change some lifetimes
						
						
						
						
						
						
					 | 
					
						2024-09-02 19:51:22 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						bcb1aa3d22
					 | 
					
						
						
							
							Find a temporary solution to par into iter on an HashMap
						
						
						
						
						
						
						
						Spoiler: Do not use an HashMap but drain it into a Vec 
						
						
					 | 
					
						2024-09-02 19:39:48 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						fa1a0beb0c
					 | 
					
						
						
							
							fix conflicts after rebase
						
						
						
						
						
						
					 | 
					
						2024-09-02 18:15:42 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						5aefe7cd17
					 | 
					
						
						
							
							add the snapshots
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:51 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						e6dd66e4a0
					 | 
					
						
						
							
							Do not fail the whole batch when a single document deletion by filter fails
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:51 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						6e3839d8b6
					 | 
					
						
						
							
							autobatch document deletion by filter
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:51 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						cd271b8762
					 | 
					
						
						
							
							stop trying to process searches after one minute
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:51 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						3ce8500d4c
					 | 
					
						
						
							
							ensure we never early exit when we have a permit and remove the warning when we implicitely drop a permit
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:51 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						588000d398
					 | 
					
						
						
							
							add a warning to help us find when we forget to drop explicitely drop a permit
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:51 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						92b151607c
					 | 
					
						
						
							
							explicitely drop the search permit
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:51 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								irevoire
							
						 
					 | 
					
						
						
							
						
						42e7499260
					 | 
					
						
						
							
							Update version for the next release (v1.10.1) in Cargo.toml
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:51 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Tamo
							
						 
					 | 
					
						
						
							
						
						41aa1e1424
					 | 
					
						
						
							
							Only spawn one search queue in actix-web
						
						
						
						
						
						
					 | 
					
						2024-09-02 16:27:50 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						9b7858fb90
					 | 
					
						
						
							
							Expose the new indexer
						
						
						
						
						
						
					 | 
					
						2024-09-02 15:21:59 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						ab01679a8f
					 | 
					
						
						
							
							Remove the useless option from the document changes
						
						
						
						
						
						
					 | 
					
						2024-09-02 15:21:00 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						521775f788
					 | 
					
						
						
							
							I push for Many
						
						
						
						
						
						
					 | 
					
						2024-09-02 15:10:21 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						72e7b7846e
					 | 
					
						
						
							
							Renaming the indexers
						
						
						
						
						
						
					 | 
					
						2024-09-02 14:42:27 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						6526ce1208
					 | 
					
						
						
							
							Fix the merging of documents
						
						
						
						
						
						
					 | 
					
						2024-09-02 14:41:20 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						24ace5c381
					 | 
					
						
						
							
							Add reindexing test
						
						
						
						
						
						
					 | 
					
						2024-09-02 13:37:01 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						21296190a3
					 | 
					
						
						
							
							Reindex embedders
						
						
						
						
						
						
					 | 
					
						2024-09-02 13:00:53 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						03fda78901
					 | 
					
						
						
							
							update other tests
						
						
						
						
						
						
					 | 
					
						2024-09-02 11:31:31 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						30a143f149
					 | 
					
						
						
							
							Test new facilities
						
						
						
						
						
						
					 | 
					
						2024-09-02 11:31:23 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						4464d319af
					 | 
					
						
						
							
							Change default template to use the new facility
						
						
						
						
						
						
					 | 
					
						2024-09-02 11:30:59 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						580ea2f450
					 | 
					
						
						
							
							Pass the fields <-> ids map with metadata to render
						
						
						
						
						
						
					 | 
					
						2024-09-02 11:30:10 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						915cf4bae5
					 | 
					
						
						
							
							Add field.is_searchable property to fields
						
						
						
						
						
						
					 | 
					
						2024-09-02 11:28:53 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						e639ec79d1
					 | 
					
						
						
							
							Move the indexers into their own modules
						
						
						
						
						
						
					 | 
					
						2024-09-02 10:42:19 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						bb885a5810
					 | 
					
						
						
							
							Fix the merge for roaring bitmap
						
						
						
						
						
						
					 | 
					
						2024-09-01 23:20:19 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						b625d31c7d
					 | 
					
						
						
							
							Introduce the PartialDumpIndexer indexer that generates document ids in parallel
						
						
						
						
						
						
					 | 
					
						2024-08-30 15:07:21 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						6487a67f2b
					 | 
					
						
						
							
							Introduce the ConcurrentAvailableIds struct and rename the other to AvailableIds
						
						
						
						
						
						
					 | 
					
						2024-08-30 15:06:50 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						271ce91b3b
					 | 
					
						
						
							
							Add the rayon Threadpool to the index function parameter
						
						
						
						
						
						
					 | 
					
						2024-08-30 14:34:24 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						54f2eb4507
					 | 
					
						
						
							
							Remove duplication of grenad merger
						
						
						
						
						
						
					 | 
					
						2024-08-30 14:34:05 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						794ebcd582
					 | 
					
						
						
							
							Replace grenad with the new grenad various-improvement branch
						
						
						
						
						
						
					 | 
					
						2024-08-30 11:53:59 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						b7c77c7a39
					 | 
					
						
						
							
							Use the latest version of the obkv crate
						
						
						
						
						
						
					 | 
					
						2024-08-30 11:53:59 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						0c57cf7565
					 | 
					
						
						
							
							Replace obkv with the temporary new version of it
						
						
						
						
						
						
					 | 
					
						2024-08-30 11:53:58 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						27df9e6c73
					 | 
					
						
						
							
							Introduce the indexer::index function that runs the indexation
						
						
						
						
						
						
					 | 
					
						2024-08-30 11:53:58 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						45c060831e
					 | 
					
						
						
							
							Introduce typed channels and the merger loop
						
						
						
						
						
						
					 | 
					
						2024-08-30 11:53:58 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						874c1ac538
					 | 
					
						
						
							
							First channels types
						
						
						
						
						
						
					 | 
					
						2024-08-30 11:53:58 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						e6ffa4d454
					 | 
					
						
						
							
							Implement the document merge function for the replace method
						
						
						
						
						
						
					 | 
					
						2024-08-30 11:53:58 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Clément Renault
							
						 
					 | 
					
						
						
							
						
						637a9c8bdd
					 | 
					
						
						
							
							Implement the document merge function for the update method
						
						
						
						
						
						
					 | 
					
						2024-08-30 11:53:58 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 | 
				
			
				
					
						
							
							
								 
								Louis Dureuil
							
						 
					 | 
					
						
						
							
						
						c683fa98e6
					 | 
					
						
						
							
							WIP
						
						
						
						
						
						
						
						Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com> 
						
						
					 | 
					
						2024-08-30 11:53:57 +02:00 | 
					
					
						
						
						
							
							
							
							
							
							
						
					 |