74c180267e
pass dimensions only when defined
2024-02-07 11:52:08 +01:00
517f5332d6
Allow actually passing dimensions for OpenAI source
...
-> make sure the settings change is rejected or the settings task fails when the specified model doesn't support
overriding `dimensions` and the passed `dimensions` differs from the model's default dimensions.
2024-02-07 11:51:44 +01:00
9ac5750096
Retrieve the overriden dimensions from the configuration when fetching settings
2024-02-07 11:51:44 +01:00
7ae4013478
Make sure the overriden dimensions are always used when embedding
2024-02-07 11:51:44 +01:00
fb705116a6
feat: add new models and ability to override dimensions
2024-02-07 11:51:42 +01:00
fbf5f2a392
Don't use a runtime in extract_embedder, use it only for OpenAI
2024-02-01 10:33:27 +01:00
1555870088
Truncate HuggingFace vectors that are too long
2024-02-01 10:33:27 +01:00
f692021bfc
Implement PR comments
2024-01-22 10:25:56 +01:00
84f49d76cd
Add cuda feature
2024-01-22 10:25:16 +01:00
0bf879fb88
Fix warning on rust stable
2023-12-20 17:48:09 +01:00
393216bf30
Flatten embedders settings
2023-12-20 17:16:43 +01:00
333ce12eb2
Fixed issue where the default revision is always the one we picked for the default model
2023-12-20 10:17:49 +01:00
eb5cb91da2
Switch default from hf to openai
2023-12-14 16:19:46 +01:00
87bba98bd8
Various changes
...
- fixed seed for arroy
- check vector dimensions as soon as it is provided to search
- don't embed whitespace
2023-12-14 16:08:42 +01:00
a4536b1381
Small adjustments to respect the spec
2023-12-14 16:08:42 +01:00
5b51cb04af
Remove some settings
2023-12-14 16:08:42 +01:00
b8e4709dfa
Remove prompt strategy and fallback
2023-12-14 16:08:41 +01:00
e0cc775dc4
Various changes
...
- DistributionShift in Search object (to be set from model in embed?)
- Fix issue where embedder index wasn't computed at search time
- Accept as default embedder either the "default" one, or the only embedder when there is only one
2023-12-14 16:08:41 +01:00
12940d79a9
WIP
...
- manual embedder
- multi embedders OK
- clippy + tests OK
2023-12-14 16:08:41 +01:00
922a640188
WIP multi embedders
...
fixed template bugs
2023-12-14 16:08:41 +01:00
65e49b7092
Remove stuff, add distribution shift (WIP)
2023-12-14 16:08:38 +01:00
13c2c6c16b
Small commit to add hybrid search and autoembedding
2023-12-14 16:07:48 +01:00
