|
|
a00f5850ee
|
Add support for placeholder search for empty queries
|
2020-10-06 20:19:50 +02:00 |
|
|
|
ce8e56ee18
|
Rewrite the indexer to use one MTBL by database
This allows us to avoid prefixing keys and appending into LMDB databases
|
2020-10-04 17:04:33 +02:00 |
|
|
|
007e647462
|
Introduce the Mdfs Iterator that explore the proximity graph using a mana DFS
|
2020-10-02 16:46:07 +02:00 |
|
|
|
d0c73564b1
|
Use the CboRoaringBitmapCodec for the word pair proximity docids
|
2020-10-02 16:46:06 +02:00 |
|
|
|
4eda149ffa
|
Rename the BoRoaringBitmap codec
|
2020-10-02 16:46:06 +02:00 |
|
|
|
bc35c9a598
|
Introduce the size_of_database infos subcommand
|
2020-10-02 16:46:05 +02:00 |
|
|
|
d6fa9c0414
|
Index the intra documents word pair proximities
|
2020-09-22 14:04:33 +02:00 |
|
|
|
e34437b2d7
|
Move the proximity function to a module
|
2020-09-22 10:54:59 +02:00 |
|
|
|
5664c37539
|
Introduce an heed codec that reduce the size of small amount of serialized integers
|
2020-09-07 20:06:23 +02:00 |
|
|
|
daa3673c1c
|
Invert the word docid positions key order
|
2020-09-06 10:30:53 +02:00 |
|
|
|
dc88a86259
|
Store the word positions under the documents
|
2020-09-05 18:03:06 +02:00 |
|
|
|
580ed1119a
|
Make the engine to return csv string records as documents and headers
|
2020-08-31 19:02:00 +02:00 |
|
|
|
bad0663138
|
Come back to the old tokenizer
|
2020-08-31 13:34:38 +02:00 |
|
|
|
ad5cafbfed
|
Introduce a database to store docids in groups of four positions
|
2020-08-29 17:42:55 +02:00 |
|
|
|
3db517548d
|
Move the documents back into the LMDB database
|
2020-08-29 15:14:04 +02:00 |
|
|
|
3fe497e129
|
Improve the Mtbl heed codec to only encode MTBL databases
|
2020-08-29 11:20:39 +02:00 |
|
|
|
0a44ff86ab
|
Put the documents MTBL back into LMDB
We makes sure to write the documents into a file before
memory mapping it and putting it into LMDB, this way we avoid
moving it to RAM
|
2020-08-28 15:43:24 +02:00 |
|
|
|
d784d87880
|
Remove the prefix LMDB databases
|
2020-08-28 14:41:43 +02:00 |
|
|
|
7cde312f14
|
Introduce the StrBEU32Codec heed codec
|
2020-08-28 14:16:37 +02:00 |
|
|
|
8806fcd545
|
Introduce a better query and document lexer
|
2020-08-16 14:36:54 +02:00 |
|
|
|
1e358e3ae8
|
Introduce the AstarBagIter that iterates through best paths
|
2020-08-15 16:24:06 +02:00 |
|
|
|
7dc594ba4d
|
Introduce the Search builder struct
|
2020-08-13 14:27:51 +02:00 |
|
|
|
bfb46cbfbe
|
Introduce the Crtierion enum
|
2020-08-12 10:43:02 +02:00 |
|
|
|
6d04a285dc
|
Retrieve and display the distances of the words found
|
2020-08-11 15:18:02 +02:00 |
|
|
|
1bd37d213a
|
Lowercase quoted words
|
2020-08-10 14:49:09 +02:00 |
|
|
|
883a8109c8
|
Show both database and documents database sizes
|
2020-08-10 14:37:18 +02:00 |
|
|
|
394844062f
|
Move the documents MTBL database inside the Index
|
2020-08-10 13:47:19 +02:00 |
|
|
|
91282c8b6a
|
Move the documents into another file
|
2020-08-07 13:11:31 +02:00 |
|
|
|
fae694a102
|
Put the documents into an MTBL database
|
2020-08-07 12:14:40 +02:00 |
|
|
|
d3b1096510
|
Compute the word attribute postings lists on each threads
|
2020-08-06 11:50:27 +02:00 |
|
|
|
9ade00e27b
|
Highlight all the matching words
|
2020-07-14 11:53:21 +02:00 |
|
|
|
3d144e62c4
|
Search for best proximities in multiple attributes
|
2020-07-13 19:06:56 +02:00 |
|
|
|
576dd011a1
|
Compute the candidates but not by attribute
|
2020-07-13 18:16:05 +02:00 |
|
|
|
6b14b20369
|
Introduce a method to retrieve the number of attributes of the documents
|
2020-07-13 17:50:16 +02:00 |
|
|
|
12358476da
|
Use the log crate instead of stderr
|
2020-07-12 10:55:09 +02:00 |
|
|
|
d31da26a51
|
Avoid cloning RoraringBitmaps when unecessary
|
2020-07-11 23:51:32 +02:00 |
|
|
|
b12bfcb03b
|
Reduce the deepness of the word position document ids
This helps reduce the number of allocations.
|
2020-07-07 12:30:05 +02:00 |
|
|
|
7178b6c2c4
|
First basic version using MTBL again
|
2020-07-07 11:32:33 +02:00 |
|
|
|
ec1023e790
|
Intersect document ids by inverse popularity of the words
This reduces the worst request we had which took 56s to now took 3s ("the best of the do").
|
2020-07-05 19:33:51 +02:00 |
|
|
|
2fcae719ad
|
Use another LRU impl which uses hashbrown
|
2020-06-29 22:26:06 +02:00 |
|
|
|
f98b615bf3
|
Replace the LRU by an Arc cache
|
2020-06-29 20:48:57 +02:00 |
|
|
|
5f0088594b
|
Index by writing directly into LMDB
|
2020-06-29 13:54:47 +02:00 |
|
|
|
63cbeca64e
|
Skip all derived words when too short
|
2020-06-28 12:13:12 +02:00 |
|
|
|
736f0f7560
|
Use the proximity instead of the attributes when searching for <= 7 proximities
|
2020-06-28 12:13:12 +02:00 |
|
|
|
fe3be8f18a
|
Replace the HashMap by a Vec for attributes documents ids
|
2020-06-28 12:13:12 +02:00 |
|
|
|
7e16afbdce
|
Ignore documents which are not part of the candidates when exploring with A*
|
2020-06-24 15:06:45 +02:00 |
|
|
|
1c7a9a4132
|
Remove the found documents from the candidates list
|
2020-06-24 15:00:26 +02:00 |
|
|
|
50169b9798
|
Compute the full list of ids we are willing to find by attribute
|
2020-06-24 14:48:04 +02:00 |
|
|
|
374ec6773f
|
Introduce a database to store all docids for a word and attribute
|
2020-06-22 19:24:20 +02:00 |
|
|
|
ba3e805981
|
Document the Index types and the internal LMDB databases
|
2020-06-22 18:09:22 +02:00 |
|
