6 Commits

Author	SHA1	Message	Date
Clément Renault	656a851830	Introduce the Transform struct transforming CSVs ... This allows us to: - transform a CSV, a JSON or a JSON lines data type into the same Grenad x Obkv streamable data type and creates the new FieldsIdsMap. - Extract all the documents user ids in advance to be able to delete the existing documents before re-indexing them. - Keep the last documents with the same user id avoiding duplicates in the same request.	2020-10-24 13:37:38 +02:00
Clément Renault	f6eecb855e	Send a basic progressing status to the updates front page	2020-10-21 15:38:28 +02:00
Clément Renault	f948a03be2	Optimise the merge functions to avoid allocations	2020-10-20 16:40:50 +02:00
Clément Renault	cde8478388	Replace the panic in the merge function by actual errors	2020-10-20 16:19:07 +02:00
Clément Renault	8ed8abb9df	Introduce an append-only indexing system	2020-10-20 15:00:58 +02:00
Clément Renault	a122d3d466	Export the indexing part into a module	2020-10-20 14:22:09 +02:00