Skip to content
This repository was archived by the owner on Feb 25, 2020. It is now read-only.
This repository was archived by the owner on Feb 25, 2020. It is now read-only.

Conversion from raw files to NihDB file format #529

@jdegoes

Description

@jdegoes

Once the analytics engine can query across raw files (e.g. JSON), as well as NihDB files, we need the ability to convert from raw formats to NihDB, to improve performance.

This needn't be integrated into the query engine (although it could be, as some sort of caching later for queries across raw files), but could instead exist as a separate process which a user could run -- a utility of sorts that would convert data from various formats to NihDB, where queries would run much more efficiently.

When this ticket is done, although users can query across their JSON / CSV files directly, they will be able to achieve increased performance by converting those files into NihDB format.

At minimum, a user will be able to convert files manually by running a command-line process, or through a REST API on the storage side which executes the conversion asynchronously.

The conversion should not delete the original files (users can always do that themselves).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions