Added ChronicleMap persistence as an alternative to SQLite

[akumaburn]

I did recently add ChronicleMap (https://github.com/OpenHFT/Chronicle-Map) as a store to CQEngine and tested it successfully. It is license compatible with this project using the same type - Apache 2.0 license: https://github.com/OpenHFT/Chronicle-Map/blob/ea/LICENSE

Usage is like:

ChroniclePersistence<Community, String> diskPersistence = new ChroniclePersistence<>((SimpleAttribute<Community, String>) GUID_ATTRIB, new File("persist.dat"),String.class,Community.class,UUID.randomUUID().toString().length()+1,64 * 1024,1000000);

ConcurrentIndexedCollection<Community> communities = new ConcurrentIndexedCollection<>(diskPersistence);

[npgall]

This is very interesting, and something I would be interested in merging.

I am hoping soon to carve out some time to take a closer look.

[jeansossmeier]

@akumaburn This is cool, are you using this sucessfully?
Have you done any benchmarking?

Another performance improvement that could be done is instead of Protostuff, you could use Apache Fury, which is crazy fast.
https://github.com/apache/fury

[akumaburn]

@akumaburn This is cool, are you using this sucessfully? Have you done any benchmarking?

Another performance improvement that could be done is instead of Protostuff, you could use Apache Fury, which is crazy fast. https://github.com/apache/fury

Sorry for the late reply, We did use this successfully, it was able to handle millions of trace logs per day with very fast search in our custom logging service; That being said I think there may be a bug when scaling the datastore to very large sizes, I don't remember if I narrowed it down though as my company switched to using OpenTelemetry/HyperDX.

It was definitely faster than what is currently in use (HyperDX/DataDog/Grafana search is much slower); might make for a good project if you feel up to it.