Heliosearch – The Next Solr
Project Heliosearch is about creating the next major evolution of Solr. To kick things off, I’ve started and implemented “off-heap” filters, as a first step toward taking performance to the next level.
For a number of reasons, I felt it best to incubate this project at github, where it could have a community dedicated solely to it’s advancement. The longer term plan is to bring it back to the ASF once it has gained enough traction.
Off-Heap Data Structures
JVMs have never been particularly good at dealing with large heaps. The larger the heap, the more resources the JVM wastes copying memory around that could otherwise be spent doing more useful work. Larger heaps also correlate with larger stop-the-world garbage collection pauses.
Update:The first heliosearch release is out, and the first tests of off-heap filters performance and off-heap fieldCache performance have been very encouraging.
Native Code Optimizations
In addition to moving more large data structures off-heap(like UnInvertedField), I am planning to implement native code optimizations for certain hotspots. Native code faceting would be an obvious first choice since it can often be a CPU bottleneck.
Update: Native code faceting for single valued string fields is in!
Community
We’re off to a fast start, with over 30 committers already, and our first pre-release out.
Stop by the and let us know what types of features you’d like to see!