David Smiley is working with Ben Lewis at Harvard Center for Geographic Analysis to build a “Billion Object Platform” or BOP. The goal of the BOP, which is funded by the Sloan Foundation, is to provide a proof-of-concept platform designed to lower the barrier for researchers who need to access big streaming spatio-temporal datasets.
The BOP is being developed specifically to provide a big data API to the Harvard Dataverse and Harvard WorldMap platforms. The platform will be loaded with streaming geo-referenced tweets but any dataset could, with a some tweaking, be substituted. Hosting of the BOP is being provided by Massachusetts Open Cloud.
The BOP builds on work David Smiley did previously, also working with the Harvard Center for Geographic Analysis, to build 2D faceting (what we are calling HHypermap Core) into Lucene/Solr as part of a map service discovery platform called HHypermap Registry, project which was funded by the National Endowment for the Humanities.
To build the BOP, David Smiley developed many new capabilities in addition to 2D faceting:(list…), scaling it up massively to create a system to allow researchers to interactively query and visualize big spatio-temporal datasets.
TODO link to conference presentation (Lucene Solr Revolution 2016). And more here ... TODO