haikuporter
haikuporter copied to clipboard
haikuports
HaikuPorter/BuildMaster: MongoDB reporter
Trying to make the data in haikuporter buildmaster more reasonable to be parsed with a web app. We're still grepping 1000's of json files to find last package builds, etc.
builders:
buildruns:
The code is pretty simplistic and tries to not arbirararly modify dictionary layouts.
Benefit is actually being able to rapidly search 1000's of buildruns for packages via an index of the json structure. Obviously more needs done here to render a web application based on the mongodb contents.
A json "reporter" allows continued "dumping" buildmaster state to on-disk json files.
Updated this one for the latest code. Fixed some old calling conventions to pymongo.
The data format sucks since it's still the internal format used by haikuporter, but we could "translate" it into something easier to consume within the "ReporterMongo" code. We could also translate the data into something easier to consume within the ReporterJson code.
For now though, this one opens the door to reporting to things other than 10,000 json files on disk which are slow to parse.
Do we have a way to import old JSONs into Mongo, too? It would be good to be able to search all old builds...
It makes some arbitrary adjustments to the dict.. so we would need a short tool to translate the existing json to the intended mongo layout.