evidence icon indicating copy to clipboard operation
evidence copied to clipboard
verified publisher icon evidence-dev

Support For Apache Pinot

Open hdulay opened this issue 2 years ago • 5 comments

What do you use this database for?

Real-Time analytical queries. Apache Pinot is an RTOLAP database (real-time online analytical process).

Are there any popular and well-maintained clients for this database?

https://socket.dev/npm/package/pinot-client

Are you interested in contributing this database connector?

yes, I'm new to this project. Any instructions for building connectors would be great. Thanks

hdulay avatar Sep 07 '23 11:09 hdulay

Hey @hdulay ,

Thanks for the suggestion!

We're currently reworking our project to allow "pluggable" connectors, and I think pinot is a really good candidate for this.

Context: currently, when we add connectors, we have to add them to the core library. This makes maintaining them a pain, as we (Evidence team) do not always have instances of these DBs to test. This new architecture will mean it's much easier for people to contribute and maintain these without being held back by the bandwidth of our team!

If you're open to it, I can update here once the new architecture is live (hopefully pretty soon!)

archiewood avatar Sep 08 '23 18:09 archiewood

Yes I look forward to it. Thanks

hdulay avatar Sep 08 '23 18:09 hdulay

Hey @archiewood I'm also really interested in a Pinot connector! Were you able to make progress on the plugin architecture?

cyrilou242 avatar Feb 15 '24 22:02 cyrilou242

Hey sorry for the delay!

We have totally re-architected Evidence since this issue was opened, and Evidence is indeed now "pluggable", meaning you can write your own connectors, publish them, and use them with Evidence.

Here are some resources that may help:

  1. Docs on creating a plugin: https://docs.evidence.dev/plugins/creating-a-plugin/datasources
  2. Template for a connector plugin: https://github.com/evidence-dev/datasource-template
  3. An example that I spun up for pulling stats from the npm API: https://github.com/archiewood/npm-stats

If you do write a Pinot plugin,

  • the suggested package name is evidence-connector-[plugin] eg evidence-connector-pinot
  • the docs are new, so let us know if there is extra information you need
  • share it on the Evidence community, and we'll feature it!

archiewood avatar Feb 16 '24 00:02 archiewood

Awesome thanks a lot @archiewood! I'll have a look at this.

cyrilou242 avatar Feb 16 '24 10:02 cyrilou242

hey @cyrilou242 just heard from another user asking about pinot - did you ever look at this?

archiewood avatar Jul 28 '24 15:07 archiewood