starbasedb Replicate data from external source to internal source with a Plugin

Is your feature request related to a problem? Please describe. StarbaseDB instances support by default an internal database (SQLite offered by the Durable Object) as well as an optional external data source. External data sources can be powered in one of two ways, both by providing values in the wrangler.toml file of the project.

Outerbase API Key
Connection details of the database

Describe the solution you'd like What would be beneficial for some use cases is the ability to bring in an external data source (e.g. a Postgres on Supabase) and have a pull mechanism where data can be brought into the internal DO SQLite so that the instance serves as a close-to-edge replica that can be queried alternatively to querying the Supabase Postgres instance.

Describe alternatives you've considered

Considering the pull vs push mechanism. A pull mechanism seems to be a better global solution where a push mechanism would be required to live elsewhere on a per provider basis.

Additional context

Might be beneficial for users to be able to define in the plugin what intervals data should be pulled at
Might be beneficial to allow users to define which tables should have data pulled into it (perhaps not all tables need replicated)
Likely need a way to know for each table what the last queried items were so you can do append-only type polling for new data. Does a user need to define a column to base this on (e.g. id or created_at columns perhaps)?

Jan 23 '25 21:01 Brayden

/bounty $250

Jan 24 '25 16:01 Brayden

💎 $250 bounty • Outerbase (YC W23)

Steps to solve:

Start working: Comment /attempt #72 with your implementation plan
Submit work: Create a pull request including /claim #72 in the PR body to claim the bounty
Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

❗ Important guidelines:

To claim a bounty, you need to provide a short demo video of your changes in your pull request
If anything is unclear, ask for clarification before starting as this will help avoid potential rework
Low quality AI PRs will not receive review and will be closed
Do not ask to be assigned unless you've contributed before

Thank you for contributing to outerbase/starbasedb!

Attempt	Started (UTC)	Solution	Actions
🟢 @Kunal-Darekar	Mar 19, 2025, 09:47:47 AM	#95	Reward
🟢 @MAVRICK-1	Jul 23, 2025, 07:54:25 AM	#96	Reward
🟢 @onyedikachi-david	Jan 24, 2025, 10:18:25 PM	#75	Reward