extensions icon indicating copy to clipboard operation
extensions copied to clipboard

[Firestore-BigQuery-Export] Is it possible to export a real time database to bigquery

Open hannahbwoods opened this issue 3 years ago • 6 comments

Hello,

I'd like to export a subset of my realtime database to big query using the Firestore-BigQuery-Export extension, do you know if thats possible? or a feature thats being added soon?

Thanks, Hannah

hannahbwoods avatar Jan 11 '22 16:01 hannahbwoods

Hi @hannahbwoods.

Currently this extension only streams through Firestore collections.

I will add this to our project board for further discussion as a potential new feature.

dackers86 avatar Jan 12 '22 09:01 dackers86

@hannahbwoods this feature for the current stream to bigquery extensions is something we're considering. Problem is if we add RTDB support to the current one all users need to have RTDB enabled since the extension will install the trigger either way. On our roadmap is conditional resources for extensions which allows selective deployment of resources. When this feature is implemented we can easily support RTDB.

i14h avatar Jan 18 '22 17:01 i14h

Thanks for the update Iman! Do you have an estimate on that feature's timeline? Or should we consider other solutions such as a cloud function.

On Tue, Jan 18, 2022 at 5:13 PM Iman Rahmatizadeh @.***> wrote:

@hannahbwoods https://github.com/hannahbwoods this feature for the current stream to bigquery extensions is something we're considering. Problem is if we add RTDB support to the current one all users need to have RTDB enabled since the extension will install the trigger either way. On our roadmap is conditional resources for extensions which allows selective deployment of resources. When this feature is implemented we can easily support RTDB.

— Reply to this email directly, view it on GitHub https://github.com/firebase/extensions/issues/843#issuecomment-1015632153, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV46ZX6XJW4LY3YRQVNLLI3UWWNTZANCNFSM5LWY4XRQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

hannahbwoods avatar Jan 20 '22 17:01 hannahbwoods

It's on our roadmap for this year but we haven't come up with a date yet sorry :-/ I guess in the interim you could use a cloud function

i14h avatar Jan 20 '22 19:01 i14h

sup guys any update about this?

Dulb26 avatar Jun 24 '22 11:06 Dulb26

Hi guys, any update to connect RTDB to Big Query? It would be a very key component.

dlondonog avatar Jun 27 '23 13:06 dlondonog