airbyte [source-hubspot] New streams: Associations between objects

Topic

Add new streams to be able to retrieve up-to-date associations with their labels between HS objects using official API endpoints

Relevant information

According to the official Hubspot documentation, there are endpoints dedicated to associations between objects. It would be interesting to collect associations only, with their labels, instead of retrieving them via usual object streams. Another advantage: associations may be dynamically retrievable for all object types via those endpoints (i.e. no need to include missing association types in existing objects streams).

Example of use case solving a current huge pain 🙏

existing object streams used in incremental only retrieve associations at the time of data extraction
if any association is created after the incremental sync, it is NOT retrieved (except if object itself is modified again, being in the scope of a subsequent incremental sync)
as Engagements are mainly never modified once created, but possibly a posteriori associated to many objects, we miss a huge number of Engagements x Deals/Contacts/Companies associations ; we also can not consider re-syncing all engagements at a regular schedule because of the data volume

-> if we have new dedicated streams for associations between usual objects (eg: Company x Contact, Deal x Engagements call, etc...), we could enable a daily full-refresh sync for all associations only in an efficient manner and then use up-to-date associations data for our internal use cases 👍

NB: above dedicated streams for associations may even evolve to incremental sync also as soon as Hubspot enables Search it in associations API endpoints (the same way as for other objects)🤞

Aug 22 '24 14:08 kev-datams

@airbytehq/dev-python can someone take a look into this Hubspot proposal?

Aug 22 '24 17:08 marcosmarxm

That's a good idea. I'm adding this to our backlog, but I don't have a timeline to offer at the moment.

Aug 23 '24 18:08 girarda

That's a good idea. I'm adding this to our backlog, but I don't have a timeline to offer at the moment.

@girarda, does no timeline at the moment mean not before 3 months for sure ? What is the frequency of backlog reviews to prioritize it in the future ?

Thanks !

Aug 24 '24 06:08 kev-datams

+1 This would be great to have!