Airbyte process error causing sync failures between Source e.g. Hubspot and MySQL

[seventhingsit]

Summary

We are experiencing an issue with Jitsu when syncing data between Hubspot and our MySQL destination. The syncs fail and are aborted due to an error related to parsing the airbyte state line. This problem has started occurring with the newer versions of Jitsu. Related issue: https://github.com/airbytehq/airbyte/issues/36893

System configuration and versions

Jitsu Version: v1.44.8 docker image: jitsucom/jitsu Self-Hosted Environment DB Dest: mariadb:10.9.3 via docker container

Artifacts (logs, etc)

Here are the relevant log entries leading up to and including the error.

2024-05-14 07:37:41 [ERROR] - Airbyte output line process error: Error parsing airbyte state line {"type": "STATE", "state": {"type": "STREAM", "stream": {"stream_descriptor": {"name": "companies", "namespace": null}, "stream_state": {}}, "sourceStats": {"recordCount": 500.0}}}: malformed state line 'data' doesn't exist. Process will be killed
2024-05-14 07:37:41 [ERROR] - Error closing airbyte runner: exit status 1

Additional context and motivation:

This issue started occurring after updating to the latest versions. The sync process previously worked without issues. The error appears consistently across multiple configurations involving Hubspot and MySQL. Please let us know if any additional information is required to troubleshoot this issue.

Thank you for your support.

[vklimontovich]

Thanks for reporting this. Have you tried Jitsu 2.0? Does this issue exists here as well?

[seventhingsit]

No. Is there a migration path from version 1 to 2? I have seen that there are fundamental, structural changes that make an update not so easy. I was therefore hoping that connectors or destination drivers might be updated. Or is the statement here that version 1 is no longer maintained?

[vklimontovich]

Not really, 2.* has a completely different architecture. The good news is that you jitsu 2.0 should be easy to deploy:

We maintain Jitsu 1.* only when it pertains to Jitsu Cloud issues. We gladly accept contributions too, but we don't have resources to do anything besides that.

Speaking of your issue, it seems like the best way to go is to rollback to a last version of Airbyte connector that used to work for you. The new version seems to be using some protocol features that are not compatible with Jitsu 1.*

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.