`source-postgres` : Throw a configuration error for "temporary file size exceeds temp_file_limit" error

[akashkulk]

Topic

Postgres connector

Revelant information

Context : OC issue https://github.com/airbytehq/oncall/issues/1822

Use the config error message framework to point users towards the documentation of how to increase the temporary file size : https://docs.airbyte.com/integrations/sources/postgres/#temporary-file-size-limit

[moreaupascal56]

Hello @akashkulk, I would love to try and contribute to this issue, however the context link is private I guess, is this available for PR? thx and have a great day ;)

[evantahler]

I think this might actually want to be a TRANSIENT_ERROR

[theyueli]

This bug was reported last year. Thanks to the recent improvements on reliable data reading in postgres source, it seems I can no longer reproduce this issue. However, the temp_file_limit issue still gets reported as a system error at destination side. This is discovered when I was testing pg <-> pg sync.

for instance,

2024-05-24 17:03:40 platform > failures: [ {
  "failureOrigin" : "destination",
  "failureType" : "system_error",
  "internalMessage" : "java.lang.RuntimeException: org.postgresql.util.PSQLException: ERROR: temporary file size exceeds temp_file_limit (0kB)",
  "externalMessage" : "Something went wrong in the connector. See the logs for more details.",
  "metadata" : {
    "attemptNumber" : 9,
    "jobId" : 15,
    "from_trace_message" : true,
    "connector_command" : "write"
  },

I agree with @evantahler , this error should be a transient error. A config error is related to the misconfiguration of our own software, not the database itself. I will send a quick pr to make it a transient error.