babelfish-for-postgresql
[Bug]: bcp giving "An existing connection was forcibly closed by the remote host." error
What happened?
I am loading a large amount of tables in a for loop with bcp. It is working well most of the time but sometimes errors out (on the same tables).
My powershell commands, on a Windows server, look like this:
-- data export command
bcp "<SELECT * FROM MYDB.MYSCHEMA.MYTABLE>" <FILEPATH> -c -t"<CUSTOM DELIM>" -r"<CUSTOM NEWLINE>" -S <RDS ENDPOINT> -U <USERNAME> -P <PASSWORD>
--data import command
bcp <MYDB.MYSCHEMA.MYTABLE> in <FILEPATH> -c -t"<CUSTOM DELIM>" -r"<CUSTOM NEWLINE>" -S <BABELFISH ENDPOINT> -U <USERNAME> -P <PASSWORD>
I can't see anything in those tables that seem special, but it consistently errors on the same ones. Does anyone have any knowledge of what might be causing this?
Version
BABEL_3_X_DEV (Default)
Extension
None
Which flavor of Linux are you using when you see the bug?
No response
Relevant log output
Starting copy...
1000 rows sent to SQL Server. Total sent: 1000
SQLState = 08S01, NativeError = 10054
Error = [Microsoft][ODBC Driver 17 for SQL Server]TCP Provider: An existing connection was forcibly closed by the remote host.
SQLState = 08S01, NativeError = 10054
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = S1000, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Text column data incomplete
SQLState = 08S01, NativeError = 0
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
SQLState = 08S01, NativeError = -2147467259
Error = [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure
BCP copy in failed
### Code of Conduct
- [X] I agree to follow this project's Code of Conduct.
Side note, is it possible to load the data via psql's \copy command, then move the data to the Babelfish DB?
@calmitchell617 can you please provide more details like the table DDL and also check if there was a crash and provide it's stack trace along with any logs you can capture.
Yes, I will provide as much as possible.
This work is for a 3rd party client - I just asked them what is OK to share. They will probably want certain things shared privately, can you please let me know how that can be accomplished? Email?
Once the problem is diagnosed, I would be happy to post a generalized example for the benefit of the viewers of this issue, but like I said, I'm not sure what the issue is, so cannot generalize it at this time.
@KushaalShroff they said it is OK to share DDL privately, as well as the stack trace.
How can I get that to you? If you want to send me an email, the contact form at https://www.sqlpipe.com/contact goes to my inbox.
@calmitchell617: Are you using Babelfish on Aurora PostgreSQL? If so, you can open a support ticket with AWS and share the materials there.
@suprio-amzn, thank you for the response. Yes, this is on Aurora PostgreSQL. Should I open the ticket on the customer's account, or in mine?
My account does not have paid AWS support, but I think the customer does.
Hello @calmitchell617
Did you manage the issue? I am getting exactly the same error when copying large data. I am thinking about exceeding destination memory, but not sure.
