vault icon indicating copy to clipboard operation
vault copied to clipboard

error creating database object: error verifying connection: failed to send SQL Batch: write tcp wsasend: An existing connection was forcibly closed by the remote host.

Open svisamsetty opened this issue 6 years ago • 4 comments

I am getting the following error when trying to write a configuration for a local MSSQL database.

error creating database object: error verifying connection: failed to send SQL Batch: write tcp 127.0.0.1:58416->127.0.0.1:1433: wsasend: An existing connection was forcibly closed by the remote host.

If I continuously try for about 50 times, the connection get written without an error randomly. 1 in about 50 times.

Steps to reproduce the behavior:

  1. Start the vault server.
  2. Enable the database secret engine from the vault ui.
  3. Run the command: vault write database/config/productdb plugin_name=mssql-database-plugin connection_url='sqlserver://{{username}}:{{password}}@127.0.0.1:1433' allowed_roles="processadmin" username="sa" password="admin@123"

Expected behavior Write the database configuration without throwing the error

Environment: Operating System: Windows 10 Professional Vault version: 0.11.4 Microsoft SQL Server Express (64-bit) Version 12.0.2000.8

Vault server configuration file(s):

Using dev mode; vault server --dev

Additional context Seems to happen when using Windows Binary.

svisamsetty avatar Nov 04 '18 13:11 svisamsetty

I got this exact problem as well, Works sometimes, I do see that the Vault > MSSQL connection jumps between addresses, IPv4 <> IPv6, maybe this is related. Running Wireshark on the traffic, i'm also seeing some SSPI message[malformed packet] errors.

Environment: Windows 10 (Version 1803, build; 17134.829) Vault v1.1.3 ('9bc820f700f83a7c4bcab54c5323735a581b34eb') Microsoft SQL Server 2019 (CTP2.4) - 15.0.1400.75 (X64)

C:\Users\kasper\Downloads\vault_1.1.3_windows_amd64>vault write database/config/my-mssql-database plugin_name=mssql-database-plugin connection_url='sqlserver://{{username}}:{{password}}@localhost:1433' allowed_roles="*" username="my_auth_creds" password="my_auth_creds!!"

C:\Users\kasper\Downloads\vault_1.1.3_windows_amd64>vault write database/config/my-mssql-database plugin_name=mssql-database-plugin connection_url='sqlserver://{{username}}:{{password}}@localhost:1433' allowed_roles="*" username="my_auth_creds" password="my_auth_creds!!"
Error writing data to database/config/my-mssql-database: Error making API request.

URL: PUT http://localhost:8200/v1/database/config/my-mssql-database
Code: 400. Errors:

* error creating database object: error verifying connection: read tcp [::1]:50625->[::1]:1433: wsarecv: An existing connection was forcibly closed by the remote host.

C:\Users\kasper\Downloads\vault_1.1.3_windows_amd64>vault write database/config/my-mssql-database plugin_name=mssql-database-plugin connection_url='sqlserver://{{username}}:{{password}}@localhost:1433' allowed_roles="*" username="my_auth_creds" password="my_auth_creds!!"

C:\Users\kasper\Downloads\vault_1.1.3_windows_amd64>

Server trace log;

2019-07-01T13:52:23.550+0200 [DEBUG] rollback: attempting rollback: path=cubbyhole/
2019-07-01T13:52:23.550+0200 [DEBUG] rollback: attempting rollback: path=sys/
2019-07-01T13:52:23.550+0200 [DEBUG] rollback: attempting rollback: path=identity/
2019-07-01T13:52:23.550+0200 [DEBUG] rollback: attempting rollback: path=database/
2019-07-01T13:52:23.550+0200 [DEBUG] rollback: attempting rollback: path=secret/
2019-07-01T13:53:04.315+0200 [TRACE] secrets.database.database_a1069c7d.mssql-database-plugin: initialize: transport=builtin status=started
2019-07-01T13:53:04.323+0200 [TRACE] secrets.database.database_a1069c7d.mssql-database-plugin: initialize: transport=builtin status=finished verify=true err="error verifying connection: read tcp [::1]:50625->[::1]:1433: wsarecv: An existing connection was forcibly closed by the remote host." took=8.1093ms
2019-07-01T13:53:04.323+0200 [TRACE] secrets.database.database_a1069c7d.mssql-database-plugin: close: transport=builtin status=started
2019-07-01T13:53:04.325+0200 [TRACE] secrets.database.database_a1069c7d.mssql-database-plugin: close: transport=builtin status=finished err=<nil> took=2.6453ms
2019-07-01T13:53:10.894+0200 [TRACE] secrets.database.database_a1069c7d.mssql-database-plugin: initialize: transport=builtin status=started
2019-07-01T13:53:10.901+0200 [TRACE] secrets.database.database_a1069c7d.mssql-database-plugin: initialize: transport=builtin status=finished verify=true err=<nil> took=7.0761ms
2019-07-01T13:53:10.901+0200 [TRACE] secrets.database.database_a1069c7d.mssql-database-plugin: close: transport=builtin status=started
2019-07-01T13:53:10.904+0200 [TRACE] secrets.database.database_a1069c7d.mssql-database-plugin: close: transport=builtin status=finished err=<nil> took=2.648ms
2019-07-01T13:53:23.516+0200 [DEBUG] rollback: attempting rollback: path=auth/token/
2019-07-01T13:53:23.516+0200 [DEBUG] rollback: attempting rollback: path=identity/
2019-07-01T13:53:23.516+0200 [DEBUG] rollback: attempting rollback: path=sys/
2019-07-01T13:53:23.516+0200 [DEBUG] rollback: attempting rollback: path=secret/
2019-07-01T13:53:23.516+0200 [DEBUG] rollback: attempting rollback: path=cubbyhole/
2019-07-01T13:53:23.517+0200 [DEBUG] rollback: attempting rollback: path=database/
2019-07-01T13:54:23.513+0200 [DEBUG] rollback: attempting rollback: path=auth/token/```

kasperbrandenburg avatar Jul 01 '19 12:07 kasperbrandenburg

I just tested with Vault Version 1.3.2 on Windows 10 Home with SQLServer running on a different host with mcr.microsoft.com/mssql/server:2017-CU17-ubuntu container. I got no connection problems at all.

Closing this issue, feel free to comment if the problem persists.

spangenberg avatar Feb 11 '20 17:02 spangenberg

I am seeing this error repeatedly with no intermittent success

  1. Vault dev server installed (version v1.3.2)
  2. SQLExpress local instance running, TCP enabled, IPAll set for 1433 port, can connect with no issues via SSMS (windows auth and sql auth)
  3. Firewall rules added InBound rules for 1433 and 8200 ports

vault write database/config/my-mssql-database plugin_name=mssql-database-plugin connection_url='sqlserver://{{username}}:{{password}}@localhost\SQLEXPRESS:1433' allowed_roles="*" username="..." password="..."

vault write database/config/my-mssql-database plugin_name=mssql-database-plugin connection_url='sqlserver://{{username}}:{{password}}@local\SQLEXPRESS:1433' allowed_roles="*" username="..." password="..."

vault write database/config/my-mssql-database plugin_name=mssql-database-plugin connection_url='sqlserver://{{username}}:{{password}}@127.0.0.1\SQLEXPRESS:1433' allowed_roles="*" username="..." password="..."

Any idea on what am I doing wrong ?

lkasiuipath avatar Apr 07 '20 18:04 lkasiuipath

I suspect this issue is no longer relevant in the most recent versions (1.11.3).

Hey @svisamsetty @ikatson @kasperbrandenburg - is this issue still applicable for you folks or have you retested since?

aphorise avatar Sep 02 '22 21:09 aphorise

I believe this issue is no longer relevant and may be closed.

aphorise avatar Sep 17 '22 20:09 aphorise

I'm going to go ahead and close this issue now. Please feel free to open a new one if needed - note, we accept bug reports and enhancement requests for "n-2" versions of Vault, which as of this writing would be 1.10+. Thanks!

heatherezell avatar Oct 27 '22 17:10 heatherezell