Troubleshooting the API execution error

[shsarava]

Trying to add a server to the backend, with the following steps. Not sure why the transaction fails.

Got a new Transaction

curl -X POST --user admin:admin -H "Content-Type: application/json" http://localhost:5555/v2/services/haproxy/transactions?version=1 {"_version":1,"id":"57768891-56ea-436b-b85e-f7450dd616c8","status":"in_progress"}

Adding a new backend server to the existing backend

curl -X POST --user admin:admin -H "Content-Type: application/json" -d '{"name": "server2", "address": "10.0.0.40", "port": 80, "check": "enabled", "maxconn": 30, "weight": 80}' "http://localhost:5555/v2/services/haproxy/configuration/servers?backend=app&transaction_id=57768891-56ea-436b-b85e-f7450dd616c8" {"address":"10.0.0.40","check":"enabled","maxconn":30,"name":"server2","port":80,"proxy-v2-options":null,"weight":80}

Committing the transaction

curl -X PUT --user admin:admin -H "Content-Type: application/json" "http://localhost:5555/v2/services/haproxy/transactions/57768891-56ea-436b-b85e-f7450dd616c8"

Getting an Error

{"code":400,"message":"14:** ERR transactionId=57768891-56ea-436b-b85e-f7450dd616c8 "}

Am I missing any needed steps ? How can I troubleshoot the error

[mjuraga]

This means that checking the the transaction file syntax has failed. Check what you have set up for your -b --haproxy-binary (default: haproxy) set up. Then you can check the failed transactions directory set up in your dataplaneapi (default: /tmp/haproxy/failed), and look for your failed transaction config file (suffix 57768891-56ea-436b-b85e-f7450dd616c8).

See what's wrong with the config and try running <haproxy-binary set up> -c -f <failed transaction file> to see why haproxy checks fails with that file.

Also try running it as the same user that is running the dataplaneapi.

[shsarava]

Thank you for the quick response. I tried to use the same failed config with haproxy binary and it ran successfully. I tried to run the dataplaneapi with trace logging but didn't get much output when the error happened. Not sure what could be the issue..

[mjuraga]

Can you paste here how you run your dataplaneapi? What CLI options do you use there?

[shsarava]

Processes I see running

root 7702 1 0 Jun21 ? 00:00:00 sudo ./dataplaneapi --port 5555 -b /usr/sbin/haproxy -c /etc/haproxy/haproxy.cfg -d 5 -r systemctl restart haproxy -s systemctl restart haproxy -u datapla

root 7703 7702 0 Jun21 ? 00:00:20 ./dataplaneapi --port 5555 -b /usr/sbin/haproxy -c /etc/haproxy/haproxy.cfg -d 5 -r systemctl restart haproxy -s systemctl restart haproxy -u dataplaneapi

CLI I used to run the DataplaneAPI

sudo ./dataplaneapi --port 5555 -b /usr/sbin/haproxy -c /etc/haproxy/haproxy.cfg -d 5 -r "systemctl restart haproxy" -s "systemctl restart haproxy" -u dataplaneapi -t /tmp/haproxy --userlist api &

[mjuraga]

And when you try to execute:

sudo /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c

What is the output?

[shsarava]

I am actually using systemd to run haproxy and the service config file is below

ec2-user@ip-10-0-0-214 ~]$ cat /etc/systemd/system/haproxy.service [Unit] Description=HAProxy Load Balancer After=network.target [Service] Environment=LD_LIBRARY_PATH=/usr/local/openssl-1.1.1c/lib/ Environment="CONFIG=/etc/haproxy/haproxy.cfg" "PIDFILE=/run/haproxy.pid" "EXTRAOPTS=-S /run/haproxy-master.sock" ExecStartPre=/usr/local/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS ExecStart=/usr/local/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE $EXTRAOPTS ExecReload=/usr/local/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS ExecReload=/bin/kill -USR2 $MAINPID KillMode=mixed Restart=always SuccessExitStatus=143 Type=notify [Install] WantedBy=multi-user.target

[mjuraga]

It seems your haproxy is located at /usr/local/sbin/haproxy

Try running dataplaneapi with -b /usr/local/sbin/haproxy

[shsarava]

Made the change

root 10495 10275 0 14:57 pts/2 00:00:00 sudo ./dataplaneapi --port 5555 -b /usr/local/sbin/haproxy -c /etc/haproxy/haproxy.cfg -d 5 -r systemctl restart haproxy -s systemctl restart haproxy -u dataplaneapi -t /tmp/haproxy --userlist api

root 10496 10495 0 14:57 pts/2 00:00:01 ./dataplaneapi --port 5555 -b /usr/local/sbin/haproxy -c /etc/haproxy/haproxy.cfg -d 5 -r systemctl restart haproxy -s systemctl restart haproxy -u dataplaneapi -t /tmp/haproxy --userlist api

Same steps of adding a server to backend and getting the same error

[ec2-user@ip-10-0-0-214 ~]$ curl -X POST --user admin:admin -H "Content-Type: application/json" http://localhost:5555/v2/services/haproxy/transactions?version=1 {"_version":1,"id":"f3a8b275-c435-428d-a17e-b7441d8effbb","status":"in_progress"}

[ec2-user@ip-10-0-0-214 ~]$ curl -X POST --user admin:admin -H "Content-Type: application/json" -d '{"name": "server3", "address": "10.0.0.41", "port": 80, "check": "enabled", "maxconn": 30, "weight": 80}' "http://localhost:5555/v2/services/haproxy/configuration/servers?backend=app&transaction_id=f3a8b275-c435-428d-a17e-b7441d8effbb" {"address":"10.0.0.41","check":"enabled","maxconn":30,"name":"server3","port":80,"proxy-v2-options":null,"weight":80}

[ec2-user@ip-10-0-0-214 ~]$ curl -X PUT --user admin:admin -H "Content-Type: application/json" "http://localhost:5555/v2/services/haproxy/transactions/f3a8b275-c435-428d-a17e-b7441d8effbb" {"code":400,"message":"14: ERR transactionId=f3a8b275-c435-428d-a17e-b7441d8effbb "}

[shsarava]

I used 2.0.0 and it worked. 2.0.3 seems to have a bug .. even the logging doesn't work..

HAProxy Data Plane API v2.0.0 448d36e

Build from: [email protected]:haproxytech/dataplaneapi.git Build date: 2020-04-27T15:28:42Z

[andyallan]

We had the same issue. With 2.0.3 every transaction commit now fails.

Rolling back to 2.0.2 solves it for us.

[mjuraga]

related to https://github.com/haproxytech/dataplaneapi/issues/119

[mjuraga]

What is your haproxy version? It seems that older versions of haproxy report results of -c differently. Please check #119 for more details.

[andyallan]

Sorry for the delay. HA-Proxy version 2.0.15-1ppa1~bionic 2020/06/13 - https://haproxy.org/