runtime error: invalid memory address or nil pointer dereference

[f1-outsourcing]

url: v2/services/haproxy/configuration/acls/1?parent_name=$server&parent_type=frontend&version=1

{"code":500,"message":"runtime error: invalid memory address or nil pointer dereference: /usr/local/go/src/runtime/panic.go:220 panicmem"}

HAProxy Data Plane API v2.5.3 a6a86c8 HAProxy version 2.2.22-16420af, released 2022/03/14

[mjuraga]

Hi @f1-outsourcing we will take a look, tnx for reporting.

[mjuraga]

Can you provide us with the log with the full stack trace oft the panic?

[f1-outsourcing]

Can you provide us with the log with the full stack trace oft the panic?

Do you have a link to a quick how to do this manual?

[mjuraga]

Just check the dataplaneapi logs, it should contain the full stack-trace in case of runtime errors.

[f1-outsourcing]

I am getting this in stdout (instead of stderr) of the container.

{"code":500,"message":"runtime error: invalid memory address or nil pointer dereference: /usr/local/go/src/runtime/panic.go:220 panicmem"}

time="2022-06-24T08:57:15Z" level=error msg="Panic runtime error: invalid memory address or nil pointer dereference" stack_trace="\n/usr/local/go/src/runtime/panic.go:220 panicmem\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/types/connection.go:92 (*Connection).String\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/tcp-request.go:89 (*Requests).Result\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/tcp-request_generated.go:135 (*Requests).ResultAll\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:123 (*configParser).writeParsers\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:48 (*configParser).String\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:87 (*configParser).StringWithHash\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:59 (*configParser).Save\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/configuration.go:201 (*client).Save\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:528 (*Transaction).createTransactionFiles\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:117 (*Transaction).startTransaction\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:97 (*Transaction).CheckTransactionOrVersion\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:97 (*Transaction).StartTransaction\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/configuration.go:2857 (*client).loadDataForChange\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/acl.go:158 (*client).EditACL\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/handlers/acl.go:226 (*ReplaceACLHandlerImpl).Handle\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/operations/acl/replace_acl.go:84 (*ReplaceACL).ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/operation.go:28 NewOperationExecutor.func1\n/usr/local/go/src/net/http/server.go:2047 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/router.go:77 NewRouter.func1\n/usr/local/go/src/net/http/server.go:2047 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/redoc.go:72 Redoc.func1\n/usr/local/go/src/net/http/server.go:2047 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/spec.go:46 Spec.func1\n/usr/local/go/src/net/http/server.go:2047 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/rs/[email protected]/cors.go:219 (*Cors).Handler.func1\n/usr/local/go/src/net/http/server.go:2047 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/adapters/adapters.go:148 ConfigVersionMiddleware.func1.1\n/usr/local/go/src/net/http/server.go:2047 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/lestrrat-go/[email protected]/logformat.go:65 (*ApacheLog).Wrap.func1\n/usr/local/go/src/net/http/server.go:2047 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/adapters/adapters.go:115 RecoverMiddleware.func1.1"

HAProxy version 2.2.24-26b8015, released 2022/05/13

[mjuraga]

Can you share the haproxy configuration file, you can redact sensitive data?

[f1-outsourcing]

Can you share the haproxy configuration file, you can redact sensitive data?

I can send via email, and not an address of a free provider that analyses email content, like gmail.com/google.

[mjuraga]

Can you check if this has been fixed in the recent version?

[f1-outsourcing]

{"code":500,"message":"runtime error: invalid memory address or nil pointer dereference: /usr/local/go/src/runtime/panic.go:259 panicmem"}

time="2022-09-16T09:12:16Z" level=error msg="Panic runtime error: invalid memory address or nil pointer dereference" stack_trace="\n/usr/local/go/src/runtime/panic.go:259 panicmem\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/types/connection.go:92 (*Connection).String\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/tcp-request.go:89 (*Requests).Result\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/tcp-request_generated.go:135 (*Requests).ResultAll\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:123 (*configParser).writeParsers\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:48 (*configParser).String\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:87 (*configParser).StringWithHash\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:59 (*configParser).Save\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/configuration.go:209 (*client).Save\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:529 (*Transaction).createTransactionFiles\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:119 (*Transaction).startTransaction\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:99 (*Transaction).CheckTransactionOrVersion\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:99 (*Transaction).StartTransaction\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/configuration.go:3054 (*client).loadDataForChange\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/acl.go:158 (*client).EditACL\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/handlers/acl.go:226 (*ReplaceACLHandlerImpl).Handle\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/operations/acl/replace_acl.go:83 (*ReplaceACL).ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/operation.go:28 NewOperationExecutor.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/router.go:78 NewRouter.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/redoc.go:72 Redoc.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/spec.go:46 Spec.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/rs/[email protected]/cors.go:219 (*Cors).Handler.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/adapters/adapters.go:148 ConfigVersionMiddleware.func1.1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/lestrrat-go/[email protected]/logformat.go:65 (*ApacheLog).Wrap.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/adapters/adapters.go:115 RecoverMiddleware.func1.1"

HAProxy Data Plane API v2.6.1 5c2258b

Build from: [email protected]:haproxytech/dataplaneapi.git Build date: 2022-09-08T09:07:27Z

[mjuraga]

Do you have any tcp-request connection lines in your config and can you paste them here?

[f1-outsourcing]

[marc@os0 temp2]$ grep 'tcp-request' haproxy.cfg.ORG
  tcp-request connection reject if blacklistweb
  tcp-request connection reject if blacklistweb
  tcp-request connection reject if blacklistabuse
  tcp-request inspect-delay 5s
  tcp-request content accept if { req.ssl_hello_type 1 }
  tcp-request connection set-dst-port int(xxxx) if !{ req.ssl_hello_type 1 } !{ payload(0,7) -m bin 5353482d322e30 } { dst_port xxx } net_office
  tcp-request connection reject if !net_office
  #tcp-request inspect-delay 2s
  #tcp-request content accept if RDP_COOKIE
  tcp-request connection reject if !net_office
  tcp-request connection reject if !net_office
  tcp-request connection reject if !net_allow !net_office !net_content
  tcp-request connection set-dst-port int(xxxx) if { dst_port xxxx }
  tcp-request connection reject if !net_office
  tcp-request connection set-dst-port int(xxxx) if { dst_port xxxx}
  tcp-request connection reject if !net_office
  tcp-request connection reject if !net_office
  tcp-request connection reject if !net_office
  tcp-request connection reject if blacklistmail
  tcp-request connection set-dst-port dst_port,add(10000)
  tcp-request connection set-dst-port dst_port,add(10000)

ps. looks like dataplaneapi keeps running, I think previously it would stop.

[f1-outsourcing]

I was just thinking, if memory cannot be allocated in the resources limited container, could this message also arise? I assume that this api does not allocate much memory, does it? Maybe few MB's not?

[f1-outsourcing]

I was doing some more testing on a docker container with a trimmed down config. I do not seem to have issues there. Returning back to the staging environment. I noticed that I have no problems with getting frontend info, getting acl info, and these errors I get when modifying the acl.

time="2022-09-16T14:23:49Z" level=error msg="Panic runtime error: invalid memory address or nil pointer dereference" stack_trace="\n/usr/local/go/src/runtime/panic.go:259 panicmem\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/types/connection.go:92 (*Connection).String\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/tcp-request.go:89 (*Requests).Result\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/parsers/tcp/tcp-request_generated.go:135 (*Requests).ResultAll\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:123 (*configParser).writeParsers\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:48 (*configParser).String\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:87 (*configParser).StringWithHash\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/config-parser/[email protected]/writer.go:59 (*configParser).Save\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/configuration.go:209 (*client).Save\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:529 (*Transaction).createTransactionFiles\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:119 (*Transaction).startTransaction\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:99 (*Transaction).CheckTransactionOrVersion\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/transaction.go:99 (*Transaction).StartTransaction\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/configuration.go:3054 (*client).loadDataForChange\n/home/mjuraga/projects/go/pkg/mod/github.com/haproxytech/client-native/[email protected]/configuration/acl.go:158 (*client).EditACL\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/handlers/acl.go:226 (*ReplaceACLHandlerImpl).Handle\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/operations/acl/replace_acl.go:83 (*ReplaceACL).ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/operation.go:28 NewOperationExecutor.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/router.go:78 NewRouter.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/redoc.go:72 Redoc.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/go-openapi/[email protected]/middleware/spec.go:46 Spec.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/rs/[email protected]/cors.go:219 (*Cors).Handler.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/adapters/adapters.go:148 ConfigVersionMiddleware.func1.1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/go/pkg/mod/github.com/lestrrat-go/[email protected]/logformat.go:65 (*ApacheLog).Wrap.func1\n/usr/local/go/src/net/http/server.go:2109 HandlerFunc.ServeHTTP\n/home/mjuraga/projects/haproxytech/release/dataplaneapi/adapters/adapters.go:115 RecoverMiddleware.func1.1"

[f1-outsourcing]

Do you have a work-a-round for this?

[mjuraga]

can you paste the API calls to reproduce this?

[f1-outsourcing]

local jsonhdr='Content-Type: application/json'
local url="v2/services/haproxy/configuration/acls/$aclindex?parent_name=$frontend&parent_type=frontend&version=$confver"
local jsondata='{"acl_name":"'${aclarr[1]}'","criterion": "src", "index": '$aclindex', "value":"'$aclips'"}'


curl -s -X PUT --user $user:$userpasswd -H "$jsonhdr" -d "$jsondata"  http://$haproxy:$dpport/$url

[f1-outsourcing]

Hi @mjuraga Marko, I was wondering what is the status of this? Maybe some background info so I can use some work-around?

[f1-outsourcing]

Do you have a work-a-round for this? Because now I am copying and reloading the config file to a container just to change one ip. 4-5 commands I could just replace by one

[mjuraga]

Hey @f1-outsourcing I think I found the issue, fix is coming soon. Thank you for your help!

[f1-outsourcing]

Cool!!!! Very nice!

[mjuraga]

This is the commit that should fix it: https://github.com/haproxytech/dataplaneapi/commit/e267b5e2e4420ec5429a72b0caeb2bcf8e9b42fa

[f1-outsourcing]

I normally use the releases.

I had to add to the startup script --restart-cmd haproxy-reload otherwise I would get a startup error.

This output is to stdout instead of stderr configuration error: the custom reload strategy requires these options to be set: ReloadCmd, RestartCmd

When I build from master, I am getting these startup errors. My binary is also 39MB, it used to be 26MB.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0xcb2444]

goroutine 1 [running]:
github.com/haproxytech/dataplaneapi/configuration.copyConfigurationToStorage(0xc000380f00)
     github.com/haproxytech/dataplaneapi/configuration/configuration_storage.go:532 +0x17c4
github.com/haproxytech/dataplaneapi/configuration.(*Configuration).Save(0xc000380f00)
     github.com/haproxytech/dataplaneapi/configuration/configuration.go:282 +0xa5
main.startServer(0xc000380f00)
     github.com/haproxytech/dataplaneapi/cmd/dataplaneapi/main.go:187 +0x176c
main.main()
     github.com/haproxytech/dataplaneapi/cmd/dataplaneapi/main.go:59 +0x3d

[f1-outsourcing]

Works perfectly now!!! Thanks a lot

[mjuraga]

Thank you for the patience.