Getting sql_exporter.yml: no such file or directory exception while running this app

[debanitaghosh]

Hello,

I am getting the below exception after doing go install -

I0506 16:57:12.334059 22679 main.go:52] Starting SQL exporter (version=, branch=, revision=) (go=go1.14.2, user=, date=) I0506 16:57:12.334665 22679 config.go:18] Loading configuration from sql_exporter.yml F0506 16:57:12.334701 22679 main.go:56] Error creating exporter: open sql_exporter.yml: no such file or directory goroutine 1 [running]: github.com/golang/glog.stacks(0xc00026dd00, 0xc000274000, 0x74, 0xd0) /Users/debanitaghosh/go/pkg/mod/github.com/golang/[email protected]/glog.go:769 +0xb8 github.com/golang/glog.(*loggingT).output(0x1cb7a40, 0xc000000003, 0xc000262a80, 0x1c500ea, 0x7, 0x38, 0x0) /Users/debanitaghosh/go/pkg/mod/github.com/golang/[email protected]/glog.go:720 +0x372 github.com/golang/glog.(*loggingT).printf(0x1cb7a40, 0x3, 0x16cd763, 0x1b, 0xc00017dee8, 0x1, 0x1) /Users/debanitaghosh/go/pkg/mod/github.com/golang/[email protected]/glog.go:655 +0x14b github.com/golang/glog.Fatalf(...) /Users/debanitaghosh/go/pkg/mod/github.com/golang/[email protected]/glog.go:1148 main.main() /Users/debanitaghosh/go/pkg/mod/github.com/free/[email protected]/cmd/sql_exporter/main.go:56 +0x542

But i do have sql_exporter.yml under documentation folder. Not sure why i am getting this error.

Thanks in advance.

[debanitaghosh]

I'm trying to use this app for getting metrics for MySQL and Oracle SQL DB.

[burningalchemist]

@debanitaghosh sql-exporter.yml by default is expected to be next to the executable, not in documentation folder. As an alternative, you can specify location with -config.file parameter. Let me know if I can help. :)

[paarth1995]

@debanitaghosh Were you able to get this to work. If so how did you do it? I am running into the same issue.

[burningalchemist]

Hey @paarth1995, please check my message. You need to copy sql_exporter.yml from the documentation folder to the place where the executable resides. Otherwise, use -config.file parameter to provide the config path.

[rashmi-kj]

@burningalchemist I am using sql_exporter as an image. How can I add this config in deployment manifest?

[burningalchemist]

Hi, if you're working with Kubernetes, you can to provide the config files via the ConfigMap object, mount it to the path where sql_exporter resides.

Otherwise you can mount it anywhere and provide the path via the -config.file parameter in your manifest.

[rashmi-kj]

@burningalchemist Yes, I am working with Kubernetes and using docker image for sql_exporter. Shouldn't this config be already present in the image itself? My exporter pod is crashing and gives me the same error "no such file or directory". I was trying to add db credentials as env variables which didn't work. Could you let me know what is happening here?

[burningalchemist]

@rashmi-kj According to the Dockerfile [1] the only present thing in the image is the binary. The rest should be attached from the user side. Please take a look at the example [2]. This file should be mounted to the location where sql_exporter binary exists.

I'd suggest though to create ConfigMap with the content of sql_exporter.yaml, mount it to the folder /config/sql_exporter.yaml and then provide it in the command: -config.file=/config/sql_exporter.yaml. You should be good to go then.

[1] https://github.com/free/sql_exporter/blob/master/Dockerfile [2] https://github.com/free/sql_exporter/blob/master/examples/sql_exporter.yml

[rashmi-kj]

Thanks @burningalchemist . I created ConfigMap still the same error.

I0824 08:56:45.260241 1 config.go:18] Loading configuration from sql_exporter.yml F0824 08:56:45.260259 1 main.go:56] Error creating exporter: open sql_exporter.yml: no such file or directory goroutine 1 [running]: github.com/free/sql_exporter/vendor/github.com/golang/glog.stacks(0xc00014de00, 0xc00013e280, 0x74, 0x123) /go/src/github.com/free/sql_exporter/vendor/github.com/golang/glog/glog.go:769 +0xb8

[burningalchemist]

@rashmi-kj Could you please share your Kubernetes manifest? Specifically for sql_exporter.

[rashmi-kj]

@burningalchemist Thanks that was helpful. There was a problem with mountPath I provided /config/sql_exporter.yml instead of /config. The exporter pod is up but it is not collecting metrics now. I did a port-forward on service of exporter application. It comes as blank screen.

I think there is an issue with the connection string here.

[burningalchemist]

@rashmi-kj yes, it could be, could you provide some details on the connection string? You can use some random names, but I'm interested in the format of DSN.

[rashmi-kj]

sqlserver://user:password@host:port This is the format which I am using. I am using name of the service as host. 'sqlserver://username:password@mssqldb:1433'

[burningalchemist]

@rashmi-kj the format seems correct, but according to the logs your error happens because there is no such DNS entry associated with the host, it's a network issue. If your MSSQL is in the different namespace, then you need to access with a different DNS rather than if sql_exporter and db share the same namespace in Kubernetes.

If your DB is hosted outside of Kubernetes, you need to make sure that your cluster has the access to the outside. Just to make sure - you're trying to get connected to the Microsoft SQL Server instance?

[rashmi-kj]

@burningalchemist Yes, both are in different namespaces. What should be the connection string in that case?

[burningalchemist]

Services in Kubernetes expose their endpoint using a common DNS pattern. It looks like this:

<Service Name>.<Namespace Name>.svc.cluster.local

Normally, you just need the Service’s name and DNS will automatically resolve to the full address. However, if you need to access a Service in another Namespace just use the Service name plus the Namespace name.

For example, if you want to connect to the “database” service in the “test” namespace, you can use the following address:

database.test

If you want to connect to the “database” service in the “production” namespace, you can use the following address:

database.production

Helpful link: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#namespaces-of-services

[rashmi-kj]

@burningalchemist This has really improved my understanding. I have my sql instances on azure(SQL Databases). I am using the server name provided by them something like abcd-defg.database.windows.net. Any suggestions?

[burningalchemist]

@rashmi-kj This means that you run your DB instances outside of Kubernetes. In this case, you need to use the format: sqlserver://username:[email protected]:1433, which you probably already do. At the same time, make sure that pods in your Kubernetes cluster have access outside of Kubernetes. This might require some debugging. In fact, I'd deploy a busybox pod and run some curl commands towards your DB instance to confirm that you have network access there and adjust configuration in Azure if not. Once it works with the busybox, it should also work for sql_exporter. :+1:

[rashmi-kj]

@burningalchemist Thanks it worked but there is some other issue which I am facing now. I had removed collector configurations from my sql_exporter.yaml assuming that they were optional. Below two lines.

Collectors (referenced by name) to execute on the target.

collectors: [mssql_standard]

Collector files specifies a list of globs. One collector definition is read from each matching file.

collector_files:

• "*.collector.yml"

Now my exporter application is not scrapping any metric. I tried port-forwarding to see the metrics. Is this happening due to missing collector configuration? What is the importance of collectors?

[rashmi-kj]

@burningalchemist Many Thanks for your guidance. It helped me a lot.

[burningalchemist]

@rashmi-kj Yeah, collector configuration should be specified. In fact, you can specify it in the sql_exporter.yaml file as well. Or in a separate file of a certain mask, e.g. *.collector.yml. I don't have an example at hand, so I'd recommend try things out. You can spin up locally any other database, like MySQL and try with it. Apart from the connection string things should be the same.

Hope it helps. :+1: