grafana-aws-athena-datasource icon indicating copy to clipboard operation
grafana-aws-athena-datasource copied to clipboard

Published 20 hours ago • verified publisher icon mtanda

"Metric request error" when query fails due to permission issues

Open faridnsh opened this issue 4 years ago • 0 comments

What happened was that my query had permission issues:

{
    "QueryExecutions": [
        {
            "QueryExecutionId": "<redacted>",
            "Query": "<redacted>",
            "StatementType": "DML",
            "ResultConfiguration": {
                "OutputLocation": "<redacted>"
            },
            "QueryExecutionContext": {},
            "Status": {
                "State": "FAILED",
                "StateChangeReason": "User: arn:aws:sts::<redacted>:assumed-role/<redacted> is not authorized to perform: athena:GetDataCatalog on resource: <redacted>",
                "SubmissionDateTime": "2021-07-21T15:35:28.336000+02:00",
                "CompletionDateTime": "2021-07-21T15:35:28.421000+02:00"
            },
            "Statistics": {
                "TotalExecutionTimeInMillis": 85,
                "QueryQueueTimeInMillis": 85
            },
            "WorkGroup": "primary"
        }
    ],
    "UnprocessedQueryExecutionIds": []
}

And that caused the following line to choke: https://github.com/mtanda/grafana-aws-athena-datasource/blob/4af26402677617f9ec709b4c3a5afbc0349fc649/pkg/query.go#L223

With the following error:

t=2021-07-21T13:35:28+0000 lvl=dbug msg="Querying for data source via SQL store" logger=datasources id=1 orgId=1
t=2021-07-21T13:35:29+0000 lvl=dbug msg="panic: runtime error: invalid memory address or nil pointer dereference" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xcba944]" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg= logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="goroutine 8 [running]:" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="main.(*AwsAthenaQuery).waitForQueryCompleted(0xc000103580, 0x1183d80, 0xc000280e10, 0xc0000b4038, 0x1, 0x1, 0x0, 0x1)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/work/grafana-aws-athena-datasource/grafana-aws-athena-datasource/pkg/query.go:223 +0x1c4" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="main.(*AwsAthenaQuery).getQueryResults(0xc000103580, 0x1183d80, 0xc000280e10, 0x1, 0xc000038380, 0x1c, 0xc00007a680, 0x0, 0xc0002a4b40, 0x0, ...)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/work/grafana-aws-athena-datasource/grafana-aws-athena-datasource/pkg/query.go:96 +0x1265" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="main.(*AwsAthenaDatasource).QueryData(0xc000213a90, 0x1183d80, 0xc000280e10, 0xc00007fb80, 0xdfeea0, 0x116c5c0, 0xbe2d9b)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/work/grafana-aws-athena-datasource/grafana-aws-athena-datasource/pkg/datasource.go:154 +0x1a7" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="github.com/grafana/grafana-plugin-sdk-go/backend.(*dataSDKAdapter).QueryData(0xc000213bd0, 0x1183d80, 0xc000280e10, 0xc00007e4b0, 0xdfeea0, 0x0, 0x116c5c0)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/github.com/grafana/[email protected]/backend/data_adapter.go:21 +0x63" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="github.com/grafana/grafana-plugin-sdk-go/backend/grpcplugin.(*dataGRPCServer).QueryData(0xc000213e80, 0x1183d80, 0xc000280e10, 0xc00007e4b0, 0xc000213e80, 0x7f490dde2030, 0xc00007a640)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/github.com/grafana/[email protected]/backend/grpcplugin/grpc_data.go:47 +0x51" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="github.com/grafana/grafana-plugin-sdk-go/genproto/pluginv2._Data_QueryData_Handler.func1(0x1183d80, 0xc000280e10, 0xf30600, 0xc00007e4b0, 0x18, 0xc00007fb30, 0x203000, 0xc00027e000)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/github.com/grafana/[email protected]/genproto/pluginv2/backend.pb.go:1306 +0x86" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1(0x1183d80, 0xc000280e10, 0xf30600, 0xc00007e4b0, 0xc00000e860, 0xc00000e880, 0xc61e8a, 0xed3880, 0xc00000e8a0, 0xc00000e860)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/server_metrics.go:107 +0xad" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1(0x1183d80, 0xc000280e10, 0xf30600, 0xc00007e4b0, 0xc000282600, 0x0, 0xc0002b5b18, 0x40e888)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25 +0x63" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1(0x1183d80, 0xc000280e10, 0xf30600, 0xc00007e4b0, 0xc00000e860, 0xc00000e880, 0xc000067b88, 0x51dd48, 0xf08360, 0xc000280e10)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:34 +0xd5" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="github.com/grafana/grafana-plugin-sdk-go/genproto/pluginv2._Data_QueryData_Handler(0xdfeea0, 0xc000213e80, 0x1183d80, 0xc000280e10, 0xc00007cd80, 0xc00023ad20, 0x1183d80, 0xc000280e10, 0xc0002b62c0, 0x29c)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/github.com/grafana/[email protected]/genproto/pluginv2/backend.pb.go:1308 +0x14b" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002364e0, 0x118bdc0, 0xc00008d980, 0xc000282600, 0xc00023b4d0, 0x1895ff0, 0x0, 0x0, 0x0)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/google.golang.org/[email protected]/server.go:1082 +0x50a" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="google.golang.org/grpc.(*Server).handleStream(0xc0002364e0, 0x118bdc0, 0xc00008d980, 0xc000282600, 0x0)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/google.golang.org/[email protected]/server.go:1405 +0xcc9" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0000b3eb0, 0xc0002364e0, 0x118bdc0, 0xc00008d980, 0xc000282600)" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/google.golang.org/[email protected]/server.go:746 +0xa1" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="created by google.golang.org/grpc.(*Server).serveStreams.func1" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="\t/home/runner/go/pkg/mod/google.golang.org/[email protected]/server.go:744 +0xa1" logger=plugins.backend pluginId=mtanda-aws-athena-datasource
t=2021-07-21T13:35:29+0000 lvl=dbug msg="plugin process exited" logger=plugins.backend pluginId=mtanda-aws-athena-datasource path=/opt/bitnami/grafana/data/plugins/grafana-aws-athena-datasource/aws-athena-plugin_linux_amd64 pid=29 error="exit status 2"
t=2021-07-21T13:35:29+0000 lvl=eror msg="Metric request error" logger=context userId=1 orgId=1 uname=admin error="failed to query data: Failed to query data: rpc error: code = Unavailable desc = transport is closing" remote_addr=31.20.25.13

I believe this is happening since the code is accessing e.Statistics.DataScannedInBytes, but the e.Statistics doesn't have a DataScannedInBytes field.

I think it'd be nice instead to forward the error message(StateChangeReason). I'd be happy to make a PR for this, however I'm not sure how to write that in idiomatic go.

faridnsh avatar Jul 21 '21 13:07 faridnsh