terraform-provider-grafana icon indicating copy to clipboard operation
terraform-provider-grafana copied to clipboard

Published 20 hours ago • verified publisher icon grafana

Organization is created even though the API return 403 - permission denied

Open emilmark-wowgroup opened this issue 4 years ago • 12 comments

Hi there,

We are trying to configure an organization but we get permission denied. The organization is created in Grafana and the terraform state contain the resource. (If we run plan or apply after this, we get permission denied again).

Terraform Version

Terraform v0.14.2

  • provider registry.terraform.io/grafana/grafana v1.8.1
  • provider registry.terraform.io/hashicorp/aws v3.24.1
  • provider registry.terraform.io/terraform-providers/grafana v1.8.1

Affected Resource(s)

Please list the resources as a list, for example:

  • grafana_organization

Terraform Configuration Files

provider "grafana" {
  url = local.grafana_url
  auth   = "redacted"
  org_id = 1
}

resource "grafana_organization" "test" {
  name = "Test"
  create_users = false
}

Terraform Output

grafana_organization.test: Creating...

Error: status: 403, body: {"message":"Permission denied"}

  on grafana_provisioning.tf line 12, in resource "grafana_organization" "test":
  12: resource "grafana_organization" "hi" {


Releasing state lock. This may take a few moments...

Steps to Reproduce

  1. terraform apply

Important Factoids

Running on Fargate + RDS with the following env. vars:

AWS_SDK_LOAD_CONFIG | true GF_AUTH_BASIC_ENABLED | true GF_AUTH_SIGV4_AUTH_ENABLED | true GF_DATABASE_HOST | redacted.eu-west-1.rds.amazonaws.com:3306 GF_DATABASE_TYPE | mysql GF_DATABASE_USER | grafana GF_INSTALL_PLUGINS | grafana-clock-panel,grafana-worldmap-panel,mtanda-heatmap-epoch-panel,grafana-piechart-panel,grafana-polystat-panel GF_PANELS_DISABLE_SANITIZE_HTML | false GF_SECURITY_ADMIN_USER | admin GF_SERVER_DOMAIN | redacted.example.com GF_SERVER_ROOT_URL | %(protocol)s://%(domain)s:%(http_port)s/grafana/ GF_SERVER_SERVE_FROM_SUB_PATH | true GF_USERS_ALLOW_ORG_CREATE | true GF_USERS_ALLOW_SIGN_UP | false

emilmark-wowgroup avatar Feb 03 '21 09:02 emilmark-wowgroup

Forgot to mention that we were using an API key as auth

emilmark-wowgroup avatar Feb 03 '21 12:02 emilmark-wowgroup

same behaviour with grafana_team resources. resources are created, but 403 return code makes them tainted.


t=2021-02-10T15:57:18+0000 lvl=dbug msg="Applying default URL parsing for this data source type" logger=datasource type=prometheus url=http://prometheusserver:8082  
t=2021-02-10T15:57:18+0000 lvl=dbug msg="Not adding CSP header to response since it's disabled" logger=http.server
t=2021-02-10T15:57:18+0000 lvl=dbug msg="Not adding CSP header to response since it's disabled" logger=http.server
t=2021-02-10T15:57:18+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=1 uname= method=GET path=/api/users status=403 remote_addr=xxx.yyy.zzz.www time_ms=25 size=31 referer=
t=2021-02-10T15:57:18+0000 lvl=dbug msg="Not adding CSP header to response since it's disabled" logger=http.server
t=2021-02-10T15:57:18+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=1 uname= method=GET path=/api/users status=403 remote_addr=xxx.yyy.zzz.www time_ms=35 size=31 referer=
t=2021-02-10T15:57:18+0000 lvl=dbug msg="Not adding CSP header to response since it's disabled" logger=http.server
t=2021-02-10T15:57:18+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=1 uname= method=GET path=/api/users status=403 remote_addr=xxx.yyy.zzz.www time_ms=27 size=31 referer=
t=2021-02-10T15:57:18+0000 lvl=dbug msg="Not adding CSP header to response since it's disabled" logger=http.server
t=2021-02-10T15:57:18+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=1 uname= method=GET path=/api/users status=403 remote_addr=xxx.yyy.zzz.www time_ms=24 size=31 referer=
t=2021-02-10T15:57:18+0000 lvl=dbug msg="Not adding CSP header to response since it's disabled" logger=http.server
t=2021-02-10T15:57:20+0000 lvl=dbug msg="Scheduling update" logger=alerting.scheduler ruleCount=0

tested with terraform 0.13.2 & 0.14.5 tested with provider 1.7.0 & 1.8.1

token is an Admin enabled api key.

keisari-ch avatar Feb 10 '21 16:02 keisari-ch

And same here ...

resource "grafana_team" "test" {
  name = "test"
}

2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: 2021/02/20 11:33:08 [DEBUG] Grafana API Request Details:
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: ---[ REQUEST ]---------------------------------------
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: GET /api/teams/12/members HTTP/1.1
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Host: my.grafana
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: User-Agent: Go-http-client/1.1
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Authorization: ......
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Content-Type: application/json
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: X-Grafana-Org-Id: 1
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Accept-Encoding: gzip
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: 
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: 
2021-02-20T11:33:08.632+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: -----------------------------------------------------
2021-02-20T11:33:08.808+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: 2021/02/20 11:33:08 [DEBUG] Grafana API Response Details:
2021-02-20T11:33:08.808+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: ---[ RESPONSE ]--------------------------------------
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: HTTP/1.1 403 Forbidden
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Content-Length: 31
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Cache-Control: no-cache
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Connection: keep-alive
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Content-Type: application/json; charset=UTF-8
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Date: Sat, 20 Feb 2021 10:33:08 GMT
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Expires: -1
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Pragma: no-cache
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: Server: nginx/1.19.2
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: X-Content-Type-Options: nosniff
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: X-Frame-Options: deny
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: X-Xss-Protection: 1; mode=block
2021-02-20T11:33:08.809+0100 [DEBUG] plugin.terraform-provider-grafana_v1.8.1.exe: ```

michalformanek avatar Feb 20 '21 10:02 michalformanek

use basic auth instead of token

ytsuhar avatar Mar 03 '21 10:03 ytsuhar

we do, but it's still a bug since the resources is created even though we get a 403 response (when using API key)

emilmark-wowgroup avatar Mar 03 '21 10:03 emilmark-wowgroup

I'm facing the same issue. Same versions, both auth via username/password and token.

wgebis avatar Mar 19 '21 13:03 wgebis

Hi, our issue from my previous comment was caused by wrong grafana configuration. We are using Rancher, ldap and prometheus operator. In our case (with mostly default setup) basic auth in grafana.ini was disabled. After enabling it, provider works as expected. It is weird, because some endpoints were working fine even with basic auth disabled...

michalformanek avatar Mar 19 '21 13:03 michalformanek

I have enabled basic auth:

#################################### Basic Auth ##########################
[auth.basic]
enabled = true

Both method fail :(. It worked in previous version of provider, when the Grafana objects have been created, but now I'm not able to modify them due to this issue.

wgebis avatar Mar 19 '21 14:03 wgebis

Prior 1.8.0 terraform plan and terraform apply work as expected without any 403 issues.

wgebis avatar Mar 19 '21 14:03 wgebis

I noticed, that it's related to org_id param. Then the param is passed in provider config, then permissions errors disappear.

wgebis avatar Jun 09 '21 12:06 wgebis

I am also having the same issue, setting up the admin user and password via env vars then having terraform create an org I get 403 when trying to configure the data source or anything with in that org.

logan-bobo avatar Dec 29 '21 11:12 logan-bobo

As @wgebis stated if you select the org you want to manage at the provider level the permission issue will go away. So if you want to create an org and then manage that org you will need two providers and use an alias. One for the base of Grafana and then a second where you specify the org ID at the provider level. You can achieve this with provider aliases. This has worked for me and I now have a fully managed Grafana via terraform

logan-bobo avatar Jan 02 '22 13:01 logan-bobo

Related to https://github.com/grafana/terraform-provider-grafana/issues/747, making the org_id setting easier to understand, or getting rid of it at a provider level

julienduchesne avatar Jan 10 '23 03:01 julienduchesne