vault icon indicating copy to clipboard operation
vault copied to clipboard

Published 20 hours ago • verified publisher icon hashicorp

Allow `identity` mount to be tunable

Open ianferguson opened this issue 2 years ago • 5 comments

resolves #14696

I didn't expect this to be so simple, but it seems like removing identity from the "non tunable" mounts list just works, at least for the audit hmac tuning I've tested.

There may be other impacts I'm not aware of due to this, if y'all can think of any you want me to test for specifically please let me know, I'm happy to do so

Testing

Setup

I built my branch locally and started a Vault test server

> bin/vault server -dev -dev-root-token-id=test

Execution

In another shell I ran the follow commands to enable audit logs, write some groups, tune identity to allow the name field to be printed in plaintext in audit logs, and then write some groups again:

# turn on audit log
>  vault audit enable file file_path=/tmp/vault.log
Success! Enabled the file audit device at: file/

# make read/write requests to show that by default audit logs still hmac the identity fields
> vault write identity/group name=a-group
Key     Value
---     -----
id      5fdab8ca-5127-97c6-38ce-0467373f91ad
name    a-group

> vault read identity/group/name/a-group
Key                  Value
---                  -----
alias                map[]
creation_time        2022-03-25T13:48:42.909741Z
id                   5fdab8ca-5127-97c6-38ce-0467373f91ad
last_update_time     2022-03-25T13:48:42.909741Z
member_entity_ids    []
member_group_ids     <nil>
metadata             <nil>
modify_index         1
name                 a-group
namespace_id         root
parent_group_ids     <nil>
policies             <nil>
type                 internal

# tune the identity mount to allow `name` to print
> vault secrets tune -audit-non-hmac-request-keys=name -audit-non-hmac-response-keys=name identity
Success! Tuned the secrets engine at: identity/

# make the same read/write requests for another group to show that now the audit logs print the `name` in response/request audit events 

> vault write identity/group name=b-group
Key     Value
---     -----
id      ff5011a0-d904-d098-63b2-501d9f497c86
name    b-group

> vault read identity/group/name/b-group
Key                  Value
---                  -----
alias                map[]
creation_time        2022-03-25T13:49:06.218824Z
id                   ff5011a0-d904-d098-63b2-501d9f497c86
last_update_time     2022-03-25T13:49:06.218824Z
member_entity_ids    []
member_group_ids     <nil>
metadata             <nil>
modify_index         1
name                 b-group
namespace_id         root
parent_group_ids     <nil>
policies             <nil>
type                 internal

Results: How that looked in the audit logs

Before tuning (name in request/response is hmac'd)
{
  "request": {
    "id": "a880573c-e072-af6c-64cc-dc63b127aeef",
    "operation": "read",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group/name/a-group",
    "remote_address": "127.0.0.1",
    "remote_port": 62417
  },
  "response": {
    "mount_type": "identity",
    "data": {
      "alias": {},
      "creation_time": "hmac-sha256:bf9f710dc5a8acd12925d6ece25a52cc3c3168bab8f7dfbafc7929a8c6a8fe2c",
      "id": "hmac-sha256:bcea8f768a75e7c291f27761ccc25f6b988fde6d61e322f077f4592ed8764e49",
      "last_update_time": "hmac-sha256:bf9f710dc5a8acd12925d6ece25a52cc3c3168bab8f7dfbafc7929a8c6a8fe2c",
      "member_entity_ids": [],
      "member_group_ids": null,
      "metadata": null,
      "modify_index": 1,
      "name": "hmac-sha256:c145de2c8cef9005a0677f858fbb89f988addc7f868cb06dad63a4cfad54ae0d",
      "namespace_id": "hmac-sha256:d2d2bee396f2bb4526845aa0736bc241d857de704d4a9d460031248c9bad2a2d",
      "parent_group_ids": null,
      "policies": null,
      "type": "hmac-sha256:793110e4ab7e6151e89f1f26aad53d4470774ec189380538abc0d0090faf65be"
    }
  }
}
{
  "time": "2022-03-25T13:48:57.720818Z",
  "type": "request",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "b09197a2-57e8-82f2-1540-60db5621a03d",
    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
    "operation": "update",
    "mount_type": "system",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "sys/mounts/identity/tune",
    "data": {
      "audit_non_hmac_request_keys": [
        "hmac-sha256:5e65419a2601b1d743429f0f63c13e54250141211ce7c7c4be0aa95bb931a3af"
      ],
      "audit_non_hmac_response_keys": [
        "hmac-sha256:5e65419a2601b1d743429f0f63c13e54250141211ce7c7c4be0aa95bb931a3af"
      ],
      "default_lease_ttl": "hmac-sha256:db1aae827d2f648e42d41235c83a3efebe28825349fb68d690662dc7fa15040f",
      "force_no_cache": false,
      "max_lease_ttl": "hmac-sha256:db1aae827d2f648e42d41235c83a3efebe28825349fb68d690662dc7fa15040f",
      "options": null
    },
    "remote_address": "127.0.0.1",
    "remote_port": 62418
  }
}
After tuning (name is printed in plaintext`):
{
  "time": "2022-03-25T13:49:06.218756Z",
  "type": "request",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "7b19034c-3cc4-c859-b314-98d4f0b46db7",
    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
    "operation": "update",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group",
    "data": {
      "name": "b-group"
    },
    "remote_address": "127.0.0.1",
    "remote_port": 62419
  }
}
{
  "time": "2022-03-25T13:49:06.218989Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "7b19034c-3cc4-c859-b314-98d4f0b46db7",
    "operation": "update",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group",
    "data": {
      "name": "b-group"
    },
    "remote_address": "127.0.0.1",
    "remote_port": 62419
  },
  "response": {
    "mount_type": "identity",
    "data": {
      "id": "hmac-sha256:aa7c5e0a8c96bb3a17b6f90e6c6f12e5c8c362e611c9014ef5426603f06d3c9b",
      "name": "b-group"
    }
  }
}
{
  "time": "2022-03-25T13:49:09.954194Z",
  "type": "request",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "41ebb69e-828e-b8fe-48a2-c73d22d58d78",
    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
    "operation": "read",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group/name/b-group",
    "remote_address": "127.0.0.1",
    "remote_port": 62420
  }
}
{
  "time": "2022-03-25T13:49:09.954345Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2022-03-25T09:48:26-04:00"
  },
  "request": {
    "id": "41ebb69e-828e-b8fe-48a2-c73d22d58d78",
    "operation": "read",
    "mount_type": "identity",
    "client_token": "hmac-sha256:8b4822c1345b3e43dba5d0aa12080981f593e478ae3dfcc232036cce52e2cd0e",
    "client_token_accessor": "hmac-sha256:7c584491e9f651d53424048d122fe9ec93297ac37900e2077891bb189df4454b",
    "namespace": {
      "id": "root"
    },
    "path": "identity/group/name/b-group",
    "remote_address": "127.0.0.1",
    "remote_port": 62420
  },
  "response": {
    "mount_type": "identity",
    "data": {
      "alias": {},
      "creation_time": "hmac-sha256:3cda9fccd26e9736444df512fc1838084bcc8e1a9df68c68a4aa0ff69a97c993",
      "id": "hmac-sha256:aa7c5e0a8c96bb3a17b6f90e6c6f12e5c8c362e611c9014ef5426603f06d3c9b",
      "last_update_time": "hmac-sha256:3cda9fccd26e9736444df512fc1838084bcc8e1a9df68c68a4aa0ff69a97c993",
      "member_entity_ids": [],
      "member_group_ids": null,
      "metadata": null,
      "modify_index": 1,
      "name": "b-group",
      "namespace_id": "hmac-sha256:d2d2bee396f2bb4526845aa0736bc241d857de704d4a9d460031248c9bad2a2d",
      "parent_group_ids": null,
      "policies": null,
      "type": "hmac-sha256:793110e4ab7e6151e89f1f26aad53d4470774ec189380538abc0d0090faf65be"
    }
  }
}

ianferguson avatar Mar 25 '22 13:03 ianferguson

Hi there! Thanks for this contribution. I'll get some eyes on it, but please don't forget to add a changelog entry. Thanks! :)

hsimon-hashicorp avatar Apr 04 '22 19:04 hsimon-hashicorp

@hsimon-hashicorp thanks! changelog added in e9114b5a9

ianferguson avatar Apr 06 '22 17:04 ianferguson

I happened to notice this, and wondered - is there any reason to keep the concept of an "untunable mount" at all?

https://github.com/hashicorp/vault/blob/254d8da255699ddebf11799431c9edb8f96a19ae/vault/mount.go#L86-L91

Wanting to tune fields displayed in the audit logs for sys/ seems quite possible. For cubbyhole/ it's less likely but technically possibly a useful debugging technique. And audit/ isn't even a path used by Vault at all, AFAIK - I'm guessing someone just stuck it in here whilst contemplating making audit backends more like secrets engines and auth methods, but the change was never implemented?

Just something to think about whilst people's thoughts are in this area.

maxb avatar May 08 '22 10:05 maxb

I happened to notice this, and wondered - is there any reason to keep the concept of an "untunable mount" at all?

https://github.com/hashicorp/vault/blob/254d8da255699ddebf11799431c9edb8f96a19ae/vault/mount.go#L86-L91

Wanting to tune fields displayed in the audit logs for sys/ seems quite possible. For cubbyhole/ it's less likely but technically possibly a useful debugging technique. And audit/ isn't even a path used by Vault at all, AFAIK - I'm guessing someone just stuck it in here whilst contemplating making audit backends more like secrets engines and auth methods, but the change was never implemented?

Just something to think about whilst people's thoughts are in this area.

Hi @maxb , some mounts such as the identity mount are more sensitive than others. As such, we'd like to make these mounts as standardized as possible from environment to environment to reduce the chances of complicated bugs arising due to, say, tuning.

HridoyRoy avatar May 10 '22 19:05 HridoyRoy

@HridoyRoy just getting back to this now

Reading the mount tuning table on Vault v1.11.1 several of the tuning values are already set by default:

> vault read sys/mounts/identity/tune
Key                            Value
---                            -----
default_lease_ttl              768h
description                    identity store
force_no_cache                 false
max_lease_ttl                  768h
passthrough_request_headers    [Authorization]

looking at the secrets mount tuning parameters they all seem like they'd have non-catastrophic and reasonable behaviors when set

I tested tuning all possible values on a build of this branch:

$ vault secrets tune \
  -allowed-response-headers="Authorization" \
  -audit-non-hmac-request-keys="id" \ 
  -audit-non-hmac-response-keys="id" \
  -default-lease-ttl="769h" \
  -description="Custom identity" \
  -listing-visibility="hidden" \
  -max-lease-ttl="780h" \
  -passthrough-request-headers "Authorization" \
  -allowed-managed-keys "something" \ 
  identity/
Success! Tuned the secrets engine at: identity/

$ vault read sys/mounts/identity/tune
Key                             Value
---                             -----
allowed_managed_keys            [something]
allowed_response_headers        [Authorization]
audit_non_hmac_request_keys     [id]
audit_non_hmac_response_keys    [id]
default_lease_ttl               769h
description                     Custom identity
force_no_cache                  false
listing_visibility              hidden
max_lease_ttl                   780h
passthrough_request_headers     [Authorization]

and at least a few entity create/lookup type commands all worked fine before and after those tuning settings.

I think it should be safe for the identity mount to be tuned the same as a "normal" secrets mount

ianferguson avatar Jul 28 '22 20:07 ianferguson

@HridoyRoy would you or someone else be able to review my last response? https://github.com/hashicorp/vault/pull/14723#issuecomment-1198603434

ianferguson avatar Sep 15 '22 21:09 ianferguson