azure-docs icon indicating copy to clipboard operation
azure-docs copied to clipboard

/var/aziot/secrets is not readable by user aziotks (uid 997, gid 998)

Open vdasovic opened this issue 2 years ago • 9 comments

Copy certificates to device

Step 3. Change the ownership and permissions of the certificates and keys.

Documentation states ownership and permissions which must be set on /var/aziot/secrets and /var/aziot/certs directories and files. If we follow the guidelines iotedge check fails with following message:

× read all preloaded key pairs from the Keys Service - Error
    /var/aziot/secrets is not readable by user aziotks (uid 997, gid 998)

Granting ownership to aziotks, or aziotcs, or iotedge group and users does not resolve this issue. Only solution I've found was to set permissions to /var/aziot/secrets directory to 777


Document Details

Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

vdasovic avatar Dec 12 '22 11:12 vdasovic

@vdasovic Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

AshokPeddakotla-MSFT avatar Dec 12 '22 12:12 AshokPeddakotla-MSFT

@jlian ,

Do you have any thoughts on this issue?

Thanks, Pat

PatAltimore avatar Dec 13 '22 22:12 PatAltimore

@PatAltimore are you able to get a repro on your end?

jlian avatar Dec 13 '22 23:12 jlian

@jlian, I haven't tried to repro. I won't be able to take a look until 12/21. I'll give it a try then.

PatAltimore avatar Dec 14 '22 00:12 PatAltimore

@jlian ,

I was able to repro.

I thought I would try setting ownership using sudo chown -R aziotks:aziotks /var/aziot/secrets and sudo chmod -R 644 /var/aziot/secrets/

$ sudo ls -l secrets/
total 16
-rw-r--r-- 1 aziotks aziotks 3326 Dec 14 00:19 azure-iot-test-only.intermediate.key.pem
-rw-r--r-- 1 aziotks aziotks 3326 Dec 14 00:19 azure-iot-test-only.root.ca.key.pem
-rw-r--r-- 1 aziotks aziotks 3243 Dec 14 00:19 iot-edge-device-ca-CACertName.key.pem
-rw-r--r-- 1 aziotks aziotks 1679 Dec 14 00:19 iot-edge-device-identity-patricka-device-msdn2.key.pem

I still get:

× read all preloaded key pairs from the Keys Service - Error /var/aziot/secrets is not readable by user aziotks (uid 997, gid 998)

PatAltimore avatar Dec 14 '22 01:12 PatAltimore

I'm guessing it's related to this https://github.com/Azure/iotedge/issues/6723#issuecomment-1298625044

Where you need to chown and chmod the directories with the right (700-level) permissions:

# Give aziotcs ownership to certificate folder
# Read/write/execute for aziotcs, read and execute access for others
sudo chown aziotcs:aziotcs /var/aziot/certs
sudo chmod 755 /var/aziot/certs

# Give aziotks ownership to private key
# Read/write/execute for aziotks, no permission for others
sudo chown aziotks:aziotks /var/aziot/secrets
sudo chmod 700 /var/aziot/secrets

According to @gordonwang0:

Since those are directories, the permissions should be 755 and 700, as in [...] below. The files in the directories should be 644 and 600 though.

/var/aziot/            root:root       0755​
/var/aziot/certs​       aziotcs:aziotcs 0755​
/var/aziot/secrets    ​ aziotks:aziotks 0700

And with ls:

$ sudo ls /var/aziot -la
total 16
drwxr-xr-x  4 root    root    4096 Dec 13 17:16 .
drwxr-xr-x 21 root    root    4096 Dec 13 17:16 ..
drwxr-xr-x  2 aziotcs aziotcs 4096 Dec 13 17:16 certs
drwx------  2 aziotks aziotks 4096 Dec 13 17:16 secrets

@vdasovic can you see this if this works?

EDIT: fix typo

jlian avatar Dec 14 '22 01:12 jlian

I'm guessing it's related to this Azure/iotedge#6723 (comment)

Where you need to chown and chmod the directories with the right (700-level) permissions:

# Give aziotcs ownership to certificate folder
# Read/write/execute for aziotcs, read and execute access for others
sudo chown aziotcs:aziotcs /var/aziot/certs
sudo chmod 755 /var/aziot/certs

# Give aziotks ownership to private key
# Read/write/execute for aziotks, no permission for others
sudo chown aziotks:aziotks /var/aziot/secrets
sudo chmod 700 /var/aziot/secrets

According to @gordon-msft:

Since those are directories, the permissions should be 755 and 700, as in [...] below. The files in the directories should be 644 and 600 though.

/var/aziot/            root:root       0755​
/var/aziot/certs​       aziotcs:aziotcs 0755​
/var/aziot/secret-keys​ aziotks:aziotks 0700

And with ls:

$ sudo ls /var/aziot -la
total 16
drwxr-xr-x  4 root    root    4096 Dec 13 17:16 .
drwxr-xr-x 21 root    root    4096 Dec 13 17:16 ..
drwxr-xr-x  2 aziotcs aziotcs 4096 Dec 13 17:16 certs
drwx------  2 aziotks aziotks 4096 Dec 13 17:16 secrets

@vdasovic can you see this if this works?

@vdasovic and I are working together on this and I just tested this on one of our edge devices. It seems to be working 😄

I'll continue testing with different x.509 setups and update this issue or open a new one. I would recommend adding these permissions and ownership settings to the official documentation.

elvisalicic avatar Dec 14 '22 12:12 elvisalicic

Oh yeah, that makes sense. Just noticed that last command in documentation changes directory permissions to 644 :smile: No wonder this doesn't work. I suggest fixing step 3 of the documentation to avoid confusing people :)

vdasovic avatar Dec 14 '22 13:12 vdasovic

I would recommend adding these permissions and ownership settings to the official documentation.

Just noticed that last command in documentation changes directory permissions to 644 😄 No wonder this doesn't work.

I suggest fixing step 3 of the documentation to avoid confusing people :)

Agree, we should

  • Create a "file and directory permission requirements" section in Manage IoT Edge certificates doc
  • Thoroughly comb through our existing docs to ensure that, everytime we get the user to touch certificates and private keys for IoT Edge, the permission should be set properly
    • With link back to the overall requirements section from above
    • With a step to get the user to check that the permissions match the requirement, maybe with ls -la

@PatAltimore since we already have this github issue here can we use it as a tracker or should we add it to the backlog some other way

jlian avatar Dec 14 '22 21:12 jlian

FYI the docs update for permissions are now done

https://learn.microsoft.com/azure/iot-edge/how-to-manage-device-certificates#permission-requirements

Thanks @PatAltimore

I think we can now close this issue #please-close

jlian avatar Jan 18 '23 22:01 jlian

The documentation still does not look ok. In the permissions requirements you state that /var/aziot/certs/ certificates directory must have drwxr-xr-x (755) permissions which is correct. Then further down you change the permissions of that directory to 644 and in the output example show them as correct.

# Give aziotcs ownership to certificates
# Read and write for aziotcs, read-only for others
sudo chown -R aziotcs:aziotcs /var/aziot/certs
sudo chmod -R 644 /var/aziot/certs
sudo find /var/aziot/certs -type f -name "*.*" -exec chmod 644 {} \;

And the output example shows 644 permissions as well

azureUser@vm-h2hnm5j5uxk2a:/var/aziot$ sudo ls -Rla /var/aziot
/var/aziot:
total 16
drwxr-xr-x  4 root    root    4096 Dec 14 00:16 .
drwxr-xr-x 15 root    root    4096 Dec 14 00:15 ..
drw-r--r--  2 aziotcs aziotcs 4096 Jan 14 00:31 certs
drwx------  2 aziotks aziotks 4096 Jan 14 00:35 secrets

vdasovic avatar Jan 20 '23 08:01 vdasovic

Hi @vdasovic ,

Thanks for catching the mistake.

The command sudo chmod -R 644 /var/aziot/certs was included by mistake. I removed it. It should publish soon. Without that line, do you agree with the steps?

Thanks, Pat

PatAltimore avatar Jan 20 '23 21:01 PatAltimore

Hi @PatAltimore,

Example of what output from ls -Rla /var/aziot should look like is still incorrect. "The output of list with correct ownership and permission is similar to the following:"

azureUser@vm-h2hnm5j5uxk2a:/var/aziot$ sudo ls -Rla /var/aziot
/var/aziot:
total 16
drwxr-xr-x  4 root    root    4096 Dec 14 00:16 .
drwxr-xr-x 15 root    root    4096 Dec 14 00:15 ..
drw-r--r--  2 aziotcs aziotcs 4096 Jan 14 00:31 certs
drwx------  2 aziotks aziotks 4096 Jan 14 00:35 secrets

/var/aziot/certs:
total 20
drw-r--r-- 2 aziotcs aziotcs 4096 Jan 14 00:31 .
drwxr-xr-x 4 root    root    4096 Dec 14 00:16 ..
-rw-r--r-- 1 aziotcs aziotcs 1984 Jan 14 00:24 azure-iot-test-only.root.ca.cert.pem
-rw-r--r-- 1 aziotcs aziotcs 5887 Jan 14 00:27 iot-device-devicename-full-chain.cert.pem

/var/aziot/secrets:
total 20
drwx------ 2 aziotks aziotks 4096 Jan 14 00:35 .
drwxr-xr-x 4 root    root    4096 Dec 14 00:16 ..
-rw------- 1 aziotks aziotks 3326 Jan 14 00:29 azure-iot-test-only.root.ca.key.pem
-rw------- 1 aziotks aziotks 3243 Jan 14 00:28 iot-device-devicename.key.pem

Directory certs is set to drw-r--r-- (644) while it should be drwxr-xr-x (755)

vdasovic avatar Jan 23 '23 08:01 vdasovic

Hi @vdasovic ,

Good catch! I've fixed the output from ls. Thanks for finding the issue and highlighting the error. The change should publish today.

Thanks, Pat

PatAltimore avatar Jan 23 '23 17:01 PatAltimore