Can not create any user on the chef server after deleting pivotal

[fstrudel]

Hello,

I've created users with the admin rights using: chef-server# chef-server-ctl user-create chef-server# chef-server-ctl user-create myadmin My Admin [email protected] mypassword -f myadmin.pem chef-server# chef-server-ctl org-user-add myorg myadmin --admin

and then deleted the original pivotal user as I did not think this was needed anymore:

chef-server# chef-server-ctl user-delete pivotal

BUT now I can't create any user:

chef-server# chef-server-ctl user-create myotheradmin My OtherAdmin [email protected] mypassword -f myotheradmin.pem -o myorg ERROR: Failed to authenticate to https://127.0.0.1:443 as pivotal with key /etc/opscode/pivotal.pem Response: Failed to authenticate as 'pivotal'. Ensure that your node_name and client key are correct.

OK expected. But it fails also with my previous admin user:

root@chef-server:~# chef-server-ctl user-create myotheradmin My OtherAdmin [email protected] mypassword -f myotheradmin.pem -o vidm -u myadmin -k myadmin.pem ERROR: You authenticated successfully to https://127.0.0.1:443 as myadmin but you are not authorized for this action Response: missing create permission

How do I set the "create" permission for users ? Looks like knife acl does not do that ?

Thanks Fanny

[stevendanna]

To grant create permission for users, you need to change is the CREATE ace for the global users container. Unfortunately, "pivotal" is the only actor in our current default setup that has this permission. @tylercloke has work in flight here: https://github.com/chef/chef-server/pull/475 to improve this situation.

@fstrudel Does this installation already have a lot of data in it? I ask because, the easiest way to proceed would be to do a fresh reinstall. I am sorry to have to recommend such an extreme fix. If this isn't a possibility, I can work up a procedure to reconstruct the pivotal user. (cc @manderson26 can fixie help with this? Perhaps re-running a portion of the initial bootstrap tool will do it).

@chef/lob Any objections to returning a 4XX-style error code on attempts to delete pivotal?

[fstrudel]

@stevendanna I was able to "re-create" the pivotal user by doing some database insert, and I hope I did not forget anything (our chef server has data in it I don't want to rebuild it).

But I agree with you, prevent the admin from deleting this pivotal user would be really helpful, and also giving him a special name like "superuser, DO NOT delete" instead of "Clark Kent" (that would have given me a hint ;-)).

[stevendanna]

@stevendanna I was able to "re-create" the pivotal user by doing some database insert, and I hope I did not forget anything (our chef server has data in it I don't want to rebuild it).

If you send the details of the insert you did we can double check it. I think @jeremiahsnapp is experimenting with the exact steps needed to recreate pivotal.

[fstrudel]

Ok so here the steps I did:

1. Extract the public key for the pivotal user:

root@chef-server:~# openssl rsa -in /etc/opscode/pivotal.pem   -pubout >  /etc/opscode/pivotal.pub

1. Insert the pivotal user with the right authz_id:

root@chef-server:~# su - opscode-pgsql
$ psql opscode_chef
opscode_chef=# insert into users VALUES ('<random UUID>', '5a53dd0b7a6fe862460ababb13244a67', 'pivotal', '[email protected]', 0, '<the content of pivotal.pub>', '{"first_name":"Clark","last_name":"Kent","display_name":"Initial user - DO NOT DELETE"}', '<random UUID>', now(), now());

where '5a53dd0b7a6fe862460ababb13244a67' is the UUID of the authz_id field the other users were created by (or the authz_id value of first row of the auth_actor table in bitfrost database):

opscode-pgsql@chef-server:~$ psql bifrost -c "select authz_id from auth_actor where id=1;"
             authz_id             
----------------------------------
 5a53dd0b7a6fe862460ababb13244a67
(1 row)

[jeremiahsnapp]

@fstrudel That's fantastic that you got all that. Nice work!

I just finished creating my own procedure which ultimately is very similar to yours but its commands are more easily reused by others. I'll leave it hear in case anyone else runs across this issue.

# create pivotal's public key from /etc/opscode/pivotal.pem and store in an accessible location

openssl rsa -in /etc/opscode/pivotal.pem -pubout > /var/opt/opscode/postgresql/9.2/data/pivotal.pub

# get the pivotal user's authz_id and store in an accessible location

echo "SELECT authz_id FROM auth_actor WHERE id = 1" | su -l opscode-pgsql -c 'psql bifrost -tA' | tr -d '\n' > /var/opt/opscode/postgresql/9.2/data/pivotal.authz_id

# create the pivotal user's record 

echo "INSERT INTO users (id, authz_id, username, email, pubkey_version, public_key, serialized_object, last_updated_by, created_at, updated_at) VALUES (md5(random()::text), pg_read_file('pivotal.authz_id'), 'pivotal', '[email protected]', 0, pg_read_file('pivotal.pub'), '{\"first_name\":\"Clark\",\"last_name\":\"Kent\",\"display_name\":\"Clark Kent\"}', pg_read_file('pivotal.authz_id'), LOCALTIMESTAMP, LOCALTIMESTAMP);" | su -l opscode-pgsql -c 'psql opscode_chef'

# delete the temporary files

rm /var/opt/opscode/postgresql/9.2/data/pivotal.pub /var/opt/opscode/postgresql/9.2/data/pivotal.authz_id

[glennmate]

Special thanks to @jeremiahsnapp - your instructions got me out of a snag this morning. Removed the pivotal user in error and was able to quickly recover.

[chilicheech]

Would these instructions work to reset/rotate the pivotal.pem?

[sakkemsu1]

Hi @jeremiahsnapp,

I got below error while I am following your instructions.

ERROR: duplicate key value violates unique constraint "users_authz_id_key" DETAIL: Key (authz_id)=(f2d6358a21a8279d3aa3d04dcb7f0609) already exists.

Thanks, Suresh

[sakkemsu1]

it works for me instead insert data I was able to update public key for 'pivotal' user.

[dvinograd]

I found an easier way to reset pivotal user when pivotal.pem is missing or invalid:

chef-server-ctl stop; rm -f /var/opt/opscode/bootstrapped /etc/opscode/pivotal*; chef-server-ctl reconfigure

This initiates Chef's bootstrap recipe (recipe["private-chef::bootstrap"]) which generates pivotal.pem and inserts it into Postgres. For the recipe to work, Postgresql has to be up and opscode-pgsql must be able to access it.

If bootstrap script isn't running despite these steps, check whether node["private-chef"]["bootstrap"]["enabled"] is set to true (you can see its value in /etc/opscode/chef-server-running.json).

This works on Chef server 12.2.0-1.

[johnnyramos]

Seems I hit the end here on trying to reset pivotal on the latest chef-server version (12.17.15). Instructions don't seem to work as the opscode-pgsql user does not exist.

Also tried @dvinograd's method but no dice. Probably because of the opscode-pgsl user.

Any advice? Or has a way been implemented already?

[PrajaktaPurohit]

• Automate this process as a part of chef-server-ctl or a fixie command so that is is available for use easily.
• The user delete should block for deleting this user.