'chef configure' command

[lamont-granquist]

We need to replace 'knife configure -i' with something a bit more opinionated (yet ultimately customizable) and which does better integration.

By default it should probably hit hosted chef, with a --server command line option to override the server location (I DO NOT WANT TO TYPE IN API.OPSCODE.COM AND YES IM SCREAMING THIS). It needs to know your organization, user and password. It should take those off of the command line, or via interactive prompting.

It should then fetch your validator, client key and setup a knife.rb for your with no additional prompting.

The default location for the knife.rb should be .chef/knife.rb, it should assume a ~/chef-repo/{cookbooks,data_bags,role,etc} directory structure. Do not prompt the user to make a choice. It should /allow/ overriding the directory structure to move it under a more ~/env/{organization}-like structure (I'd like to be able to set ~/env/{organization} as a preference in a .chefrc file)

The way I see this working for a new hosted user:

$ chef configure --organization acmeinc --user bob
password: ******
- creating ~/.chef/knife.rb
- creating ~/.chef/bob.pem
- creating ~/.chef/acmeinc-validator.pem
- creating ~/chef-repo/{cookbooks,data_bags,roles,environments}

The way I see this working for a new private EC user:

$ chef configure --server chef.acmeinc.com --organization acmeinc --user bob
password: ******
- creating ~/.chef/knife.rb
- creating ~/.chef/bob.pem
- creating ~/.chef/acmeinc-validator.pem
- creating ~/chef-repo/{cookbooks,data_bags,roles,environments}

The way I'd like to be able to use it after some minor config tweaking:

$ chef configure --organization acmeinc --user lamont
password: ******
- creating ~/env/acmeinc/.chef/knife.rb
- creating ~/env/acmeinc/.chef/bob.pem
- creating ~/env/acmeinc/.chef/acmeinc-validator.pem
- creating ~/env/acmeinc/chef-repo/{cookbooks,data_bags,roles,environments}

[danielsdeleo]

Overall, yes.

• We need to get this on the roadmap for the server
• We want to move to a world where the validator is not used for bootstrapping. Chef metal already works without the validator. If chef metal isn't going to be ready enough soon enough then we should remove the need for the validator from knife bootstrap.
• The chef-repo setup seems to overlap with the generators. I'd prefer that functionality be kept separate. Especially since the learn chef flow now starts without a "real" chef server, I would expect the happy path to be: create content, run in local VM or cloud via chef-zero; get HEC/EC access (via web or whatever); chef configure; continue.

[adamedx]

@tduffield, this might be of interest for you for the container issues you were dealing with.

@danielsdeleo , the server already supports an endpoint for creating a new client without the validator, right?

[danielsdeleo]

@adamedx I should explain a little how it works now and point out what need to change to make validator-less bootstraps work:

• Do a thing that makes the machine you want to bootstrap exist (by hand, cloud API, local VM API, out-of-band automated process, etc.)
• log in to the box, install chef-client
• copy credentials
• create client config
• chef-client determines client and node name based on user config (the node_name setting) or automatically from ohai hostname/fqdn.
• chef-client creates the client and node objects by making a POST to /clients and /nodes. Note that because of authz (bifrost) authZ model, the POST to /nodes has to be done using the identity of the client created in the previous step.

knife or any API client can hit the right endpoints to make the POST requests to /clients and /nodes, but you have to know what node_name you want. In the current scheme, this is something that happens very late in the process when you let chef pick the node names for you.

Depending on how we judge the tradeoffs, we could:

• Drop support for node name auto detection in bootstrapping (my preference). This is the simplest option. The user will have to indicate what node_name they want ahead of time.
• Run a shell script over SSH to detect the node name once the host is online (but before we install chef). You'd still have to wait on your cloud provider to spin up the instance, but you wouldn't have to wait for chef to install. On the downside, this wouldn't be completely cross-platform.
• Run a ruby script over SSH after chef-client is installed, e.g., chef-detect-node-name. Increases time you have to wait until a failure happens, but the code would be more cross-platform.

Some things we might do to make the experience nicer for users:

• Make validator-based bootstrap a non-default, optional behavior, e.g., knife bootstrap --use-validator so you can switch back to the old behavior if you prefer.
• If --use-validator isn't given then -N NODE_NAME is required.
• Add an option to let chef generate a unique "slug" for the node name which is appended to the name you specify. E.g., you run knife bootstrap -N chat-server --uniqify-name and knife picks a node name like chat-server-RANDOMHEX for you.
• Add an option to use the FQDN argument to knife bootstrap FQDN as the node name rather than detecting it client side, e.g., knife bootstrap my-host.example.com --use-fqdn-as-node-name.

[lamont-granquist]

I'd say we just mandate -N and offer --use-validator for old behavior.

The uniqify problem is already something that exists, it seems a bit orthogonal. The workaround to the last thing is just knife bootstrap FQDN -N FQDN -- we could think about just making that the default unless you pass in an explicit -N.

[danielsdeleo]

@lamont-granquist yeah, I was thinking about use cases like knife $cloud create where we sometimes use unique identifiers from the cloud service as node names. For a cloud bootstrap we have to make a choice about the trade-offs of picking a node name before we ask the cloud API for a new instance. If there is any issue with client creation, like invalid credentials (bad username/user key, not authorized to create client, duplicate node/client name), invalid user config, chef-server availability problem, the user will find out about it before they waste time (and money) spinning up a cloud instance that they cannot actually bootstrap. The downside is that you can't use your cloud provider's IDs for your node names, so you have to rely on getting this data from ohai. I think adding custom ID suffixes as a feature would make it more convenient to do the chef credential creation up-front, which I think would be a better UX overall.

[tyler-ball]

Triaging this issue for the Chef Workstation team. This overlaps with some of the unified CLI work we have been talking about, and highlights the disparity between different products packaged inside Workstation. Namely - that they all use different configuration files and configuration methods.

I would think, in a world where a chef command gives users all the functionality they need to manage their Chef Software installation, that users would only have 1 configuration file to manage.

[tyler-ball]

This also highlights the issue users have of "I just installed Chef Workstation, what do I do now". Our post install command should point them to something like chef configure or chef init so they have a breadcrumb of steps to follow