ux: broken default `notation verify` experience

[shizhMSFT]

The default experience is broken in beta.1.

Expected:

1. Download and install the notation binary
2. Run notation verify against a public registry
3. notation should verify the specified artifact using a default policy with system trusted root CAs.

Actual:

1. Download and install the notation binary
2. Run notation verify against a public registry fails with the following log.

$ notation verify someregistry.io/somerepo:sometag
Error: open /home/exampleuser/.config/notation/trustpolicy.json: no such file or directory

[priteshbandi]

The user needs to configure trust-store and trust-policy to start using verify operation.

1. Default policy would differ from usecase to usecase. E.g. during build time, customers might want to have strong validations than deployment time.
2. We took a conscious call to not trust the system's roots

[iamsamirzon]

@shizhMSFT - Was this working in prior version of Notation? I concur with @priteshbandi that users must "opt in" explicitly via trust policy to which root CA they want to trust.

[shizhMSFT]

The previous version of notation requires a cert but it is relatively easy to do so with a --cert option or notation cert add. Now, we need to configure the trust policy, which is complex for most new users.

Basically, I'm proposing that we should have an out of box experience for notation.

[FeynmanZhou]

Same feeling here. The prerequisites of notation verify require configuring the Trust Store and Trust Policy since Beta.1. See the Beta.1 quick start. It provides fine-grained authenticity and integrity validation capabilities which enhance the security level indeed, but it's too complicated for dummy users to learn and understand the TS/TP concepts.

As we are developing an open-source project, the users are not limited to security engineers but also include application/platform developers, DevOps engineers, etc, we really need to think about how to balance security and usability (proposed by @SteveLasker ).

Can we abstract two types of users in two different environments to relax the notation verify cumbersome user experience and provide different levels of security?

Two types of users:

• Normal users (Application/platform developers, DevOps engineers)
• Professional users (System administrator, Security engineers)

Two major environments (scenarios)

• Production
• Development and testing / PoC

Four cases:

	Testing and Development	Production
Normal users	No need to configure TS/TP	a. SHOULD use a default trust policy with system trusted root CA b. Also support configuring TP/TS and a certificate issued by CA
Professional users	SHOULD use a default trust policy with system trusted root CA	MUST configure TP/TS and use a certificate issued by CA

[dtzar]

This is related to these two issues to make the experience better:

1. https://github.com/notaryproject/notaryproject/issues/95 - this should fix the situation so if someone does notation verify someregistry.io/somerepo:sometag AND someregistry.io has let's say posted a public key certificate to the hosted location we say it should be... then we could display a response like: You don't trust the public key from someregistry.io. Would you like to download/inspect this public key? Y/N There could be a different response if we know the public key doesn't exist for that registry/domain.
2. The auto-creation of the trust policy template when one doesn't exist at the start. https://github.com/notaryproject/notation/issues/398

[iamsamirzon]

IMHO, we should not add anything unique for different users - Normal or Professional. How will Notation know if it is a normal or professional user?

I personally like the idea of a default trust policy, perhaps if that trust policy has examples inside ( which are commented out.. so a user has to "uncomment" or refer to use as examples).

[FeynmanZhou]

@iamsamirzon Notation doesn't need to differentiate the user role. The point of abstracting two different types of users is to enable a non-TS/TP option for users who don't have requirements for the fine-grained validation policies in testing & development scenarios and provide strict validation policies for advanced cases.

Providing a default trust policy is a good potion and is already on the roadmap. I am thinking if any possibility to relax the TS/TP requirement for a non-mission-critical environment, which is similar to alpha.4 experience.

[vaninrao10]

Based on last week discussion we will stick with the user guide with example along with the documentation created as per https://github.com/notaryproject/notation/pull/441. Moving this issue to post RC-1.