[token-cli] upgrade to clap-v3

[samkim-crypto]

Problem

There has been useful updates to solana-clap-v3-utils in the monorepo, but the token-cli still uses clap-v2. For example:

• https://github.com/solana-labs/solana-program-library/issues/4097 should be resolved by switching to clap-v3-utils with version 1.16.0
• Support for encryption keys needed for token-2022 is only available in solana-clap-v3-utils

Proposed changes

Upgrade token-cli to use clap v3 and solana-clap-v3-utils in a sequence of steps suggested in the clap change logs.

Migrating From clap v2

1. Add CLI tests, -h and --help output at a minimum (recommendation: trycmd for snapshot testing)
2. Update your dependency i. If you use no-default-features: add the std feature
3. Resolve compiler errors
4. Resolve behavior changes i. Refactor your App creation to a function and add a test similar to the one below, resolving any of its assertions ii. Look over the "subtle changes" under BREAKING CHANGES iii. If using builder: test your application under various circumstances to see if ArgMatches asserts regarding AllowInvalidUtf8.
5. At your leisure: resolve deprecation notices

Status:

• https://github.com/solana-labs/solana-program-library/pull/4506 implements steps 1-4 above
• Step 5 can be implemented in follow-up PRs that are split in digestible chunks

[samkim-crypto]

From clap change logs v3.0.0:

Subtle changes (i.e. compiler won't catch):

• [x] AppSettings::UnifiedHelpMessage is now default behaviour
  • {flags} and {unified} will assert if present in App::help_template
  • See https://github.com/clap-rs/clap/issues/2807
  • COMPLETED: this is a cosmetic change and is acceptable behavior (description of UnifiedHelpMessage)
• [x] AppSettings::EnableColoredHelp is now the default behavior but can be opted-out with AppSettings::DisableColoredHelp (https://github.com/clap-rs/clap/issues/2806)
  • COMPLETED: this is a cosmetic change and is acceptable behavior)
• [x] App::override_usage no longer implies a leading \t, allowing multi lined usages
  • COMPLETED: App::override_usage is not used in the token-cli
• [x] Arg::require_equals no longer implies ArgSettings::ForbidEmptyValues (https://github.com/clap-rs/clap/issues/2233)
  • COMPLETED: Arg::require_equals is not used in the token-cli
• [x] Arg::require_delimiter no longer implies ArgSettings::TakesValue and ArgSettings::UseValueDelimiter (https://github.com/clap-rs/clap/issues/2233)
  • COMPLETED: Arg::require_delimiter is not used in the token-cli
• [x] Arg::env, Arg::env_os, Arg::last, Arg::require_equals, Arg::allow_hyphen_values, Arg::hide_possible_values, Arg::hide_default_value, Arg::hide_env_values, Arg::case_insensitive and Arg::multiple_values no longer imply ArgSettings::TakesValue (https://github.com/clap-rs/clap/issues/2233)
  • COMPLETED: None of Arg::* above are used in the token-cli
• [x] ArgMatches::is_present no longer checks subcommand names
  • COMPLETED: ArgMathces::is_present is never used on subcommand names in the token-cli
• [ ] Some env variable values are now considered false for flags, not just "not-present" (https://github.com/clap-rs/clap/issues/2539)
• [x] Changed ...s meaning in usage parser. Before, it always meant multiple which is still true for --option [val].... Now [name]... --option [val] results in ArgSettings::MultipleOccurrences.
  • COMPLETED: ... is not used in the token-cli
• [ ] Usage exit code changed from 1 to 2 (https://github.com/clap-rs/clap/issues/1327)
• [x] Reject --foo=bar when takes_value(false) (https://github.com/clap-rs/clap/issues/1543)
  • COMPLETED: this is a bug fix and is acceptable behavior
• [x] No longer accept an arbitrary number of - for long arguments (-----long)
  • COMPLETED: acceptable behavior

[samkim-crypto]

From clap change logs v3.0.0:

Easier to catch changes:

• [ ] When using no-default-features, you now have to specify the std feature (reserved for future work)
• Gated env support behind env feature flag
  • Impacts Arg::env, Arg::env_os, Arg::hide_env_values, ArgSettings::HideEnvValues
  • See https://github.com/clap-rs/clap/pull/2694
• [ ] Gated crate information behind cargo feature flag
  • Impacts crate_name!, crate_version!, crate_authors!, crate_description!, app_from_crate!
• [ ] AppSettings::StrictUtf8 is now default behaviour and asserts if AppSettings::AllowInvalidUtf8ForExternalSubcommands and ArgSettings::AllowInvalidUtf8 and ArgMatches::value_of_os aren't used together
  • AppSettings::AllowInvalidUtf8 has been removed
  • https://github.com/clap-rs/clap/issues/751
• [x] Arg::short and Arg::value_delimiter now take a char instead of a &str
  • COMPLTED in https://github.com/solana-labs/solana-program-library/pull/4506 commit: d93af20
• [ ] ArgMatches panics on unknown arguments
• [ ] Removed VersionlessSubcommands, making it the default (see https://github.com/clap-rs/clap/issues/2812)
• [ ] Completion generation has been split out into clap_complete.
• [ ] Removed ArgSettings::EmptyValues in favor of ArgSettings::ForbidEmptyValues
• [x] Validator signatures have been loosed:
  • Arg::validator now takes first argument as Fn(&str) -> Result<O, E: ToString> instead of Fn(String) -> Result<(), String>
  • Arg::validator_os now takes first argument as Fn(&OsStr) -> Result<O, OsString> instead of Fn(&OsStr) -> Result<(), OsString>
  • COMPLETED in https://github.com/solana-labs/solana-program-library/pull/4506 commit: 3171490
• [ ] Arg::value_name now sets, rather than appends (see https://github.com/clap-rs/clap/issues/2634)
• [ ] Upgrade yaml-rust from 0.3 to 0.4
• [ ] Replaced ArgGroup::from(BTreeMap) to ArgGroup::from(yaml)
• [ ] Replaced ArgMatches::usage with App::generate_usage
• [ ] Replaced Arg::settings with Arg::setting(Setting1 | Setting2)
• [x] App and Arg now need only one lifetime
  • COMPLETED in https://github.com/solana-labs/solana-program-library/pull/4506 commits: 0062096, c296c2e
• [ ] Removed deprecated App::with_defaults, replaced with app_from_crate
• [ ] Removed deprecated AppSettings::PropagateGlobalValuesDown (now the default)
• [ ] Some App functions, like App::write_help now take &mut self instead of &self
• [ ] Error::message is now private, use Error::to_string
• [ ] Arg::default_value_if, Arg::default_value_if_os, Arg::default_value_ifs, Arg::default_value_ifs_os now takes the default value parameter as an option (https://github.com/clap-rs/clap/issues/1406)
• [ ] Changed App::print_help & App::print_long_help to now return std::io::Result
• [ ] Changed App::write_help & App::write_long_help to now return std::io::Result
• [x] Changed Arg::index, Arg::number_of_values, Arg::min_values, Arg::max_values to taking usize instead of u64
  • COMPLETED in https://github.com/solana-labs/solana-program-library/pull/4506 commit: 3273d86
• [ ] Changed Error::info to type Vec<String> instead of Option<Vec<String>>
• [x] Changed ArgMatches::subcommand to now return Option<(&str, &ArgMatches)>
  • COMPLETED in https://github.com/solana-labs/solana-program-library/pull/4506 commit: 6f851be
• [ ] Renamed ErrorKind::MissingArgumentOrSubcommand to ErrorKind::DisplayHelpOnMissingArgumentOrSubcommand
• [ ] Renamed ErrorKind::HelpDisplayed to ErrorKind::DisplayHelp
• [ ] Renamed ErrorKind::VersionDisplayed to ErrorKind::DisplayVersion
• [ ] Added #[non_exhaustive] to clap::{ValueHint, ErrorKind, AppSettings, ArgSettings} (https://github.com/clap-rs/clap/pull/3167)

[samkim-crypto]

From clap change logs v3.1.0:

Compatibility: Changes in behavior of note that are not guaranteed to be compatible across releases:

• [ ] (help) help subcommand shows long help like --help, rather than short help (-h), deprecated clap::AppSettings::UseLongFormatForHelpSubcommand (#3440)
• [ ] (help) Pacman-style subcommands are now ordered the same as usage errors (#3470)
• [ ] (help) Pacman-style subcommands use standard alternate syntax in usage (#3470)

Deprecations:

• [ ] clap::Command is now preferred over clap::App (#3089 in #3472)
  • clap::command! is now preferred over clap::app_from_crate (#3089 in #3474)
  • clap::CommandFactory::command is now preferred over clap::IntoApp::into_app (#3089 in #3473)
• [ ] (help) help subcommand shows long help like --help, rather than short help (-h), deprecated clap::AppSettings::UseLongFormatForHelpSubcommand (#3440)
• [ ] (error) Deprecate clap::AppSettings::WaitOnError, leaving it to the user to implement
• [ ] (validation) clap::Command::subcommand_required(true).arg_required_else_help(true) is now preferred over clap::AppSettings::SubcommandRequiredElseHelp (#3280)
• [ ] (builder) clap::AppSettings are nearly all deprecated and replaced with builder methods and getters (#2717)
• [ ] (builder) clap::ArgSettings is deprecated and replaced with builder methods and getters (#2717)
• [ ] (builder) clap::Arg::id and clap::ArgGroup::id are now preferred over clap::Arg::name and clap::ArgGroup::name (#3335)
• [ ] (help) clap::Command::next_help_heading is now preferred over clap::Command::help_heading (#1807, #1553)
• [ ] (error) clap::error::ErrorKind is now preferred over clap::ErrorKind (#3395)
• [ ] (error) clap::Error::kind() is now preferred over clap::Error::kind
• [ ] (error) clap::Error::context() is now preferred over clap::Error::info (#2628)

[samkim-crypto]

From clap change logs v3.2.0:

Compatibility MSRV is now 1.56.0 (#3732)

Behavior

• [ ] Defaults no longer satisfy required and its variants (#3793)
• [ ] When misusing ArgMatches::value_of and friends, debug asserts were turned into panics

Moving (old location deprecated)

• [ ] clap::{PossibleValue, ValueHint} to clap::builder::{PossibleValue, ValueHint}
• [ ] clap::{Indices, OsValues, ValueSource, Values} to clap::parser::{Indices, OsValues, ValueSource, Values}
• [ ] clap::ArgEnum to clap::ValueEnum (#3799)

Replaced

• [ ] Arg::allow_invalid_utf8 with Arg::value_parser(value_parser!(PathBuf)) (#3753)
• [ ] Arg::validator / Arg::validator_os with Arg::value_parser (#3753)
• [ ] Arg::validator_regex with users providing their own builder::TypedValueParser (#3756)
• [ ] Arg::forbid_empty_values with builder::NonEmptyStringValueParser / builder::PathBufValueParser (#3753)
• [ ] Arg::possible_values with Arg::value_parser([...]), builder::PossibleValuesParser, or builder::EnumValueParser (#3753)
• [ ] Arg::max_occurrences with arg.action(ArgAction::Count).value_parser(value_parser!(u8).range(..N)) for flags (#3797)
• [ ] Arg::multiple_occurrences with ArgAction::Append or ArgAction::Count though positionals will need Arg::multiple_values (#3772, #3797)
• [ ] Command::args_override_self with ArgAction::Set (#2627, #3797)
• [ ] AppSettings::NoAutoVersion with ArgAction or Command::disable_version_flag (#3800)
• [ ] AppSettings::NoHelpVersion with ArgAction or Command::disable_help_flag / Command::disable_help_subcommand (#3800)
• [ ] ArgMatches::{value_of, value_of_os, value_of_os_lossy, value_of_t} with ArgMatches::{get_one,remove_one} (#3753)
• [ ] ArgMatches::{values_of, values_of_os, values_of_os_lossy, values_of_t} with ArgMatches::{get_many,remove_many} (#3753)
• [ ] ArgMatches::is_valid_arg with ArgMatches::{try_get_one,try_get_many} (#3753)
• [ ] ArgMatches::occurrences_of with ArgMatches::value_source or ArgAction::Count (#3797)
• [ ] ArgMatches::is_present with ArgMatches::contains_id or ArgAction::SetTrue (#3797)
• [ ] ArgAction::StoreValue with ArgAction::Set or ArgAction::Append (#3797)
• [ ] ArgAction::IncOccurrences with ArgAction::SetTrue or ArgAction::Count (#3797)
• [ ] (derive) #[clap(parse(...))] replaced with: (#3589, #3794)
  • For default parsers (no parse attribute), deprecation warnings can be silenced by opting into the new behavior by adding either #[clap(action)] or #[clap(value_parser)] (ie requesting the default behavior for these attributes). Alternatively, the unstable-v4 feature changes the default away from parse to action/value_parser.
  • For #[clap(parse(from_flag))] replaced with #[clap(action = ArgAction::SetTrue)] (#3794)
  • For #[clap(parse(from_occurrences))] replaced with #[clap(action = ArgAction::Count)] though the field's type must be u8 (#3794)
  • For #[clap(parse(from_os_str)] for PathBuf, replace it with #[clap(value_parser)] (as mentioned earlier this will call value_parser!(PathBuf) which will auto-select the right ValueParser automatically).
  • For #[clap(parse(try_from_str = ...)], replace it with #[clap(value_parser = ...)]
  • For most other cases, a type implementing TypedValueParser will be needed and specify it with #[clap(value_parser = ...)]