Unquoted Numerical Default Parameter - TypeError expected string or bytes-like object

[sc-alscient]

I have a parameter in my template that contains a string made up of numbers and dashes as below:

  TagReviewDate:
    Type: String
    Description: The review date - to be used by tags
    Default: 2021-03-25

This causes Taskcat to fail.

·[0;30;41m[ERROR  ]·[0m : TypeError expected string or bytes-like object
Traceback (most recent call last):
  File "/root/.pyenv/versions/3.8.1/lib/python3.8/site-packages/taskcat/_cli.py", line 70, in main
    cli.run()
  File "/root/.pyenv/versions/3.8.1/lib/python3.8/site-packages/taskcat/_cli_core.py", line 235, in run
    return getattr(command(), subcommand)(**args)
  File "/root/.pyenv/versions/3.8.1/lib/python3.8/site-packages/taskcat/_cli_modules/test.py", line 82, in run
    parameters = config.get_rendered_parameters(buckets, regions, templates)
  File "/root/.pyenv/versions/3.8.1/lib/python3.8/site-packages/taskcat/_config.py", line 355, in get_rendered_parameters
    parameters[test_name][region_name] = ParamGen(
  File "/root/.pyenv/versions/3.8.1/lib/python3.8/site-packages/taskcat/_template_params.py", line 62, in __init__
    self.transform_parameter()
  File "/root/.pyenv/versions/3.8.1/lib/python3.8/site-packages/taskcat/_template_params.py", line 96, in transform_parameter
    self._regex_replace_param_value(self.RE_GENNUMB, self._gen_rand_num(20))
  File "/root/.pyenv/versions/3.8.1/lib/python3.8/site-packages/taskcat/_template_params.py", line 376, in _regex_replace_param_value
    if self.regxfind(regex_pattern, self.param_value):
  File "/root/.pyenv/versions/3.8.1/lib/python3.8/site-packages/taskcat/_common_utils.py", line 87, in regxfind
    security_group = re_object.search(data_line)
TypeError: expected string or bytes-like object

If I add quotes around the default parameter, Taskcat will deploy successfully. e.g.

  TagReviewDate:
    Type: String
    Description: The review date - to be used by tags
    Default: "2021-03-25"

Tascat has been installed using pip3. It is using the latest version and tsted both locally using WSL & in AWS Codebuild. Both profile and no profile has been tested.

[andrew-glenn]

Turns out it's a datetime object that's being returned from upstream cfnlint.

# cat t.template.yaml
Parameters:
  TagReviewDate:
    Type: String
    Description: The review date - to be used by tags
    Default: 2021-03-25

>>> import pprint as pp
>>> import cfnlint
>>> t = cfnlint.decode.cfn_yaml.load('t.template.yaml')
>>> pp.pprint(t)
{'Parameters': {'TagReviewDate': {'Default': datetime.date(2021, 3, 25),
                                  'Description': 'The review date - to be used '
                                                 'by tags',
                                  'Type': 'String'}}}

It's easy enough to convert to a string.

>>> p = t['Parameters']['TagReviewDate']['Default']
>>> p
datetime.date(2021, 3, 25)
>>> str(p)
'2021-03-25'
>>>

I think the appropriate path forward here is an upstream fix. A quick conversion to a string on a datetime object is fine. However, IMO, cfnlint shouldn't be returning a datetime object to begin with. The type is defined as a string. We also run into supportability issues when we expand this out to all CFN Parameter types.

Update: Turns out this is upstream of cfnlint, too. YAML parsing is causing this.

>>> with open('t.template.yaml') as f:
...     rr = f.read()
...
...
>>> pp.pprint(yaml.safe_load(rr))
{'Parameters': {'TagReviewDate': {'Default': datetime.date(2021, 3, 25),
                                  'Description': 'The review date - to be used '
                                                 'by tags',
                                  'Type': 'String'}}}
>>>

A workaround is proposed here. - but it basically says "quote your strings", or do a custom schema.

Leaving this up for discussion: How do we want to handle situations like this?

Do we want to do basic type-checks on strings and convert? Just throw an exception?

/cc @jaymccon @tonynv

[jaymccon]

My gut is to follow whatever the yaml spec says should be done with interpretation of unquoted strings. However, I would not be opposed to a pr that converts every type not parsed by cfn to a string, which seems to be the underlying behavior in the service.

Implementation may be hard seeing as we're just handing over a file to cfn-lint/pyyaml and then working with the returned object. So we could get into a world of hurt if we try to reconstruct the original string value (think different region/os rules on date formatting).

If implementation is hard, and adds a lot of parsing logic, I'd be inclined to leave the user to follow proper yaml rules for when you need to quote a value