aiohttp-swagger icon indicating copy to clipboard operation
aiohttp-swagger copied to clipboard

Using regex with escaped characters crashes with "yaml.scanner.ScannerError: while scanning a double-quoted scalar"

Open iTaybb opened this issue 4 years ago • 1 comments

When using classes with regexes, aiohttp-swagger attempts to convert these to patterns in the openapi v3 format. If the regex got any escaped characters, such as \w, the code will crash:

Traceback (most recent call last):
  File "/c/git/app/rest/__init__.py", line 31, in setup_swagger  
    aiohttp_swagger.setup_swagger(app, api_version=__api_version__,
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/aiohttp_swagger/__init__.py", line 66, in setup_swagger
    swagger_info = generate_doc_from_each_end_point(
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/aiohttp_swagger/helpers/builders.py", line 129, in generate_doc_from_each_end_point
    swagger = yaml.full_load(swagger_base)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/__init__.py", line 142, in full_load
    return load(stream, FullLoader)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/composer.py", line 64, in compose_node
    if self.check_event(AliasEvent):
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/parser.py", 
line 98, in check_event
    self.current_event = self.state()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/parser.py", 
line 449, in parse_block_mapping_value
    if not self.check_token(KeyToken, ValueToken, BlockEndToken):
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 251, in fetch_more_tokens
    return self.fetch_double()
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 655, in fetch_double
    self.fetch_flow_scalar(style='"')
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 666, in fetch_flow_scalar
    self.tokens.append(self.scan_flow_scalar(style))
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 1149, in scan_flow_scalar
    chunks.extend(self.scan_flow_scalar_non_spaces(double, start_mark))
  File "/home/itay/.pyenv/versions/3.9.5/lib/python3.9/site-packages/yaml/scanner.py", line 1223, in scan_flow_scalar_non_spaces
    raise ScannerError("while scanning a double-quoted scalar", start_mark,
yaml.scanner.ScannerError: while scanning a double-quoted scalar
  in "<unicode string>", line 272, column 26:
                    pattern: "^s3://[\w]+/"
                             ^
found unknown escape character 'w'
  in "<unicode string>", line 272, column 35:
                    pattern: "^s3://[\w]+/"

This happens because the openapi.yaml jinja manifest exports the pattern as:

s3_bucket_key:
                title: "S3 Bucket Key"
                pattern: "^s3://[\\w]+/"
                type: "string"

and causes an error, since \ in double quote is parsed as escape sequence by YAML.

The solution would probably be to use single quotes instead of double quotes, probably in the nesteddict2yaml function.

iTaybb avatar Aug 01 '21 15:08 iTaybb

This is still relevant.

Fix using sed: sed -i 's|" \\"" + str(value) + "\\"\\n"|" \\"" + str(value).replace("\\\\", "\\\\\\\\") + "\\"\\n"|g' $(python -c 'import os.path, aiohttp_swagger; print(os.path.dirname(aiohttp_swagger.__file__))')/helpers/builders.py

iTaybb avatar Feb 09 '24 07:02 iTaybb