Yamale
Yamale copied to clipboard
Adding external includes from existing schema
Hi and thank you for yamale :)
I would like to reuse some definitions from an old schema to generate a new schema. From your documentation it indeed looks like this should be possible:
Adding external includes
After you construct a schema you can add extra, external include definitions by calling
schema.add_include(dict)
. This method takes a dictionary and adds each key as another include.
I thought it could be done like shown in this dummy example:
old_schema.yaml
def_from_old_schema:
element : str()
other_element : num()
new_schema.yaml
list(include('def_from_old_schema'))
import yamale
old_schema = yamale.make_schema('old_schema.yaml')
new_schema = yamale.make_schema('new_schema.yaml')
new_schema.add_include(old_schema.dict)
However this raises a rather convoluted error message (see below). I thought the old_schema.dict
would have had the correct structure to be used for Schema.add_include()
, however values like 'String((), {})'
throws it off. What is the simplest way to generate a dict
of the right structure?
Traceback (most recent call last):
File "/home/c71chilltown/.local/lib/python3.10/site-packages/yamale/syntax/parser.py", line 39, in parse
tree = ast.parse(validator_string, mode='eval')
File "/usr/lib/python3.10/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
TypeError: compile() arg 1 must be a string, bytes or AST object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/c71chilltown/.local/lib/python3.10/site-packages/yamale/schema/schema.py", line 47, in _parse_schema_item
return syntax.parse(expression, validators)
File "/home/c71chilltown/.local/lib/python3.10/site-packages/yamale/syntax/parser.py", line 46, in parse
raise SyntaxError(
SyntaxError: Invalid schema expression: 'String((), {})'. compile() arg 1 must be a string, bytes or AST object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/c71chilltown/.local/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "
Unsure whether intended, but I think yamale.schema.Schema._process_schema()
mutates self.dict
in the process of making self._schema
, leading to your situation.
Had never noticed, since my includes are never in the primary document:
schema_for_standalone_use: stuff()
---
# includes go here
---
# more includes
So I just do main_schema.add_include(other_schema.includes)
and go on my merry way. That might be a decent workaround for you.
Some possible solutions:
-
self.dict
shouldn't be mutated at__init__()
,dict()
orcopy()
when passing it out, treat as a bug - Add an
@property
(e.g.self.raw_schema
) that serves__init__()
'sschema_dict
unchanged, since someone might depend onself.dict
being equivalent toself._schema
, given it's not marked private
Neutral on what to do--I'd lean for the latter since self.dict
is a public part of Schema
. Either way it's a tiny change.
Thank you for the swift response :)
old_schema.yaml
random: int()
#include in different document now
---
def_from_old_schema:
element : str()
other_element : num()
new_schema.yaml
list(include('def_from_old_schema'))
import yamale
old_schema = yamale.make_schema('old_schema.yaml')
new_schema = yamale.make_schema('new_schema.yaml')
new_schema.add_include(old_schema.includes) # Changed from .dict to .includes
Unfortunately the work around does not appear to be work for me as the old_schema.includes
contains a Schema
object as its item, which gives the same type of parsing error as above.
I'm loading old_schema.yaml through yamale.make_schema()
, is that a weird way of doing it?
It, indeed, works if I bypass making Schema
and generate a raw_schema
from old_schema.yaml and use that as a start point (i.e. borrowing from yamale's source code) it just feels like I'm doing something clumsy by reading old_schema.yaml in like that.
I'm of course willing to contribute, if you want me to look more into it?