dead_simple_owl_design_patterns
dead_simple_owl_design_patterns copied to clipboard
INCATools
Release schema + validator (+doc gen) on PyPi
structure
src/schema/dosdp_schema.yaml # = spec/DOSDP_schema_full.yaml
src/schema/test/
src/schema/test/generic_test # Is it valid JSON schema.
src/schema/test/positive_test_set/ # Some list of passing files to test
src/schema/test/negative test_set/# Some list of failing files to test
src/dosdp/dosdp.py # wrapper script with argparse (or click)
src/dosdp/validator.py # sub module based on existing code Allow setting threshold for hard vs soft fail # based on src/dosdp/simple_pattern_tester.py
src/dosdp/document.py # sub module - patterns as input and generates markdown doc (@matentzn has code)
# dosdp = package name
Lets try to adhere to good CLI practices: https://clig.dev/
Mainly:
- use flags over args
- use verbs for commands
In case test config is needed: https://github.com/mapping-commons/sssom-py/blob/master/tests/test_config.yaml
Also, can we make sure we can use these as a library in python directly:
from dosdp import validate
validate("test.yaml")
pypi package released: https://pypi.org/project/dosdp/0.1.0.dev1/
If ok, we can release a stable version and update usage scenarios at README.md
Usage
From the CLI
$ pip3 install dosdp==0.1.0.dev1
$ dosdp -v <test.yaml>
From Python
from dosdp import validator
validator.validate("test.yaml")
Hey @hkir-dev! Great push forward; I would like to request one change. I am trying to coordinate a standardisation of parameter across all the tools we are involved in. I will share the full list with you when it is finalised. But generally I would like to always use the same grammar:
toolname function_verb {-x/--xxx}*
So in this case:
dosdp validate -i pattern
or in the case of documentation
dosdp document -i pattern
What do you think? Sorry to spring this on you after the pypi release, I was not paying enough attention :)
It's clearly flagged as a dev & pre-release on PypI, so I think no expectation of stability yet
Note - Setup.py should be commited at root and should include link to doc
Potentially relevant: Can use GitHub actions to standardise build and link to release. e.g. see https://github.com/schlegelp/navis/blob/master/.github/workflows/publish-to-pypi.yml or https://github.com/pypa/gh-action-pypi-publish
Cli interface updated. All code is in branch pypi_release
Usage
From the CLI
$ pip3 install dosdp==0.1.0.dev1
$ dosdp validate -i <test.yaml>
$ dosdp document -i <schema.yaml> -o <schema.md>
From Python
from dosdp import validator
validator.validate("test.yaml")
Awesome! Thanks!
The document command will have to do something more than just documenting the schema, but we can discuss this next week. Thanks!
Package release github action added. Now releasing to test pypi repository. We can change repo and credentials when ready for stable release.
New python package released
- Documentation support for pattern files added
- Schema documentation interface changed. No need to give schema as input, uses schema provided by the python package
- Generated schema documentation updated to increase readability: 'array' changed to 'list', 'object' changed to 'dict', 'Refer to *#/definitions/x' changed to 'Values from x'
Usage
From the CLI
$ dosdp validate -i <test.yaml or 'test folder'>
$ dosdp document --schema
$ dosdp document --schema -o <schema.md>
$ dosdp document -i <pattern.yaml> -o <pattern.md>
From Python
from dosdp import validator
validator.validate("test.yaml")
Awesome! This is super cool, agree!
One thing missing:
dosdp document -i <test.yaml or 'test folder'> -o <'out folder'>
So that this overview table of all yamls in a folder and the overview is created
Update implemented. If input is a folder, an "overview.md" file is generated in the output folder along with the pattern documentations.
Given the document -i, how is the data preview created? shouldn't there be a link to a data directory as well?
Final repo cleanup.
Move lib, patterns, spec & any stray unused files -> attic.
@matentzn inorder to make pattern documentation interface compatible with the previous schema documentation interface, I had removed the sample data parameter. Now I added that sample data parameter back and made a new relase
https://test.pypi.org/project/dosdp/0.1.7.dev1/
Usage
From the CLI
$ dosdp validate -i <test.yaml or 'test folder'>
$ dosdp document --schema
$ dosdp document --schema -o <schema.md>
$ dosdp document -i <pattern.yaml or pattern folder> -o <pattern.md or output folder>
$ dosdp document -i <pattern.yaml or pattern folder> -d <sample data folder> -o <pattern.md or pattern folder>
From Python
from dosdp import validator
validator.validate("test.yaml")
Awesome, thank you! I will have to integrate this at some point into ODK, when I make a larger push on pattern infrastructure.