schema_salad icon indicating copy to clipboard operation
schema_salad copied to clipboard

Glob

Open pjotrp opened this issue 3 years ago • 6 comments

Globbing multiple documents.

  • [x] Fixed test that allowed 2 schemas to be passed in
  • [x] Multiple documents on command line
  • [x] Solve RDF compression because of shared placeholder ID (PubSeq style) or emit warning
  • [x] Add real filename globbing

Later we may need to add chunking when these documents take all RAM.

pjotrp avatar May 29 '21 13:05 pjotrp

Codecov Report

Merging #392 (b8d9433) into main (012c5a7) will decrease coverage by 9.38%. The diff coverage is 60.00%.

:exclamation: Current head b8d9433 differs from pull request most recent head 1869e96. Consider uploading reports for the commit 1869e96 to get more accurate results Impacted file tree graph

@@            Coverage Diff             @@
##             main     #392      +/-   ##
==========================================
- Coverage   79.29%   69.90%   -9.39%     
==========================================
  Files          18       16       -2     
  Lines        3260     2911     -349     
  Branches      879      766     -113     
==========================================
- Hits         2585     2035     -550     
- Misses        442      677     +235     
+ Partials      233      199      -34     
Impacted Files Coverage Δ
schema_salad/main.py 59.67% <60.00%> (-15.61%) :arrow_down:
schema_salad/makedoc.py 10.22% <0.00%> (-72.78%) :arrow_down:
schema_salad/jsonld_context.py 74.09% <0.00%> (-5.43%) :arrow_down:
schema_salad/utils.py 80.35% <0.00%> (-2.98%) :arrow_down:
schema_salad/avro/schema.py 61.14% <0.00%> (-1.60%) :arrow_down:
schema_salad/validate.py 70.50% <0.00%> (-0.62%) :arrow_down:
schema_salad/python_codegen.py 95.20% <0.00%> (-0.07%) :arrow_down:
schema_salad/__init__.py
schema_salad/schema.py
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 012c5a7...1869e96. Read the comment docs.

codecov[bot] avatar May 29 '21 13:05 codecov[bot]

Added multiple document support and globbing. Also added a PubSeq test. Ready for review!

pjotrp avatar May 30 '21 10:05 pjotrp

Will fix. Are you OK with the test data hosted in test/data/pubseq?

pjotrp avatar May 31 '21 20:05 pjotrp

Will fix. Are you OK with the test data hosted in test/data/pubseq?

I can't think of a reason not to be. What's the total size? We install our tests so users can verify post installation - (via pip, apt, whatever)

mr-c avatar Jun 03 '21 06:06 mr-c

Ping :-)

mr-c avatar Jun 24 '21 09:06 mr-c

On my todo list. Probably end of July because of holiday and other things.

pjotrp avatar Jun 24 '21 11:06 pjotrp