bdq TG2-TO DO List for standard submission

TG2-TO DO List for standard submission

Open Tasilee opened this issue 6 months ago • 1 comments

A list of things for @ArthurChapman, @Tasilee, @tucotuco, @chicoreus and @ymgan to complete the BDQ Core standard submission.

Introduction Document and Landing Pages

(Done) Replicate target directory structure for the standard within _review @tucotuco
(Done) Files under _review directory that require some manual content @chicoreus (content parsed out of supplemental into users guides and other documents).
(Done) Diagrams showing a generic approach to using Validations, Measures, Amendments, Validations, Measures @ymgan
(Done) Place diagram svg files within _review directory. in _review/docs/intro @chicoreus
(Done) Modfications to target directory structure to include derivative files for review @chicoreus See: _review/README for current state of files and directories
(Done) Identify files that are currently maintained outside of _review and copied into it. @chicoreus see: https://github.com/tdwg/bdq/blob/master/tg2/_make_review/copy_files.sh
(Done) Pare down Introduction document to minimum normative and non-normative needed @tucotuco (done through section 1)
(Done) Add landing pad pages for bdqffdq (required), and bdqcore (desirable). @chicoreus
- (Done) Create bdqcore landing page. @chicoreus
  - (Done) Refine which text goes on bdqcore landing page and which on bdqcore term-list @Tasilee @chicoreus
- (Done) Create bdqffdq landing page. @chicoreus
- (Done) Add normative guidance about use of bdqffdq terms to bdqffdq landing page. @chicoreus
(Done) Add list of documents with target audience for each to end of introduction @chicoreus
(Done) Add in the Acknowledgements and Contributions @ArthurChapman
(Done) Update Glossary @ArthurChapman
(Done) Check and edit Section 6 Glossary as new section added @Tasilee
(Done) Add Oracle mark text for Java to glossary @ArthurChapman

Supplementary Document

(Done) Edit Supplementary Document @Tasilee
(Done) Put supplementary information into supplementary document @Tasilee
(Done) Move normative information from supplementary document to appropriate other documents, limit content of supplementary to just supplementary. @chicoreus
(Done) Format supplementary document @ArthurChapman
(Done) Copy the bdqTag vocabulary into a markdown table in the supplementary document and remove from vocabulary/glossary_terms.csv @chicoreus
(Done) Copy the bdqTestField vocabulary into a markdown table in the supplementary document and remove from vocabulary/glossary_terms.csv. @chicoreus
(Done) Add history of the development of the tests to Section 3.
(Done) Merge the three duplicative sets of Test Field descriptions in the supplementary documentation in to one.
(Done) Add Competency Questions section for bdqffdq and bdqcore to the supplementary documentation. @chicoreus
(Done) Write additional sparql queries on bdqcore RDF and use these as competency questions in supplement @chicoreus
(Done) Review Supplement Document @ArthurChapman
(Done) Review document @Tasilee
(Done) Add discussion of bdq_issue_to_csv, bdqtestrunner, and kurator-ffdq to supplement. @chicoreus

Vocabularies and termlist documents

(Done) Flesh out Vocabulary landing page @ArthurChapman, @tucotuco
(Done) Review which terms go where @ArthurChapman @chicoreus
(Done) Review bdq and bdqdim term definitions @ArthurChapman
(Done) Update bdqffdq: Ontology @chicoreus @ArthurChapman
- (Done) Review bdq/bdqffdq vocabulary definitions @ArthurChapman @chicoreus
- (Done) Move ontology into tdw/bdq respository @chicoreus
- (Done) Update bdqffdq ontology skos:definitions and skos:comments @chicoreus
- (Done) Give each bdqffdq term a skos:prefLabel, skos:definition, rdfs:label, and rdfs:comment @chicoreus
- (Done) Create csv export of ontology @chicoreus (note, needs manual edit with each export from owl).
- (Done) Integrate First draft Python code to produce human readable display of ontology into _review/build/ , done as draft_build_bdqffdq.py @chicoreus
- (Done) Add Argument and related terms for Parameters and Arguments on Specifications to bdqffdq @chicoreus
(Done) Provide rdf distributions of Vocabulary Term List documents following SDS section 2.2.4. (owl document for bdqffdq)
- (Done) rdf artifacts for Vocabulary Terms lists of bdq, bdqdim, bdqenh, bdqcrit @chicoreus
- (Done) Update rdf artifact for bdqcore generated with kurator-ffdq @chicoreus
  - (Done) Multiple tests in each Policy @chicoreus,
  - (Done) Class names matched to bdqffdq. @chicoreus
  - (Done) Add Criterion, Enhancement, DataQualityDimension instances from bdqcrit, bdqenh, bdqdim vocabularies @chicoreus
  - (Done) Stable identifiers for methods, specifications, information elements in RDF generated from CSV @chicoreus (tg2/core/information_element_guids.csv and tg2/core/TG2_tests_additional_guids.csv used by kurator-ffdq with -guidFile and -ieGuidFile options)
    - (Done) Provide for reuse of uuids between builds of RDF artifact from configuration files @chicoreus
    - (Done) stable identifier file for core test methods/specifications/profiles @chicoreus
    - (Done) stable identifier file for information elements @chicoreus
    - (Done) add stable identifiers for methods/specifications/profiles for multirecord measures @chicoreus
    - (Done) Add Arguments and Parameters to bdqcore RDF serialization @chicoreus
    - (Done) Add and use stable identifier file for arguments @chicoreus guids stable in reexports of tests from markdown tables and rebuilds of rdf as of 4c56c3c11b see that commit message for commits in bdq_issue_to_csv and kurator-ffdq.
(Done) Add a vocabulary extension file for bdqffdq @chicoreus
(Done) Build Vocabulary term list files in _review/docs/list/{term}/index.md files @chicoreus
- (Done) Build script for term list files modified to run with local configuration files for draft standard @chicoreus
- (Done) Configure and run for bdq, bdqdim, bdqenh, bdqcrit vocabularies. @chicoreus
- (Done) Update Vocabulary Landing page, point links to correct targets (generated landing pages and term-list documents)
(Done) Review human readable term-list files in
- (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdq/index.md @ArthurChapman
- (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqdim/index.md @ArthurChapman
- (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqcrit/index.md @ArthurChapman
- (Done) https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/list/bdqenh/index.md @ArthurChapman
- (Done) Checked for consistency of headings between all four documents. Header files updated @ArthurChapman @chicoreus
(Done) Document and close Issue #152 as this vocabulary has been split into separate files and added to _review @ArthurChapman
(Done) Add remaining glossary items from #152 that aren't in vocabularies into glossary section of introduction @chicoreus
(Done) Add vocabularies and pages for terms to apply to Criterion and Enancement fundamental concepts @chicoreus
(Done) Hold discussion with Cecilie Svenningsen (GBIF) on Vocabularies @ArthurChapman
(Done) Provide GBIF with list of future Vocabularies needed for Data Quality @ArthurChapman
(Done) Review bdq term list header document @Tasilee
(Done) Build term key for each vocabulary, or pull in a term table from relevant vocabs. @chicoreus, @ArthurChapman, @Tasilee, @tucotuco
Confirm that names of vocabularies, their terms, associated documents, and versioned terms conform to the expectations of the standards metadata document https://tdwg.github.io/rs.tdwg.org/ and the SDS @chicoreus @tucotuco

Quick Reference Guide

(Done) Create Preface for the Quick Reference Guide @Tasilee
(Done) Decide what descriptors that will be in the Quick Reference Guide @Tasilee
(Done) Build bdqcore Quick Reference Guide @Tasilee
(Done) Create draft QRG Footer (Citation based on Darwin Core QRG) @Tasilee
(Done) Python script to produce [quick reference guide] @chicoreus (https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/terms/bdqcore/index.md) for combined singlerecord and multirecord bdqcore tests @chicoreus .
(Done) Incorporate this script with header/footer assembly @tucotuco @chicoreus

Users Guide

(Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
(Done) Generate TOC (sections for populating) @chicoreus
(Done) Script to build and assemble materials into _review/docs/guides directory @chicoreus
(Done) Review/edit document @Tasilee
(Done) Review/edit document @ArthurChapman
(Done) Request review of sections by team members @chicoreus
Review content to confirm no missing assumptions in understanding and clarity of text for consumers.

Implementer's Guide

(Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
(Done) Generate TOC (sections for populating) @chicoreus
(Done) Script to build and assemble materials into _review/docs/guides directory @chicoreus
(Done) Review document @Tasilee
(Done) Review/edit document @ArthurChapman
(Done) Fix Sections 8.2 and 8.3 @Tasilee @chicoreus
(Done) Add in Namespace Abbreviations (section 1.4) @chicoreus
(Done) Allocate sections to team members (https://github.com/tdwg/bdq/blob/master/tg2/_review/docs/implementers/index.md)
(Done) Add in section 6.1 Notes on how to change a Parameter @chicoreus
(Done) Add note on stub code generation by kurator-ffdq @chicoreus

bdqffdq Guide

(Done) Complete template in _review/build/template/guide/bdqffdq/ @chicoreus
(Done) Create visualizations to explain ontology, add to bdqffdq guide page. @chicoreus (files are in _review/build/template/guide/bdqffdq/)
(Done) Edit the above ontology visualizations @ArthurChapman
(Done) Script to build and assemble materials into _review/docs/guides directory @chicoreus

Standards Maintenance Document

(Done) Stub for Standard Maintenance document
(Done) Add a section on Test Data @Tasilee
(Done) Add a section on supporting code (bdq_issue_to_csv, bdqtestrunner, kurator-ffdq) @chicoreus

Tests

(Done) Review/edit (English) Description in the light of current Expected Responses (CORE) @ArthurChapman
(Done) Add skos:prefLabel for multi-record measures in bdq_issue_to_csv @chicoreus
(Done) Build script that copies latest validation data files (test data and non-printing characters) into _review directory. @chicoreus See _make_review/copy_files.sh
(Done) Add implementation for #287 run against validation data. @chicoreus
(Done) Check generated Examples in Test Data match those in the Tests and align accordingly @Tasilee
(Done) Check and align inconsistencies in phrasing in the Specification of "in bdq:sourceAuthority" vs "in the bdq:sourceAuthority" @Tasilee
(Done) Check consistency of use of bdq:sourceAuthority and ISO references in all tests and align @Tasilee
(Done) Complete review of FAIL cases in validation data, update test specifications, validation data, or implementation as needed (also under Test Data below) @chicoreus @Tasilee @ArthurChapman ** Filteredpush implementations are passing all validation rows as of v93
(Done) Replace all instances of EMPTY and NOT EMPTY, etc. in the test specifications with bdq:Empty @Tasilee @ArthurChapman
(Done) Complete conversion of CSV list of tests (produced by bdq_issue_to_csv) into a term-version file.**
- (Done)Add status column, with 'recommended' as value. @chicoreus
- (Done) Rename bdqcore_terms.csv to bdqcore_term_versions.csv. @chicoreus
- (Done) Dowstream changes from changes to file. @chicoreus
Update and add links to source code for implementations in test descriptors for filtered push libraries @chicoreus
Freeze editing of markdown table representations of tests (decision by all)
- Final export from markdown tables with bdq_issue_to_csv @chicoreus
- Then: Update rdf artifact for bdqcore generated with kurator-ffdq @chicoreus

Bibliography (References)

(Done) Prepare intro for Bibliography @Tasilee
(Done) Format RFC 2119 reference and include in reference list @Tasilee
(Done) @Tasilee to send @ArthurChapman a list of References that need correction
(Done) Correct References in the Test Issues @ArthurChapman, @Tasilee
(Done) Prepare a combined list of References @Tasilee
(Done) Update with additional new references @Tasilee
(Done) Move references to a template, add header/footer/document configuration files, setup for either self standing references page or references included in pages when built @chicoreus current state is references embedded at build time into introduction and references as a separate document.
New releases of related code, update references with links to zenodo, make sure each piece of software has text in the appropriate document with a citation. @chicoreus
- rdfbeans
- ffdq-api (v2.0.2) Lowery & Morris 2024 https://doi.org/10.5281/zenodo.14031588
- kurator-ffdq (v3.0.0) Lowery et al. 2024 https://doi.org/10.5281/zenodo.14026643
- bdq_issue_to_csv (v1.0.0) Morris 2024 https://doi.org/10.5281/zenodo.14057551
- bdqtestrunner
- event_date_qc (v3.0.5) Morris & Lowery 2024 https://doi.org/10.5281/zenodo.14056663
- sci_name_qc (v1.1.2) Morris & Dou 2024 https://doi.org/10.5281/zenodo.14052951
- geo_ref_qc metadata in annotations updated to current. geodeticDatum issues outstanding.
- rec_occur_qc metadata in annotations updated to current.

Authors and Contributors

(Done) Prepare file on Authors and Contributions @chicoreus, @Tasilee, @ArthurChapman
(Done) Add code library functions to insert authors/contributors sections in consistent form into templates @chicoreus .
(Done) Chase up missing authors and ORCID_IDs @ArthurChapman @chicoreus
(Done) Decide on breakdown of Authors, Contributors and Acknowledgements @tucotuco, @ArthurChapman, @Tasilee, @chicoreus**
(Done) Assemble acknowlegements in introduction document. (Add configuration file for acknowledgements? No Seems a good idea that works with authors configuration file, but only needs listing in the intro/index.md document). @chicoreus @ArthurChapman @Tasilee
(Done) Review/edit/re-order and standardize details on Authors and Contributors

Test Data

(Done) Add new column for 'Date Last Updated' and populate with current date @Tasilee
(Done) Rationalize (align for consistency) of Response.comments and align with Test Examples (e.g., tense, phrasing) @Tasilee
(Done) Complete review of FAIL cases in validation data, update test specifications, validation data, or implementation as needed @chicoreus @Tasilee @ArthurChapman
(Done) Resolve 3 apparent duplicate rows @Tasilee (in v91, fixed in v93) Note: All filtered push qc implementations passing all validation rows except the 3 that appear as duplicates, and the annotation test for which there is no implementation.

Consistency Checks

(Done) Align (2) full and (2) summary sections about Test Types across 4 documents @Tasilee
(Done) Review and make consistent descriptive text about test types on bdqcore landing page, bdqcore term-list page, supplement, users guide, implementers guide. @Tasilee @chicoreus
(Done) Refine descriptive text about test types to match needs of local context and add normative statements where appropriate to bdqcore landing page, bdqcore term-list page, users guide, and implementers guide. @Tasilee @chicoreus
Consistent Headings
- (Done) Purpose and Audience statements for each document (@chicoreus added), needs refinement (@Tasilee).
- (Done) Consistent parts and ordering for section 1 of documents, following expectations of SDS (first pass by @chicoreus done, second pass by @Tasilee done, third pass by @chicoreus done).
Check links
- linkchecker https://github.com/tdwg/bdq/blob/master/tg2/_review/index.md
- Check links to anchors in documents (multiple anchor references are broken).

General

(Done) Identify potential Review Managers
(Done) Prepare and send off Report to Executive (All)
(Done) Make verbal report and summary to TDWG Business Meeting at TDWG2024 @ArthurChapman
(Done) Prepare diagrams for each of the test types (NAME, SPACE, TIME, OTHER) showing linkages between the tests @ArthurChapman
(Done) Templates and build scripts for all /docs/ pages except landing page for standard, maintinance document, and references page. @chicoreus
(Done) Add definitions for IRI, term_localName, termName, etc. to build_term_key() in function_lib.py @chicoreus
(Done) Add Section in all Footer Documents for Acronyms and Glossary linking to Tables in Introduction and for References linking to the Reference document @ArthurChapman
(Done) Link to glossary from all footer pages.
(Done) Check all documents for Acronyms and add any missing ones to the Table in 5 Acronyms @Tasilee
(Done) Run a spell/typo check on all major text documents @Tasilee
(Done) Simplify and standardize usage of "framework", "Framework", "bdqffdq: Framework" and "Framework ontology" etc. All to agree, and then @Tasilee to carefully align text
Produce RDF triples from the diagrams using showing linkages between the tests using improvedBy, targetedIssue, targetedMeasure, targetedValidation terms @ArthurChapman (??I'm still not sure what needs to be done here @chicoreus?? @chicoreus has resent email describing this.)
Identify potential Maintenance Managers
Final (editorial) review of all documents :) @tucotuco
Final check of links in documents, in case of moved targets etc @chicoreus

External

File issue against TDWG SDS for need to have a key for terms
Confirm that Issues related to Darwin Core have been filed or commented on in dwc repository

Aug 22 '24 17:08 Tasilee

bdq bdq copied to clipboard

TG2-TO DO List for standard submission

Introduction Document and Landing Pages

Supplementary Document

Vocabularies and termlist documents

Quick Reference Guide

Users Guide

Implementer's Guide

bdqffdq Guide

Standards Maintenance Document

Tests

Bibliography (References)

Authors and Contributors

Test Data

Consistency Checks

General

External

bdq
bdq copied to clipboard