obi icon indicating copy to clipboard operation
obi copied to clipboard

Published 20 hours ago • verified publisher icon obi-ontology

Revisit our use of reasoners

Open jamesaoverton opened this issue 3 years ago • 5 comments

OWL has a handful of "profiles" that specify subsets of the language. In OBO we tend to use the EL and DL profiles. The DL profile has most OWL features. The EL profile does not include negation, cardinality, or many of the logical characteristics of object properties. You can use Protege or robot validate-profile to figure out which profile your ontology uses.

Why use EL instead of DL? A DL reasoner has to do a lot more work than an EL reasoner, so EL can be much faster. To some extent, EL reasoners can also make better use of parallel processing. Although there are many OWL reasoners out there, in OBO we tend to use ELK for EL reasoning and HermiT for DL.

OBI has always used DL and has various axioms outside of the EL profile. But running HermiT on OBI takes 12 minutes on my laptop, while running ELK takes 4.6 seconds. I ran robot diff to compare the results, and found that only a few dozen axioms are different (see below). Specifically:

$ time java -jar build/robot.jar --prefix "REO: http://purl.obolibrary.org/obo/REO_" reason --input build/obi_merged.owl --reasoner ELK --output obi-elk.owl

real    0m4.620s
user    0m29.234s
sys     0m1.058s

$ time java -jar build/robot.jar --prefix "REO: http://purl.obolibrary.org/obo/REO_" reason --input build/obi_merged.owl --reasoner HermiT --output obi-hermit.owl

real    12m6.825s
user    12m47.018s
sys     0m4.947s

$ java -jar build/robot.jar diff --left obi-elk.owl --right obi-hermit.owl --labels true --output obi-elk.diff

Arguably, some of the inferences that HermiT makes but ELK doesn't should just be asserted. For instance, where ELK only sees subClassOf owl:Thing we could assert the parent that HermiT infers. We could also just assert some of the 'processed material' parents.

This was a surprise to me. When I'm working on OBI I often run HermiT (and wait and wait) to make sure that I'm seeing all the inferences. But it turns out that ELK is covering almost everything. So the lesson seems to be that we don't need to run HermiT as often to get good results, especially if we assert just a few more things that we are right now.

ROBOT plain diff with labels

24 axioms in Ontology 1 but not in Ontology 2:
- AnnotationAssertion(<IAO:0000412>[imported from] _:genid2147502004 <uberon.owl>[obo:uberon.owl])
- AnnotationAssertion(<owl:minQualifiedCardinality> _:genid2147502005 "2"^^xsd:nonNegativeInteger)
- SubClassOf(<GO:0019815>[B cell receptor complex] <GO:0032991>[protein-containing complex])
- SubClassOf(<GO:0042101>[T cell receptor complex] <GO:0032991>[protein-containing complex])
- SubClassOf(<OBI:0000038>[chromatography column] <OBI:0000968>[device])
- SubClassOf(<OBI:0000410>[recombinant plasmid] <BFO:0000002>[continuant])
- SubClassOf(<OBI:0000419>[reverse transcriptase] <owl:Thing>)
- SubClassOf(<OBI:0000427>[enzyme] <owl:Thing>)
- SubClassOf(<OBI:0000430>[plasmid] <owl:Thing>)
- SubClassOf(<OBI:0000793>[double blind study execution] <owl:Thing>)
- SubClassOf(<OBI:0000822>[single blind study execution] <owl:Thing>)
- SubClassOf(<OBI:0000970>[scalar score from composite inputs] <IAO:0000109>[measurement datum])
- SubClassOf(<OBI:0000995>[administration of material to specimen] <OBI:0000652>[material combination])
- SubClassOf(<OBI:0001151>[genetically modified material] <BFO:0000040>[material entity])
- SubClassOf(<OBI:0001192>[epitope specific immune intervention] <OBI:0600007>[administering substance in vivo])
- SubClassOf(<OBI:0001870>[Illumina methylation BeadChip] <owl:Thing>)
- SubClassOf(<OBI:0001948>[sequence assembly name] <owl:Thing>)
- SubClassOf(<OBI:0100010>[Epstein Barr virus transformed B cell] <BFO:0000040>[material entity])
- SubClassOf(<OBI:0100017>[blood serum specimen] <OBI:0100051>[specimen])
- SubClassOf(<OBI:0200036>[Benjamini and Hochberg false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
- SubClassOf(<OBI:0200049>[Benjamini and Yekutieli false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
- SubClassOf(<OBI:0302716>[pool of specimens] <owl:Thing>)
- SubClassOf(<OBI:0302859>[genetically modified organism] <OBI:0000047>[processed material])
- SubClassOf(<OBI:1110022>[adaptive immune receptor] <BFO:0000040>[material entity])

32 axioms in Ontology 2 but not in Ontology 1:
+ AnnotationAssertion(<IAO:0000412>[imported from] _:genid2147520362 <uberon.owl>[obo:uberon.owl])
+ AnnotationAssertion(<owl:minQualifiedCardinality> _:genid2147520363 "2"^^xsd:nonNegativeInteger)
+ SubClassOf(<GO:0042575>[DNA polymerase complex] <OBI:0000427>[enzyme])
+ SubClassOf(<OBI:0000038>[chromatography column] <OBI:0000932>[material separation device])
+ SubClassOf(<OBI:0000410>[recombinant plasmid] <OBI:0000731>[recombinant vector])
+ SubClassOf(<OBI:0000419>[reverse transcriptase] <OBI:0000427>[enzyme])
+ SubClassOf(<OBI:0000427>[enzyme] <BFO:0000040>[material entity])
+ SubClassOf(<OBI:0000430>[plasmid] <CHEBI:4705>[double-stranded DNA])
+ SubClassOf(<OBI:0000701>[self-organizing map] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0000793>[double blind study execution] <OBI:0000822>[single blind study execution])
+ SubClassOf(<OBI:0000822>[single blind study execution] <OBI:0000826>[treatment portion of study execution])
+ SubClassOf(<OBI:0000970>[scalar score from composite inputs] <IAO:0000032>[scalar measurement datum])
+ SubClassOf(<OBI:0000995>[administration of material to specimen] <OBI:0000274>[adding a material entity into a target])
+ SubClassOf(<OBI:0001151>[genetically modified material] <OBI:0000047>[processed material])
+ SubClassOf(<OBI:0001192>[epitope specific immune intervention] <OBI:0001174>[passive immunization])
+ SubClassOf(<OBI:0001870>[Illumina methylation BeadChip] <OBI:0001869>[Illumina BeadChip])
+ SubClassOf(<OBI:0001873>[number of errors] <OBI:0001933>[value specification])
+ SubClassOf(<OBI:0001948>[sequence assembly name] <IAO:0000300>[textual entity])
+ SubClassOf(<OBI:0100010>[Epstein Barr virus transformed B cell] <OBI:0000047>[processed material])
+ SubClassOf(<OBI:0100017>[blood serum specimen] <OBI:0000953>[processed specimen])
+ SubClassOf(<OBI:0200001>[geometric mean calculation] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0200028>[quantile transformation] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0200073>[family wise error rate correction method] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0200079>[arithmetic mean calculation] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0200117>[mode calculation] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0200119>[median calculation] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0200123>[skewness calculation] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0200163>[false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
+ SubClassOf(<OBI:0302716>[pool of specimens] <OBI:0000953>[processed specimen])
+ SubClassOf(<OBI:0400106>[centrifuge] <OBI:0000932>[material separation device])
+ SubClassOf(<OBI:1110022>[adaptive immune receptor] <GO:0032991>[protein-containing complex])
+ SubClassOf(<PR:000023089>[DNA ligase] <OBI:0000427>[enzyme])

ROBOT Markdown diff

Old

  • Ontology IRI: http://purl.obolibrary.org/obo/obi/obi_merged.owl
  • Version IRI: http://purl.obolibrary.org/obo/obi/2021-04-02/obi_merged.owl
  • Loaded from: file:/Users/james/Repositories/github/obi-ontology/obi/obi-elk.owl

New

  • Ontology IRI: http://purl.obolibrary.org/obo/obi/obi_merged.owl
  • Version IRI: http://purl.obolibrary.org/obo/obi/2021-04-02/obi_merged.owl
  • Loaded from: file:/Users/james/Repositories/github/obi-ontology/obi/obi-hermit.owl

Ontology imports

Ontology annotations

B cell receptor complex http://purl.obolibrary.org/obo/GO_0019815

Removed

Benjamini and Hochberg false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200036

Removed

Benjamini and Yekutieli false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200049

Removed

DNA ligase http://purl.obolibrary.org/obo/PR_000023089

Added

DNA polymerase complex http://purl.obolibrary.org/obo/GO_0042575

Added

Epstein Barr virus transformed B cell http://purl.obolibrary.org/obo/OBI_0100010

Removed

Added

Illumina methylation BeadChip http://purl.obolibrary.org/obo/OBI_0001870

Removed

Added

T cell receptor complex http://purl.obolibrary.org/obo/GO_0042101

Removed

_:genid2147502004

Removed

_:genid2147502005

Removed

_:genid2147520362

Added

_:genid2147520363

Added

adaptive immune receptor http://purl.obolibrary.org/obo/OBI_1110022

Removed

Added

administration of material to specimen http://purl.obolibrary.org/obo/OBI_0000995

Removed

Added

arithmetic mean calculation http://purl.obolibrary.org/obo/OBI_0200079

Added

blood serum specimen http://purl.obolibrary.org/obo/OBI_0100017

Removed

Added

centrifuge http://purl.obolibrary.org/obo/OBI_0400106

Added

chromatography column http://purl.obolibrary.org/obo/OBI_0000038

Removed

Added

double blind study execution http://purl.obolibrary.org/obo/OBI_0000793

Removed

Added

enzyme http://purl.obolibrary.org/obo/OBI_0000427

Removed

Added

epitope specific immune intervention http://purl.obolibrary.org/obo/OBI_0001192

Removed

Added

false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200163

Added

family wise error rate correction method http://purl.obolibrary.org/obo/OBI_0200073

Added

genetically modified material http://purl.obolibrary.org/obo/OBI_0001151

Removed

Added

genetically modified organism http://purl.obolibrary.org/obo/OBI_0302859

Removed

geometric mean calculation http://purl.obolibrary.org/obo/OBI_0200001

Added

median calculation http://purl.obolibrary.org/obo/OBI_0200119

Added

mode calculation http://purl.obolibrary.org/obo/OBI_0200117

Added

number of errors http://purl.obolibrary.org/obo/OBI_0001873

Added

plasmid http://purl.obolibrary.org/obo/OBI_0000430

Removed

Added

pool of specimens http://purl.obolibrary.org/obo/OBI_0302716

Removed

Added

quantile transformation http://purl.obolibrary.org/obo/OBI_0200028

Added

recombinant plasmid http://purl.obolibrary.org/obo/OBI_0000410

Removed

Added

reverse transcriptase http://purl.obolibrary.org/obo/OBI_0000419

Removed

Added

scalar score from composite inputs http://purl.obolibrary.org/obo/OBI_0000970

Removed

Added

self-organizing map http://purl.obolibrary.org/obo/OBI_0000701

Added

sequence assembly name http://purl.obolibrary.org/obo/OBI_0001948

Removed

Added

single blind study execution http://purl.obolibrary.org/obo/OBI_0000822

Removed

Added

skewness calculation http://purl.obolibrary.org/obo/OBI_0200123

Added

jamesaoverton avatar Apr 02 '21 13:04 jamesaoverton

Very cool! Thanks for sharing.

Best, Mathias

On Fri, Apr 2, 2021 at 8:28 AM James A. Overton @.***> wrote:

OWL has a handful of "profiles" https://www.w3.org/TR/owl2-profiles/ that specify subsets of the language. In OBO we tend to use the EL and DL profiles. The DL profile has most OWL features. The EL profile does not include negation, cardinality, or many of the logical characteristics of object properties. You can use Protege or robot validate-profile http://robot.obolibrary.org/validate-profile to figure out which profile your ontology uses.

Why use EL instead of DL? A DL reasoner has to do a lot more work than an EL reasoner, so EL can be much faster. To some extent, EL reasoners can also make better use of parallel processing. Although there are many OWL reasoners out there, in OBO we tend to use ELK for EL reasoning and HermiT for DL.

OBI has always used DL and has various axioms outside of the EL profile. But running HermiT on OBI takes 12 minutes on my laptop, while running ELK takes 4.6 seconds. I ran robot diff to compare the results, and found that only a few dozen axioms are different (see below). Specifically:

$ time java -jar build/robot.jar --prefix "REO: http://purl.obolibrary.org/obo/REO_" reason --input build/obi_merged.owl --reasoner ELK --output obi-elk.owl

real 0m4.620s user 0m29.234s sys 0m1.058s

$ time java -jar build/robot.jar --prefix "REO: http://purl.obolibrary.org/obo/REO_" reason --input build/obi_merged.owl --reasoner HermiT --output obi-hermit.owl

real 12m6.825s user 12m47.018s sys 0m4.947s

$ java -jar build/robot.jar diff --left obi-elk.owl --right obi-hermit.owl --labels true --output obi-elk.diff

Arguably, some of the inferences that HermiT makes but ELK doesn't should just be asserted. For instance, where ELK only sees subClassOf owl:Thing we could assert the parent that HermiT infers. We could also just assert some of the 'processed material' parents.

This was a surprise to me. When I'm working on OBI I often run HermiT (and wait and wait) to make sure that I'm seeing all the inferences. But it turns out that ELK is covering almost everything. So the lesson seems to be that we don't need to run HermiT as often to get good results, especially if we assert just a few more things that we are right now.

ROBOT plain diff with labels

24 axioms in Ontology 1 but not in Ontology 2:

  • AnnotationAssertion(IAO:0000412[imported from] _:genid2147502004 <uberon.owl>[obo:uberon.owl])
  • AnnotationAssertion(owl:minQualifiedCardinality _:genid2147502005 "2"^^xsd:nonNegativeInteger)
  • SubClassOf(GO:0019815[B cell receptor complex] GO:0032991[protein-containing complex])
  • SubClassOf(GO:0042101[T cell receptor complex] GO:0032991[protein-containing complex])
  • SubClassOf(OBI:0000038[chromatography column] OBI:0000968[device])
  • SubClassOf(OBI:0000410[recombinant plasmid] BFO:0000002[continuant])
  • SubClassOf(OBI:0000419[reverse transcriptase] owl:Thing)
  • SubClassOf(OBI:0000427[enzyme] owl:Thing)
  • SubClassOf(OBI:0000430[plasmid] owl:Thing)
  • SubClassOf(OBI:0000793[double blind study execution] owl:Thing)
  • SubClassOf(OBI:0000822[single blind study execution] owl:Thing)
  • SubClassOf(OBI:0000970[scalar score from composite inputs] IAO:0000109[measurement datum])
  • SubClassOf(OBI:0000995[administration of material to specimen] OBI:0000652[material combination])
  • SubClassOf(OBI:0001151[genetically modified material] BFO:0000040[material entity])
  • SubClassOf(OBI:0001192[epitope specific immune intervention] OBI:0600007[administering substance in vivo])
  • SubClassOf(OBI:0001870[Illumina methylation BeadChip] owl:Thing)
  • SubClassOf(OBI:0001948[sequence assembly name] owl:Thing)
  • SubClassOf(OBI:0100010[Epstein Barr virus transformed B cell] BFO:0000040[material entity])
  • SubClassOf(OBI:0100017[blood serum specimen] OBI:0100051[specimen])
  • SubClassOf(OBI:0200036[Benjamini and Hochberg false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200049[Benjamini and Yekutieli false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0302716[pool of specimens] owl:Thing)
  • SubClassOf(OBI:0302859[genetically modified organism] OBI:0000047[processed material])
  • SubClassOf(OBI:1110022[adaptive immune receptor] BFO:0000040[material entity])

32 axioms in Ontology 2 but not in Ontology 1:

  • AnnotationAssertion(IAO:0000412[imported from] _:genid2147520362 <uberon.owl>[obo:uberon.owl])
  • AnnotationAssertion(owl:minQualifiedCardinality _:genid2147520363 "2"^^xsd:nonNegativeInteger)
  • SubClassOf(GO:0042575[DNA polymerase complex] OBI:0000427[enzyme])
  • SubClassOf(OBI:0000038[chromatography column] OBI:0000932[material separation device])
  • SubClassOf(OBI:0000410[recombinant plasmid] OBI:0000731[recombinant vector])
  • SubClassOf(OBI:0000419[reverse transcriptase] OBI:0000427[enzyme])
  • SubClassOf(OBI:0000427[enzyme] BFO:0000040[material entity])
  • SubClassOf(OBI:0000430[plasmid] CHEBI:4705[double-stranded DNA])
  • SubClassOf(OBI:0000701[self-organizing map] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0000793[double blind study execution] OBI:0000822[single blind study execution])
  • SubClassOf(OBI:0000822[single blind study execution] OBI:0000826[treatment portion of study execution])
  • SubClassOf(OBI:0000970[scalar score from composite inputs] IAO:0000032[scalar measurement datum])
  • SubClassOf(OBI:0000995[administration of material to specimen] OBI:0000274[adding a material entity into a target])
  • SubClassOf(OBI:0001151[genetically modified material] OBI:0000047[processed material])
  • SubClassOf(OBI:0001192[epitope specific immune intervention] OBI:0001174[passive immunization])
  • SubClassOf(OBI:0001870[Illumina methylation BeadChip] OBI:0001869[Illumina BeadChip])
  • SubClassOf(OBI:0001873[number of errors] OBI:0001933[value specification])
  • SubClassOf(OBI:0001948[sequence assembly name] IAO:0000300[textual entity])
  • SubClassOf(OBI:0100010[Epstein Barr virus transformed B cell] OBI:0000047[processed material])
  • SubClassOf(OBI:0100017[blood serum specimen] OBI:0000953[processed specimen])
  • SubClassOf(OBI:0200001[geometric mean calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200028[quantile transformation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200073[family wise error rate correction method] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200079[arithmetic mean calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200117[mode calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200119[median calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200123[skewness calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200163[false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0302716[pool of specimens] OBI:0000953[processed specimen])
  • SubClassOf(OBI:0400106[centrifuge] OBI:0000932[material separation device])
  • SubClassOf(OBI:1110022[adaptive immune receptor] GO:0032991[protein-containing complex])
  • SubClassOf(PR:000023089[DNA ligase] OBI:0000427[enzyme])

ROBOT Markdown diff Old

  • Ontology IRI: http://purl.obolibrary.org/obo/obi/obi_merged.owl
  • Version IRI: http://purl.obolibrary.org/obo/obi/2021-04-02/obi_merged.owl
  • Loaded from: file:/Users/james/Repositories/github/obi-ontology/obi/obi-elk.owl

New

  • Ontology IRI: http://purl.obolibrary.org/obo/obi/obi_merged.owl
  • Version IRI: http://purl.obolibrary.org/obo/obi/2021-04-02/obi_merged.owl
  • Loaded from: file:/Users/james/Repositories/github/obi-ontology/obi/obi-hermit.owl

Ontology imports Ontology annotations B cell receptor complex http://purl.obolibrary.org/obo/GO_0019815 Removed

Benjamini and Hochberg false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200036 Removed

Benjamini and Yekutieli false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200049 Removed

DNA ligase http://purl.obolibrary.org/obo/PR_000023089 Added

DNA polymerase complex http://purl.obolibrary.org/obo/GO_0042575 Added

Epstein Barr virus transformed B cell http://purl.obolibrary.org/obo/OBI_0100010 Removed

Added

Illumina methylation BeadChip http://purl.obolibrary.org/obo/OBI_0001870 Removed

Added

T cell receptor complex http://purl.obolibrary.org/obo/GO_0042101 Removed

_:genid2147502004 Removed

_:genid2147502005 Removed

_:genid2147520362 Added

_:genid2147520363 Added

adaptive immune receptor http://purl.obolibrary.org/obo/OBI_1110022 Removed

Added

administration of material to specimen http://purl.obolibrary.org/obo/OBI_0000995 Removed

Added

arithmetic mean calculation http://purl.obolibrary.org/obo/OBI_0200079 Added

blood serum specimen http://purl.obolibrary.org/obo/OBI_0100017 Removed

Added

centrifuge http://purl.obolibrary.org/obo/OBI_0400106 Added

chromatography column http://purl.obolibrary.org/obo/OBI_0000038 Removed

Added

double blind study execution http://purl.obolibrary.org/obo/OBI_0000793 Removed

Added

enzyme http://purl.obolibrary.org/obo/OBI_0000427 Removed

Added

epitope specific immune intervention http://purl.obolibrary.org/obo/OBI_0001192 Removed

Added

false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200163 Added

family wise error rate correction method http://purl.obolibrary.org/obo/OBI_0200073 Added

genetically modified material http://purl.obolibrary.org/obo/OBI_0001151 Removed

Added

genetically modified organism http://purl.obolibrary.org/obo/OBI_0302859 Removed

geometric mean calculation http://purl.obolibrary.org/obo/OBI_0200001 Added

median calculation http://purl.obolibrary.org/obo/OBI_0200119 Added

mode calculation http://purl.obolibrary.org/obo/OBI_0200117 Added

number of errors http://purl.obolibrary.org/obo/OBI_0001873 Added

plasmid http://purl.obolibrary.org/obo/OBI_0000430 Removed

Added

pool of specimens http://purl.obolibrary.org/obo/OBI_0302716 Removed

Added

quantile transformation http://purl.obolibrary.org/obo/OBI_0200028 Added

recombinant plasmid http://purl.obolibrary.org/obo/OBI_0000410 Removed

Added

reverse transcriptase http://purl.obolibrary.org/obo/OBI_0000419 Removed

Added

scalar score from composite inputs http://purl.obolibrary.org/obo/OBI_0000970 Removed

Added

self-organizing map http://purl.obolibrary.org/obo/OBI_0000701 Added

sequence assembly name http://purl.obolibrary.org/obo/OBI_0001948 Removed

Added

single blind study execution http://purl.obolibrary.org/obo/OBI_0000822 Removed

Added

skewness calculation http://purl.obolibrary.org/obo/OBI_0200123 Added

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/obi-ontology/obi/issues/1328, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACF6DLRT7L7TKWHWSOI3INTTGXBBXANCNFSM42I33NBA .

mbrochhausen avatar Apr 02 '21 13:04 mbrochhausen

Thanks for raising this James. We have an interesting history here, in that we first assumed reasoners could fully 'understand' the logical term definitions if we made them complex enough. But then realized the limits of OWL, and adjusted our expectations downards. Moving from DL to EL would be another step 'down' in terms of reasoning. But I think we have realized that the use of common templates that structure how to write definitions for sets of terms is a lot more powerful. If I am not mistaken, there isn't a single assay term in your list of things missed by ELK. So if we aim to have templates for most (if not all) terms in OBI, I believe that we can switch to ELK, and have better results in terms of reasoning than we have now with a more complex reasoner.

In terms of how to do the switch: How about we keep both reasoners for releases, but switch to ELK for development?

Hope this is making sense.

  • Bjoern

On Fri, Apr 2, 2021 at 3:33 AM Mathias Brochhausen @.***> wrote:

Very cool! Thanks for sharing.

Best, Mathias

On Fri, Apr 2, 2021 at 8:28 AM James A. Overton @.***> wrote:

OWL has a handful of "profiles" https://www.w3.org/TR/owl2-profiles/ that specify subsets of the language. In OBO we tend to use the EL and DL profiles. The DL profile has most OWL features. The EL profile does not include negation, cardinality, or many of the logical characteristics of object properties. You can use Protege or robot validate-profile http://robot.obolibrary.org/validate-profile to figure out which profile your ontology uses.

Why use EL instead of DL? A DL reasoner has to do a lot more work than an EL reasoner, so EL can be much faster. To some extent, EL reasoners can also make better use of parallel processing. Although there are many OWL reasoners out there, in OBO we tend to use ELK for EL reasoning and HermiT for DL.

OBI has always used DL and has various axioms outside of the EL profile. But running HermiT on OBI takes 12 minutes on my laptop, while running ELK takes 4.6 seconds. I ran robot diff to compare the results, and found that only a few dozen axioms are different (see below). Specifically:

$ time java -jar build/robot.jar --prefix "REO: http://purl.obolibrary.org/obo/REO_" reason --input build/obi_merged.owl --reasoner ELK --output obi-elk.owl

real 0m4.620s user 0m29.234s sys 0m1.058s

$ time java -jar build/robot.jar --prefix "REO: http://purl.obolibrary.org/obo/REO_" reason --input build/obi_merged.owl --reasoner HermiT --output obi-hermit.owl

real 12m6.825s user 12m47.018s sys 0m4.947s

$ java -jar build/robot.jar diff --left obi-elk.owl --right obi-hermit.owl --labels true --output obi-elk.diff

Arguably, some of the inferences that HermiT makes but ELK doesn't should just be asserted. For instance, where ELK only sees subClassOf owl:Thing we could assert the parent that HermiT infers. We could also just assert some of the 'processed material' parents.

This was a surprise to me. When I'm working on OBI I often run HermiT (and wait and wait) to make sure that I'm seeing all the inferences. But it turns out that ELK is covering almost everything. So the lesson seems to be that we don't need to run HermiT as often to get good results, especially if we assert just a few more things that we are right now.

ROBOT plain diff with labels

24 axioms in Ontology 1 but not in Ontology 2:

  • AnnotationAssertion(IAO:0000412[imported from] _:genid2147502004 <uberon.owl>[obo:uberon.owl])
  • AnnotationAssertion(owl:minQualifiedCardinality _:genid2147502005 "2"^^xsd:nonNegativeInteger)
  • SubClassOf(GO:0019815[B cell receptor complex] GO:0032991[protein-containing complex])
  • SubClassOf(GO:0042101[T cell receptor complex] GO:0032991[protein-containing complex])
  • SubClassOf(OBI:0000038[chromatography column] OBI:0000968[device])
  • SubClassOf(OBI:0000410[recombinant plasmid] BFO:0000002[continuant])
  • SubClassOf(OBI:0000419[reverse transcriptase] owl:Thing)
  • SubClassOf(OBI:0000427[enzyme] owl:Thing)
  • SubClassOf(OBI:0000430[plasmid] owl:Thing)
  • SubClassOf(OBI:0000793[double blind study execution] owl:Thing)
  • SubClassOf(OBI:0000822[single blind study execution] owl:Thing)
  • SubClassOf(OBI:0000970[scalar score from composite inputs] IAO:0000109[measurement datum])
  • SubClassOf(OBI:0000995[administration of material to specimen] OBI:0000652[material combination])
  • SubClassOf(OBI:0001151[genetically modified material] BFO:0000040[material entity])
  • SubClassOf(OBI:0001192[epitope specific immune intervention] OBI:0600007[administering substance in vivo])
  • SubClassOf(OBI:0001870[Illumina methylation BeadChip] owl:Thing)
  • SubClassOf(OBI:0001948[sequence assembly name] owl:Thing)
  • SubClassOf(OBI:0100010[Epstein Barr virus transformed B cell] BFO:0000040[material entity])
  • SubClassOf(OBI:0100017[blood serum specimen] OBI:0100051[specimen])
  • SubClassOf(OBI:0200036[Benjamini and Hochberg false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200049[Benjamini and Yekutieli false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0302716[pool of specimens] owl:Thing)
  • SubClassOf(OBI:0302859[genetically modified organism] OBI:0000047[processed material])
  • SubClassOf(OBI:1110022[adaptive immune receptor] BFO:0000040[material entity])

32 axioms in Ontology 2 but not in Ontology 1:

  • AnnotationAssertion(IAO:0000412[imported from] _:genid2147520362 <uberon.owl>[obo:uberon.owl])
  • AnnotationAssertion(owl:minQualifiedCardinality _:genid2147520363 "2"^^xsd:nonNegativeInteger)
  • SubClassOf(GO:0042575[DNA polymerase complex] OBI:0000427[enzyme])
  • SubClassOf(OBI:0000038[chromatography column] OBI:0000932[material separation device])
  • SubClassOf(OBI:0000410[recombinant plasmid] OBI:0000731[recombinant vector])
  • SubClassOf(OBI:0000419[reverse transcriptase] OBI:0000427[enzyme])
  • SubClassOf(OBI:0000427[enzyme] BFO:0000040[material entity])
  • SubClassOf(OBI:0000430[plasmid] CHEBI:4705[double-stranded DNA])
  • SubClassOf(OBI:0000701[self-organizing map] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0000793[double blind study execution] OBI:0000822[single blind study execution])
  • SubClassOf(OBI:0000822[single blind study execution] OBI:0000826[treatment portion of study execution])
  • SubClassOf(OBI:0000970[scalar score from composite inputs] IAO:0000032[scalar measurement datum])
  • SubClassOf(OBI:0000995[administration of material to specimen] OBI:0000274[adding a material entity into a target])
  • SubClassOf(OBI:0001151[genetically modified material] OBI:0000047[processed material])
  • SubClassOf(OBI:0001192[epitope specific immune intervention] OBI:0001174[passive immunization])
  • SubClassOf(OBI:0001870[Illumina methylation BeadChip] OBI:0001869[Illumina BeadChip])
  • SubClassOf(OBI:0001873[number of errors] OBI:0001933[value specification])
  • SubClassOf(OBI:0001948[sequence assembly name] IAO:0000300[textual entity])
  • SubClassOf(OBI:0100010[Epstein Barr virus transformed B cell] OBI:0000047[processed material])
  • SubClassOf(OBI:0100017[blood serum specimen] OBI:0000953[processed specimen])
  • SubClassOf(OBI:0200001[geometric mean calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200028[quantile transformation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200073[family wise error rate correction method] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200079[arithmetic mean calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200117[mode calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200119[median calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200123[skewness calculation] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0200163[false discovery rate correction method] <APOLLO_SV:00000796>[dataset creating])
  • SubClassOf(OBI:0302716[pool of specimens] OBI:0000953[processed specimen])
  • SubClassOf(OBI:0400106[centrifuge] OBI:0000932[material separation device])
  • SubClassOf(OBI:1110022[adaptive immune receptor] GO:0032991[protein-containing complex])
  • SubClassOf(PR:000023089[DNA ligase] OBI:0000427[enzyme])

ROBOT Markdown diff Old

  • Ontology IRI: http://purl.obolibrary.org/obo/obi/obi_merged.owl
  • Version IRI: http://purl.obolibrary.org/obo/obi/2021-04-02/obi_merged.owl
  • Loaded from: file:/Users/james/Repositories/github/obi-ontology/obi/obi-elk.owl

New

  • Ontology IRI: http://purl.obolibrary.org/obo/obi/obi_merged.owl
  • Version IRI: http://purl.obolibrary.org/obo/obi/2021-04-02/obi_merged.owl
  • Loaded from: file:/Users/james/Repositories/github/obi-ontology/obi/obi-hermit.owl

Ontology imports Ontology annotations B cell receptor complex http://purl.obolibrary.org/obo/GO_0019815 Removed

Benjamini and Hochberg false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200036 Removed

Benjamini and Yekutieli false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200049 Removed

DNA ligase http://purl.obolibrary.org/obo/PR_000023089 Added

DNA polymerase complex http://purl.obolibrary.org/obo/GO_0042575 Added

Epstein Barr virus transformed B cell http://purl.obolibrary.org/obo/OBI_0100010 Removed

Added

Illumina methylation BeadChip http://purl.obolibrary.org/obo/OBI_0001870 Removed

Added

T cell receptor complex http://purl.obolibrary.org/obo/GO_0042101 Removed

_:genid2147502004 Removed

_:genid2147502005 Removed

_:genid2147520362 Added

_:genid2147520363 Added

adaptive immune receptor http://purl.obolibrary.org/obo/OBI_1110022 Removed

Added

administration of material to specimen http://purl.obolibrary.org/obo/OBI_0000995 Removed

Added

arithmetic mean calculation http://purl.obolibrary.org/obo/OBI_0200079 Added

blood serum specimen http://purl.obolibrary.org/obo/OBI_0100017 Removed

Added

centrifuge http://purl.obolibrary.org/obo/OBI_0400106 Added

chromatography column http://purl.obolibrary.org/obo/OBI_0000038 Removed

Added

double blind study execution http://purl.obolibrary.org/obo/OBI_0000793 Removed

Added

enzyme http://purl.obolibrary.org/obo/OBI_0000427 Removed

Added

epitope specific immune intervention http://purl.obolibrary.org/obo/OBI_0001192 Removed

Added

false discovery rate correction method http://purl.obolibrary.org/obo/OBI_0200163 Added

family wise error rate correction method http://purl.obolibrary.org/obo/OBI_0200073 Added

genetically modified material http://purl.obolibrary.org/obo/OBI_0001151 Removed

Added

genetically modified organism http://purl.obolibrary.org/obo/OBI_0302859 Removed

geometric mean calculation http://purl.obolibrary.org/obo/OBI_0200001 Added

median calculation http://purl.obolibrary.org/obo/OBI_0200119 Added

mode calculation http://purl.obolibrary.org/obo/OBI_0200117 Added

number of errors http://purl.obolibrary.org/obo/OBI_0001873 Added

plasmid http://purl.obolibrary.org/obo/OBI_0000430 Removed

Added

pool of specimens http://purl.obolibrary.org/obo/OBI_0302716 Removed

Added

quantile transformation http://purl.obolibrary.org/obo/OBI_0200028 Added

recombinant plasmid http://purl.obolibrary.org/obo/OBI_0000410 Removed

Added

reverse transcriptase http://purl.obolibrary.org/obo/OBI_0000419 Removed

Added

scalar score from composite inputs http://purl.obolibrary.org/obo/OBI_0000970 Removed

Added

self-organizing map http://purl.obolibrary.org/obo/OBI_0000701 Added

sequence assembly name http://purl.obolibrary.org/obo/OBI_0001948 Removed

Added

single blind study execution http://purl.obolibrary.org/obo/OBI_0000822 Removed

Added

skewness calculation http://purl.obolibrary.org/obo/OBI_0200123 Added

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/obi-ontology/obi/issues/1328, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACF6DLRT7L7TKWHWSOI3INTTGXBBXANCNFSM42I33NBA

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/obi-ontology/obi/issues/1328#issuecomment-812530929, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADJX2IU24H4TIFENWSBFYYTTGXBT3ANCNFSM42I33NBA .

-- Bjoern Peters Professor La Jolla Institute for Immunology 9420 Athena Circle La Jolla, CA 92037, USA Tel: 858/752-6914 Fax: 858/752-6987 http://www.liai.org/pages/faculty-peters

bpeters42 avatar Apr 09 '21 03:04 bpeters42

Discussed on the OBI call 2021-04-12: I plan to go through this small number of axioms and see what we can either:

  1. simplify: if the axiom is just more complicated than it needs to be, we can simplify the axiom to remove negations and cardinality restrictions
  2. manually assert axioms that we definitely want

In the end, I propose that we use ELK most of the time, and periodically check to see if HermiT is inferring more than ELK is, then repeat those two steps above.

Help with this would be appreciated.

jamesaoverton avatar Apr 12 '21 16:04 jamesaoverton

we can successfully assert the following inferences that HermiT makes:

image

hectorguzor avatar Jun 07 '21 15:06 hectorguzor

As discussed on 2021-06-07: We have no plans to convert OBI to an EL ontology. But the axioms listed here are expensive and complex in some way, so it's worth reviewing them to see if they're really needed.

I had initially thought that we could just assert a few more axioms that would give ELK "hints" at what HermiT infers, but Hector has worked on this issue for a while, and now I'm less optimistic.

jamesaoverton avatar Jun 07 '21 16:06 jamesaoverton