Payload Redesign

[hepengfe]

This PR fixes https://github.com/asyml/forte/issues/841.

Description of changes

• create base payload ontologies with modalities (TextPayload, AudioPayload, ImagePayload in forte/data/ontology/top.py)
• add some example ontologies generated by json files such as JpegPayload and SoundFilePayload in ft/onto/payload_ontology.py
• add forte/utils/payload_factory.py.
  • PayloadFactory a variable holds the mapping from payload to its loading function across modules
  • register() function that can be used as a decorator for payload classes to register the function mapping in PayloadFactory
• add test case for audio and image payloads

Possible influences of this PR.

Describe what are the possible side-effects of the code change.

Test Conducted

Describe what test cases are included for the PR.

An implementation template:

Payload:
 - uri
 - meta
 - index in data pack
 
 
TextPayload(Payload) # Json schema
​
​
{
  "entry_name": "ft.onto.ImagePayload"
  "parent_entry": "forte.data.ontology.top.Payload"
}
​
{
  "entry_name": "ft.onto.JpegMeta"
   "attributes":{
    "extension": str,
    "mime": str,
    "type_code": str,
    "version": str,
  }
}
​
{
  "entry_name": "ft.onto.JpegPayload"
  "parent_entry": "ft.onto.ImagePayload"
  "attributes":{
    {
       "attribute_name": "meta",
       "attribute_type": "ft.onto.JpegMeta",
    }
  }
}
​
​
"Jpegload" <--> "Loading function"
​
class PayloadFactory:
   def register(meta_name, func/class):
       # check whether meta_name is a real meta data
       # remember that this func will handle this meta
       
       
class Payloading():
    def route(meta):
      
class CustomPayloading(Payloading):
    import jpeg_lib
    def route_meta(meta):
       # if else conditions of meta:
          func(meta)
          
       return obj # to be store in cache.
       
       
       
pack = DataPack()
​
payload_entry = pack.add(JpegPayload("url"))
​
payload_entry = pack.get_payload_at(1)
​
payload_entry.get_type()
​
# enum
payload_entry.get_modality()
​
image1 = pack.get_payload_at(1)
​
pack.get("Sentence") # iterating all payload inside the pack.
​
pack.get_payload_at(0).get("Sentence") # iterating all payload of of text at 0
​
pack.get_payload_at(0).get("ft.onto.image.Object") # 1) showing nothing 2) raise error
​
for obj in image1.get("ft.onto.image.Object"):
  print(obj.class)

[hepengfe]

A review on the draft would be helpful for further implementation.

[codecov[bot]]

Codecov Report

Merging #859 (83c454e) into master (fd717ff) will increase coverage by 0.06%. The diff coverage is 93.85%.

@@            Coverage Diff             @@
##           master     #859      +/-   ##
==========================================
+ Coverage   80.93%   80.99%   +0.06%     
==========================================
  Files         253      256       +3     
  Lines       19677    19821     +144     
==========================================
+ Hits        15925    16055     +130     
- Misses       3752     3766      +14     

Impacted Files	Coverage Δ
forte/data/ontology/core.py	80.22% <ø> (ø)
ft/onto/base_ontology.py	95.25% <ø> (-0.27%)	:arrow_down:
ft/onto/wikipedia.py	94.11% <ø> (ø)
ftx/onto/ag_news.py	100.00% <ø> (ø)
ftx/onto/race_qa.py	100.00% <ø> (ø)
tests/forte/data/readers/audio_reader_test.py	95.34% <ø> (-0.11%)	:arrow_down:
forte/data/data_pack.py	85.00% <71.42%> (+0.10%)	:arrow_up:
tests/forte/image_annotation_test.py	80.76% <87.50%> (-0.32%)	:arrow_down:
forte/data/ontology/ontology_code_generator.py	89.75% <92.00%> (-0.25%)	:arrow_down:
forte/data/ontology/top.py	79.25% <92.59%> (+1.08%)	:arrow_up:
... and 8 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[hepengfe]

Reminder for changes https://github.com/asyml/forte/pull/891#discussion_r931670605