List of potential libraries and API services to add

[tyarkoni]

This is a standing issue for tracking potential libraries and API services to wrap in pliers.

Multimodal/major APIs

• [ ] Amazon Rekognition
• [x] Microsoft Azure APIs

Audio

• [x] Librosa
• [ ] A-weighting of AudioStims
• [ ] madmom

Image

• [x] face_recognition (this one should be trivial)
• [ ] neural-doodle and neural-enhance
• [ ] OpenPose (with Python API)

Language

• [x] spaCy
• [ ] DeepSpeech

[qmac]

https://github.com/abhishekbanthia/Public-APIs

[adelavega]

Amazon Rekognition - https://aws.amazon.com/rekognition/

[tyarkoni]

@qmac--I think we should make Amazon Rekognition our highest priority for addition. The video features look amazing.

[PeerHerholz]

Ahoi hoi folks,

here are some more ideas/suggestions. Spoiler: all auditory!

• [ ] remaining librosa features (beat/tempo)
• [ ] bregman toolkit (might need python 2 -> 3 conversion)
• [ ] essentia -> cool stuff like mood detection, segmentation and classification, integration with AcousticBrainz
• [ ] pyaudioanalysis -> feature extraction, segmentation, classification, pre-trained models included, speaker diarization, emotion recognition, etc.
• [x] AudioSet -> (pre-trained) CNNs and huge data set
• [ ] aubio -> feature extraction

[rbroc]

Pointer to new potentially interesting library tokenizers, although some tokenization options area will already be covered by the transformers wrapper.

[rbroc]

Library for co-reference resolution which works as pipeline extension for SpaCy: neuralcoref.

Finds expressions in text that refer to same entity and groups them in clusters.

At token level, can yield info of which reference cluster(s) (if any) the token expression is part of. At document level, can be used to extract metrics such as number of entities named, and number of mentions per entity.

Potential use case: tracking mentions of characters along a narrative/dialogue.

[rbroc]

New pretty cool object recognition model trained with contrastive learning: https://github.com/google-research/simclr. Paper: https://arxiv.org/abs/2002.05709

[rbroc]

Auditory English Lexicon Project (auditory lexical norms): haven't taken a close look yet, but for future reference, paper here: https://doi.org/10.3758/s13428-020-01352-0, website with data here: https://inetapps.nus.edu.sg/aelp/