Tensorflow Libraries

TFLibrary is an open-source toolkit based on Tensorflow, with the design goals of modularity, flexibility and extensibility. It is in active development and used extensively in my own researches.

• Data utility functions for tf.data, inspired from Google's NMT

• Metrics simple abstractions of metrics, including implementations of a few commonly-used metrics.

• Models simple wrapper class around models.

• Modules a simplified version of sonnet.modules for modular model design.

  • Modules.attention attention modules takes in as input multiple inputs and compute attended features, currently includes matrix cross attention and BiDAF-style attention.
  • Modules.embedding embedding modules takes texts or text-ids and produce dense representations, currently includes matrix embedding, ELMO embedding, and GLOVE embedding.
  • Modules.encoders encoder modules encodes dense representations of input texts, currently includes LSTM encoder and BiDAF-style encoder.
  • Modules.transformer transformer modules are simplified / decomposed version of Transformer in Tensor2Tensor library, currently include Transformer encoder.

• Remotes simple wrappers for method calls that enable them to be executed remotely.

• Tuner Hyper-parameter tuner using grid search as well as Bayesian Optimization, with support for multi-GPUs in parallel computations settings.

• utils a collection of various utility functions.

  • utils.TrainingManager a light helper class for monitoring training progress, and providing signals for early stopping.

• Bandit multi-armed bandit.

• misc there are also a few other models, including DDPG, pointer network and its rnn_cell_impl.RNNCell wrapper version.

Installation

pip install -e .
pip install -r REQUIREMENTS.txt

Docker

# using `--no-cache` to avoid accidentally missing changes in parent tensorflow image
Tags="tf2.0.0-torch1.3.1"
GitBranch=`git rev-parse --abbrev-ref HEAD`
docker build -t tf-library:${GitBranch}-${Tags} .

Versions

Version 0.9

• Moved Tuner from utils/ to its own directory Tuner/, and changed both the interface and implementation of Tuner to support Bayesian Optimization based tuning and more flexible extension.