swift-models

swift-models copied to clipboard

Add tensorflow/swift-apis as a SwiftPM dependency.

2

## Motivation This enables building `tensorflow/swift-models` using stock toolchains from [swift.org/download](https://swift.org/download/#snapshots). `swift build` will clone and build `tensorflow/swift-apis` as a regular SwiftPM dependency. Eventually, we would like to stop releasing...

dan-zheng

Find a more descriptive word for `numericalized`

2

In a few places ([1](https://github.com/tensorflow/swift-models/blob/master/Datasets/LanguageModelDataset.swift#L17), [2](https://github.com/tensorflow/swift-models/blob/master/Datasets/WordSeg/WordSegRecord.swift#L19)), we've started using the term `numericalized`, which might be definitionally accurate, but is a bit cumbersome. Let's come up with a better term to...

texasmichelle

Add transfer learning support to image classification models

5

It would be super helpful if we could use the predefined models in this repository with pretrained weights for transfer learning like keras-applications does.

rickwierenga

As brought up by Xander in [swift-apis issue #25](https://github.com/tensorflow/swift-apis/issues/25), our efforts on checkpointing have thus far focused on model state checkpoints, but there are other state parameters that should be...

BradLarson

Better error message on unzip failure

2

I attempted to run benchmarks on a fresh instance that did not have `unzip` installed. The error message is misleading, stating that the file does not exist, even though the...

texasmichelle

Complex Loss Functions Inside TrainingLoop

3

Thanks to @xihui-wu's talk earlier today, I learned about the [TrainingLoop](https://github.com/tensorflow/swift-models/blob/542a03c09aa7bd6e326a18b5d086f2bec8b4b24c/TrainingLoop/TrainingLoop.swift#L180) struct. I had essentially replicated this functionality in a messier way in my code, so I'm looking at it...

xanderdunn

GPT-2 currently exhausts all available GPU memory on an 8 GB GPU

2

In testing PR #671, we noticed that the GPT-2 model now exhausts all available memory on 8 GB GPUs (example: GTX 1080) for both eager mode and X10 runtimes. It...

BradLarson

Add logging callbacks for training statistics

1

The training loop abstraction has a visual progress bar for tracking statistics during training. In many cases, it's desirable to log or store training statistics for later analysis, or to...

BradLarson

Transformer model requires more parameters than supported on TPU

2

It [has been pointed out by Wojtek Czarnowski](https://groups.google.com/a/tensorflow.org/d/msg/swift/NyAg6MV5PT0/0KjPYSGdBQAJ) that in specific cases the Transformer model (or components used within it) can trigger a compilation error in X10 on TPU: ```...

BradLarson

Implementing graph convolutional neural networks

1

Given the recent popularity of graph convolutional neural networks (i.e. https://github.com/tkipf/gcn), would it be worth implementing a swift paradigmatic version? I have a semi-working version, but would defer to those...

hector-crean