Augmentor
Augmentor copied to clipboard
mdbloice
Using Augmentor with TensorFlow
How can i use Augmentor on the fly with tensorflow? There's an example with Keras. Can you provide an example with tensor flow too?
Hi there, as I do not have TensorFlow currently installed on my main dev machine, I can't really add this functionality at present. However, it is something I definitely want to add, so I will take a look at doing this some time soon. I can't make any promises regarding when this might be unfortunately. That said, I'd be happy to take a look at a PR you might want to make!
you may like miniconda to setup tensorflow with cpu / very simple to get up and running. https://gist.github.com/johndpope/187b0dd996d16152ace2f842d43e3990
@benjaminbarbe If I have correctly understood your question, I guess that there is no too much difference in case of tensorflow. At least, when you're writing a training loop by hand, like in this snippet (probably has typos, but I guess you've got the idea):
from Augmentor.Operations import RotateRange
from PIL import Image
op = RotateRange(probability=0.5, max_left_rotation=30, max_right_rotation=30)
with tf.Session() as session:
tf.global_variables_initialized().run()
for epoch in range(n_epochs):
for x_batch, y_batch in training_batches:
pil_images = [Image.fromarray(x) for x in x_batch]
augmented = op.perform_operation(pil_images)
arrays = [np.asarray(img, dtype=float) for img in augmented]
sess.run([train_op], feed_dict={x: x_batch, y: y_batch})
Probably there are some helpful utils in the library to make this process more simple. Though if you want to augment images right on GPU, then you need something more involved.
Tensorflow has the standard data processing API. I'm interested to interface Augmentor with the API.
As I can see, Augmentor is implemented entirely in Python. It should be possible to use tf.py_func to wrap Augmentor's functionality and provide data mappers that can be used with tf.data.Dataset.
@mdbloice, I'm not very familiar with Augmentor, so what would be a good point to start? Also, can Augmentor process images on a GPU?
I came across this repo the other week - https://github.com/tueimage/SE2CNN it will build a roto-translation covariant convolutional network with tensorflow. (so you don't need to add extra images to training augmentation - the neural net will allow invariants in rotation / translation) https://arxiv.org/pdf/1804.03393
fyi - Jonas Adler @adler-j has created a tensorflow example for deforming images with his own function. There's a tensorflow sample. Should be able to cherry pick / glue tensorflow code https://github.com/adler-j/tfdeform
Hi @alshedivat, sorry for the delay in replying, work duties have kept me from doing any work on Augmentor for the past few weeks. Actually, what I am working on next is proper support for TensorFlow. Right now, using Augmentor with Keras should work, and PyTorch should also work. Next step is to get Augmentor working with TF properly without workarounds. Expect it in the next release (although I cannot say for sure when that might be, unfortunately).
@mdbloice, cool, thanks for your reply! Do you have a roadmap for TF support? I'm pretty familiar with TF and it's Dataset API, so happy to contribute if that accelerates development.
Also, it looks like Augmentor doesn't use opencv, does it? Another image augmentation library shows that using highly optimized opencv gives best throughput.
HI @alshedivat, no I do not really have a roadmap right now for TF support, I am only able to work on Augmentor when time allows. And it's not really predictable when that might be, so I cannot say for sure when I might be able to work on new features or bugs unfortunately!
As for OpenCV, no I do not use it because it is a rather large dependency that I wanted to avoid. The library you linked to looks good though! I think with multi-threading the throughput in Augmentor is pretty good.
As for contributing, that would be great - I presume TF expects a generator of some kind, and if you check out the code for the keras_generator() function, you'll see how it works:
If you can write a function like tf_generator or something similar that would output images in the format that is expected by TensorFlow, then I think that would be all there is to it! Let me know what you think.