Matthew R. Becker
Matthew R. Becker
So 1 sounds interesting. For 2 sklearn's preferred serialization format is through joblib. How would this interact with that?
Mmmmk. Make a PR and we can work on it.
I finally did some benchmarking here. For smallish batches sizes, we will see ~20% percent performance improvements. At large batch sizes this API is actually slower from what I can...
Yep just a CPU. I don’t follow your logic here. How does tensorflow deal with memory management and transfers from the cpu to the gpu using the datasets api? Why...
I did some reading. The dataset API currently runs only on the CPU. Apparently people use it to help stage data onto the gpu efficiently while the gpu is executing...
I just set the prefetched amount to 10x the batch size. Usually unless the code is doing a lot of memory allocation, competition for memory is not a problem on...
I can check tensorboard at some point but TBH I am not sure it is worth it right now.
Also to clarify, this test was for data already in memory as opposed to commming off of disk.
Looks like tf 1.5 now supports sparse tensors in the dataset api. This could make implementing this easier.
Yup. We will def start using some of these. Circle is killing our jobs a lot.