steppy

steppy copied to clipboard

bcottman

Structure of steps - ideas for making it cleaner

2

@kamil-kaczmarek, @jakubczakon I know it is a bunch of different ideas and suggestions clustered in one issue. Let me know which of those are compatible with the current roadmap. (I...

stared

kamil-kaczmarek

Hi, I have a proposal: let's make it possible to dump adapted input of a step to disk. It's very handy when you are working on a 5th or 10th...

apyskir

have to clean cache manually

2

Having to call `step.clean_cache()` is error-prone. Ideally, we should have automatic cache invalidation.

mromaniukcdl

Should Kaggle solutions become step-examples?

1

bcottman

Hi! Let's assume I'm training a huge network for a lot of epochs and it saves checkpoints in `checkpoints` folder. I suggest to prepare a possibility to run `transform` on...

apyskir

Do all Steps execute parallel?

2

Is it necessary to divide executions inside my class to be separate Thread or just divide them between Steps? For example, I can to fit KNN, PCA in one class...

denyslazarenko

check shapes of: * Transformer's input * Transformer's output * adapter input * adapter output these inputs / outputs may be images, Pandas DataFrames, Numpy arrays, Tensors, lists, tuples, etc.

kamil-kaczmarek

* in the logs -> mean cpu/gpu utilization per batch/epoch * in the neptune monitor callback -> print data on graph * use this [post with code snippet](https://community.neptune.ml/t/htop-for-cpu-and-memory-inspection/334)

kamil-kaczmarek