Soojeong Kim

### Things to Change Add test set to test: - [ ] Correctness (make models deterministic, and see compare the exact numbers) - [ ] Throughput - [ ] Rule-wise...

### Things to Change To implement RNN models, a user can represent the RNN hidden state with Variable to correctly pass it between multiple session runs. However, current parallax just...

### Things to Change Since current in-graph replication logic makes both system and user-API complicated, we can explore if between-graph replication for PS would make sense in terms of performance....

### Things to Change According to NMT, embedding parameter goes to a worker if the number of embedding partitions is one. It happens in the Parallax, too. ### Current Behavior...

GPU layer tests free allocated memory explicitly, because the memory is allocated by native codes. Destroying layer related memory is already done in the code, but input or expectedError matrices...

DNN gpu version integration test with ImageNet data

3

The gpu side of dolphin-dnn needs integration tests on moderately sized clusters. ImageNet is a good candidate as a test dataset.

Implement multi-threaded Trainer.

2

As discussed in offline, multi -threaded Trainer should be implemented. Each Trainer thread shared training data to compute and they run until all of the current batch data is computed.

All of parameter gradients are collected from all layers and send them at the same time in regular sequence. Parameter gradients have no relationship each other, which means any gradients...

Clock for SSP is a unit of iteration(mini-batch), but it's not reflected in the dolphin code now. Considering both Async and SSP, clock should be redesigned.