Emile van Krieken

Should we save the originally passed tensor in `StochasticTensor`? This uses more memory, but could be safer.

Now that wrappers are mostly outside of the API (outside exception cases), having @cost unwrap and then add as a cost seems unnecessary. The wrapper could be removed in favour...

It might be possible that you sample two different `storch.Tensor`s that 'belong' to the same sampling step. For example, you sample halve of the dimensions of a variable using a...

We can safely override many methods from torch.Tensor. However, PyTorch contains many typechecks that explicitly want to find a torch.Tensor. So while storch.Tensors could be safely used there, it would...

See http://docs.pyro.ai/en/dev/ops.html#pyro.ops.indexing.Vindex

Reshape is quite difficult to make parallelizable: `logits = logits.reshape(logits.shape[:-1] + (20, 10))` works though. Maybe make it possible to use `logits.reshape[..., 20, 10]`?

Monte Carlo Gradient Estimation for Machine Learning discusses a method to test Gradient Implementation in Section 9.1. Should be easy enough to test.

Right now, one has to manually implement a loop to compute variances between estimators. Can this be automated?

Possible ideas for examples - [x] Simple VAE with gaussian latents - [ ] Bayes by backprop - [ ] Reinforcement learning/MDPs/Policy gradients, possibly with actor-critic architecture. - [ ]...

- [ ] Add support for automatic (k-history) markov decision processes for online RL. Probably something like ``` mdp = MDP(transition_function: Callable[(BlackboxTensor, BlackboxTensor), Tensor], action_function: Callable[Tensor, Tensor]) reward = mdp.simulate(initial_state,...