Vedant Roy

Cannot read from parquet files that contain binary

2

### 🐛 Describe the bug I create some parquet files with the following: ``` def save_tensor(t): buf = io.BytesIO() th.save(t, buf) return buf.getvalue() ``` ``` for idx, batch in enumerate(tqdm(dl,...

Assorted Issues

12

** Environment ** - OS: Ubuntu 22.04 LTS - GPU: ``` *-display description: VGA compatible controller product: GP102 [GeForce GTX 1080 Ti] vendor: NVIDIA Corporation physical id: 0 bus info:...

[Q] Clarify how EMA plugin works?

4

I have a few questions about the EMA docs: https://docs.mosaicml.com/en/latest/method_cards/ema.html - Will the EMA version of the model automatically be used during inference time? - What if I want to...

[Q] How is the output of validate mapped to metrics?

6

I'm trying to do something simple: log validation loss every N batches. Reading through the docs I find this: ``` def validate(self, batch): labels = batch.pop('labels') output = self.forward(batch) output...

How to ignore errors from gulp-typescript?

2

I have a legacy project with broken/missing types, but I want to emit the code anyway. Here's a small bit of code: ```typescript function tsPipeline(src, dst, extraLibs) { return function...

The SimCLR states the importance of global batch norm: > In distributed training with > data parallelism, the BN mean and variance are typically > aggregated locally per device. In...

Read videos from memory

1

### 🚀 The feature I have a lot of small videos (3 million video files), in an attempt to reduce disk bottleneck, I store all the videos in LMDB using:...

Interrupted System call when doing multi-GPU training

5

** Environment ** - 8 A100s although also happens on other GPU types ``` No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.4 LTS Release: 20.04 Codename: focal...

Support Cloudflare R2 for object store check-pointing

7

Cloudflare R2 is cheaper than S3 (no egress fees) and is very easy to use. It is fully S3 compatible, so it should be possible to just use the S3...

yaque drops items?

2

Here's an example: ```rust use std::path::Path; use tokio::{ sync::mpsc::{channel, Receiver, Sender}, }; use yaque::{self, TryRecvError, recovery}; const CHAN_BUF_SIZE: usize = 32; const BATCH_SIZE: usize = 50; const QUEUE_PATH: &str =...