decord icon indicating copy to clipboard operation
decord copied to clipboard

VideoLoader memory leak until program crash

Open MattsonCam opened this issue 2 years ago • 1 comments

While iterating through the batches of the VideoLoader, the memory approaches 100% until eventually crashing. This occurs for all possible values of the shuffle parameter.

Code to Reproduce:

import numpy as np from sklearn.preprocessing import OneHotEncoder import decord as dc

ctx = dc.cpu(0) batchsz = 10 shape = (batchsz, height, width, 3)

interval = 20 skip = 50

vl = dc.VideoLoader(videopaths, ctx=ctx, shape=shape, interval=interval, skip=skip, shuffle=2)

def frameGenerator(): while True: vl.reset() i = 1 for batch in vl: result = i % 15 #k = [[video2label[p]] for p in np.array(batch[1][:,0])] k = [[video2label[p]] for p in batch[1].asnumpy()[:,0]] y_train = onehot.transform(k) x_train = batch[0] i += 1 yield x_train.asnumpy(), y_train

traingen = frameGenerator() while True: next(traingen)

MattsonCam avatar Jun 09 '22 04:06 MattsonCam

This may be related to #208 and #197.

sueskind avatar Jun 26 '22 15:06 sueskind