Tae Hwan Jung
Tae Hwan Jung
@seongpyoHong I've fixed ER Diagram like below: ```sql CREATE TABLE bucket ( id varchar(255) primary key not null, additional text not null, dataset_name varchar(255) not null, endpoint varchar(255) not null,...
Versions prior to 0.4.0 manage metadata in json format, so using only s3 could maintain the shape. However, since metadata is managed by the RDBMS, it is necessary to write...
@jinserk No, a lot of questions on this project don't bother me. Rather, I am happy to think that this project can be improved. First question: If there is no...
@jinserk Thanks for the great suggestion. As you suggested, adding it to `DataSaver.__del__()` doesn't seem to have any problem in terms of concurrency(multiprocessing). I will reflect on this. Thanks!
@jinserk The python destructor is not a function that is triggered when the class ends. Therefore, it seems more efficient to manage with python's Context Manager (`__enter()__`, `__exit__`) : ```python...
Like the previous issue (#21), this error seems to be a problem of the `list_object` function overridden in the NAS. If that doesn't work, please ask again for me. Thanks
Write the following code to check the inference time of one transformer layer: ```python import torch from transformers.configuration_gpt2 import GPT2Config from transformers.modeling_gpt2 import Block, GPT2Model def count_parameters(model): return sum(p.numel() for...
@jinserk Thank you for your interest in the project! - can I save any objects other than tensors, i.e. a tuple of tensors, a dict or a sparse tensors? In...
@jinserk Matrices with atypical shapes are difficult to store regardless of sparse. Moreover, sparse is not difficult to implement because it is guaranteed by through `scipy`(https://github.com/appier/h5sparse). However, it is very...
If so, how about storing the fixed tensor itself that goes into the model's input in matorage? This is the core idea of matorage. Also using high compressor leve(7~9) can...