Hariharan Devarajan
Hariharan Devarajan
### Description Currently, unifyfs clients read node-local data from the server thread which is a bottleneck. In the case of Laminated files, we can fetch the node-local extents from the...
## Description In the newer version of glibc this is a compile-time error. ``` client_read.c:697:30: error: argument 1 range [18446744071562067968, 18446744073709551614] exceeds maximum object size 9223372036854775807 [-Werror=alloc-size-larger-than=] reqs = (read_req_t*)...
### Describe the problem you're observing In the case of producer-consumer workloads where a subset of producers produce and then another subset of processes on a different set of nodes...
### Describe the problem you're observing Currently, unifyfs supports several tools such as transferring data from pfs to unifyfs. These tools are extremely useful but there is no programmatic way...
### System information Type | Version/Name --- | --- Operating System | ubuntu OS Version | 18.04 Architecture | x86_64 UnifyFS Version | 0.9.2 ### Describe the problem you're observing...
### System information Lassen Machine [link](https://hpc.llnl.gov/hardware/compute-platforms/lassen) ### Describe the problem you're observing The code hangs and timeouts on two UnifyFS calls. `unifyfs_dispatch_transfer` and `unifyfs_finalize` for `unifyfs_dispatch_transfer` it hangs for the...
In my use case, I have the following project structure. - My project - external - SCR - venv - scripts Currently, there is no way to pass the INSTALL...
1. Rename current data loaders and reader to DLIO 2. Create native implementation for data loaders and readers. Fixes #80
@zhenghh04 there is an option with Pytorch data loader to make works persistent using `persistent_workers=True`. I believe this will reduce overhead of spawning workers within Pytorch and help the data...
@zhenghh04 Can you please help me describe at a high-level what are the fields we need for post processing and what are the queries we need to calculate?