data-juicer

data-juicer copied to clipboard

cyruszhang

Would it be more appropriate to change the return type of the read, read_json, and read_webdataset functions in the RayDataSet class from RayDataSet to ray.data.Dataset? Because the data returned by...

cn-robot-ai

### Before Reporting 报告之前 - [x] I have pulled the latest code of main branch to run again and the bug still existed. 我已经拉取了主分支上最新的代码，重新运行之后，问题仍不能解决。 - [x] I have read the...

kongzhinvwang2

你好，最近使用data-juicer，由于公司的网络是隔离的，不能访问外网，请问可以把能用的模型放到一个文件夹？方便一次性下载上传到内部空间。不慎感激。如果用docker镜像还需要下载这些模型？

zhanglv0209

在使用提供的的jupyterLab提供的palyground时候，网站报错

2

cyrilmyself

[Bug]: video_split_by_scene_mapper Runing with: [INFO] Process ffmpeg is running without libra.

1

### Before Reporting 报告之前 - [x] I have pulled the latest code of main branch to run again and the bug still existed. 我已经拉取了主分支上最新的代码，重新运行之后，问题仍不能解决。 - [x] I have read the...

Cccccc0630

As title says.

chenyushuo

Add Operator-Level Parallel Data Processing with Ray Actors

1

This PR introduces significant performance optimizations for parallel data processing using Ray Actors and multi-threading. The following changes have been implemented: 1. **Dynamic Resource Allocation**: - Multiple Actors per operator...

Cccccc0630

后续是否会为DefaultExecutor提供stop方法

3

### Before Asking 在提问之前 - [x] I have read the [README](https://github.com/alibaba/data-juicer/blob/main/README.md) carefully. 我已经仔细阅读了 [README](https://github.com/alibaba/data-juicer/blob/main/README_ZH.md) 上的操作指引。 - [x] I have pulled the latest code of main branch to run again and...

HunterLG

Reaching out from the Ray Team!

3

### Search before continuing 先搜索，再继续 - [x] I have searched the Data-Juicer issues and found no similar feature requests. 我已经搜索了 Data-Juicer 的 issue 列表但是没有发现类似的功能需求。 ### Description 描述 Hi! This is...

richardliaw