oneflow

oneflow copied to clipboard

mosout

## Summary Pytorch kernel implements the “round half to even” to break ties when a number is equidistant from two integers (e.g. round(2.5) is 2). （ https://pytorch.org/docs/stable/generated/torch.round.html#torch.round ） ## Code...

wyg1997

hjchen2

The custom lr_scheduler form of oneflow and pytorch is different

4

pytorch version: 1.12.1+cu102 oneflow: ``` version: 0.8.1+cu112.git.c0811b327a git_commit: c0811b327a cmake_build_type: Debug rdma: False mlir: False ``` As is shown in the next code, the args of func `get_lr` is different...

clackhan

MARD1NO

Use straighten algorithm for memory. And also compress the memory through adjustment. Can reduce memory in some cases.

Yipeng1994

RankTaskGraph

4

将TaskGraph的逻辑拆解成BoxingTaskGraph和RankTaskGraph。BoxingTaskGraph负责构建boxing相关的task graph子图，然后序列化到BoxingTaskGraphProto。RankTaskGraph负责两点：1）构建指定rank的CompTaskNode；2）从BoxingTaskGraphProto恢复属于boxing部分的子图； 分布式编译的大体过程将会是： 1. 在main线程（或master进程）上由OpGraph构建BoxingTaskGraph，并序列化成BoxingTaskGraphProto； 2. 在线程池里的各个worker线程（或worker进程）上由OpGraph/BoxingTaskGraphProto/rank等信息构建属于该rank的RankTaskGraph，然后生成该rank的plan。 本pr实现的是分离编译的中间状态版本：即BoxingTaskGraph在main线程上构建，而RankTaskGraph在线程池里构建。 后续pr再实现彻底的分离编译，即BoxingTaskGraph在master进程上构建，而RankTaskGraph在worker进程上构建。

lixinqi

Fix https://github.com/Oneflow-Inc/OneTeam/issues/1686

mosout

https://github.com/Oneflow-Inc/OneTeam/issues/1644#issuecomment-1225386553 中描述的实现。

leaves-zwx

pytorch version: 1.12.1+cu102 oneflow: ``` version: 0.8.1+cu112.git.c0811b327a git_commit: c0811b327a cmake_build_type: Debug rdma: False mlir: False ``` ```python >>> import oneflow as flow >>> x = flow.randint(3, 9, (1,)) >>> y...

clackhan