relax icon indicating copy to clipboard operation
relax copied to clipboard

Published 20 hours ago • verified publisher icon tlc-pack

[ROADMAP] Milestones

Open YuchenJin opened this issue 3 years ago • 3 comments

Functionality and Robustness

  • [x] M0: relax lowest form ⇒ VM
    • [x] M0b: IRBuilder/mutator interface initial finalization
    • [x] M0c: build mixed TIR/Relax IRModule
  • [x] M1: MVP lowering mechanism
    • [x] M1a: lower a program with call_dps, packedfunc only
    • [x] M1b: lower a program with call_dps + tir
    • [x] M1c: be able to build a IRModule that contains mixed relax and tir func
  • [x] M2: Code robustification and cleanup
    • [x] M2a: Sync with upstream meta-schedule, with a clean parser implementation
    • [x] M2b: BlockBuilder and Normalizer cleanup
    • [x] M2c: ExprMutator cleanup
    • [x] M2d: Refactor the MVP lowering mechanism
    • [x] M2e: End to end build flow IRModule => runtime.Module
    • [x] M2f: Migrate relax passes to Pass Manager infra
  • [ ] M3: End to end workload
    • [x] M3a: EmitTE mechanism for building workloads using TE and topi
    • [x] M3b: end to end inference workload through EmitTE
    • [ ] M3c: op => tir lowering rough sketch
    • [ ] M3d: op reg + op gradient
    • [x] M3e: end to end fine-tuning workload

Passes and Performance

Can be done in parallel after M2

  • [ ] Memory optimization and match graph runtime strategy for static shape
  • [x] AutoTIR integration
  • [ ] Fusion choices

YuchenJin avatar Nov 03 '21 15:11 YuchenJin

For M2, we might want to migrate current passes to the existing Pass Manager infra; For M3d and M3e, I am wondering what the plan is for AD algorithm, would it be a easy task?

ZihengJiang avatar Nov 03 '21 18:11 ZihengJiang

For M2, we might want to migrate current passes to the existing Pass Manager infra; For M3d and M3e, I am wondering what the plan is for AD algorithm, would it be a easy task?

Sounds good, I think we can migrate the passes to the Pass Manager Infra after M2. For the AD plan, I think in the short term, we want to write a relax gradient pass to generate the backward pass to demonstrate the transformer fine-tuning workload; in the long term, we want to enable AD on the TIR level. cc @altanh

YuchenJin avatar Nov 03 '21 19:11 YuchenJin

Would be good to add the "relax gradient pass" as a separate entry since I guess it will take some time.

ZihengJiang avatar Nov 03 '21 20:11 ZihengJiang