torchchat icon indicating copy to clipboard operation
torchchat copied to clipboard

[Hackability Refactor] File structure Re-organization under a torchchat/ folder

Open Jack-Khuu opened this issue 5 months ago • 1 comments

This PR is a major restructuring of the files in the torchchat repo. In order to make this safe and easy to review, each of the commits are squashed into themes and intentionally avoids making any behavior changes wherever avoidable.

The largest overarching changes are:

  1. Most of the top level files/folders are now under a torchchat directory. This makes the repo more approachable and collects the files into rough themes.
  2. The ability to call export.py/generate.py/eval.py directly has been removed. While this may be a feature we reintroduce (improve), it currently introduces complexity and inconsistency (multiple entry points/imports).

Click through the final structure in the branch: https://github.com/pytorch/torchchat/tree/hackability_refactor_base

Note: This PR aims to establish a general file structure of the repo (compared to the relatively flat structure previously). It does not prescribe a rigid system.

Some of the intentionally excluded changes:

  • distributed files are mostly untouched; these will eventually be integrated into torchchat, but this is not the time/place
  • runner/tokenizer are mostly untouched; these are relatively self contained and will be refactored separately.
  • torchchat.py remains on root; this will eventually be replaced by a contained executable

Jack-Khuu avatar Aug 28 '24 20:08 Jack-Khuu