torchtitan
                                
                                 torchtitan copied to clipboard
                                
                                    torchtitan copied to clipboard
                            
                            
                            
                        remove PP tracer
Stack from ghstack (oldest at bottom):
- -> #555
Discussed with @wconstab and @kwen2501 , it seems PP tracer has two limitations right now:
- It doesn't support init_weights, thus requiring a seed checkpoint to do init, which is something we probably will deprecate soon after we have the init support for manual splitting.
- It doesn't support mixed precision training, as the tracer requires that the model being traced be consistent with the model during forward.
We came to the conclusion to remove PP tracer for now, to have a clean version of torchtitan.