chore: Introduce CLI tests to ensure regression is caught in CI
Currently, the scaffolding CLI testing relies heavily on manual processes, which has led to several challenges:
- Issues with file/directory generation are only caught through manual testing
- The testing process is time-consuming
- Edge cases may be missed due to the manual nature of testing
- No automated verification of scaffolding output
Current Test Coverage Limitations
While we have integration tests using tryorama that verify the generated hApp's zome functions work as expected, these tests do not verify the CLI's code generation functionality. This has led to issues:
Recent example: The ui directory was completely omitted due to Crane lib filtering out .hbs files, but existing tests didn't catch this
Current tests focus on zome function behavior but don't verify the actual scaffolding output
No automated verification of template processing and file generation
Proposed Solution
Implement automated CLI tests in our CI pipeline to:
- Verify correct file/directory generation
- Ensure scaffolding creates expected structure
- Validate command outputs
- Catch regressions early
Scope
Phase 1: Basic CLI Testing
- [ ] Set up CLI tests
- [ ] Implement tests for basic file/directory generation
- [ ] Add assertions for common scaffolding patterns
- [ ] Verify command execution results
- [ ] Add tests for error cases and edge scenarios
Phase 2 (Optional): UI Snapshot Testing
- [ ] #421
- [ ] Set up snapshot comparison in CI
Technical Details
Test Coverage Should Include
- Directory structure validation
- File content verification
- Command execution flows
- Error handling scenarios
Example Test Cases
- Verify scaffold command creates expected directory structure
- Validate file permissions and contents
- Test command option combinations
- Verify error messages for invalid inputs
Success Criteria
- [ ] All CLI commands have automated tests
- [ ] CI pipeline includes CLI testing
- [ ] Reduced time spent on manual testing
- [ ] Improved confidence in CLI functionality
- [ ] Documentation for writing new CLI tests
Prioritizing #421