shipkit
shipkit copied to clipboard
Downstream testing improvement suggestions
I was brainstorming about the improvements to the behavior and logging of downstream testing. Here's what we could do to make this feature great (the feature is already pretty amazing!!!).
Logging
- when the build runs with -i show the child build output directly in the terminal
- add more lifecycle messages explaining what happens (Perhaps we should ask some other engineer who does not know Shipkit to explain what happens when he reads the log. If he cannot explain, let's add more descriptive information, like: "In order to test downstream dependency we are cloning remote repository first")
- when child build fails, dump entire build log to the lifecycle output (or perhaps last xxx lines so that we see the error / stack trace)
- can we use relative paths for logging lifecycle messages? Gradle Project has a way to show relative path instead of absolute path. This will make the log shorter, easier to read and understand.
- we could use GitHub Gist to store logs of invocations
- in the PR description, put the link to the Travis CI job that initiated the PR
- more descriptive start of the test:
//instead:
Run test of mockitoMockito. The output will be saved in /Users/sfaber/mockito/release/testDownstream/build/mockitoMockito-build.log
//we could:
Test downstream project 'mockito/mockito’\n
- build log: ...
- work dir: ...
Naming
Making some names simpler, shorter, easier to use:
- cloneProjectFromGitHubMockitoShipkitExample -> cloneMockitoShipkitExample
- cloneProjectToWorkDirMockitoShipkitExample -> copyMockitoShipkitExample
Incremental build
- can we get testDownstream incremental? E.g. when I run this task twice in a row without code changes, can this be UP-TO-DATE? We can use the build log as an output from execution, the inputs we know.
Nitpick
- about cloneProjectToWorkDirMockitoShipkitExample task - does it make more sense to use copy task instead of clone task? Just curious.
@wwilk, wdyt?
Related to #435 , I will consider the suggestions made here when working on it