shipkit icon indicating copy to clipboard operation
shipkit copied to clipboard

Downstream testing improvement suggestions

Open mockitoguy opened this issue 8 years ago • 2 comments

I was brainstorming about the improvements to the behavior and logging of downstream testing. Here's what we could do to make this feature great (the feature is already pretty amazing!!!).

Logging

  • when the build runs with -i show the child build output directly in the terminal
  • add more lifecycle messages explaining what happens (Perhaps we should ask some other engineer who does not know Shipkit to explain what happens when he reads the log. If he cannot explain, let's add more descriptive information, like: "In order to test downstream dependency we are cloning remote repository first")
  • when child build fails, dump entire build log to the lifecycle output (or perhaps last xxx lines so that we see the error / stack trace)
  • can we use relative paths for logging lifecycle messages? Gradle Project has a way to show relative path instead of absolute path. This will make the log shorter, easier to read and understand.
  • we could use GitHub Gist to store logs of invocations
  • in the PR description, put the link to the Travis CI job that initiated the PR
  • more descriptive start of the test:
//instead:
Run test of mockitoMockito. The output will be saved in /Users/sfaber/mockito/release/testDownstream/build/mockitoMockito-build.log

//we could:
Test downstream project 'mockito/mockito’\n
   - build log: ...
   - work dir: ...

Naming

Making some names simpler, shorter, easier to use:

  • cloneProjectFromGitHubMockitoShipkitExample -> cloneMockitoShipkitExample
  • cloneProjectToWorkDirMockitoShipkitExample -> copyMockitoShipkitExample

Incremental build

  • can we get testDownstream incremental? E.g. when I run this task twice in a row without code changes, can this be UP-TO-DATE? We can use the build log as an output from execution, the inputs we know.

Nitpick

  • about cloneProjectToWorkDirMockitoShipkitExample task - does it make more sense to use copy task instead of clone task? Just curious.

mockitoguy avatar Aug 30 '17 16:08 mockitoguy

@wwilk, wdyt?

mockitoguy avatar Aug 30 '17 16:08 mockitoguy

Related to #435 , I will consider the suggestions made here when working on it

wwilk avatar Oct 06 '17 22:10 wwilk