build icon indicating copy to clipboard operation
build copied to clipboard

How to test Resources?

Open devkabiir opened this issue 6 years ago • 7 comments

  • Dart SDK Version (dart --version)
    • Dart VM version: 2.3.1 (Tue May 21 19:28:38 2019 +0200) on "windows_x64"
  • What package(s) from this repo you are using, and the version
    • build: 1.1.4
    • build_test: 0.10.8
    • build_config: 0.4.0
  • What builder(s) you are using (or writing yourself). Try to give a short summary of what they do. I have two builders, collector and generator
    • collector collects some meta info and runs before generator. Stores that meta data in a Resource which is a Map
    • generator then acts on that Map and generates single output.
  • Whether you are using Windows, MacOSX, or Linux (if applicable)
    • Windows 10 x64
  • Whether you are using Chrome, Safari, Firefox, Edge (if applicable)
    • Not applicable
  • Any other packages or constraints we should know about
    • Not that I can think of

How to test the Resource populated by collector?. How to write test in general for a builder that depends on another builder.

Both of the builders work as expected. But I want to write tests since I'll be adding more features to them.

devkabiir avatar Jun 21 '19 06:06 devkabiir

You can look at the DevCompilerBuilder unit test for an example.

The key is the testBuilderAndCollectAssets utility method which runs a builder using the normal testBuilder command and after it is done updates the original assets map to include all the outputs.

We could probably move that to utility to the build_test package, but you can just copy it for now as its pretty trivial.

jakemac53 avatar Jun 21 '19 13:06 jakemac53

In terms of directly testing a Resource they should be testable independently... it is really just giving you a handle on a persistent object. The thing you are actually returning should hopefully be unit testable.

jakemac53 avatar Jun 21 '19 14:06 jakemac53

@jakemac53 Thanks, The way I understand it, testBuilderAndCollectAssets is useful when Builder A depends on outputs/assets from builder B.

In my case collector does not produce any outputs it always updates a Resource and it is set to runs_before: ['my_package|generator']. Since generator runs in the same "execution" It has access to the Resource populated by collector. This resource is just a Map.

How do I run collector builder and test that the Resource is populated with correct values. And how to run generator with pre-populated/mocked Resource.

devkabiir avatar Jun 24 '19 17:06 devkabiir

Ah, I see what you mean now. In order to test a more real world scenario where resources are shared between testBuilder calls, or maybe even mock one, you would need to provide your own ResourceManager.

We would have to add an option to testBuilder which would plumb that through to its call to runBuilder. Otherwise it creates a new resource manager each time which means it always disposes the resources between calls.

Note however that the general request here is a bit of a red flag - resources need to work for incremental builds. In that case we would only rerun your collector builder when its inputs change, so its resource might not have a chance to collect whatever information you are trying to collect (it might not run at all, or may only run on a few of your primary inputs, depending on what changed). For that reason you should always be serializing this information to disk as well, so that your Resource can fall back on reading from disk the information it hasn't directly collected.

I do think it would be useful to be able to write tests with a custom ResourceManager though, as at a minimum you can probably run your tests faster.

jakemac53 avatar Jun 24 '19 18:06 jakemac53

@jakemac53 - We should probably expand the docs and add some warnings about putting information in a Resource that isn't reflected in outputs or inputs. We could also consider publishing a package for the DecodingCache implementation we have...

natebosch avatar Jun 24 '19 19:06 natebosch

I like the idea of publishing the DecodingCache. I guess the main question is where, it could potentially just be added to the build package.

jakemac53 avatar Jun 24 '19 19:06 jakemac53

I guess it would probably make sense for it to work off of bytes only, and maybe accept an encoder/decoder or something.

jakemac53 avatar Jun 24 '19 19:06 jakemac53