python-docs-samples icon indicating copy to clipboard operation
python-docs-samples copied to clipboard

[FIrestore Samples] Update tests to be self-contained and verified for correctness.

Open frankyn opened this issue 5 years ago • 8 comments

In which file did you encounter the issue?

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/firestore/cloud-client/snippets_test.py

Describe the issue

Tests aren't self contained and also not being verified for correctness. At the moment, tests only run the samples and verify that they execute without failure. Although, tests do provide some level of support using this method, they aren't verifying state correctness of Firestore post after testing a given sample.

frankyn avatar Oct 24 '18 23:10 frankyn

This may be fixed in the updated tests from @BenWhitehead.

gguuss avatar Oct 10 '19 20:10 gguuss

There are a few older tests that need to have assertions added to them.

BenWhitehead avatar Oct 11 '19 15:10 BenWhitehead

@BenWhitehead do we have the individual tests we should fix?

crwilcox avatar Dec 31 '19 00:12 crwilcox

@crwilcox I think with your updated bootstrapping for the tests that should hopefully transparently take care of this.

@frankyn Do you have specific tests that we should double check to make sure the fix from @crwilcox addresses things?

BenWhitehead avatar Jan 02 '20 17:01 BenWhitehead

@crwilcox PTAL

andrewferlitsch avatar Mar 16 '20 18:03 andrewferlitsch

Apologies, I missed this in email.

The tests are not validating correctness and only making sure samples execute without failures.

Here's an example of a test being verified: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/firestore/cloud-client/snippets_test.py#L74-L80

Here's an example of test not being verified: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/firestore/cloud-client/snippets_test.py#L70-L71

frankyn avatar Mar 17 '20 17:03 frankyn

I've been in touch with @BenWhitehead recently, and it is still in the pipeline.

leahecole avatar May 14 '20 19:05 leahecole