Test suite fails if run with random ordering

[andyw8]

RSpec defaults to --order defined.

With --order random, I see varying failures, such as below.

I ran into this because I have a .rspec config in my home directory. My dotfiles are based on thoughtbot's which contain this:

https://github.com/thoughtbot/dotfiles/blob/aaa2c3cd09eb2148e19268e5ef827fab0292b7cd/rspec#L1-L2

In Effective Testing with RSpec 3 the authors recommend using random ordering:

As we discussed in ​Testing the Invalid Case​, RSpec is capable of running your specs in :random order, and we generally recommend you do that.

---

asset_cloud master % be rspec --order=random

Randomized with seed 32601
....................................................................FFFFF............................................................F..........

Failures:

  1) AssetCloud::GCSBucket #read raises AssetCloud::AssetNotFoundError if the file is not found
     Failure/Error:
       expect do
         @bucket.read(key)
       end.to raise_error(AssetCloud::AssetNotFoundError)

       expected AssetCloud::AssetNotFoundError, got #<NoMethodError: undefined method `gcs_bucket' for #<GCSCloud:0x00007f808ba17558>> with backtrace:
         # ./lib/asset_cloud/buckets/gcs_bucket.rb:45:in `bucket'
         # ./lib/asset_cloud/buckets/gcs_bucket.rb:63:in `find_by_key!'
         # ./lib/asset_cloud/buckets/gcs_bucket.rb:10:in `read'
         # ./spec/gcs_bucket_spec.rb:117:in `block (3 levels) in <top (required)>'
         # ./spec/gcs_bucket_spec.rb:116:in `block (2 levels) in <top (required)>'
     # ./spec/gcs_bucket_spec.rb:116:in `block (2 levels) in <top (required)>'

  2) AssetCloud::GCSBucket #delete removes the file from the bucket
     Failure/Error:
       expect do
         @bucket.delete(key)
       end.not_to raise_error

       expected no Exception, got #<NoMethodError: undefined method `gcs_bucket' for #<GCSCloud:0x00007f808ba17558>> with backtrace:
         # ./lib/asset_cloud/buckets/gcs_bucket.rb:45:in `bucket'
         # ./lib/asset_cloud/buckets/gcs_bucket.rb:63:in `find_by_key!'
         # ./lib/asset_cloud/buckets/gcs_bucket.rb:25:in `delete'
         # ./spec/gcs_bucket_spec.rb:99:in `block (3 levels) in <top (required)>'
         # ./spec/gcs_bucket_spec.rb:98:in `block (2 levels) in <top (required)>'
     # ./spec/gcs_bucket_spec.rb:98:in `block (2 levels) in <top (required)>'

  3) AssetCloud::GCSBucket #ls with arguments returns the file
     Failure/Error: @bucket ||= cloud.gcs_bucket

     NoMethodError:
       undefined method `gcs_bucket' for #<GCSCloud:0x00007f808ba17558>
     # ./lib/asset_cloud/buckets/gcs_bucket.rb:45:in `bucket'
     # ./lib/asset_cloud/buckets/gcs_bucket.rb:63:in `find_by_key!'
     # ./lib/asset_cloud/buckets/gcs_bucket.rb:6:in `ls'
     # ./spec/gcs_bucket_spec.rb:36:in `block (2 levels) in <top (required)>'

  4) AssetCloud::GCSBucket #stat returns information on the asset
     Failure/Error: @bucket ||= cloud.gcs_bucket

     NoMethodError:
       undefined method `gcs_bucket' for #<GCSCloud:0x00007f808ba17558>
     # ./lib/asset_cloud/buckets/gcs_bucket.rb:45:in `bucket'
     # ./lib/asset_cloud/buckets/gcs_bucket.rb:63:in `find_by_key!'
     # ./lib/asset_cloud/buckets/gcs_bucket.rb:35:in `stat'
     # ./spec/gcs_bucket_spec.rb:132:in `block (2 levels) in <top (required)>'

  5) AssetCloud::GCSBucket #write writes a file into the bucket with content_disposition
     Failure/Error: @bucket ||= cloud.gcs_bucket

     NoMethodError:
       undefined method `gcs_bucket' for #<GCSCloud:0x00007f808ba17558>
     # ./lib/asset_cloud/buckets/gcs_bucket.rb:45:in `bucket'
     # ./lib/asset_cloud/buckets/gcs_bucket.rb:17:in `write'
     # ./spec/gcs_bucket_spec.rb:90:in `block (2 levels) in <top (required)>'

  6) BasicCloud#find should return the appropriate asset when one exists
     Failure/Error: raise AssetCloud::AssetNotFoundError, key unless @memory.has_key?(key)

     AssetCloud::AssetNotFoundError:
       Could not find asset products/key.txt
     # ./lib/asset_cloud/buckets/memory_bucket.rb:17:in `read'
     # ./lib/asset_cloud/base.rb:157:in `read'
     # ./lib/asset_cloud/asset.rb:108:in `value'
     # ./lib/asset_cloud/base.rb:111:in `find'
     # ./spec/base_spec.rb:133:in `block (3 levels) in <top (required)>'

Deprecation Warnings:

Using `should_receive` from rspec-mocks' old `:should` syntax without explicitly enabling the syntax is deprecated. Use the new `:expect` syntax or explicitly enable `:should` instead. Called from /Users/andyw8/src/github.com/Shopify/asset_cloud/spec/asset_spec.rb:49:in `block (3 levels) in <top (required)>'.

Using `should` from rspec-expectations' old `:should` syntax without explicitly enabling the syntax is deprecated. Use the new `:expect` syntax or explicitly enable `:should` with `config.expect_with(:rspec) { |c| c.syntax = :should }` instead. Called from /Users/andyw8/src/github.com/Shopify/asset_cloud/spec/versioned_memory_bucket_spec.rb:25:in `block (3 levels) in <top (required)>'.


If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
`config.raise_errors_for_deprecations!`, and it will turn the
deprecation warnings into errors, giving you the full backtrace.

2 deprecation warnings total

Finished in 0.08327 seconds (files took 1.1 seconds to load)
144 examples, 6 failures

Failed examples:

rspec ./spec/gcs_bucket_spec.rb:113 # AssetCloud::GCSBucket #read raises AssetCloud::AssetNotFoundError if the file is not found
rspec ./spec/gcs_bucket_spec.rb:93 # AssetCloud::GCSBucket #delete removes the file from the bucket
rspec ./spec/gcs_bucket_spec.rb:32 # AssetCloud::GCSBucket #ls with arguments returns the file
rspec ./spec/gcs_bucket_spec.rb:121 # AssetCloud::GCSBucket #stat returns information on the asset
rspec ./spec/gcs_bucket_spec.rb:80 # AssetCloud::GCSBucket #write writes a file into the bucket with content_disposition
rspec ./spec/base_spec.rb:132 # BasicCloud#find should return the appropriate asset when one exists

Randomized with seed 32601

asset_cloud master %