Add warning about the costs involved with Google BigQuery

[craigfrancis]

trafficstars

With the Getting Started guide at:

https://github.com/HTTPArchive/httparchive.org/blob/master/docs/gettingstarted_bigquery.md

Can it begin with a warning about the potential costs involved with Google BigQuery.

I'm not impressed with my recent £126.73 bill ($166.62), after I simply ran a couple of SELECT statements; or how Google charged that with no warnings, or getting any kind of approval from me.

This is related to the issue I raised on https://github.com/HTTPArchive/almanac.httparchive.org/issues/617

[fhoffa]

For closure, since HTTPArchive/almanac.httparchive.org#617 was closed:

• I'm glad methodology.html now links to https://timkadlec.com/remembers/2019-12-10-using-bigquery-without-breaking-the-bank/.

In summary:

• If you use BigQuery without a credit card, there's nothing to worry about. You'll get a free terabyte for queries every month.

• The day you decide you want to pay to go beyond the terabyte of free queries, please-please-please set up cost controls.

I wish BigQuery already had these cost controls by default the moment you put your credit card (stay tuned), but in the meantime: if you add your credit card, make sure to have cost controls on.

Also: There is a third mode, where you sign-up for the free trial. Then you have to add a credit card to get $300 of free credits - but you will never be charged until the day you click a button saying you want to go beyond these $300.

[craigfrancis]

Unfortunately, like many, that first one won't apply to me (I'm someone who had already put a credit card against my generic Google account), there is no mention of cost controls during setup (unfortunately I'm not psychic), and I never got that button (or $300 of credits) when I did my 1 month free trial of Google services (that I used with the Google Maps API, not BigQuery).

---

For reference, I pay for other Google services (mainly Google Mail, aka G Suite), and I use the Google Maps API a small amount (within the free usage tier)... so I was not aware that enabling the BigQuery API, to access this public data, for a couple of quick/simple queries, would result in any charges, let alone £126.73 ($166.62).

Personally I think this is something Google should fix, so I really do hope your "stay tuned" means something.

I really feel the current setup is incredibly deceptive; and there's a part of me still wondering if I should take Google to court (I did waste some time querying this with the billing department, with no luck; and unfortunately I don't think I've got the time or money to take this any further).

From my point of view, this setup is like taking a family to a restaurant, having a nice meal, expecting to pay for that meal, then finding out the kids play area, that was open to everyone, with no pricing at the entrance, had a £12 per minute charge (I only used BigQuery for ~10 minutes, I pay £4.14 for G Suite per month, and my 1 month "free trial" was used several years ago on the Google Maps API).

[tunetheweb]

There are works in progress to try to make it cheaper as discussed in this tweet and the Document it links to. Hopefully that should make your experience less likely.

[craigfrancis]

Thanks Barry, I think I'm hoping that Google will make some changes to the BigQuery system itself, so everyone knows what it's about to cost (i.e. they are informed about the charges, and they accept/approve them).

[tunetheweb]

Yup should tackle it from both ends.

BTW I wouldn't close this @craigfrancis - it's still relevant until the above two things happen. Suggest you reopen?