Error while provisioning spark cluster with terraform

[topefolorunso]

I used e2-standard-2 for the kafka and airflow instances I then used e2-medium for both master and worker in the spark cluster Apparently, the kafka and airflow instances were successful.

google_dataproc_cluster.mulitnode_spark_cluster: Creating...
╷
│ Error: Error creating Dataproc cluster: googleapi: Error 400: Multiple validation errors:
│  - Insufficient 'CPUS' quota. Requested 6.0, available 4.0.
│  - Insufficient 'IN_USE_ADDRESSES' quota. Requested 3.0, available 2.0.
│  - This request exceeds CPU quota. Some things to try: request fewer workers (a minimum of 2 is required), 
use smaller master and/or worker machine types (such as n1-standard-2)., badRequest
│
│   with google_dataproc_cluster.mulitnode_spark_cluster,
│   on main.tf line 93, in resource "google_dataproc_cluster" "mulitnode_spark_cluster":
│   93: resource "google_dataproc_cluster" "mulitnode_spark_cluster" {
│

[ankurchavda]

Hi Tope,

Have you availed the 300$ in credit by creating a new GCP account? Can you check this link and see if you find something here?

[topefolorunso]

Thanks. The problem was with the quota available for the free trial. Apparently, I can't increase it.

What I did instead was to use 1 master which acts as worker as well. I hope that works as well.

[ankurchavda]

Glad you got it to work. I guess it should work alright.