streamify

streamify copied to clipboard

The number of topics in kafka control center did not increase

1

After creating docker container for the kafka instance and running the script to send message to it with the eventsim, the number of topics in the control center (9021) would...

topefolorunso

Error while provisioning spark cluster with terraform

3

I used e2-standard-2 for the kafka and airflow instances I then used e2-medium for both master and worker in the spark cluster Apparently, the kafka and airflow instances were successful....

topefolorunso

Command for spark-submit

27

I have a question regarding the command for spark-submit: ``` spark-submit \ --packages org.apache.spark:spark-sql-kafka-0-10_2.12:3.1.2 \ stream_all_events.py ``` What is the meaning of the packages `org.apache.spark:spark-sql-kafka-0-10_2.12:3.1.2`?

stephenllh

Do we need two separate VMs for this project?

3

From the setup.md, it seems that you have two VMs set up. Just curious, is it necessary?

stephenllh

I have an error when I launch the terraform apply

6

│ Error: Invalid value for network: project: required field is not set │ │ with google_compute_firewall.rules, │ on main.tf line 19, in resource "google_compute_firewall" "port_rules": │ 19: resource "google_compute_firewall" "port_rules"...

mbiombani

I am getting the below error in airflow interface airflow.exceptions.AirflowException: Bash command failed. The command returned a non-zero exit code 1. please help me to resolve the issue

HarikaK23

Error creating Dataproc cluster

2

Hi, in the last step: Terraform apply I have this error: **Error:** Error creating Dataproc cluster: googleapi: Error 400: User not authorized to act as service account '[email protected]'. To act...

rafical