quickstart-examples
quickstart-examples copied to clipboard
snowplow
Terraform configuration for Docker
From the README, it seems that the configuration in this repository only supports the 3 big cloud providers.
I would like to try to setup things locally first before attempting to do it in the cloud.
Is it possible to create a Terraform config using development using Docker?
Hi @arvinsim Snowplow only really deploys into those three major clouds currently - most of our loaders are predicated on needing Cloud Blob Storage to stage and load into warehouses like Snowflake and Databricks. It is possible to run parts of the pipeline now with Kafka as a connector but the loaders again would require Azure connectivity to be able to load into a warehouse.
If you are looking to explore Snowplow locally before moving to a full pipeline in the cloud I would suggest checking out Snowplow Micro (https://docs.snowplow.io/docs/testing-debugging/snowplow-micro/what-is-micro/) which is designed for a local experience.
From there you can also check out out trial here (https://try.snowplowanalytics.com/). We offer both a 14 day guided trial loading into a hosted postgres as well as a 35 day trial where we can load into the warehouse of your choice (Snowflake, Databricks or Redshift currently).
It is on my radar to provide a better local experience using something like Docker Compose / Swarm or even a Helm Chart style one-click local deployment but as so much of our functionality is still tied to the cloud this is difficult to achieve.
@arvinsim you might want to check this out for a local dev experience: https://github.com/snowplow-incubator/snowplow-local