data-engineer-roadmap

data-engineer-roadmap copied to clipboard

Agile ways of working

3

Not really a technical skill, but considering that most tech companies have adopted agile methodologies, I think having some knowledge of how Scrum or Kanban works is also an important...

akrstova

Concurrency models are missing

2

For a modern data engineer knowledge of concurrency models is important. 1. A data engineer should know the difference between **concurrency** and **parallelism**. 2. A data engineer should know the...

Vlad-Radz

Hi, First and foremost, nice job on characterizing concepts and the fields. I really liked the picture. On the issue itself: Why do you characterize DynamoDB as Key-Value and not...

sofg0mes

What about modeling / simulation?

2

Right now this is more an invitation to discussion than a request. What modeling techniques does a data engineer need and for what use cases? Does anybody do simulation before...

Vlad-Radz

Tasks manager

1

Hello I haven't seen frameworks like Celery https://docs.celeryproject.org/en/stable/index.html Or spring https://spring.io/projects/spring-cloud-dataflow#overview In my personal experience I had to create many batch pipelines, using these. Now with airflow I'm planning to...

silviodc

Encryption in Transit and at Rest

1

Hello, I would suggest the inclusion of encryption in transit SSL/TLS in the networking part and refine the data security & privacy with encryption at Rest

silviodc

Opennebula ?

1

Where does tools like Openebula fit in? https://opennebula.io/

Sheetalr24

Russian translation

1

Отлично, хорошо бы на русском! Nice.

amoraroma

Why Infrastructure as a code

7

This is an interesting point and bit controversial but why IAAC. Well I am very active on this but I really found, this is still not that much established and...

abhishek-ch

Cloud specific versions

5

Would there be interest in creating Cloud specific versions of the roadmap that goes into more specific details for each product choice? I work at Google Cloud so would be...

tfayyaz