pdr-backend
                                
                                 pdr-backend copied to clipboard
                                
                                    pdr-backend copied to clipboard
                            
                            
                            
                        Issue1002 - Create lake & etl readme
Fixes #1002
Changes proposed in this PR:
- [x] Communicated basics of lake, etl, and overall predictoor job into a readme.
- [x] Communicated how these components are different from each other, and how they work together.
- [x] Communicated checkpoint details about [GQLDF CSV + RAW] providing end-to-end requirements for how we keep this updated.
- [ ] Communicated checkpoint details about [RAW => ETL Tables] providing end-to-end requirements for how we keep this updated.
- [ ] Communicate a bit about CLI commands, and how they work.
- [ ] An end-to-end example with lake update raw
- [ ] An end-to-end example with lake update etl
- [ ] Update vs. Drop vs. other commands
- [ ] Create basic visuals for each section, to provide a mental model.
Reviewed the README and have some feedback:
- [ ] Add information about where the data comes from and what kind of data we are ingesting into the lake
- [ ] There is no information about the data available, tables etc. I would expect to visually see the data that I'm going to get before going trough the lake setup process
- [ ] The README is long and we don't have any legend on the top to make it easy to see the content and navigate trough different sections. Add a legend on the top of the page, example:
I'm having to draw a bunch of diagrams... targeting to deliver this when @kdetry is further along w/ integration. Currently focusing on #1001 and advancing the ETL.
README has been concluded inside PR #1001, closing PR and updating ticket