danny-baker
Missing Data Files
Hi,
I have succesfully built the docker image. Thanks a lot. But now I have 2 questions pertaining data sets. First one is that data files (parquet) are missing so menus above the map don't return any result on the map. And my second question is that how can i organize custom data files in order to display them on the map.
Thanks in Advance,
Hi Thundersoft. This is still related to https://github.com/danny-baker/atlas/issues/2.
Just confirming the data is there, it's just the map is not displaying properly. If you select a dataset and then click the LINE chart button bottom left, you will see the data. This is actually an issue affecting my production app as well so there must have been a dependency change in the last 3 months that has broken something.
Until I resolve this issue it is not possible to get the main map displaying anything. It's likely because I'm using an old version of Dash and Plotly charts. I'm looking into this issue currently and will post an update here when it's resolved. Might be a week or two.
All the processed master parquet files are stored in /data_lakehouse/titanium/
If you look in there (on the main_static_data branch) you will see the json data (for the map and globe) as well as the statistics files. You can open the statistics directly into a pandas dataframe with a pandas.read_parquet(statistics.parquet).
Dan.
Hi Danny,
I just wanted to remind the issue. Is there any update on your code to display statistics on the map ? Also Do you have a detailed guideline describing how to install new data sets to display on the map ?
Thanks in Advance,
Hi Thundersoft. FYI I am still working on this issue on a feature branch. Turns out I've had to do a major refactor of the code base to bring it up to Dash 2.0 readyness. I'm almost there but probably need another month or so.
At this point, no I don't have a guide or anything on the data pipeline. You'd need to inspect the code to see how it works.