RocketPy

RocketPy copied to clipboard

Pull request type

• [X] Code base additions (bugfix, features)

Pull request checklist

• Code base maintenance (refactoring, formatting, renaming):

  • [ ] Docs have been reviewed and added / updated if needed
  • [ ] Lint (black rocketpy) has passed locally and any fixes were made
  • [ ] All tests (pytest --runslow) have passed locally

• Code base additions (for bug fixes / features):

  • [ ] Tests for the changes have been added
  • [x] Docs have been reviewed and added / updated if needed
  • [x] Lint (black rocketpy) has passed locally and any fixes were made
  • [x] All tests (pytest --runslow) have passed locally

What is the current behavior?

The EnvironmentAnalysis class was an important project that lead us to our last milestone Environment Analysis SACup 2022. However, it is time to work hard again in order to fill some blank spaces we left last time, and also to help reaching the newest milestone of our team: RocketPy at EuroC 2022

What is the new behavior?

Features that are being required:

• [X] 1 - Create a function to export environment object to be used as customAtmosphere in other simulations
• [X] 2 - Create a function to export environment_analysis to be used as customAtmosphere in Environment
• [x] 3 - Catch Reanalysis Data for EuroC's site and save it on data folder (#238 )
• [ ] 4 - Run EnvAnalys example for EuroC's site and export to a proper format

Optional features and tasks:

• [ ] I - Add tests to EnvironmentAnalysis Class
• [ ] II - Create Wind Heading profiles (#197)
• [x] III - Save EnvironmentAnalysis processed data so it can be loaded later in order to speed up by skipping the parse steps (#237)
• [ ] V - Check if we can aggregate both surface and pressure level data from environment analysis into a flight simulation
• [ ] VI - Integrate with wind.tv files and other webscraping tools (https://github.com/RocketPy-Team/Hackathon_2020/pull/95/files)
• [ ] VII - Automate data input by using OpenDap protocol (check if it is possible with Reanalysis)

Does this introduce a breaking change?

• [ ] Yes
• [X] No

I guess you can see the two new methods that are already working: Environment.exportEnvironment() and EnvironmentAnalysis.exportMeanProfiles(). These two will allow us to save already processed atmospheric profiles and load them into new simulations . They have the same behavior, but one exports data from Env. and other from EnvAnal., I hope this is the only difference. The selected file format as .json. I had to think a little bit when choosing it, csv and xml were also option. But at the end I believe this will allow us to be more flexible and easy-to-use in the future. A quick reference: https://pythonspot.com/save-a-dictionary-to-a-file/

After that, I've tried to export self.surfaceDataDict and self.pressureLevelDataDict from Environment Anaysis so we could use them to skip the parse steps on EnvironmentAnalysis, thus speeding up the process (that's the task III). However, the json.dumps() (which converts py dictionaries to .json) method is leading me to some problems since two important object types are not JSON serializable, they are datetime and function. The datetime is easy to solve, since I can convert datetime <> string types really easy. However, dealing with function objects is special hard, for instance, we have the following objects on the self.pressureLevelDataDict :

I believe we, at this point, have three different options to solve:

• Try to create a maner to encode/decode Function objects when converting dicts to .json files (see https://stackoverflow.com/questions/51763570/error-converting-dict-to-json-in-python)
• Convert Function to lists and then use on json.dumps() since lists are JSON serializable. This is actually already being done on EnvironmentAnalysis.exportMeanProfiles().
• Forget the idea of saving Environment Analysis processed data.

I'm accepting suggestions and an extra hand here. @MateusStano @FranzYuri @PatrickSampaioUSP let me know if you guys understood the problem.

Currently requesting data on Copernicus website:

• Pressure Level Data: https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-pressure-levels?tab=form
• Surface Level Data: https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=overview

Had to download several files, each one with 2 years long, on the pressure level data scenrario.

@Gui-FernandesBR do you need help merging these files into one after they are downloaded?

I had to do the same for SACup, so I know how that can be accomplished.

@Gui-FernandesBR do you need help merging these files into one after they are downloaded?

I had to do the same for SACup, so I know how that can be accomplished.

Thank you @giovaniceotto , I'm using the following code to merge, it has started 12 minutes ago and is still running, so I'm afraid it doesn't work properly, let's see. Do you have any pther code?

Btw, we probalbly should add a quick tutorial describing how to download and merge such files...

source-code: https://neetinayak.medium.com/combine-many-netcdf-files-into-a-single-file-with-python-469ba476fc14

@giovaniceotto didn't work, help needed then

I only have the surface data file working . The pressure Level is divided in 11 files.

@giovaniceotto didn't work, help needed then

I only have the surface data file working . The pressure Level is divided in 11 files.

Solved! It turns out that the range that I chose for lat/lon was too large, I was dealing with really large files then

@Gui-FernandesBR do you need help merging these files into one after they are downloaded? I had to do the same for SACup, so I know how that can be accomplished.

Thank you @giovaniceotto , I'm using the following code to merge, it has started 12 minutes ago and is still running, so I'm afraid it doesn't work properly, let's see. Do you have any pther code?

Btw, we probalbly should add a quick tutorial describing how to download and merge such files...

source-code: https://neetinayak.medium.com/combine-many-netcdf-files-into-a-single-file-with-python-469ba476fc14

I use the exact same code.

I am having issues with the windRose plots, we need to adrress that before merging.

@PatrickSampaioUSP could you take a look at the windrose bug please?

Hey! I ran the simulation and couldn't replicate the error. All worked fine over here

Just in order to not merging without review... @PatrickSampaioUSP could you approve and merge please ?