gempy-project
progress towards using rasterio in create_topography
cropping probably does not work yet but will actually be easy to get working, replaced prints with logging
removed tmp files with just doing it all in memory, removed really slow pandas read call in favor of just using numpy meshgrid
likely not correct yet, but close to it
Description
Please include a summary of the changes.
Relates to
Checklist
- [ ] My code follows the PEP 8 style guidelines.
- [ ] My code uses type hinting for function and method arguments and return values.
- [ ] My code contains descriptive and helpful docstrings which are formatted per the Google Python Style Guidelines.
- [ ] I have created tests which entirely cover my code.
- [ ] The test code either 1. demonstrates at least one valuable use case (e.g. integration tests) or 2. verifies that outputs are as expected for given inputs (e.g. unit tests).
- [ ] New and existing tests pass locally with my changes.
This looks really good. Does rasterio depend on gdal and it is just a friendlier api or it is its own unique thing? I am thinking on terms of installation
Rasterio has a much simpler/easier user experience/ it is easier to use and currently serves all my purposes. Installation with Anaconda is also relatively easy.
basically yes, rasterio makes it actually easy to use/install gdal with python interpreters managed by pip because they bundle prebuilt binarys. But because of this they don't directly distribute the python gdal api, as they use cython to link the underlying c/c++ to python. Otherwise you have to do a system level install of gdal or use conda, which can be tricky/problematic depending on the platform. So by using rasterio instead you save yourself a lot of headaches like #506 indicates. Unless you are really needing the full capabilities of gdal it isn't worth depending on GDAL directly. IIRC the failing test is because the DEM used in the test doesn't have CRS information which isn't a realistic scenario, or rather it doesn't make sense to use gdal to read DEM products without CRS information.
okay you convinced me (it was not too difficult given the overhead of GDAL). @AlexanderJuestel for gemgis is GDAL a dependency? Ideally if gemgis is made easier to install I would take away all the topographic code from gempy and rely on gemgis to pass a numpy array
@Leguark, as @AndrewAnnex mentioned, GDAL is a dependency of rasterio but they made it relatively easy to install it. I really do like the idea of passing a rasterio object or array to GemPy. That would make things a little bit easier and more convenient :)
@Leguark I think the best path forward would be to merge this PR in first to remove GDAL, so that #506 can move forward. Independently another PR can move the topography module to gemgis.
@Leguark could you take a look again at the missing CRS information for this test data, I saw some recent commits related to gdal install difficulties and this pr could fix that problem
Hey @AndrewAnnex,
do you need any help here? I could try contributing a little.
Cheers Alex
@AlexanderJuestel yes it would be great for someone to track down the correct crs information for the DEM used in the failing test(s). I think I tried whenever I first contributed this but found that the raster had no crs information attached to it so I wasn't sure what crs to give it to allow for the tests to pass unmodified. Whoever made the original contribution of the tests should have that information
@AndrewAnnex, do you mean the failing checks in the PR here?
They are related to an error thrown out by Theano because of a line of code that stopped working with newer versions of NumPy:
https://github.com/Theano/Theano/blob/74b66d9a78c8b7e7189a9a0850feaf83e3e0ce05/theano/configdefaults.py#L1284
The current line (np.distutils.__config__.blas_opt_info) would have to be replaced with np.distutils.__config__.blas_ilp64_opt_info
@AlexanderJuestel IIRC when the tests ran on travis ci the failures were due to the missing CRS info, but those were run in 2020 so the logs are gone
I am closing this for now because after all the refactoring to gempy v3, main gempy repo dependencies are very controlled. Any rasterio or gdal code should go as a gempy_plugin