How to create a mesh and import data in GemGis

[jc-barreto]

I'm getting started with GemGis and I would like to reproduce something similar with this example, however, I don't know how to create the mesh in vtk file that is imported in the example. I have the topography of the place understudy in .dwg (autocad) file and I have cptu tests in excel (one sheet for each cptu), I have imported the cptu as pandas data frame in Jupyter, and I would like to plot all the cptu I have with the topography in 3D using GemGis. Is this possible? How can I do that? There is any Youtube Channel with tutorial videos?

Thank you

[AlexanderJuestel]

Hello @jc-barreto,

with mesh, do you mean the surface that is displayed?

I have not worked with dwg files before. Can you post a sample of your data so I can test it?

Unfortunately, we haven't had the time to create any YouTube videos yet, sorry.

Cheers Alex

[jc-barreto]

Hello @AlexanderJuestel Thank you for answering.

Yes, the mesh is the surface displayed. I put the files here. What I wanna do, like in your example, is to plot the columns x,y, and elevation with the colors of the column label from all excel sheets.

I really appreciate your help.

Cheers,

[bluetyson]

So you want a 3d plot based on elevation colormapped to the cptu?

[jc-barreto]

@bluetyson Yes, like the example 35

[jc-barreto]

I did this in Jupyter notebook, with matplotlib, but without the surface from topography, I guess with GemGis will be bettter

[RichardScottOZ]

Ok, I doubt too many of us have autocad - if you can convert the dwg into a standard sort of raster this is pretty straightforward.

Or something along these lines if not a tif:-

def read_raster_depth(filename):
    """
    Helpful: http://xarray.pydata.org/en/stable/auto_gallery/plot_rasterio.html
    """
    # Read in the data
    data = xr.open_rasterio(filename)
    values = np.asarray(data)
    nans = values == data.nodatavals
    if np.any(nans):
        values = np.ma.masked_where(nans, values)
        values[nans] = np.nan
    # Make a mesh
    xx, yy = np.meshgrid(data['x'], data['y'])
    zz = values.reshape(xx.shape) # will make z-comp the values in the file
    # zz = np.zeros_like(xx) # or this will make it flat
    mesh = pv.StructuredGrid(xx, yy, zz)
    mesh['data'] = values.ravel(order='F')
    mesh['data'] = mesh['data'] * -1
    return mesh

dtm = read_raster_depth('topography.dwg')
dtm

[RichardScottOZ]

[RichardScottOZ]

That would be assuming it is some sort of format like this, and not some multilayer vector type thing, like Alex, don't know about them.

[RichardScottOZ]

If that is the case and you can convert it to dxf you can do something like here:- https://gemgis.readthedocs.io/en/latest/getting_started/tutorial/27_opening_obj_and_dxf_files.html

[jc-barreto]

I have uploaded the topography as .dxf file here, but I don't understand how to import to gemgis my 2D topograph

[jc-barreto]

Is this the kind of data I must have ?

[RichardScottOZ]

I think the Gemgis function wants all geometries the same @AlexanderJuestel ? so it would have to match that

Is the topography something you created yourself?

[RichardScottOZ]

[jc-barreto]

No, this topography is from some project. I manage to import to ArcGis, I have georeferenceated, and now I have a raster and Tiff that I have exported from ArcGis, is this what is necessary? But the extension of the files after raster I don't understand, I have never worked with this before. Those are the files that I have exported when rastering with ArcGis

and those are the one I get when exporting as Tiff

What should I import em GemGis?

[AlexanderJuestel]

Hello @jc-barreto,

these lines you are showing there look much like vector data and hence shape files. These can be used to create your topography. However, what you can do (at least in QGIS) is loading the .adf file and export it as .tif file, what you seem to have dones as well. This can then be loaded into GemGIS

[jc-barreto]

@AlexanderJuestel, thank you for the help. Now I have the .tif file and I can import with rasterio, but I don't know how to create the mesh .vtk and and import the surface. I have updated the files here, could you please help me reproduce example 35?

[AlexanderJuestel]

@jc-barreto,

You can follow this Tutorial that explains how to make a mesh out of your topography with PyVista: https://gemgis.readthedocs.io/en/latest/getting_started/tutorial/14_visualizing_topography_and_maps_with_pyvista.html

[jc-barreto]

Thank you, it works! Now, I'm finally following example 35, and I'm facing problems with the function gg.visualization.add_row_to_boreholes. I have my topography

and my data

but when I try to run this part of the code: list_df = gg.visualization.add_row_to_boreholes(df_groups) it gives this error: The name of the columns in my table/data must be the same as in the example?

[bluetyson]

Or change it to reference the right column in your case

[jc-barreto]

Now, I have transformed my data to the same format/labeling that the example data, and still getting problems... And I don't understand this error, because I have a pandas dataframe, and as it is printed, I have 8320 rows x 7 columns...

[AlexanderJuestel]

You are missing column Z, which is equal to the position of your stratigraphic interface. The depth column indicates the total depth of the well and should be the same for stratigraphic interfaces belonging to one well

[jc-barreto]

Z is not the altitude? I would like to plot my data for altitude, not for Z. and the depth is the total length of my borehole, for example, if I started to collect data from depth -16m to -24m, my depth is 8m? Those are my data, plotted with matplotlib(X,Y, Altitude, colored by formation), I would like to do the same but with GemGis, so I can include the topography

[jc-barreto]

I must say that GemGis is not straightforward, after a while I manage ( I think) to put my data in the right format:

But I think I don't understand well the name of each variable, what is the 'altitude' in the code? because is always the same for all points in each borehole in the example and in my case change for each point. Usually the z = top elevation - depth of the point. Is that the case? If I fix the elevation for each borehole as the top elevation in each borehole, I can run the code, but the final plot doesn't show the borehole.

If I comment the line with the code about label, I can see something, like points at the place where should be the boreholes

I already have tried to reduce, increase the radius of the tube, the min_length but still not plotting properly. Please help me, I have spent so much time trying to use GemGis I don't wanna give up now

[RichardScottOZ]

Altitude or RL can be the height of the location of the drillhole at the surface.

Print the topo mesh and the tubes mesh out for us so we can see the dimensions/coordinates.

print(topo)

[RichardScottOZ]

or just that in a cell.

[AlexanderJuestel]

@jc-barreto I think you are almost there, would you mind sending me your data and I will have a look. It is sometimes easier to look at an issue locally :)

[jc-barreto]

@RichardScottOZ and @AlexanderJuestel Thank you for the help. Here is the topo

And I put my data and the notebook I'm using here

[RichardScottOZ]

thanks, what about the tubes?

Like Alex said, not too far off getting this to work

[jc-barreto]

[RichardScottOZ]

something in the labels is odd by the looks?... min/max had nans anyway

as this works