pyNastran icon indicating copy to clipboard operation
pyNastran copied to clipboard
verified publisher icon SteveDoyle2

Issue when build_dataframe=True with transient analysis.

Open marcodesiderio opened this issue 3 months ago • 6 comments

When building a dataframe with the transient analysis, I think the df.index is being built incorrectly.

given:

df = op2.get_result(result_name)[1].dataframe

in my case the dataframe - if filtered by element ID 1, has a total of 80 rows - which is expected: 5 nodes per element (N1, N2, N3, N4) x 2 Location (top, bottom) x 8 outputs ('fiber_distance', 'oxx', 'oyy', 'txy', 'angle', 'omax', 'omin', 'von_mises').

However, let's now see how many of these rows are unique:

tuple_set = set() for item in df.index: if item[0] == 1: print(item) tuple_set.add(item)

now len(tuple_set) yields 10 and the following is output:

Image

clearly some duplicate rows. I suspect the fix should be made here:

https://github.com/SteveDoyle2/pyNastran/blob/main/pyNastran/op2/tables/oes_stressStrain/real/oes_plates.py

Essentially what I believe is happening is that for each combination of Node and Location, only one of the ouputs is output - e.g. 1, 'CEN', 'Top' only yields 'fiber_distance' 1, 'CEN', 'Bottom' only yields 'oxx'

There is probably some small bug where when the new MultiIndex tuples are created, something is being mixed up.

UPDATE:

After doing some digging, it appears that this issue is caused by the way in which the eid_nid_item vector is flattened.

eid_nid_item.append(eidi.ravel())

Which flattens it row-wise, while apparently it should be flattened column-wise:

eid_nid_item.append(eidi.ravel(order='F'))

seems to solve the issue.

I suspect this might need to be changed for other outputs as well.

marcodesiderio avatar Sep 24 '25 00:09 marcodesiderio

I’m not following. It seems like you’re modifying the dataframe and that’s maybe where the error is coming from.

Can you make a small problem that shows this?

On Tue, Sep 23, 2025 at 5:43 PM Marco Desiderio @.***> wrote:

marcodesiderio created an issue (SteveDoyle2/pyNastran#845) https://github.com/SteveDoyle2/pyNastran/issues/845

When building a dataframe with the transient analysis, I think the df.index is being built incorrectly.

given:

df = op2.get_result(result_name)[1].dataframe

in my case the dataframe - if filtered by element ID 1, has a total of 80 rows - which is expected: 5 nodes per element (N1, N2, N3, N4) x 2 Location (top, bottom) x 8 outputs ('fiber_distance', 'oxx', 'oyy', 'txy', 'angle', 'omax', 'omin', 'von_mises').

However, let's now see how many of these rows are unique:

tuple_set = set() for item in df.index: if item[0] == 1: print(item) tuple_set.add(item)

now len(tuple_set) yields 10 and the following is output: image.png (view on web) https://github.com/user-attachments/assets/5a0833a8-538d-4b79-b5bd-cc13451250c8

clearly some duplicate rows. I suspect the fix should be made in the op2_object.py script but I am unsure where exactly.

Essentially what I believe is happening is that for each combination of Node and Location, only one of the ouputs is output - e.g. 1, 'CEN', 'Top' only yields 'fiber_distance' 1, 'CEN', 'Bottom' only yields 'oxx'

There is probably some small bug where when the new MultiIndex tuples are created, something is being mixed up.

— Reply to this email directly, view it on GitHub https://github.com/SteveDoyle2/pyNastran/issues/845, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAICUWIZLPJNV2YBFZJ4LX33UHSJ5AVCNFSM6AAAAACHKKSIO2VHI2DSMVQWIX3LMV43ASLTON2WKOZTGQ2DOMBZHAYTGMI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

SteveDoyle2 avatar Sep 24 '25 14:09 SteveDoyle2

Sure! I could have been more clearer in the initial thread.

Let's consider specifically the cquad4_stress output. For both a static, and a transient analysis. Use the following code.

op2 = OP2(debug=False) op2.read_op2(op2_path, build_dataframe=True) result_name = 'cquad4_stress' df = op2.get_result(result_name)[1].dataframe

This yields a pandas dataframe as expected.

  1. In a static analysis, the dataframe has a multi-index, where the index of each row is the tuple (ElementID, NodeID, Location) and each column will represent in turn the stress outputs ('fiber_distance', 'oxx', 'oyy', 'txy', 'angle', 'omax', 'omin', 'von_mises')
  2. In a multi-step or transient analysis, these stress outputs are now added to the rows' multiindex, while each column becomes a point in time.

Now, I run the following loop, where df is the pandas dataframe for a transient analysis (FEMAP Example 17 if you have access to it) - only print results for elementID 1:

for item in df.index: if item[0] == 1: print(item)

This prints:

(1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx')

You can see how the index highlighted in bold is repeated multiple times - the fiber_distance output is only given for the element centroid and top surface - not any other node or location (well I suspect the numerical value themselves are different, their index is just incorrect). The same is true for every single other output (it essentially is a block repeated 10 times).

Once we have established that there is indeed rows with duplicate indexes, we can compare the values of those rows:

  • Iterate thru all rows and only extract the output for ElementID 233, element centroid, bottom surface, sigma xx:
  • Make the sum of the array for comparison purposes.

for idx_tuple, row in df.iterrows(): if idx_tuple == (233, 'CEN', 'Bottom', 'oxx'): print(row.to_numpy().sum())

prints:

3383.1592 38.24394 331.9271 9.325302 174.79813 857.2373 1940.3357 2626.4119

So, clearly there are rows in the dataframe that have the same index, but output different results.

Using order='F' in np.ravel() - suggested fix:

(1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Top', 'oxx') (1, 'CEN', 'Top', 'oyy') (1, 'CEN', 'Top', 'txy') (1, 'CEN', 'Top', 'angle') (1, 'CEN', 'Top', 'omax') (1, 'CEN', 'Top', 'omin') (1, 'CEN', 'Top', 'von_mises') (1, 'CEN', 'Bottom', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 'CEN', 'Bottom', 'oyy') (1, 'CEN', 'Bottom', 'txy') (1, 'CEN', 'Bottom', 'angle') (1, 'CEN', 'Bottom', 'omax') (1, 'CEN', 'Bottom', 'omin') (1, 'CEN', 'Bottom', 'von_mises') (1, 64, 'Top', 'fiber_distance') (1, 64, 'Top', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Top', 'txy') (1, 64, 'Top', 'angle') (1, 64, 'Top', 'omax') (1, 64, 'Top', 'omin') (1, 64, 'Top', 'von_mises') (1, 64, 'Bottom', 'fiber_distance') (1, 64, 'Bottom', 'oxx') (1, 64, 'Bottom', 'oyy') (1, 64, 'Bottom', 'txy') (1, 64, 'Bottom', 'angle') (1, 64, 'Bottom', 'omax') (1, 64, 'Bottom', 'omin') (1, 64, 'Bottom', 'von_mises') (1, 14, 'Top', 'fiber_distance') (1, 14, 'Top', 'oxx') (1, 14, 'Top', 'oyy') (1, 14, 'Top', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Top', 'omax') (1, 14, 'Top', 'omin') (1, 14, 'Top', 'von_mises') (1, 14, 'Bottom', 'fiber_distance') (1, 14, 'Bottom', 'oxx') (1, 14, 'Bottom', 'oyy') (1, 14, 'Bottom', 'txy') (1, 14, 'Bottom', 'angle') (1, 14, 'Bottom', 'omax') (1, 14, 'Bottom', 'omin') (1, 14, 'Bottom', 'von_mises') (1, 15, 'Top', 'fiber_distance') (1, 15, 'Top', 'oxx') (1, 15, 'Top', 'oyy') (1, 15, 'Top', 'txy') (1, 15, 'Top', 'angle') (1, 15, 'Top', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Top', 'von_mises') (1, 15, 'Bottom', 'fiber_distance') (1, 15, 'Bottom', 'oxx') (1, 15, 'Bottom', 'oyy') (1, 15, 'Bottom', 'txy') (1, 15, 'Bottom', 'angle') (1, 15, 'Bottom', 'omax') (1, 15, 'Bottom', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Top', 'oxx') (1, 25, 'Top', 'oyy') (1, 25, 'Top', 'txy') (1, 25, 'Top', 'angle') (1, 25, 'Top', 'omax') (1, 25, 'Top', 'omin') (1, 25, 'Top', 'von_mises') (1, 25, 'Bottom', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 25, 'Bottom', 'oyy') (1, 25, 'Bottom', 'txy') (1, 25, 'Bottom', 'angle') (1, 25, 'Bottom', 'omax') (1, 25, 'Bottom', 'omin') (1, 25, 'Bottom', 'von_mises')

now each index is unique.

marcodesiderio avatar Sep 24 '25 15:09 marcodesiderio

I don’t have that example. Can you attach it?

It seems like your issue is due to the way you’re iterating across the dataframe though.

On Wed, Sep 24, 2025 at 8:23 AM Marco Desiderio @.***> wrote:

marcodesiderio left a comment (SteveDoyle2/pyNastran#845) https://github.com/SteveDoyle2/pyNastran/issues/845#issuecomment-3329339022

Sure! I could have been more clearer in the initial thread.

Let's consider specifically the cquad4_stress output. For both a static, and a transient analysis. Use the following code.

op2 = OP2(debug=False) op2.read_op2(op2_path, build_dataframe=True) result_name = 'cquad4_stress' df = op2.get_result(result_name)[1].dataframe

This yields a pandas dataframe as expected.

  1. In a static analysis, the dataframe has a multi-index, where the index of each row is the tuple (ElementID, NodeID, Location) and each column will represent in turn the stress outputs ('fiber_distance', 'oxx', 'oyy', 'txy', 'angle', 'omax', 'omin', 'von_mises')
  2. In a multi-step or transient analysis, these stress outputs are now added to the rows' multiindex, while each column becomes a point in time.

Now, I run the following loop, where df is the pandas dataframe for a transient (FEMAP Example 17 if you have access to it):

for item in df.index: if item[0] == 1: print(item)

This prints:

(1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Bottom', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Bottom', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Bottom', 'oxx')

You can see how the index highlighted in bold is repeated multiple times - the fiber_distance output is only given for the element centroid and top surface - not any other node or location (well I suspect the numerical value themselves are different, their index is just incorrect). The same is true for every single other output (it essentially is a block repeated 10 times).

Using order='F' in np.ravel():

(1, 'CEN', 'Top', 'fiber_distance') (1, 'CEN', 'Top', 'oxx') (1, 'CEN', 'Top', 'oyy') (1, 'CEN', 'Top', 'txy') (1, 'CEN', 'Top', 'angle') (1, 'CEN', 'Top', 'omax') (1, 'CEN', 'Top', 'omin') (1, 'CEN', 'Top', 'von_mises') (1, 'CEN', 'Bottom', 'fiber_distance') (1, 'CEN', 'Bottom', 'oxx') (1, 'CEN', 'Bottom', 'oyy') (1, 'CEN', 'Bottom', 'txy') (1, 'CEN', 'Bottom', 'angle') (1, 'CEN', 'Bottom', 'omax') (1, 'CEN', 'Bottom', 'omin') (1, 'CEN', 'Bottom', 'von_mises') (1, 64, 'Top', 'fiber_distance') (1, 64, 'Top', 'oxx') (1, 64, 'Top', 'oyy') (1, 64, 'Top', 'txy') (1, 64, 'Top', 'angle') (1, 64, 'Top', 'omax') (1, 64, 'Top', 'omin') (1, 64, 'Top', 'von_mises') (1, 64, 'Bottom', 'fiber_distance') (1, 64, 'Bottom', 'oxx') (1, 64, 'Bottom', 'oyy') (1, 64, 'Bottom', 'txy') (1, 64, 'Bottom', 'angle') (1, 64, 'Bottom', 'omax') (1, 64, 'Bottom', 'omin') (1, 64, 'Bottom', 'von_mises') (1, 14, 'Top', 'fiber_distance') (1, 14, 'Top', 'oxx') (1, 14, 'Top', 'oyy') (1, 14, 'Top', 'txy') (1, 14, 'Top', 'angle') (1, 14, 'Top', 'omax') (1, 14, 'Top', 'omin') (1, 14, 'Top', 'von_mises') (1, 14, 'Bottom', 'fiber_distance') (1, 14, 'Bottom', 'oxx') (1, 14, 'Bottom', 'oyy') (1, 14, 'Bottom', 'txy') (1, 14, 'Bottom', 'angle') (1, 14, 'Bottom', 'omax') (1, 14, 'Bottom', 'omin') (1, 14, 'Bottom', 'von_mises') (1, 15, 'Top', 'fiber_distance') (1, 15, 'Top', 'oxx') (1, 15, 'Top', 'oyy') (1, 15, 'Top', 'txy') (1, 15, 'Top', 'angle') (1, 15, 'Top', 'omax') (1, 15, 'Top', 'omin') (1, 15, 'Top', 'von_mises') (1, 15, 'Bottom', 'fiber_distance') (1, 15, 'Bottom', 'oxx') (1, 15, 'Bottom', 'oyy') (1, 15, 'Bottom', 'txy') (1, 15, 'Bottom', 'angle') (1, 15, 'Bottom', 'omax') (1, 15, 'Bottom', 'omin') (1, 15, 'Bottom', 'von_mises') (1, 25, 'Top', 'fiber_distance') (1, 25, 'Top', 'oxx') (1, 25, 'Top', 'oyy') (1, 25, 'Top', 'txy') (1, 25, 'Top', 'angle') (1, 25, 'Top', 'omax') (1, 25, 'Top', 'omin') (1, 25, 'Top', 'von_mises') (1, 25, 'Bottom', 'fiber_distance') (1, 25, 'Bottom', 'oxx') (1, 25, 'Bottom', 'oyy') (1, 25, 'Bottom', 'txy') (1, 25, 'Bottom', 'angle') (1, 25, 'Bottom', 'omax') (1, 25, 'Bottom', 'omin') (1, 25, 'Bottom', 'von_mises')

now each index is unique.

— Reply to this email directly, view it on GitHub https://github.com/SteveDoyle2/pyNastran/issues/845#issuecomment-3329339022, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAICUWJZA4YVCYFDXVLZDDT3UKZKHAVCNFSM6AAAAACHKKSIO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMRZGMZTSMBSGI . You are receiving this because you commented.Message ID: @.***>

SteveDoyle2 avatar Sep 24 '25 20:09 SteveDoyle2

Thanks for the fast response! Here is the example:

Hinge.zip

I am not exactly sure what you mean by 'iterating across the dataframe': in the example I provided, I am taking the dataframe as it is, and then I am simply looping thru the index of dataframe (essentially a list of tuples) - essentially going row by row. If each row is a unique output (and row.to_numpy()) will yield the time-history of that output), there should not be any duplicate rows, which is in fact the case?

As an example, if I get the dataframe as it is, then there would be no way for me to know which of the rows is the time history for 1, 'CEN', 'Top', 'oxx', since it's not the index of any of the rows (but, reasonably, it must exist).

marcodesiderio avatar Sep 24 '25 20:09 marcodesiderio

Hello Steve! Did you get a chance to take a look at this issue?

marcodesiderio avatar Nov 19 '25 20:11 marcodesiderio

Sorry. I'm pretty busy. I forgot.

Yeah, that looks terrible...it's clearly making something that's ntimes*nresults rows long with a 3d index long instead of ntimes results with a 4d array.

I didn't code the MultiIndex code, but it seems there are 3 input arrays when there should be 4. The fourth is dynamically created using:

  • nlayerheaders = nelementnnode*2 * headers The issue seems to be the sorting order.

I think it need an array that does this... for N elements: for 5 nodes: for result in [fiber_distance, oxx, oyy, txy, angle, omax, omin, von_mises]: for 2 layers: append(result) which is not great to build. It gets even worse when you work with composites and the number of layers is changing.

The data is: type=RealPlateStressArray ntimes=100 nelements=240 nnodes_per_element=5 nlayers=2 ntotal=2400 data: [ntimes, ntotal, 8] where 8=[fiber_distance, oxx, oyy, txy, angle, omax, omin, von_mises] element_node.shape = (2400, 2) data.shape=(100, 2400, 8) element type: CQUAD144-144 s_code: 1 sort1 dts = [0. 0.0005 0.001 ... 0.0485 0.049 0.0495]; dtype=float32

SteveDoyle2 avatar Nov 21 '25 05:11 SteveDoyle2