activitysim
activitysim copied to clipboard
pandas 1.3 PerformanceWarning
trafficstars
Running with pandas version 1.3 emits PerformanceWarnings at various points, they look like this:
WARNING - activitysim/activitysim/abm/models/util/mode.py:123: PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling
frame.insertmany times, which has poor performance. Consider using pd.concat instead. To get a de-fragmented frame, usenewframe = frame.copy()choosers[trace_column_names] = choosers.index
As noted in #449 the defrag approach suggested in the warning does not work on DataFrames with a lot of bool columns. We should identify whether this is actually a meaningful performance problem for us, and if it is then investigate possible solutions.