Program is significantly slower importing files -- optimization needed

[RDmitchell]

Describe the bug Several cities using SEED for benchmarking compliance has noticed that the program is taking significantly longer to import files, to the point that they think the program has crashed (see #2515 )

Expected Behavior Import, map and match files in a reasonable amount of time.

Actual Behavior see Describe the Bug

Steps to Reproduce Import a file that has 2000+ records into a cycle that has 2000+ existing records, and at least 1 other cycle.

Instance Information Instance: LBNL production server SHA: e8d5cde48

Additional Context Most cities using that latest SEED on the LBNL instance are reporting this

[adrian-lara]

The biggest pain point seems to be the start_system_matching_and_geocoding endpoint. I'd suspect that initial first steps would be to confirm that, then identify the slowest queries within the underlying task.

[RDmitchell]

is geocoding a significant time sink?

Since we have the option now to turn geocoding off in Settings, would turning if off speed things up? I'm not sure all of our current users are interested in geocoding (although I could be wrong).

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity within 60 days. It will be closed if no further activity occurs. Thank you for your contributions.

[isalanglois]

@RDmitchell Is this still relevant?

[RDmitchell]

@isalanglois -- I suspect it is since no work was done on it. You could put it on the new board, in the Test column, assign it to me, and if it is still a problem we can set it to To Do. How does that sound?

[isalanglois]

@RDmitchell That's great, thanks for the quick follow-up!

[RDmitchell]

I imported 4500 records (and 45 fields, so not as large as an ESPM file) into a new organization. From the initial import to the records being added to the Property table took approximately 7 minutes, which seems pretty reasonable.

The bulk of the time, probably more than half of it, was step 6/6, matching / merging

The geocoding time was trivial in this particular case

I am going to close this issue and we can always make another one if we find a file that takes a very long time to import