interp-net icon indicating copy to clipboard operation
interp-net copied to clipboard

Should the step "python src/mimic_data_extraction.py" be very slow?

Open ruilialice opened this issue 3 years ago • 1 comments

Hi, when I run the script python src/mimic_data_extraction.py it works very slow, after 4 hours, it only processed about 170 visits, but the total number of visits is more than 20000, so I am wondering that whether it should be that slow or there is something wrong with my database constructed with postgres. Thanks.

ruilialice avatar Apr 01 '21 01:04 ruilialice

You can try to run the data extraction for each variable in a parallel way. Basically, every curr.execute command in the code can be executed parallelly.

satyanshukla avatar Apr 01 '21 21:04 satyanshukla