Dazhong Xia
Dazhong Xia
We don't have concrete plans to actively work on this, but if this bugs you and you have a couple hours to throw at it, go for it @krivard !...
I'm excited about these (proposed) changes! I wonder if it makes sense to set specific goals around speedup & ergonomic improvements, so we know when the performance is "good enough"...
We can consider following a similar pattern to #4731 where we don't tie ourselves to the `GenericExtractor` logic, which might give us some freedom to implement this logic without having...
@e-belfer is this blocking for the 176 work? If so, let's put this in the current sprint, otherwise we should icebox it.
This may be only possible if we use the `forkserver` approach detailed in #4619 . I've tried a number of different approaches with `spawn`ed processes, including: * just start a...
Hi! I'm back from my trip and just took a look at the current draft - so far everything looks good, thanks for taking this on and congrats on getting...
[Profiling results for all the `raw_*__all_dfs` run](https://github.com/user-attachments/files/23172943/memray-flamegraph-key_raw____all_dfs_-2025-10-24T21.04.56.html) summary: ``` Command line: ./memory_profile.py --directory profiles/ --aggregate -a key:"raw_*__all_dfs" Start time: Fri Oct 24 2025 17:04:56 GMT-0400 (Eastern Daylight Time) End time:...
More narrow profile for just EIA 930: summary: ``` Command line: ./memory_profile.py --directory profiles/ --aggregate -a raw_eia930__all_dfs Start time: Mon Oct 27 2025 16:38:00 GMT-0400 (Eastern Daylight Time) End time:...
That's great news, @carlopi ! Thanks for getting a fix working so quickly, do you have any updates on the PR / is there anything we can do to help...
Poked around at the gcp_pudl_etl.sh. I think there's lots of room for improvement with minimal effort. I think we want this general structure for the file: ```bash # set BUILD_TYPE...