"Could not create BatchIterator temporary file" on Windows fix

[WiktorPrzetacznik]

Description:

There's an issue with from csv on Windows, following command:

./webgraph.exe from csv <input.csv --num_nodes 6573 output

results in:

[2024-07-27T20:43:04Z INFO webgraph.exe] Reading arcs CSV [2024-07-27T20:43:04Z INFO webgraph.exe] Completed. [2024-07-27T20:43:04Z INFO webgraph.exe] Elapsed: 44ms [10,000 lines, 222602.13 lines/s, 4.49 μs/lines]; res/vir/avail/free/total mem 0.00B/0.00B/8.52GB/8.52GB/17.11GB [2024-07-27T20:43:04Z INFO webgraph::cli::from::csv] Arcs read: 10000 [2024-07-27T20:43:04Z DEBUG webgraph::utils::sort_pairs] Sorted 10000 arcs in 2.4946ms thread 'main' panicked at src\cli\from\csv.rs:122:14: called Result::unwrap() on an Err value: Could not create BatchIterator temporary file C:\Users\wikto\AppData\Local\Temp\FromCsvPairs8e4rM5\000000

Caused by: The system cannot find the path specified. (os error 3) note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Cause:

std::fs::File::create fails on some platforms if the directory doesn't exist.

Solution:

The issue is resolved by making sure temporary file's directory exists (not only on Windows).

Question:

As I only use from csv, I'm not sure if other places suffer from it, too. Should this PR include fixes in other places?

[vigna]

@zommiommy you are the main author of the CLI—thoughts?

[zommiommy]

Yeah, it's a general problem that I'll fix. Thank you!

[zommiommy]

But the error you posted shouldn't be related to that. The path C:\Users\wikto\AppData\Local\Temp\FromCsvPairs8e4rM5 is generated at line 59 of src/cli/from/csv.rs:

let dir = tempfile::Builder::new().prefix("FromCsvPairs").tempdir()?;

It should have created the folder. When dir goes out of scope, tempfile deletes the dir, so are you sure the error is that one?

[zommiommy]

Now on main all CLI commands create the dirs, if needed, of the files it creates.

Regarding your problem, does C:\Users\wikto\AppData\Local\Temp exist? That path should be the result of calling GetTempPath2A which on windows can be customized by setting the TMP and TEMP environment variables.

[WiktorPrzetacznik]

@zommiommy Yes, it exists. Also, I'm not really sure why let dir = tempfile::Builder::new().prefix("FromCsvPairs").tempdir()?; continues if it doesn't create that dir, I haven't examined tempfile create's source - maybe some platfrom-specific behavior? For sure it's not about lack of permissions, I run it as an admin.

What I did is testing out that if I create the dir manually - it no longer panics with the error above. You can easily test it by yourself if you have any Windows device/dualboot.

I just tested main branch and it still panics:

C:\Users\wikto\RustroverProjects\webgraph-rs-fixed\target\debug>webgraph.exe from csv <output.csv --num-nodes 6573 edges10000 [2024-07-28T12:12:46Z INFO webgraph.exe] Reading arcs CSV [2024-07-28T12:12:46Z INFO webgraph.exe] Completed. [2024-07-28T12:12:46Z INFO webgraph.exe] Elapsed: 38ms [9,999 lines, 261039.83 lines/s, 3.83 μs/lines]; res/vir/avail/free/total mem 0.00B/0.00B/6.19GB/6.19GB/17.11GB [2024-07-28T12:12:46Z INFO webgraph::cli::from::csv] Arcs read: 9999 [2024-07-28T12:12:46Z DEBUG webgraph::utils::sort_pairs] Sorted 9999 arcs in 3.6666ms thread 'main' panicked at src\cli\from\csv.rs:123:14: called Result::unwrap() on an Err value: Could not create BatchIterator temporary file C:\Users\wikto\AppData\Local\Temp\FromCsvPairsisCfxy\000000

Caused by: The system cannot find the path specified. (os error 3) note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

[vigna]

Sorry, this slipped away somehow. Do you still have the issue?