Is Windows (no WSL) supported?

[matteodefelice]

I think the answer is "no", but I just want to be sure, perhaps someone had success in using PyPSA-EUR on windows. I can compile the tutorial (electricity) but if I try something larger I get:

1. A ProtectedOutputException because the files in data\bundle are read-only. After removing the read-only...
2. It says there are incomplete files that are actually complete:

IncompleteFilesException:
The files below seem to be incomplete. If you are sure that certain files are not incomplete, mark them as complete with

    snakemake --cleanup-metadata <filenames>

To re-generate the files rerun your command with the --rerun-incomplete flag.
Incomplete files:
cutouts/europe-2013-sarah.nc
cutouts/europe-2013-era5.nc
resources/load.csv
results/config.yaml

I cannot clean up the metadata because snakemake says that the metadata is not present.

[fneum]

We don't really test it without WSL.

Could you, nonetheless, try the following:

• Just remove the .snakemake folder. This will clear the metadata the hard way.
• Ensure the cutouts match the size they have on zenodo: https://zenodo.org/records/6382570
• In rules/retrieve.smk, you can remove the write-protection: protected(...) if that causes trouble.

[matteodefelice]

Thank you, deleting the .snakemake folder unblocked. I could continue the building but I got an error after a few minutes:

[Thu Jan  4 21:10:52 2024]
Error in rule build_hydro_profile:
    jobid: 20
    input: resources/country_shapes.geojson, data/eia_hydro_annual_generation.csv, cutouts/europe-2013-era5.nc
    output: resources/profile_hydro.nc
    log: logs/build_hydro_profile.log (check log file(s) for error details)
    conda-env: C:\Users\matte\local_work\pypsa-eur\.snakemake\conda\0acec97e4fa0e090fb66494ff5c3f997_

RuleException:
CalledProcessError in file C:\Users\matte\local_work\pypsa-eur\rules/build_electricity.smk, line 346:
Command 'C:/Users/matte/miniconda3/envs/pypsa-eur/python.exe "C:\Users\matte\local_work\pypsa-eur\.snakemake\scripts\tmp8bj6d337.build_hydro_profile.py"' returned non-zero exit status 1.
  File "C:\Users\matte\local_work\pypsa-eur\rules/build_electricity.smk", line 346, in __rule_build_hydro_profile
  File "C:\Users\matte\miniconda3\envs\pypsa-eur\Lib\concurrent\futures\thread.py", line 58, in run
Benchmark: unable to collect cpu and memory benchmark statistics
[Thu Jan  4 21:11:53 2024]
Finished job 14.
3 of 16 steps (19%) done
Benchmark: unable to collect cpu and memory benchmark statistics
[Thu Jan  4 21:12:07 2024]
Finished job 7.
4 of 16 steps (25%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake\log\2024-01-04T210956.884707.snakemake.log

The build_hydro_profile.log says: WARNING:country_converter.country_converter:Eurasia not found in regex

[matteodefelice]

I have switched to WSL to compile everything and then I perform the solve step on Windows (I don't have Gurobi for Linux).

UPDATE: I get the same error on WSL (using cbc), so it's not Windows-specific I get this error:

rule solve_network:
    input: resources/networks/elec_s_128_ec_lv1.5_.nc, results/config.yaml
    output: results/networks/elec_s_128_ec_lv1.5_.nc
    log: logs\solve_network\elec_s_128_ec_lv1.5__solver.log, logs/solve_network/elec_s_128_ec_lv1.5__python.log
    jobid: 1
    benchmark: benchmarks/solve_network/elec_s_128_ec_lv1.5_
    reason: Missing output files: results/networks/elec_s_128_ec_lv1.5_.nc; Input files updated by another job: resources/networks/elec_s_128_ec_lv1.5_.nc
    wildcards: simpl=, clusters=128, ll=v1.5, opts=
    threads: 4
    resources: tmpdir=C:\Users\matte\AppData\Local\Temp, mem_mb=104880, mem_mib=100022, walltime=12:00:00

Changing to shadow directory: C:\Users\matte\local_work\pypsa-eur\.snakemake\shadow\tmpvuygbosw
INFO:pypsa.io:Imported network elec_s_128_ec_lv1.5_.nc has buses, carriers, generators, global_constraints, lines, links, loads, storage_units, stores
  File "C:\Users\matte\local_work\pypsa-eur\.snakemake\shadow\tmpvuygbosw\.snakemake\scripts\tmpcakhfupl.solve_network.py", line 954, in <module>
    n = solve_network(
        ^^^^^^^^^^^^^^
  File "C:\Users\matte\local_work\pypsa-eur\.snakemake\shadow\tmpvuygbosw\.snakemake\scripts\tmpcakhfupl.solve_network.py", line 890, in solve_network
    status, condition = n.optimize(**kwargs)
                        ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\matte\miniconda3\envs\pypsa-eur\Lib\site-packages\pypsa\optimization\optimize.py", line 592, in __call__
    return optimize(self._parent, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\matte\miniconda3\envs\pypsa-eur\Lib\site-packages\pypsa\optimization\optimize.py", line 562, in optimize
    m = create_model(
        ^^^^^^^^^^^^^
  File "C:\Users\matte\miniconda3\envs\pypsa-eur\Lib\site-packages\pypsa\optimization\optimize.py", line 304, in create_model
    define_loss_constraints(n, sns, c, transmission_losses)
  File "C:\Users\matte\miniconda3\envs\pypsa-eur\Lib\site-packages\pypsa\optimization\constraints.py", line 898, in define_loss_constraints
    assert isfinite(
AssertionError: Loss approximation requires finite 's_nom_max' for extendable branches:
 Line
0      inf
1      inf
10     inf
100    inf
101    inf
      ...
95     inf
96     inf
97     inf
98     inf
99     inf
Name: s_nom_max, Length: 228, dtype: float64
[Sat Jan  6 20:25:06 2024]
Error in rule solve_network:
    jobid: 1
    input: resources/networks/elec_s_128_ec_lv1.5_.nc, results/config.yaml
    output: results/networks/elec_s_128_ec_lv1.5_.nc
    log: logs\solve_network\elec_s_128_ec_lv1.5__solver.log, logs/solve_network/elec_s_128_ec_lv1.5__python.log (check log file(s) for error details)
    conda-env: C:\Users\matte\local_work\pypsa-eur\.snakemake\conda\0e95f95f0ae8597c6e893b0fc8d8460d_

RuleException:
CalledProcessError in file C:\Users\matte\local_work\pypsa-eur\rules/solve_electricity.smk, line 37:
Command 'C:/Users/matte/miniconda3/envs/pypsa-eur/python.exe "C:\Users\matte\local_work\pypsa-eur\.snakemake\shadow\tmpvuygbosw\.snakemake\scripts\tmpcakhfupl.solve_network.py"' returned non-zero exit status 1.
  File "C:\Users\matte\local_work\pypsa-eur\rules/solve_electricity.smk", line 37, in __rule_solve_network
  File "C:\Users\matte\miniconda3\envs\pypsa-eur\Lib\concurrent\futures\thread.py", line 58, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake\log\2024-01-06T202331.616449.snakemake.log

(BTW, I am running the version 0.9.0)

[fneum]

Ah, in your config, set lines: max_extension: 20000. The transmission loss approximation, which is now the default, requires an upper limit to the expansion of individual lines.

Fixed in 647a988c

[matteodefelice]

Added it to my config file, deleted the folder .snakemake and pulled the latest commit but I still get the same error. What am I missing?

[euronion]

Have you tried re-creating the networks used as input for the rule?

[matteodefelice]

@euronion how should I do that?

[euronion]

@matteodefelice You'd have to rerun the execution graph including the rule(s) which use the config entry, i.e. for lines: max_extension: 20000 the rule prepare_network. Since it's the rule directly preceeding solve_network in the DAG, the simplest thing is to just delete resources/networks/elec_s_128_ec_lv1.5_.nc and run the workflow again.

@fneum max_extensions is not listed under params for rule prepare_network, is that intentional?

[fneum]

Closing since it seems to be stale. But reopen if issue persists.