CliMA
PySDM and ClimateMachine coupling examples in Kinematic setup
This PR includes a coupling logic for ClimateMachine.jl and PySDM.
PySDM is a particle-based aerosol/cloud microphysics package written entirely in Python. This PR depicts how Python modules can be leveraged within ClimateMachine.jl including the continuous integration setup.
The initial set of tests included here is based on the kinematic 2D example previously used as a test case in both PySDM and ClimateMachine.jl. In the tests added in this PR, ClimateMachine.jl handles air motion and total water transport, while PySDM handles representation of aerosol and liquid water transport as well as phase changes leading to formation of cloud water.
Output from PySDM is handled using VTK files. Example animation with an evolution of radius computed from particle properties is shown below:
https://user-images.githubusercontent.com/63914185/138484760-045285c2-b544-46bc-a801-6e6499e04145.mp4
What one can already be achieved with the code in this PR:
- Explicit (unparameterised) representation of activation and deactivation/resuspension of CCN allowing comparison with activation parameterization for non-trivial setups.
- Aerosol-coupled (and aerosol-budget coupled) prediction of cloud droplet spectral properties such as effective radius, droplet spectral width, radar reflectivity, …
- Lagrangian (particle-based) transport of aerosol and water (positive definite, numerical diffusion free)
Credits:
-
Implementation: Oleksii Bulenok (CS student at Jagiellonian University, Kraków, Poland)
-
Support: Anna Jaruga (Caltech) & Sylwester Arabas (UIUC & Jagiellonian Univ.)
-
[X] Code follows the style guidelines OR N/A.
-
[X] Unit tests are included OR N/A.
-
[X] Code is exercised in an integration test OR N/A.
-
[X] Documentation has been added/updated OR N/A.
Co-authored-by: Oleksii Bulenok [email protected] Co-authored-by: Anna Jaruga [email protected] Co-authored-by: Sylwester Arabas [email protected]
try
Build succeeded:
- format
- test-os (windows-latest)
- test-os (ubuntu-latest)
- buildkite/climatemachine-docs
- buildkite/climatemachine-ci
- test-os (macos-latest)
![bors[bot] avatar](https://avatars.githubusercontent.com/in/1847?v=4 "Published by a pub.dev verified publisher"){kind=small}
Some additional comments summarising our discussions with @abulenok and @trontrytel:
Coupling logic outline:
- Thermodynamics: ClimateMachine.jl is run without any moisture model. The total specific humidity is not changed in the simulation. The phase partitioning between water vapour and liquid water resulting from the condensation and evaporation processes is computed based on the PySDM state.
- Hydrodynamics/grid: quantities defined on ClimateMachine.jl grid are interpolated onto Arakawa-C regular grid in each timestep and passed to PySDM.
- Technicalities: Julia-Python coupling is realised using PyCall (https://github.com/JuliaPy/PyCall.jl). Additional dependencies (PyCall, Conda) are collected in Project.toml file in the test folder. Conda is used to install PySDM (through pip) in runtests.jl
- Output format: PySDM particles state is accessible through additional VTK files
Known limitations / TODOs / kludges:
- PySDM’s native water variables are mass mixing ratios (mass per mass of dry air), ClimateMachine.jl uses specific quantities (mass per total mass of air), for now, this discrepancy is ignored.
- Likewise, PySDM’s density variable is the dry density, while ClimateMachine.jl uses total density, and this discrepancy is ignored for now.
- For now, the phase changes represented with PySDM are not reflected in any change of state in ClimateMachine.jl - even though there are clouds present and the water vapour is thus depleted, other components of ClimateMachine see the state as dry.
- ClimateMachine.jl solves for the diffusion of total water, while PySDM does not feature representation of diffusion of liquid water - inconsistent
- The newly added KM_CliMa_no_saturation_adjustment.jl and KM_saturation_adjustment.jl code from pre-existing tests should be refactored removing code duplication
- While PySDM does support GPU-resident calculations (using CUDA through the ThrustRTC package, https://pypi.org/p/ThrustRTC), this PR depicts a CPU setup only.
Low-hanging fruits for future development:
- Enabling other processes in PySDM including: (i) coalescence (requires a source term for total water and internal energy, note that grid interpolation needs to be taken into account); (ii) breakup (see @edejong-caltech fork, work in progress)
More challenging potential developments:
- aqueous chemistry (tested with parcel setup only so far in PySDM)
- GPU-enabled setup
- 3D non-kinematic simulations (PySDM has not been used yet in 3D setups)
- Boundary conditions for particles (eg., kinematic squall line case is hard to implement due to the need to specify particle properties when handling inflow through domain boundary)
@trontrytel @slayoo @abulenok
Fantastic work! This is very exciting. One step closer to having a PySDMachine superdroplet LES!
The animation looks really cool! Bravo! tada
I think the main thing that I would suggest we change is related to code loading. There are several files, e.g.,
KM_PySDM.jl, that are included in multiple modules, which are then included in tests. I think it's preferred to load each block once. We can do that by either moving some of the test modules outside of modules, and include the shared code at the top of theruntests.jlfile, or by putting the shared code in a module and usingimport ..SharedModuleinside the test modules.
Thanks for the suggestion. The code loading is changed in https://github.com/CliMA/ClimateMachine.jl/pull/2244/commits/690618ac30cad43f4964e82f66bb2c8e414f19e5.