idaes-pse
idaes-pse copied to clipboard
IDAES
conda forge install of idaes
Here's some notes that I haven't tried yet but got from William Krinsman (a summer student of Shreyas')
OK basically I think it comes down to (you need to have conda build installed first,
conda install conda-build)conda skeleton pypi `package-name` conda build `package-name`What conda skeleton does is create a directory in the pwd containing the so-called "recipe" (meta.yaml and build.sh) for the package. Then conda build attempts to create the package. If it's successful, the package will be some tar.gz. file stored in one of the current environment's directories (something like
path/to/environment/conda-bld). Then you can upload the package to you or your organization's Anaconda channel using the anaconda client (conda install anaconda-client,anaconda upload path/to/package.tar.gz) or you can install it using something likeconda install path/to/package/tar.gz --use-local.It almost always works for Python packages, and even for R packages works 90% of the time (it used to be worse). The only times there seems to be difficulty is when the package requires C/C++ or Fotran code to be compiled, since sometimes conda build doesn't seem to look for the header files in the right locations. But that usually isn't an issue.
I think this is where I learned all of this from, so it might be more helpful https://docs.conda.io/projects/conda-build/en/latest/user-guide/tutorials/build-pkgs-skeleton.html
I started the process of setting up the conda-forge install of IDAES back in May. I did not get terribly far though. See this PR in conda-forge/staged-recipes: https://github.com/conda-forge/staged-recipes/pull/8398. I probably won’t be able to continue working on this until mid-July. If it needs done before that, I am happy to walk someone through what I have done so far.
With the pip install and the get-extensions script, is this necessary? It may just be extra work we don't need.
@eslickj I agree. I think we should close this until we have a compelling reason to support conda distribution.
@ksbeattie @jghouse88 @dangunter @eslickj I am scheduled to teach an optimization graduate elective starting January 2021. I plan to use IDAES selectively in the class. It will make my life easier if I can stick with 100% conda for students. In my experience, mixing pip and conda can lead to some weird things.
@adowling2, I can confirm the weird things. For me, I've had problems with Jupyter, Jupyter Qt-Console, and Qt, but I'm sure other things can have trouble too. If you are mostly installing stuff with Conda, then later installing IDAES, this seems like a good reason to take another look at a Conda install for IDAES.
This mostly bit me in the beginning were we didn't have a clear install procedure and I had a weird mix of Conda and Pip installed stuff. This was also before I really started keeping everything nicely separated in environments. In a class were the students probably will just install everything in one environment, it could get ugly.
And I’m tech support for the students.
On Tue, Sep 1, 2020 at 9:47 AM John Eslick [email protected] wrote:
@adowling2 https://github.com/adowling2, I can confirm the weird things. For me, I've had problems with Jupyter, Jupyter Qt-Console, and Qt, but I'm sure other things can have trouble too. If you are mostly installing stuff with Conda, then later installing IDAES, this seems like a good reason to take another look at a Conda install for IDAES.
This mostly bit me in the beginning were we didn't have a clear install procedure and I had a weird mix of Conda and Pip installed stuff. This was also before I really started keeping everything nicely separated in environments. In a class were the students probably will just install everything in one environment, it could get ugly.
I like the idea of having a pure conda install of things, but haven't found the time to look into it. Perhaps @lbianchi-lbl will?
The other advantage is that we might be able to promote IDAES within the Anaconda desktop app.
I'm going to add some resources below for whoever ends up working on an IDAES conda package. Both Pyomo and PyUtilib are distributed as conda packages on conda-forge. The feedstock repos for these packages are linked below in case they are helpful: https://github.com/conda-forge/pyutilib-feedstock https://github.com/conda-forge/pyomo-feedstock
@blnicho: thanks for the pointers! I've used the Pyomo configuration as a reference, and I found it very useful since this was my first time creating a Conda package from scratch.
I managed to get started on this, and I was able to successfully build and install an idaes-pse Conda package locally. I've started a Wiki page that at the moment contains the conda-build recipes and my notes on the steps needed to build, and possible ways to distribute, the package.
Please feel free to add your feedback or thoughts on this; my impression is that from here setting up the package distribution via conda-forge shouldn't be too problematic, but any input from maintainers with experience will be very welcome.
@lbianchi-lbl I'd like to revisit this after the great repo migration of 2020. I think I just hosed another conda environment mixing pip (for IDAES) with everything else...
I'm teaching a graduate elective starting in February 2020; I'd really to be able to install idaes via conda by then.
@adowling2 yes, I definitely agree. We actually did some progress on this with @ksbeattie before the Great Migration ( :laughing: ) started, and I'm looking forward to finishing once things settle down on that front. I don't think we're very far off, and having things ready and tested by that date seems like a fully realistic goal.
@lbianchi-lbl @ksbeattie I'm teaching a graduate elective (starts in 3 weeks) in which I'd like to use IDAES. Is this something we can get working before then?
Hi @adowling2, we recently made progress on this and we were able to successfully build the package with all-Conda dependencies and install it from a dedicated channel. We plan to create a new build of the Conda package ready for internal testing later this week when we cut the release.
In addition to the general IDAES Conda package, more specifically for the purpose of your course, we were thinking that it could be worth to look into having a dedicated Conda environment.yml file. This is essentially the Conda version of requirements.txt, allowing to create a complete Conda environment from scratch in one shot with conda env create --file environment.yml.
In my experience, this is a simple but effective way to manage setups with non-trivial dependencies: since all dependencies are known when the environment is created, there is less room for conflict compared to creating an empty environment and installing individual packages one at the time. Additionally, if the platforms where the environment will be installed are known and homogeneous enough (e.g. all attendants will be using a Windows machine), it is also possible to create the environment file with "strict" dependencies pinning directly from an existing environment that's known to be working, which would reduce the possibility of conflicts even more.
If this sounds like a workable solution, I'm happy to help with setting this up and testing it before the start of the course.
@ksbeattie @lbianchi-lbl Is this something I can test tomorrow or Monday?
Edit: I'll likely be supporting Winows and macOS users. I'll try to direct people to Colab too.
Edit: I verified this works!
Update: as of IDAES 1.9.0, we have a working idaes-pse package (see https://idaes-pse.readthedocs.io/en/stable/getting_started/index.html?highlight=Conda#generic-install). Currently, this package is built manually and distributed via a dedicated IDAES-PSE channel on Anaconda.org.
The next step for this issue would be to enable the conda-forge build so that the idaes-pse package is built and hosted directly by conda-forge (i.e. without requiring the additional IDAES-PSE channel).
