[WIP] Frequency Flattening Optimizer

[rohithsrinivaas]

This PR seeks to port the original FFOpt workflows from atomate to atomate2. This workflow is used to generate molecular structures that correspond to a global minima instead of a higher-order saddle point in the Potential Energy Surface (PES). Q-Chem is the DFT calculator used in the workflow. Tagging @utf @rdguha1995 @rkingsbury since they have been using FFOpt from atomate.

Reference:

Evan Walter Clark Spotte-Smith et.al

Completed

Flow for Double Opt (similar to VASP workflow) Flow for Optimizer followed by Frequency calculation Flow for Frequency Flattening Optimizer Removed small bugs in OptMaker and FreqMaker

TODO

Finish tests

[esoteric-ephemera]

@rdguha1995 and @rkingsbury: this is also a good time to deprecate the prev_qchem_dir kwarg in the QChem makers in favor of the generic prev_dir. The other code packages currently use the generic prev_dir.

I've added a feature so that both kwargs can currently be set, with prev_dir taking precedence.

Also, it seems like the QCInputs were not being correctly parsed in the QCInputSet.from_directory method: the input_dict field should pull from mol.qin right?

@janosh: what do you think is a good deprecation period for this?

[janosh]

@janosh: what do you think is a good deprecation period for this?

depends on the size of the user base. these flows are quite new right? small user base can migrate more quickly

[rkingsbury]

@rohithsrinivaas thanks very much for your work on this! Having FFOpt in atomate2 is the main feature I've been waiting on before migrating workflows away from atomate 1.

Flow for Double Opt (similar to VASP workflow)

Did we have a double opt in the original atomate1 workflows for Q-Chem? I don't recall that we did, and I'm not sure a double opt really makes sense for a molecular calculation. Maybe @espottesmith or @rdguha1995 or @sblau have other thoughts?

@rdguha1995 and @rkingsbury: this is also a good time to deprecate the prev_qchem_dir kwarg in the QChem makers in favor of the generic prev_dir. The other code packages currently use the generic prev_dir.

Thanks, I'm highly in favor of moving to prev_dir. To my knowledge, there are very few people regularly using the atomate2 Q-Chem input sets / makers, so I think the migration can happen quickly. That said, there isn't much harm in leaving the warning in place for a few releases just to ensure smooth migration.

[espottesmith]

We never really used double-opt for any of the datasets that we developed (LIBE, MADEIRA, now radQM9), but that doesn't mean that it's useless. Certainly in the literature folks do things like pre-opting with a force-field or with xTB or something. I can also imagine wanting to pre-opt with a cheap DFT level of theory.

Honestly, my biggest critique of this PR, and one that I hope offends no one is: is this actually necessary? As I understand it, this is basically a replication of the atomate1 workflows that doesn't add any features, and it also duplicates some work in QuAcc, which is where @sblau and (to a lesser extent) I have been developing/running our workflows for a while now. The PR does no harm (other than potentially confusing new users trying to figure out how to run Q-Chem in high throughput), but it's also unclear who it helps. I suppose you said you might shift over, @rkingsbury. Perhaps that's enough.

Agreed that I don't think deprecation of prev_qchem_dir will cause harm.

[janosh]

As I understand it, this is basically a replication of the atomate1 workflows that doesn't add any features, and it also duplicates some work in QuAcc

i have no stake in this but would like to point out that given the long-term goal of fully replacing atomate1 with atomate2 requires that all flows be migrated (regardless of whether new features are added or not). from that perspective, even though i don't plan to use this flow, it sounds like a valuable contribution

unless there's some unspoken agreement about how to split up functionality between atomate2 and quacc from @utf, @Andrew-S-Rosen, @computron, i'd say the fact that it's also implemented in quacc has no bearing. maybe a plan for atomate2 + quacc codevelopment is something to discuss in next MP foundation meeting? (@JaGeo)

[espottesmith]

Yeah, there's a larger discussion here that's worth having, somewhere other than GitHub PR comments. I will say that I'm not advocating that atomate2 and QuAcc need to coordinate with one another, though they certainly can if everyone would be happy that way.

[JaGeo]

@janosh I will try to plan the next foundation meeting in two weeks. I hope I can think clearly again at that point.... I will also send an agenda. Let me then know if I should add anything.

[rkingsbury]

We never really used double-opt for any of the datasets that we developed (LIBE, MADEIRA, now radQM9), but that doesn't mean that it's useless. Certainly in the literature folks do things like pre-opting with a force-field or with xTB or something. I can also imagine wanting to pre-opt with a cheap DFT level of theory.

Point well taken!

As I understand it, this is basically a replication of the atomate1 workflows that doesn't add any features, and it also duplicates some work in QuAcc

i have no stake in this but would like to point out that given the long-term goal of fully replacing atomate1 with atomate2 requires that all flows be migrated (regardless of whether new features are added or not). from that perspective, even though i don't plan to use this flow, it sounds like a valuable contribution

Yes, for me the primary motivation for this is to enable full migration away from atomate1. So I think this PR should exist solely because atomate1 is/was the only other place that FFOpt was available.

unless there's some unspoken agreement about how to split up functionality between atomate2 and quacc from @utf, @Andrew-S-Rosen, @computron, i'd say the fact that it's also implemented in quacc has no bearing. maybe a plan for atomate2 + quacc codevelopment is something to discuss in next MP foundation meeting? (@JaGeo)

I was not aware of the work in quacc (though I may check it out) but up to this point we have not made any effort to coordinate the two.

[Andrew-S-Rosen]

This seems pretty clear cut to me.

Atomate1 is a thing of the past now. It's not maintained. If people want to use something from atomate1, it should be ported to atomate2 even if it is "duplicating functionality". Consider it a refactor if you wish. Refactoring is still worthwhile, as it helps with long-term maintainability. While it may be replicating old features, that is true for many of the workflows in atomate2. To put it bluntly, I am never going to use atomate1 again, so if a workflow exists there that's not in atomate2, then it might as well not exist to me.

With regards to quacc, it is largely irrelevant to the conversation. If there is similar content in both, then so be it. In the end, the functionality is quite different because one of the main changes is that @samblau wanted to build his Q-Chem workflows around Sella features (an ASE optimizer). That means the backend is built around ASE, hence why quacc was more natural for him. In any case, it's a non-issue IMO. There are intentionally no plans for cross development, and people can feel free to do as they please.

Proceed!

[esoteric-ephemera]

@rohithsrinivaas I made a few changes:

• Added a task_type attribute to BaseQCMaker that allows one to set the task_type of a calculation explicitly. Previously, this was set by the job's name, which doesn't make jobs with multiple repeated steps (like Frequency Analysis 1, Frequency Analysis 2,...) easy to label
• Made the FrequencyOptFlatteningMaker perform a Frequency Analysis step only if it's the initial step and the molecule has no negative frequencies. This prevents duplicate analysis steps from being run
• Added tests for the new flow. We just need one for the double relax maker

[samblau]

I'm running around like a crazy person right now on partial paternity leave and so don't have the time I'd like to dig into this, but a couple key points to make:

• FFopt is almost entirely a custodian feature instead of an atomate1 feature. The only thing you should need to port over to atomate2 specifically for FFopt is FrequencyFlatteningOptimizeFW.
• I wrote a handful of atomate1 workflows back in the day, but I pretty quickly reverted to just using FrequencyFlatteningOptimizeFW and passing structures between individual fireworks via script as necessary. Building a larger workflow just didn't end up being sufficiently useful because each application necessitated a slightly different procedure. Easier to just write modular code into the main branch and stick modular pieces together as necessary. All this to say - I would not advocate for porting all atomate1 Q-Chem functionality just because it's there, especially the workflows.
• @rkingsbury and anyone who actually wants to be doing science involving geometry optimization with Q-Chem (or with any electronic structure calculations on molecules): I stroooooongly suggest you switch over to QuAcc so that you can use Sella (with Q-Chem or xTB or ORCA or whatever). I literally don't even need to FFopt anymore because I never encounter negative frequencies when optimizing with Sella.

[rkingsbury]

@rkingsbury and anyone who actually wants to be doing science involving geometry optimization with Q-Chem (or with any electronic structure calculations on molecules): I stroooooongly suggest you switch over to QuAcc so that you can use Sella (with Q-Chem or xTB or ORCA or whatever). I literally don't even need to FFopt anymore because I never encounter negative frequencies when optimizing with Sella.

!!! Thanks for the tip @samblau! 👍🏻

[rdguha1995]

Apologies for being late to this! Firstly, a huge shout out to @rohithsrinivaas for his work on this. I think there has already been an extensive discussion on the overall applicability of this PR and I would not want to belabor my views on that. Personally I agree with @Andrew-S-Rosen - any QChem related development on atomate2 should be done with a view that one year down the line, there is a good chance that atomate1 will not be used. In that time period, even if everyone doing molecular calculations in the current ecosystem move over to quacc, it should not mean that these workflows should not exist in atomate2.

Coming to the question by @rkingsbury on a double-opt workflow, I have no strong opinion. I agree with @espottesmith that it is quite common to combine successive optimizations at increasing levels of theory. It can be included as a basic implementation of a flow, but I don't think it's absolutely necessary, because there is no "dynamic" aspect to it. From my understanding, the behavior can be duplicated if you append two opt jobs.

Lastly, absolutely agree with @esoteric-ephemera , we should move over to prev_dir. The code base I was working with had prev_vasp_dir, which naturally led to the prev_qchem_dir. Haha.

[esoteric-ephemera]

Hey @utf, @JaGeo, @janosh, chatted with @rohithsrinivaas and this should be ready for review / merging. Just want to make sure that we meet the deadline for adding WFs / material to the atomate2 paper

[utf]

Thanks, this looks great to me.