RosettaCommons
Question: Exploring the Optimal Range for Denoising Timesteps (Iterations) in Binder Design
Dear RFdiffusion Developers
First, my sincere thanks for creating such a powerful tool for protein design. I am writing to seek your expert guidance on a subtle observation we've made regarding the iterations hyperparameter (diffuser.T) for a de novo protein binder design task.
Observation: In our experiments, we have found that a higher number of iterations does not necessarily guarantee a better success rate. Specifically, we observed: A run with 200 iterations produced a good set of candidate designs with favorable RMSD and i_pae scores.
When we increased the iterations to 500, the overall quality and success rate of the designs slightly decreased. Although the difference was not dramatic, it was contrary to our initial expectation that more extensive sampling would be beneficial.
This leads us to believe that there might be a "sweet spot" for the number of iterations that is specific to the design problem.
My Questions
-
Is this a commonly observed phenomenon? We hypothesize that beyond a certain point, a higher iteration count might not add significant value and could slightly increase the probability of exploring less productive regions of the conformational space.
-
Could you share any recommended strategies or heuristics for efficiently identifying this optimal iteration range for a new binder design project?
-
Are there any general principles regarding how factors like binder length or target surface complexity might influence this optimal range?
Thank you for your time and for sharing your expertise. Any insights you could provide would be immensely valuable.
I don't have extensive experience with changing the number of iteration steps, but I will note that RFdiffusion was originally trained with a diffuser.T of 200. It could very well be that adding more iterations wouldn't really help much, as there might be a lower limit to the "granularity" at which the model learned to denoise. (I don't know how much this has been studied -- I'd probably recommend looking at general (e.g. image) diffusion model research to see what that has to say.)
It's also worth pointing out that the default diffuser.T is now 50, as it's been seen that reducing this does not degrade performance all that much in the general case, and (generally speaking), it's probably worth generating 3-4 times as many outputs and then throwing away most of them, rather than spending a large amount of compute time on any single model. -- Of course, this trade off may vary based on the particular biological system you're working with. (General recommended approach is probably guess-and-check. Try it with the default, and if you're getting quality issues with the designs, see if adjusting the parameter changes the quality metrics you're interested in meaningfully.)
I don't have extensive experience with changing the number of iteration steps, but I will note that RFdiffusion was originally trained with a diffuser.T of 200. It could very well be that adding more iterations wouldn't really help much, as there might be a lower limit to the "granularity" at which the model learned to denoise. (I don't know how much this has been studied -- I'd probably recommend looking at general (e.g. image) diffusion model research to see what that has to say.)
It's also worth pointing out that the default diffuser.T is now 50, as it's been seen that reducing this does not degrade performance all that much in the general case, and (generally speaking), it's probably worth generating 3-4 times as many outputs and then throwing away most of them, rather than spending a large amount of compute time on any single model. -- Of course, this trade off may vary based on the particular biological system you're working with. (General recommended approach is probably guess-and-check. Try it with the default, and if you're getting quality issues with the designs, see if adjusting the parameter changes the quality metrics you're interested in meaningfully.)
That's very clear and helpful, thank you. The T=200 training detail and the new T=50 default are the key pieces of information we needed. We'll proceed with the recommended strategy of generating more samples with a lower T. Much appreciated!
Hi Shoyeon,
When you said you run RFdiffusion with 200 or 500 iterations, do you mean the noising step or the denoising step? It seems to me that changing diffusion.T=200 alone would not change the number of timesteps in the inference (denoising) step, and it will also result in this error: Traceback (most recent call last): File "~/RFdiffusion/./scripts/run_inference.py", line 103, in main denoised_xyz_stack = torch.stack(denoised_xyz_stack) RuntimeError: stack expects a non-empty TensorList Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Would you mind enlightening me as to how we can run RFdiffusion with increased iterations as you have said? What parameters are we to modify in the HYDRA configs before running the inference: ../scripts/run_inference.py inference.output_prefix=example_outputs/design_ppi inference.input_pdb=input_pdbs/insulin_target.pdb 'contigmap.contigs=[A1-150/0 70-100]' 'ppi.hotspot_res=[A59,A83,A91]' inference.num_designs=10 denoiser.noise_scale_ca=0 denoiser.noise_scale_frame=0
Many thanks!
Hi Shoyeon,
When you said you run RFdiffusion with 200 or 500 iterations, do you mean the noising step or the denoising step? It seems to me that changing diffusion.T=200 alone would not change the number of timesteps in the inference (denoising) step, and it will also result in this error: Traceback (most recent call last): File "~/RFdiffusion/./scripts/run_inference.py", line 103, in main denoised_xyz_stack = torch.stack(denoised_xyz_stack) RuntimeError: stack expects a non-empty TensorList Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Would you mind enlightening me as to how we can run RFdiffusion with increased iterations as you have said? What parameters are we to modify in the HYDRA configs before running the inference: ../scripts/run_inference.py inference.output_prefix=example_outputs/design_ppi inference.input_pdb=input_pdbs/insulin_target.pdb 'contigmap.contigs=[A1-150/0 70-100]' 'ppi.hotspot_res=[A59,A83,A91]' inference.num_designs=10 denoiser.noise_scale_ca=0 denoiser.noise_scale_frame=0
Many thanks!
Thank you very much for clarifying the difference between T and the inference steps. I appreciate the detailed solution. I will implement the change using inf_steps=500 and begin testing immediately.