dflow icon indicating copy to clipboard operation
dflow copied to clipboard

MEMOIZATION cache does not work with other namespace except 'argo'

Open heidongxianhua opened this issue 1 year ago • 5 comments

Checklist

  • [ ] Checked the syntax is legal within dflow.
  • [ ] Tested using the latest version.

Summary

What happened/what you expected to happen?

What version are you running?

Diagnostics

Paste the smallest python script that reproduces the bug when I run the examples/test_big_parameter.py, and change the namespace to test, the cache can not hit, but it work well when the namespace is 'argo'.


with the namespace 'test' image

with the namespace 'argo' image

heidongxianhua avatar Nov 28 '23 04:11 heidongxianhua

Could you please provide details on how you change the namespace? Do you change the config['namespace'] from dflow.config, or do you use Workflow(...,namespace='test')? If it’s the latter, please try adding dflow.config['namespace']='test' in the script and then check if there are any remaining issues.

saltball avatar Nov 28 '23 10:11 saltball

Could you please provide details on how you change the namespace? Do you change the config['namespace'] from dflow.config, or do you use Workflow(...,namespace='test')? If it’s the latter, please try adding dflow.config['namespace']='test' in the script and then check if there are any remaining issues.

yes, I change the namespace in config, https://github.com/deepmodeling/dflow/blob/master/src/dflow/config.py#L33C40-L33C40

heidongxianhua avatar Nov 29 '23 03:11 heidongxianhua

Are your Argo components installed in the test namespace as well? Because Argo will attempt to look for memoized configmaps in the same namespace by default.

zjgemi avatar Dec 20 '23 08:12 zjgemi

Are your Argo components installed in the test namespace as well? Because Argo will attempt to look for memoized configmaps in the same namespace by default.

yeah, maybe it is. And, which components should I install in the test namespace too? @zjgemi

heidongxianhua avatar Dec 28 '23 02:12 heidongxianhua

https://argo-workflows.readthedocs.io/en/latest/managed-namespace/ If you only run workflows that are maintained by the same team that manages the Argo Workflows installation, you may want a namespace install (install all Argo components in the test namespace). If you want some users to run workflows in the test namespace without granting them privileges in the namespace where Argo components is installed. Things are more complicated. You may try to submit memoized configmaps to the namespace where Argo components is installed and submit workflows to the test namespace to see if it works.

zjgemi avatar Jan 05 '24 07:01 zjgemi