kfp-tekton
kfp-tekton copied to clipboard
Volume claim is generated with access mode ReadWriteMany for default storage class
/kind bug
Here is a simple KFP application:
from kfp.components import InputPath, InputTextFile, OutputPath, OutputTextFile
from kfp.components import func_to_container_op
from kfp_tekton.compiler import TektonCompiler
# Writing many numbers
@func_to_container_op
def write_numbers(numbers_path: OutputPath(str), start: int = 0, count: int = 10):
with open(numbers_path, 'w') as writer:
for i in range(start, count):
writer.write(str(i) + '\n')
# Reading and summing many numbers
@func_to_container_op
def sum_numbers(numbers_path: InputPath(str)) -> int:
sum = 0
with open(numbers_path, 'r') as reader:
for line in reader:
sum = sum + int(line)
return sum
# Printing
@func_to_container_op
def print_text(text_path: InputPath()): # The "text" input is untyped so that any data can be printed
'''Print text'''
with open(text_path, 'r') as reader:
for line in reader:
print(line, end = '')
# Pipeline to sum 100000 numbers
def sum_pipeline(count: 'Integer' = 100000):
numbers_task = write_numbers(count=count)
print_text(numbers_task.output)
sum_task = sum_numbers(numbers_task.outputs['numbers'])
print_text(sum_task.output)
yamlFile = 'sum_pipeline.yaml'
TektonCompiler().compile(sum_pipeline, yamlFile)
The execution requires a PVC for execution.
But if you look at generated yaml file, you will see
..................
workspaces:
- name: sum-pipeline
volumeClaimTemplate:
spec:
accessModes: [ReadWriteMany]
resources:
requests: {storage: 2Gi}
Which requires read write many access mode for a default storage class, which fails on PVC binding
What do I expext
..................
workspaces:
- name: sum-pipeline
volumeClaimTemplate:
spec:
accessModes: [ReadWriteOnce]
resources:
requests: {storage: 2Gi}
Which actually works
Environment:
- Python Version 3.7
- SDK Version: 0.8.1
- Tekton Version (use
tkn version): v0.24 - Kubernetes Version (use
kubectl version): v1.19.0+d856161 - OS (e.g. from
/etc/os-release):
You can configure the workspace mounting strategy using these env variables https://github.com/kubeflow/kfp-tekton/tree/master/sdk#big-data-passing-workspace-configuration
Thanks @Tomcli, it works great
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
/close fixed
@Tomcli: Closing this issue.
In response to this:
/close fixed
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.