Support deploymentType PERSISTENT_2

[ittus]

Is your feature request related to a problem?/Why is this needed PERSISTENT_2 deployment type has some advantages: better throughput, auto synchronize from LustreFSX -> S3. It's not supported in aws-fsx-csi-driver now.

deploymentType (Optional) - FSx for Lustre supports three deployment types, SCRATCH_1, SCRATCH_2 and PERSISTENT_1. Default: SCRATCH_1.

/feature

Describe the solution you'd like in detail Support PERSISTENT_2 as an option in deploymentType

More information FSx Lustre released support for a new deployment type "Persistent_2" at re:Invent 2021. It provides long term storage with higher throughput tiers (125, 250, 500, 1000 MiB/s/TiB). See https://docs.aws.amazon.com/fsx/latest/LustreGuide/using-fsx-lustre.html for more details.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[ittus]

/remove-lifecycle stale

[kwohlfahrt]

FWIW, this seems to be purely a documentation issue. I was able to create a storage class (and a volume from it) with the following parameters:

      deploymentType: "PERSISTENT_2"
      perUnitStorageThroughput: "125"
      storageType: "SSD"

[ilkarataev]

FWIW, this seems to be purely a documentation issue. I was able to create a storage class (and a volume from it) with the following parameters:

      deploymentType: "PERSISTENT_2"
      perUnitStorageThroughput: "125"
      storageType: "SSD"

If you not use S3 import it work)

[jacobwolfaws]

/lifecycle frozen Yes, PERSISTENT_2 support is purely a documentation issue at this point. Freezing and will close once that documentation change has been made

[ittus]

I've tried to create PERSISTENT_2 with s3ImportPath and there is an error

 BadRequest: Linking a Persistent 2 file system to an S3 bucket using the LustreConfiguraton is not supported. Create a file system and then create a data repository association to link S3 buckets to the file system. For more details, visit https://docs.aws.amazon.com/fsx/latest/LustreGuide/create-dra-linked-data-repo.html

If s3ImportPath is removed, then volume is created successfully.

Is there anyway to declare the s3 bucket association as PERSISTENT_1?

[PatrickGhadban]

You can't create a data repo association w/ PERSISTENT_1 (P_1) file systems but you can create an S3 linked P_1 file system with autoImports, for example .

Reference: https://docs.aws.amazon.com/fsx/latest/LustreGuide/create-dra-linked-data-repo.html

Data repository associations, automatic export, and multiple data repositories are only supported on Persistent 2 file systems. If you're using a file system with an older FSx for Lustre deployment type, see Working with older deployment types.

[ittus]

One idea is changing API for PERSISTENT_2 to support multiple dataRepositories, for example:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: fsx-sc
provisioner: fsx.csi.aws.com
parameters:
  subnetId: <subnet-id>
  securityGroupIds: <security-group-id>
  deploymentType: PERSISTENT_2
  dataRepositories:
  - fileSystemPath: /ns1/path1
    dataRepositoryPath: s3://s3-path-1
    BatchImportMetaDataOnCreate: true
  - fileSystemPath: /ns1/path2
    dataRepositoryPath: s3://s3-path-2
    BatchImportMetaDataOnCreate: true

Then call CreateDataRepositoryAssociation API in CreateFileSystem

(cc: @jacobwolfaws )

[PatrickGhadban]

That actually a plan we have moving forward but not something we have resourced at the time being. When it is resourced, we will let you know

[jacobwolfaws]

We've updated the documentation to show P2 support. @ittus feel free to create a new issue dedicated to s3ImportPath

[jacobwolfaws]

/close

[k8s-ci-robot]

@jacobwolfaws: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jacobwolfaws]

/remove-lifecycle frozen