coldfront

coldfront copied to clipboard

Add a proper REST API to ColdFront. Features should include:

• Authentication: users should be able to create app credentials that will allow access to only their data.
• Read access to most data elements: resources, allocations, attributes, grants, pubs, users, etc.
• Write access where appropriate
• Admins should be able to create user accounts?

This independently landed on my TODO list - is anyone working on that? I willing to work on that, but I don't want to duplicate effort :)

@mdzik Would be great to get some help on this and welcome anything you're willing to contribute (code, architecture design suggestions, etc.). At this point I don't think there would be much duplicated effort. We have done some preliminary work on code for authenticating the API (token based auth backend) but it's in the very early stages. @brisco17 is leading the effort on that and may have some code up in the coming weeks. So feel free to hack away on this and we're more than happy to collaborate.

We already use Django REST (https://www.django-rest-framework.org/) framework internally, with its token authentication, for slurm synchronization. It is really mature framework and works like a charm. I would prefer to build in same eco-system, but you probably have some reasons behind not using it?

We could utilize OAuth/OIDC "offline token" mode, as mobile apps are often using. Or JWT-based one.

Our main goal is to build Go app for, mainly, MOTD for users - stuff like "You have account XX-XXXX at this cluster, with XXX CPUhrs left" or "You need to review your projects" etc. Other use-case is the OnDemand - to provide users drop-down with Accounts to be used in job submission.

I was thinking to start with something simple like (quick draft, rewritten from our internal spec for slurm, sorry for typos etc.):

paths:
  /api/allocations/:
    get:
      operationId: listUserAllocations
      description: ''
      parameters: []
      responses:
        '200':
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Allocation'


  /api/allocations/{id}/:
    get:
      operationId: retrieveUserAllocation
      description: ''
      parameters:
      - name: id
        in: path
        required: true
        description: "A unique integer value identifying this Allocation."
        schema:
          type: int
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Allocation'
    post:
      operationId: renevUserAllocation
      description: ''
      parameters:
      - name: id
        in: path
        required: true
        description: "A unique integer value identifying this Allocation."
        schema:
          type: int
      - name: action
        in: path
        required: true
        description: "Action to take"
        schema:
          type: string
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Allocation'

Allocation:
      type: object
      properties:
       - resource: Recource
       - attributes: Array(Attribute)
       - id
      required:
      - id

We already use Django REST (https://www.django-rest-framework.org/) framework internally, with its token authentication, for slurm synchronization. It is really mature framework and works like a charm. I would prefer to build in same eco-system, but you probably have some reasons behind not using it?

Yes, sounds great. We were planning on using Django REST so this is good with us.

We could utilize OAuth/OIDC "offline token" mode, as mobile apps are often using. Or JWT-based one.

Sounds good. Even if we start with something simple initially that's fine too.

Our main goal is to build Go app for, mainly, MOTD for users - stuff like "You have account XX-XXXX at this cluster, with XXX CPUhrs left" or "You need to review your projects" etc. Other use-case is the OnDemand - to provide users drop-down with Accounts to be used in job submission.

I was thinking to start with something simple like (quick draft, rewritten from our internal spec for slurm, sorry for typos etc.):

paths:
  /api/allocations/:
    get:
      operationId: listUserAllocations
      description: ''
      parameters: []
      responses:
        '200':
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Allocation'


  /api/allocations/{id}/:
    get:
      operationId: retrieveUserAllocation
      description: ''
      parameters:
      - name: id
        in: path
        required: true
        description: "A unique integer value identifying this Allocation."
        schema:
          type: int
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Allocation'
    post:
      operationId: renevUserAllocation
      description: ''
      parameters:
      - name: id
        in: path
        required: true
        description: "A unique integer value identifying this Allocation."
        schema:
          type: int
      - name: action
        in: path
        required: true
        description: "Action to take"
        schema:
          type: string
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Allocation'

Allocation:
      type: object
      properties:
       - resource: Recource
       - attributes: Array(Attribute)
       - id
      required:
      - id

This all looks good to me!

How is this coming along? We're hoping to implement coldfront but without a REST API, it's much harder to import/export data.

For example, we need to sync Active Directory groups with coldfront projects, so I need a programmatic way to get a list of projects with their PI's, managers, and users (ideally in one call 🙂).

If there's some existing code written, I'm happy to attempt to contribute!

@lcrownover This is a high priority issue for us and we're very interested in getting a proper API implemented. There has been some initial work done by @mdzik in #425 and we're hoping to start reviewing that in the coming weeks. We'll gladly accept any help with this and welcome any contributions!