DataFed icon indicating copy to clipboard operation
DataFed copied to clipboard

[DAPS-1517] Add Handling to CPP Core Service

Open AronPerez opened this issue 5 months ago • 5 comments

Ticket

DAPS-1517

Description

This PR implements a new repository type field with two supported values: "globus" (default, for traditional repositories) and "metadata_only" (for metadata-only repositories).

Implementation Details

Protocol Buffer Changes

  • SDMS.proto: Added optional string type = 12 to RepoData message
  • SDMS_Auth.proto: Added optional string type = 13 to RepoCreateRequest and RepoUpdateRequest

C++ Core Service (DatabaseAPI.cpp)

  • repoCreate: Handles type field from request, forwards to Foxx API
  • repoUpdate: Handles type field from request, forwards to Foxx API
  • setRepoData: Reads type field from database response and populates RepoData

How Has This Been Tested?

  • Unit tests: core/database/foxx/tests/operations.test.js & core/database/foxx/tests/repo.test.js
  • End-to-end tests: Added comprehensive tests in tests/end-to-end/test_api_repo.py
  • Integration: Verified type field flows correctly through Python SDK → C++ Core → Database → Response

Summary by Sourcery

Add handling for the repository type field in the core service, update the DatabaseAPI client to pass and parse the type, and cover the new behavior with end-to-end tests.

New Features:

  • Add support for a new repository 'type' field with values 'globus' and 'metadata_only'

Enhancements:

  • Extend the C++ DatabaseAPI to include the type field in repoCreate, repoUpdate, and response parsing

Tests:

  • Add end-to-end tests for creating repositories with explicit types, defaulting behavior, and updating the type field

AronPerez avatar Aug 18 '25 03:08 AronPerez

Reviewer's Guide

This PR introduces a new “type” attribute for repositories across the C++ core service, protobuf schemas, and end-to-end SDK tests, defaulting to “globus” and supporting a “metadata_only” mode.

Class diagram for updated Repository-related protobuf messages

classDiagram
    class RepoCreateRequest {
        +string type
        ...
    }
    class RepoUpdateRequest {
        +string type
        ...
    }
    class RepoDataReply {
        +RepoData repo
        ...
    }
    class RepoData {
        +string type
        ...
    }
    RepoCreateRequest --> RepoData
    RepoUpdateRequest --> RepoData
    RepoDataReply --> RepoData

File-Level Changes

Change Details Files
Integrate the new “type” field into DatabaseAPI
  • Include type in repoCreate payload when present
  • Include type in repoUpdate payload when present
  • Populate RepoDataReply with type from stored JSON
core/server/DatabaseAPI.cpp
Extend protobuf schemas to define repository type
  • Add new type field to SDMS.proto
  • Add new type field to SDMS_Auth.proto
common/proto/common/SDMS.proto
common/proto/common/SDMS_Auth.proto
Add end-to-end tests for repository type handling
  • Test explicit globus type on create
  • Test explicit metadata_only type on create
  • Verify default type when unspecified
  • Test updating type via repoUpdate
tests/end-to-end/test_api_repo.py

Possibly linked issues

  • #1517: The PR implements the repository type system with 'globus' and 'metadata_only' enums as specified in the issue.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an issue from a review comment by replying to it. You can also reply to a review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull request title to generate a title at any time. You can also comment @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in the pull request body to generate a PR summary at any time exactly where you want it. You can also comment @sourcery-ai summary on the pull request to (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the pull request to resolve all Sourcery comments. Useful if you've already addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull request to dismiss all existing Sourcery reviews. Especially useful if you want to start fresh with a new review - don't forget to comment @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

  • Contact our support team for questions or feedback.
  • Visit our documentation for detailed guides and information.
  • Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai[bot] avatar Aug 18 '25 03:08 sourcery-ai[bot]

@AronPerez are you wanting to finish this feature? There is of course no pressure to do so other than if you are it still needs to get done within the timeline. Otherwise, I will be reassigning these items to myself.

JoshuaSBrown avatar Sep 24 '25 12:09 JoshuaSBrown

Sure. There's currently a bug with the core service not receiving the request for allocation creation with the added type to give you an idea of where I'm at with that

AronPerez avatar Sep 24 '25 23:09 AronPerez

Sure. There's currently a bug with the core service not receiving the request for allocation creation with the added type to give you an idea of where I'm at with that

Just to be clear that's a yes, you do want to wrap it up.

JoshuaSBrown avatar Sep 25 '25 12:09 JoshuaSBrown

I am assigning all of these items to myself.

JoshuaSBrown avatar Sep 25 '25 15:09 JoshuaSBrown