go-sdk icon indicating copy to clipboard operation
go-sdk copied to clipboard

feat: streamed list objects endpoint

Open SoulPancake opened this issue 1 month ago • 3 comments

Description

Generated from : https://github.com/openfga/sdk-generator/pull/651

This PR implements support for the StreamedListObjects endpoint in the Go SDK, providing a true streaming interface using Go channels. The API layer is generated from OpenAPI, but this PR adds the streaming functionality to both the API and client layers. The API surface is extended to expose a channel-based, non-blocking streaming interface that yields objects as they arrive from the server (NDJSON). The generated OpenAPI client already provided the endpoint; this change wires in the streaming parsing, context-aware cancellation, and an error channel to surface asynchronous errors.

How is it being solved?

Streaming parser: the HTTP response body is scanned line-by-line and each parsed JSON line is pushed to a typed Objects channel immediately. Channel API: responses include two channels — Objects (stream of parsed items) and Errors (async error reporting). The channels are closed when the stream ends or on cancellation. Context support: request execution respects the provided context; cancelling the context closes the response channels and cancels the HTTP request. Error handling: parsing and I/O errors are sent to the Errors channel; callers can read Errors after the Objects channel closes to detect failures.

Streaming Architecture The implementation uses Go channels to provide a non-blocking streaming interface:


response, err := client.StreamedListObjects(ctx).
    Body(ClientStreamedListObjectsRequest{
        Type:     "document",
        Relation: "viewer",
        User:     "user:anne",
    }).
    Execute()

// Stream objects as they arrive
for obj := range response.Objects {
    fmt.Println(obj.Object)
}

// Check for errors
if err := <-response.Errors; err != nil {
    log.Fatal(err)
}

What changes are made to solve it?

Key Features True Streaming: Processes NDJSON responses line-by-line, yielding results as they're received Channel-Based: Uses Go channels for async, non-blocking consumption Context Support: Full support for context cancellation and timeouts Error Handling: Dedicated error channel for async error reporting Clean API: Follows existing SDK patterns and conventions API Layer (streaming.go) Added StreamedListObjectsWithChannel method that:

Opens HTTP connection to streaming endpoint Parses NDJSON response line-by-line using bufio.Scanner Sends parsed objects through Objects channel Reports errors through Errors channel Supports graceful cancellation via context

Additional notes :

  1. No limit on result set size (only limited by server timeout)
  2. Idiomatic Go: Uses channels and context as Go developers expect

References

Review Checklist

  • [x] I have clicked on "allow edits by maintainers".
  • [ ] I have added documentation for new/changed functionality in this PR or in a PR to openfga.dev [Provide a link to any relevant PRs in the references section above]
  • [x] The correct base branch is being used, if not main
  • [x] I have added tests to validate that the change in functionality is working as expected

Summary by CodeRabbit

  • New Features

    • Added streaming support for listing objects, enabling efficient retrieval of large result sets with real-time response streaming and built-in error handling.
  • Documentation

    • Added comprehensive documentation and usage examples for the new streaming object listing functionality.
  • Other

    • Updated project contribution guidelines.

SoulPancake avatar Oct 25 '25 04:10 SoulPancake

[!IMPORTANT]

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

[!NOTE]

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

This PR introduces streaming support for listing objects in the OpenFGA Go SDK. It adds new data models, extends the API client with StreamedListObjects methods, implements streaming response handling with NDJSON parsing and error propagation, provides client-level abstractions with proper resource cleanup, and includes comprehensive documentation and usage examples.

Changes

Cohort / File(s) Summary
New Models
model_stream_result_of_streamed_list_objects_response.go, model_streamed_list_objects_response.go
Adds two new model types for streaming responses: StreamResultOfStreamedListObjectsResponse (wraps result and error) and StreamedListObjectsResponse (wraps object string). Includes constructors, accessors, JSON marshaling, and nullable wrappers.
Core API Layer
api_open_fga.go
Extends OpenFgaApi interface with StreamedListObjects and StreamedListObjectsExecute methods. Implements ApiStreamedListObjectsRequest builder pattern with Body, Options, and Execute methods. Handles NDJSON endpoint, validates inputs, applies headers and retry logic.
Streaming Infrastructure
streaming.go, streaming_test.go
Introduces StreamedListObjectsChannel type with Object/Error channels and Close method. Provides ProcessStreamedListObjectsResponse to parse NDJSON lines and ExecuteStreamedListObjects to handle HTTP streaming. Tests cover lifecycle, success, errors, empty lines, invalid JSON, and context cancellation scenarios.
SDK Client Layer
client/client.go, client/streaming_test.go
Adds SdkClient interface methods and OpenFgaClient implementation for streaming. Introduces ClientStreamedListObjectsRequest/Response/Options types and SdkClientStreamedListObjectsRequest builder. Implements Execute to translate contextual tuples and options. Tests verify successful streaming, option handling, error responses, and missing store ID validation.
Documentation
docs/OpenFgaApi.md, docs/StreamResultOfStreamedListObjectsResponse.md, docs/StreamedListObjectsResponse.md, README.md
Documents new StreamedListObjects endpoint with usage examples and parameters. Adds model documentation for StreamResultOfStreamedListObjectsResponse and StreamedListObjectsResponse. Updates README with new API method entry.
Project Files
CHANGELOG.md, CONTRIBUTING.md, .openapi-generator/FILES
Updates changelog with v0.7.3 date and streamed-list-objects details. Removes PR submission restriction note from contributing guidelines. Registers new model files in generator manifest.
Example & Utilities
example/streamed_list_objects/main.go, example/streamed_list_objects/README.md, example/streamed_list_objects/go.mod
Adds complete example demonstrating streaming in sync and async modes, with store/model setup, tuple creation, and streaming consumption patterns. Includes configuration documentation and module dependencies.
Test Formatting
utils_test.go
Minor formatting and indentation adjustments with no functional changes to test logic or assertions.

Sequence Diagram(s)

sequenceDiagram
    participant Client as OpenFGA Client
    participant API as HTTP API (Streaming)
    participant Parser as NDJSON Parser
    participant Channel as StreamedListObjectsChannel

    Client->>API: POST /stores/{store_id}/streamed-list-objects
    API-->>Client: HTTP 200 (NDJSON stream)
    
    rect rgb(200, 220, 255)
    Note over Parser,Channel: Streaming Loop (in goroutine)
    
    loop For each NDJSON line
        API-->>Parser: JSON line
        activate Parser
        Parser->>Parser: Unmarshal to StreamResult
        alt Success
            Parser->>Channel: Send StreamedListObjectsResponse to Objects channel
        else Error in line
            Parser->>Channel: Send error to Errors channel
            Parser->>Channel: Continue processing
        else Invalid JSON
            Parser->>Channel: Send parsing error to Errors channel
        end
        deactivate Parser
    end
    end
    
    Note over Channel: Context cancellation or EOF
    Parser->>Channel: Close channels
    
    Client->>Channel: Consume Objects and Errors channels
    Client->>Channel: Call Close() when done
    activate Channel
    Channel->>Channel: Cancel streaming context
    deactivate Channel

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

  • Areas requiring extra attention:
    • streaming.go and streaming_test.go: Concurrency patterns, goroutine lifecycle, channel closure semantics, and NDJSON parsing robustness with error handling mid-stream
    • client/client.go: Integration of streaming abstractions with existing client patterns, contextual tuple propagation, and response mapping logic
    • api_open_fga.go: Request validation, retry logic application to streaming endpoints, and consistency with existing ListObjects patterns
    • Interaction between StreamedListObjectsChannel lifecycle and context cancellation across multiple call sites

Possibly related PRs

  • openfga/go-sdk#233: Streaming additions directly extend the per-request RequestOptions/headers mechanism used for request customization in api_open_fga.go and client paths.

Suggested reviewers

  • tylernix
  • jimmyjames

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 19.23% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'feat: streamed list objects endpoint' clearly and concisely describes the main change: adding support for a new StreamedListObjects endpoint to the Go SDK.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Oct 25 '25 04:10 coderabbitai[bot]

Codecov Report

:x: Patch coverage is 35.94249% with 401 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 33.82%. Comparing base (3759712) to head (3112003).

Files with missing lines Patch % Lines
api_client.go 42.29% 121 Missing and 25 partials :warning:
api_open_fga.go 0.00% 110 Missing :warning:
...stream_result_of_streamed_list_objects_response.go 0.00% 66 Missing :warning:
model_streamed_list_objects_response.go 0.00% 45 Missing :warning:
streaming.go 69.13% 19 Missing and 6 partials :warning:
client/client.go 87.32% 5 Missing and 4 partials :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #252      +/-   ##
==========================================
- Coverage   33.90%   33.82%   -0.09%     
==========================================
  Files         111      114       +3     
  Lines       10614    10990     +376     
==========================================
+ Hits         3599     3717     +118     
- Misses       6663     6911     +248     
- Partials      352      362      +10     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar Oct 25 '25 04:10 codecov-commenter

Can you also review @daniel-jonathan 's PR here: https://github.com/openfga/js-sdk/pull/280

Ideally both would have the same semantics, example, tests, config options, README, etc..

rhamzeh avatar Oct 29 '25 16:10 rhamzeh

[!WARNING] Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn Critical
Critical CVE: Misuse of ServerConfig.PublicKeyCallback may cause authorization bypass in golang golang.org/x/crypto

CVE: GHSA-v778-237x-gjrc Misuse of ServerConfig.PublicKeyCallback may cause authorization bypass in golang.org/x/crypto (CRITICAL)

Affected versions: < 0.31.0

Patched version: 0.31.0

From: ?golang/golang.org/x/[email protected]

ℹ Read more on: This package | This alert | What is a critical CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at [email protected].

Suggestion: Remove or replace dependencies that include known critical CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore golang/golang.org/x/[email protected]. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
License policy violation: golang github.com/hashicorp/errwrap under MPL-2.0

License: MPL-2.0 (LICENSE)

From: ?golang/github.com/hashicorp/[email protected]

ℹ Read more on: This package | This alert | What is a license policy violation?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at [email protected].

Suggestion: Find a package that does not violate your license policy or adjust your policy to allow this package's license.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore golang/github.com/hashicorp/[email protected]. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
License policy violation: golang github.com/hashicorp/go-multierror under MPL-2.0

License: MPL-2.0 (LICENSE)

From: ?golang/github.com/hashicorp/[email protected]

ℹ Read more on: This package | This alert | What is a license policy violation?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at [email protected].

Suggestion: Find a package that does not violate your license policy or adjust your policy to allow this package's license.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore golang/github.com/hashicorp/[email protected]. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
License policy violation: golang gonum.org/v1/gonum under SunPro

License: SunPro (THIRD_PARTY_LICENSES/Sun-LICENSE)

From: ?golang/gonum.org/v1/[email protected]

ℹ Read more on: This package | This alert | What is a license policy violation?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at [email protected].

Suggestion: Find a package that does not violate your license policy or adjust your policy to allow this package's license.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore golang/gonum.org/v1/[email protected]. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

socket-security[bot] avatar Nov 17 '25 12:11 socket-security[bot]