dotnet-sdk icon indicating copy to clipboard operation
dotnet-sdk copied to clipboard
verified publisher icon openfga

feat: add StreamedListObjects support

Open daniel-jonathan opened this issue 3 months ago • 1 comments

StreamedListObjects Implementation Summary

Issue: openfga/dotnet-sdk#110

Overview

Implemented StreamedListObjects functionality in the .NET SDK, providing feature parity with the JavaScript and Python SDKs.

What Was Implemented

Core Functionality

  • StreamedListObjects method returning IAsyncEnumerable<StreamedListObjectsResponse>
  • NDJSON stream parser in BaseClient
  • Support for all ListObjects parameters (authorization model ID, consistency, contextual tuples, context)
  • Proper resource cleanup on early termination and cancellation
  • Full CancellationToken support

Cross-Framework Support

  • netstandard2.0
  • net48
  • net8.0
  • net9.0

Files Changed

New Files

  • src/OpenFga.Sdk/Model/StreamedListObjectsResponse.cs
  • src/OpenFga.Sdk.Test/ApiClient/StreamingTests.cs (11 tests)
  • src/OpenFga.Sdk.Test/Client/StreamedListObjectsTests.cs (11 tests)
  • example/StreamedListObjectsExample/ (complete example application)

Modified Files

  • src/OpenFga.Sdk/ApiClient/BaseClient.cs - NDJSON streaming support
  • src/OpenFga.Sdk/ApiClient/ApiClient.cs - Streaming request handling
  • src/OpenFga.Sdk/Api/OpenFgaApi.cs - API endpoint
  • src/OpenFga.Sdk/Client/Client.cs - Client-level method
  • src/OpenFga.Sdk/Telemetry/Attributes.cs - Telemetry support
  • src/OpenFga.Sdk/OpenFga.Sdk.csproj - Added Microsoft.Bcl.AsyncInterfaces package
  • CHANGELOG.md - Added feature entry

Test Results

All 22 tests passing (10 streaming + 11 client integration):

  • NDJSON parsing (single/multi-line, chunked, invalid JSON handling)
  • Empty lines and whitespace handling
  • Resource cleanup and early termination
  • Cancellation token support
  • Error handling (HTTP errors, validation errors)
  • Large dataset efficiency (100+ objects)
  • All ListObjects options (consistency, contextual tuples, etc.)

Usage Example

await foreach (var response in fgaClient.StreamedListObjects(
    new ClientListObjectsRequest {
        User = "user:anne",
        Relation = "can_read",
        Type = "document"
    },
    new ClientListObjectsOptions {
        Consistency = ConsistencyPreference.HIGHERCONSISTENCY
    },
    cancellationToken)) {
    
    Console.WriteLine($"Received: {response.Object}");
}

Key Benefits

  • No pagination limits - only constrained by server timeout
  • Memory efficient - objects processed as they arrive
  • Early termination - can break early with automatic cleanup
  • Idiomatic .NET - uses IAsyncEnumerable<T> pattern

References

  • API Documentation
  • Example Application
  • JavaScript SDK implementation review
  • Python SDK implementation review

Summary by CodeRabbit

  • New Features

    • Introduced StreamedListObjects API method enabling incremental object retrieval as items arrive
    • Supports all ListObjects parameters: authorization model ID, consistency preferences, contextual tuples, and context data
    • Includes resource cleanup on early termination and cancellation token support

  • Documentation

    • Added comprehensive StreamedListObjects usage example with streaming patterns and best practices

daniel-jonathan avatar Nov 05 '25 20:11 daniel-jonathan

[!IMPORTANT]

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Introduces streaming support to the OpenFGA .NET SDK via a new StreamedListObjects method returning IAsyncEnumerable<StreamedListObjectsResponse>. Adds NDJSON parsing infrastructure, cancellation support, resource cleanup, and comprehensive test coverage across API, client, and infrastructure layers.

Changes

Cohort / File(s) Summary
Streaming Infrastructure
src/OpenFga.Sdk/ApiClient/BaseClient.cs, src/OpenFga.Sdk/ApiClient/ApiClient.cs		 Adds streaming API methods SendStreamingRequestAsync<T> to parse NDJSON responses line-by-line, handle cancellation, propagate errors, and yield parsed objects incrementally; supports optional header injection and error handling consistent with existing patterns.
API Layer
src/OpenFga.Sdk/Api/OpenFgaApi.cs		 Introduces StreamedListObjects method mirroring ListObjects but streaming results via /stores/{store_id}/streamed-list-objects endpoint; validates store ID, constructs query parameters, and yields items via SendStreamingRequestAsync.
Client Facade
src/OpenFga.Sdk/Client/Client.cs		 Adds StreamedListObjects method to OpenFgaClient that delegates to api.StreamedListObjects with request/options conversion and EnumeratorCancellation support.
Model
src/OpenFga.Sdk/Model/StreamedListObjectsResponse.cs		 Introduces StreamedListObjectsResponse class with Object property (serialized as "object"), JSON (de)serialization, equality/hashing, and validation support for streamed items.
Streaming Infrastructure Tests
src/OpenFga.Sdk.Test/ApiClient/StreamingTests.cs		 Comprehensive unit tests for BaseClient.SendStreamingRequestAsync covering NDJSON parsing, single/multiple lines, empty/whitespace handling, cancellation, HTTP error propagation, early resource disposal, and invalid JSON edge cases.
Integration Tests
src/OpenFga.Sdk.Test/Client/StreamedListObjectsTests.cs		 Adds 12 test cases for OpenFgaClient.StreamedListObjects validating incremental streaming, parameter inclusion (authorization model ID, consistency, contextual tuples), error handling, resource cleanup, cancellation, empty responses, validation errors, and efficient large-scale streaming.
Example & Documentation
example/StreamedListObjectsExample/StreamedListObjectsExample.cs, example/StreamedListObjectsExample/README.md, example/StreamedListObjectsExample/StreamedListObjectsExample.csproj		 Adds runnable example program demonstrating StreamedListObjects usage patterns including normal streaming, early termination, cancellation via CancellationTokenSource, and cleanup; includes project file with .NET 9.0 target and dependency configuration.
Configuration & Telemetry
src/OpenFga.Sdk/OpenFga.Sdk.csproj, src/OpenFga.Sdk/Telemetry/Attributes.cs		 Adds Microsoft.Bcl.AsyncInterfaces 8.0.0 package dependency; expands telemetry attribute processing to include StreamedListObjects for request body and user extraction.
Changelog
CHANGELOG.md		 Documents new StreamedListObjects feature in Unreleased section with description of streaming behavior, parameter support, and resource cleanup guarantees.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant OpenFgaClient
    participant OpenFgaApi
    participant ApiClient
    participant BaseClient
    participant HttpClient
    participant NDJSON Stream

    Caller->>OpenFgaClient: StreamedListObjects(request, options)
    OpenFgaClient->>OpenFgaApi: StreamedListObjects(storeId, body, options, cancellationToken)
    OpenFgaApi->>ApiClient: SendStreamingRequestAsync<ListObjectsRequest, StreamedListObjectsResponse>
    ApiClient->>ApiClient: GetAuthenticationToken()
    ApiClient->>BaseClient: SendStreamingRequestAsync<ListObjectsRequest, StreamedListObjectsResponse>
    BaseClient->>HttpClient: SendAsync(request, HttpCompletionOption.ResponseHeadersRead)
    HttpClient->>NDJSON Stream: Stream response body (NDJSON)
    
    loop For each line in stream
        NDJSON Stream->>BaseClient: Read line
        alt Valid JSON with "result"
            BaseClient->>BaseClient: Parse JSON, extract result
            BaseClient->>ApiClient: Yield<StreamedListObjectsResponse>
            ApiClient->>OpenFgaApi: Yield<StreamedListObjectsResponse>
            OpenFgaApi->>OpenFgaClient: Yield<StreamedListObjectsResponse>
            OpenFgaClient->>Caller: Yield<StreamedListObjectsResponse>
        else Empty/Whitespace line
            BaseClient->>BaseClient: Skip line
        else Invalid JSON
            BaseClient->>BaseClient: Skip line (log if applicable)
        else Cancellation requested
            BaseClient->>BaseClient: Break loop, dispose stream
        else HTTP error status
            BaseClient->>BaseClient: Throw ApiException via CreateSpecificExceptionAsync
        end
    end

    Caller->>Caller: Complete iteration or break early

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45–75 minutes

  • BaseClient.cs: Dense NDJSON streaming logic with line-by-line parsing, JSON deserialization, error recovery (skipping invalid lines), cancellation handling, and response status validation requires careful verification of edge cases.
  • StreamingTests.cs: Comprehensive test suite covering 10 scenarios (single/multiple lines, empty lines, cancellation, HTTP errors, early disposal, whitespace, invalid JSON) with mocked HTTP responses; verify test coverage completeness and mocking strategy.
  • StreamedListObjectsTests.cs: 12 integration tests with mock HTTP handler and NDJSON response creation; validate request construction (path, method, headers), parameter encoding, and error/edge-case assertions.
  • ApiClient.cs and OpenFgaApi.cs: Verify token acquisition, header construction, and delegation patterns align with existing non-streaming request paths; confirm no regression in authentication flow.
  • Telemetry attributes expansion: Confirm StreamedListObjects is added to all relevant conditions consistently.

Possibly related PRs

  • openfga/dotnet-sdk#133: Modifies per-request header construction and propagation in ApiClient, OpenFgaApi, and Client classes, which overlaps with the streaming header-building logic in this PR.

Suggested labels

enhancement

Suggested reviewers

  • evansims
  • rhamzeh
  • ewanharris

Pre-merge checks and finishing touches

✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: add StreamedListObjects support' directly and clearly describes the main feature addition in this PR: implementing StreamedListObjects streaming API support.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Nov 05 '25 20:11 coderabbitai[bot]