Surface the distinction between an explicit "null" value and an absent value

[madsodgaard]

When using a schema that has a field from a reference that is both non-required and "nullable" this is a common way of defining it in OpenAPI: 3.1.0

UpdateTaskOccurrenceCompletionResponse:
      properties:
        old_task:
          $ref: '#/components/schemas/TaskOccurenceDTO'
        next_task:
          oneOf:
            - $ref: '#/components/schemas/TaskOccurenceDTO'
            - type: 'null'
      type: object
      required:
        - old_task

but this currently generates the following API:

/// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse`.
        public struct UpdateTaskOccurrenceCompletionResponse: Codable, Hashable, Sendable {
            /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/old_task`.
            public var old_task: Components.Schemas.TaskOccurenceDTO
            /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/next_task`.
            @frozen public enum next_taskPayload: Codable, Hashable, Sendable {
                /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/next_task/case1`.
                case TaskOccurenceDTO(Components.Schemas.TaskOccurenceDTO)
                /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/next_task/case2`.
                case case2(OpenAPIRuntime.OpenAPIValueContainer)
            }
            /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/next_task`.
            public var next_task: Components.Schemas.UpdateTaskOccurrenceCompletionResponse.next_taskPayload

This is quite confusing syntax at the callsite, since we have this weird case2, instead of just having only an optional TaskOccurenceDTO. Is this expected behaviour or could we do something to improve the ergonomics before the 1.0.0 release? I'll happily work on a PR, if you could point me in the right direction to where in the codebase the relevant parts are!

[czechboy0]

Hi @madsodgaard,

what's the meaning of marking the value as nullable beyond just defining as

old_task:
  $ref: '#/components/schemas/TaskOccurenceDTO'

and not putting it in the required list?

How would you want to see it represented in Swift, if not as a simple optional property of TaskOccurenceDTO?

[madsodgaard]

@czechboy0 One place where I use this pattern a lot is when designing PATCH endpoints that allows partial updates of models. If I have field name that can actually be NULL in the database, I would give that the type:

type:
  - string
  - "null"

since this is a partial update endpoint, I want to differentiate between the user providing a "null" value (thus updating the value of name to NULL) and not providing a value at all (not updating name). That's why I don't include it in the required. From what I understand required means that the property must be present, regardless of the value. As you said I could just not include it in the required and remove the null type. But then I would only be doing that because I know I am going to use this library to generate a client, and I'll get a nicer API.

The ideal situtation, in my mind, is that the library could somehow recognize this pattern and just provide an optional property of TaskOccurenceDTO, as it does currently when not specifying the property in the required list and with only one type. Hope that makes some sense:)

[czechboy0]

Swift OpenAPI Generator delegates JSON parsing to automatic Codable implementations (wherever possible) and to Foundation's JSONDecoder, which doesn't make the distinction between a null value and an absent value, so unfortunately what you're trying to do won't work. Several pieces of the stack, including those not owned by this tool (such as Foundation itself), would have to change.

You'll need to make resetting values somehow an explicit part of the API, not relying on the distinction between null and an absent value. And for values you're not trying to overwrite, just define them as regular properties and omit them from the required array.

[madsodgaard]

@czechboy0 Thanks! Of course the fact that Codable does not differentiate between absent values and null is quite challenge to this issue. But, this pattern is so common in APIs today, that perhaps we should find a way to support? By "this pattern", I am referring to PATCH endpoints for partial updates instead of PUT endpoints to replace the entire resource. I have used this property wrapper previously in server-side applications. Do you think such a type or something similar would have a place in the library? (or is it too niché?) It would be used in places where you have a non-required nullable field.

@propertyWrapper
struct PotentiallyAbsent<T: Codable>: Codable {
    var wrappedValue: T? {
        switch state {
        case .present(let value):
            return value
        default:
            return nil
        }
    }
    
    enum State {
        case absent
        case null
        case present(T)
    }
    
    var projectedValue: PotentiallyAbsent<T> {
        return self
    }
    
    var state: State
    
    init(_ state: State) {
        self.state = state
    }
    
    static var null: PotentiallyAbsent<T> {
        .init(.null)
    }
    
    static var absent: PotentiallyAbsent<T> {
        .init(.absent)
    }
    
    static func present(_ value: T) -> PotentiallyAbsent<T> {
        .init(.present(value))
    }
    
    init(from decoder: Decoder) throws {
        fatalError("Attempted to decode `PotentiallyAbsent` as a standalone value")
    }
    
    func encode(to encoder: Encoder) throws {
        fatalError("Attempted to encode `PotentiallyAbsent` as a standalone value")
    }
}

extension KeyedDecodingContainer {
    func decode<T>(_ type: PotentiallyAbsent<T>.Type, forKey key: K) throws -> PotentiallyAbsent<T> {
        do {
            if try self.decodeNil(forKey: key) {
                return .init(.null)
            } else {
                return .init(.present(try decode(T.self, forKey: key)))
            }
        } catch DecodingError.keyNotFound {
            return .init(.absent)
        }
    }
}

extension KeyedEncodingContainer {
    mutating func encode<T>(_ value: PotentiallyAbsent<T>, forKey key: KeyedEncodingContainer<K>.Key) throws {
        switch value.state {
        case .absent:
            break
        case .null:
            try encodeNil(forKey: key)
        case let .present(value):
            try encode(value, forKey: key)
        }
    }
}

[czechboy0]

I don't think it's likely for us to try to solve this in Swift OpenAPI Generator. The fix should be in Codable and JSONDecoder. Once it's there, we can see what'd remain for Swift OpenAPI Generator to have to do to support this case.

But only doing it here would be a significant amount of work that introduces a lot of complexity.

One way this could be better represented that'd work today would be a PATCH call that takes two values, a "toSet" dictionary, which has keys matching the property to set, and the value to set. And if a value is nil for a property, the property isn't changed.

And the second value would be called "toReset", which would just be a list of property names whose values to reset to their default.

[czechboy0]

Renamed the issue to reflect that this is a missing feature. See my comment above explaining that this goes deeper in the stack we're based on, so would likely need some improvements from JSONDecoder first, but keeping open to allow tracking progress.

[brandonbloom]

I ran into this (or a similar) issue. The encoded output of my server omits a field with a nil value, but some code consuming this service expects an explicit null value.

One idea is this property wrapper: https://stackoverflow.com/a/53142581/67280

[czechboy0]

I think it'd be best to first help Foundation get the desired behavior, and then Swift OpenAPI Generator and all other projects relying on JSONDecoder get it for free.

https://github.com/apple/swift-foundation