cbor-gen icon indicating copy to clipboard operation
cbor-gen copied to clipboard

Correctly marshal/unmarshal struct fields of slice type with nil values to/from CborNull

Open NexZhu opened this issue 5 years ago • 4 comments

Currently struct fields of slice type with nil values will be marshalled into empty slice, not Null representation in CBOR.

For example:

type Example struct {
	slice []byte
}

e1 := Example{
	slice: nil,
}

e2 := Example{
	slice: []byte{},
}

Currently, e1 and e2 will be marshalled into the same byte sequence, which will all be unmarshalled into the same value as e2. So below will fail, which is undesirable IMO:

assert.Equal(e1, e1MarshalledThenUnmarshalled)

This PR fix it by correctly marshal/unmarshal struct fields of slice type with nil values to/from CborNull

NexZhu avatar Jul 10 '20 18:07 NexZhu

I believe this is intentional given that go treats nil and [] the same way. Otherwise, some empty arrays will encode to an empty cbor array, and others to Null. We also reject Null on decode.

Stebalien avatar Jul 24 '20 13:07 Stebalien

I believe this is intentional given that go treats nil and [] the same way. Otherwise, some empty arrays will encode to an empty cbor array, and others to Null. We also reject Null on decode.

fmt.Println([]int{} == nil) // prints false
fmt.Println(len([]int{})) // prints 0
fmt.Println(len(nil)) // error

I think these all show that Go doesn't treat them exactly the same way, the behavior of append is just a special case.

Also, it can be useful to represent them differently in CBOR (which is my case), for example:

b1 := Block{
	Timestamp: []byte,
        ... some other block header fields...
	Transactions: nil, // means b1 is used to transfer block header fields, transactions are omitted
}

b2 := Block{
	Timestamp: []byte,
        ... some other block header fields...
	Transactions: []Transaction{}, // means block b2 contains 0 transaction
}

NexZhu avatar Jul 24 '20 14:07 NexZhu

While it is possible to distinguish between a nil slice and a non-nil slice, doing so in this case would just lead to more confusion given that most go programs use them interchangeably. If you need to distinguish between "empty" and "nil", I recommend implementing support for *[]foo (pointer to slice).

Note: nil and a nil slice aren't quite the same thing. nil is 0. ([]T)(nil) (nil slice) is actually SliceHeader{Data: 0, Len: 0, Cap: 0}.

fmt.Println([]int{} == []int{}) // doesn't compile
fmt.Println(len([]int{})) // prints 0
fmt.Println(len(([]int)(nil))) // prints 0

Stebalien avatar Jul 24 '20 15:07 Stebalien

While it is possible to distinguish between a nil slice and a non-nil slice, doing so in this case would just lead to more confusion given that most go programs use them interchangeably. If you need to distinguish between "empty" and "nil", I recommend implementing support for *[]foo (pointer to slice).

SGTM, is there a plan to implement support for pointer to slice? @whyrusleeping

NexZhu avatar Jul 25 '20 09:07 NexZhu