[http] Strange behaviour around sending reference types

[dignifiedquire]

I am trying to implement a command which has Type either *big.Int or big.Cid.

1. When using no Type and only MakeEncoder and send big.NewInt(0) I end up with the error unexpected type float64 (expected interface {})
2. When I try to wrap the result in something like this:

type result struct {
  myint *big.Int
  cid *cid.Cid
}

and use Type: &result{}, I end up with result being nil when sending result{myint: big.NewInt(0)}

[whyrusleeping]

unexported fields can't be json marshaled. Other packages can't read fields of your struct if you don't give them that ability.

[dignifiedquire]

That explains case 2, but what about case 1, float vs big.Int?

[whyrusleeping]

I don't know what youre expecting to happen. What types does json have? If you don't tell the commands lib what type to use, youre going to get json types. json has floats, json doesnt have 'big integers'

[Kubuxu]

@dignifiedquire your struct result has two unexported fields: myint and cid. You either have to export them or implement manual marshalling. See go-cid as an example: https://github.com/ipfs/go-cid/blob/master/cid.go#L379-L415

[dignifiedquire]

Yeah, I got that, but it is super surprising and not at all clear that a json encoder is used from the current docs :/

[dignifiedquire]

also the first issue still is there, cmds lib shouldn't freak out on a bigInt when getting a float just because json is unable to represent that properly

[whyrusleeping]

@dignifiedquire use your brain. a big int encodes to json as a float. Then, without telling the commands lib how to intepret whatever response it gets, you just generically try to decode the thing. So the json unmarshaler, without any type hints, says "this is a float" and decodes it as such.

This is the whole reason we have the type field on the commands.

[dignifiedquire]

I understand the reason but I think the library should do better and allow the conversion from float to bigint automatically or use a better encoder that encodes bigints as string with enough information to get them back exactly.

On 17. Mar 2018, 19:51 +0100, Whyrusleeping [email protected], wrote:

@dignifiedquire use your brain. a big int encodes to json as a float. Then, without telling the commands lib how to intepret whatever response it gets, you just generically try to decode the thing. So the json unmarshaler, without any type hints, says "this is a float" and decodes it as such. This is the whole reason we have the type field on the commands. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[Stebalien]

Ideally, we'd be using CBOR or something but we'd need to add support for that.

[whyrusleeping]

@dignifiedquire I don't think youre understanding this here. The explicit contract of the library is "SET THE TYPE ON THE COMMAND". I guess we could check to see if its unset and do panic("youre being pretty dumb right now")

[keks]

@whyrusleeping @dignifiedquire Actually JSON doesn't use floats but Numbers, and big.Int implements json.Marshaler and json.Unmarshaler so you can actually do it that way if you expect big.Int. It's just that Go parses JSON Numbers into float64 if no type indication is given. So to accomodate this you'll have to write your own option type (like you did) that implements json.(Unm|M)arshaler:

type Result struct {
Int *big.Int
Cid *cid.Cid
isInt bool // to decide which one is used
}

func (r *Result) JSONMarshal() ([]byte, error) {
  if r.isInt {
    return json.Marshal(r.Int)
  } else {
    return json.Marshal(r.Cid)
  }
}

func (r *Result) JSONUnmarshal(data []byte) error {
  if data[0] >= '0' && data[0] <= '9' {
    r.isInt = true
    r.Int = big.NewInt(0)
    return json.Unmarshal(data, r.Int)
  } else {
    r.isInt = false
    r.Cid = cid.NewCid() // or sth like that
    return json.Unmarshal(data, r.Cid)
  }
}