go-json icon indicating copy to clipboard operation
go-json copied to clipboard

does not build with Go HEAD

Open ianlancetaylor opened this issue 1 year ago • 8 comments

This package does some extremely unsafe and unsupported operations. One of them has broken with Go HEAD, the future 1.23 release.

# github.com/goccy/go-json/internal/encoder.test
github.com/goccy/go-json/internal/encoder.(*Compiler).structCode: relocation target reflect.ifaceIndir not defined
github.com/goccy/go-json/internal/encoder.(*Compiler).isNilableType: relocation target reflect.ifaceIndir not defined
FAIL	github.com/goccy/go-json/internal/encoder [build failed]

The internal function reflect.ifaceIndir no longer exists. This package should not be referencing it.

If there are performance issues with the reflect package, please file them upstream. Please don't try to work around them using unsupported operations. Thanks.

ianlancetaylor avatar May 08 '24 13:05 ianlancetaylor

This library has 47 go:linkname lines, particularly in https://github.com/goccy/go-json/blob/5e2ae3f23c1db71990844a230a4d11559efe128e/internal/runtime/rtype.go, which I find very surprising for a JSON library. Please transition away from them like Ian says - if the standard library is lacking features or isn't as performant as you would like it to be, file bugs or patches upstream.

Alternatively, I'm not sure that much can be done in terms of new Go versions not breaking you. It is bound to happen when you depend on dozens of internal reflect and runtime implementation details. @ianlancetaylor personally, I would not restore functions like ifaceIndir, not even for one release cycle - we might otherwise give the impression that go:linkname is somehow covered by the Go1 compatibility guarantee.

mvdan avatar May 10 '24 07:05 mvdan

@ianlancetaylor Thank you for notification to me. Since the current code related to the reflect package can be replaced with the use of only public APIs, I plan to address this soon.

goccy avatar May 10 '24 08:05 goccy

Previously, there was a trick implemented to avoid the forced escaping of argument values with reflect.ValueOf. However, since that code is no longer present, we can now use the reflect package normally.

goccy avatar May 10 '24 08:05 goccy

Also, as a general recommendation. Have every linkname & poking into Go runtime protected by a specific Go version tag, because there's no guarantee on how map, reflect etc. internals are implemented in the future. (e.g. see upcoming swissmap in https://go-review.googlesource.com/c/go/+/580778).

egonelbre avatar May 10 '24 08:05 egonelbre

Thank you. I'm already aware of that, and I implemented it with that understanding.

goccy avatar May 10 '24 08:05 goccy

Is there a way to iter a map with zero-alloc? reflect.Value.MapRange or reflect.Value.MapKeys both alloc O(len(m))

trim21 avatar Jul 16 '24 05:07 trim21

@trim21 Yes: use the MapIter.SetIterKey or MapIter.SetIterValue method.

ianlancetaylor avatar Jul 16 '24 21:07 ianlancetaylor

Thanks. for future readers, it would look like this:

it will have 3 allocs/op for map with any size

func(ctx *Ctx, b []byte, rv reflect.Value) ([]byte, error) {
		if rv.IsNil() {
			return appendNull(b), nil
		}

		size := rv.Len()
		b = appendArrayBegin(b, int64(size))

		iter := rv.MapRange()

		kv := reflect.New(keyType).Elem()
		vv := reflect.New(valueType).Elem()

		for iter.Next() {
			kv.SetIterKey(iter)
			b, err = keyEncoder(ctx, b, kv)
			if err != nil {
				return b, err
			}

			vv.SetIterValue(iter)
			b, err = valueEncoder(ctx, b, vv)
			if err != nil {
				return b, err
			}
		}

		return append(b, '}'), nil
	}

trim21 avatar Jul 16 '24 23:07 trim21