hashstructure icon indicating copy to clipboard operation
hashstructure copied to clipboard

Different slice in struct result in the same hash

Open hunghuy201280 opened this issue 1 year ago • 6 comments

Here's how to reproduce it:

package main

import (
	"fmt"
	"github.com/mitchellh/hashstructure/v2"
)

type TempStruct struct {
	Strings []string `hash:"set"`
}

func main() {
	channelIds := &[]string{
		"66693f7ecdd2e6e2b6f30c18", "66693f7ecdd2e6e2b6f30c19",
	}

	structt := TempStruct{Strings: *channelIds}

	hashInt, _ := hashstructure.Hash(structt, hashstructure.FormatV2,
		&hashstructure.HashOptions{SlicesAsSets: true})
	channelIds1 := &[]string{
		"66759857b42b04c45ed0c6e6", "66759857b42b04c45ed0c6e7",
	}
	struct1 := TempStruct{Strings: *channelIds1}

	hashInt1, _ := hashstructure.Hash(struct1, hashstructure.FormatV2, nil)
	fmt.Println(hashInt)
	fmt.Print(hashInt1)
}

which will print:

4385387346745637338
4385387346745637338

hunghuy201280 avatar Jun 22 '24 04:06 hunghuy201280

This fork seems to notice the difference: https://github.com/creker/hashstructure

see: https://go.dev/play/p/7NGEOYu9AZX

MJacred avatar Jul 09 '24 11:07 MJacred

This issue seems like a duplicate of or related to https://github.com/mitchellh/hashstructure/issues/36

and was apparently fixed in this fork: https://github.com/michaelbeaumont/hashstructure/commit/d0db0908675fc419b4cdef999bac1833147de87d

EDIT: tested the commit and it does NOT fix this issue

EDIT2: I'm afraid this is simply the limit of the used FNV-1 algorithm. See also here.
If you change fnv.New64() to fnv.New64a() in hashstructure.go, then the collision does not occur, simply because it's has slightly better avalanche characteristics (according to wikipedia).
The reason why creker's approach should always work is because they use sha256, which is the slowest, but also collision-free way

MJacred avatar Jul 09 '24 11:07 MJacred

Currently I need to iterate through the input slice and use sha256 to hash every element before using hashstructure.Hash, this way will prevent collision temporarily

hunghuy201280 avatar Jul 10 '24 04:07 hunghuy201280

That's certainly one way.
It really depends on your needs. Maybe creker's way would also work for you and faster? Faster than sha256 would be shake128: https://pkg.go.dev/golang.org/x/crypto/sha3#ShakeSum128
But in those cases, you get an []byte with size 32 or 64, no uint64...

MJacred avatar Jul 10 '24 15:07 MJacred

Has this project been abandoned? For what I gather here the owner does not care about fixing the issues :(

lordofscripts avatar Jul 22 '24 20:07 lordofscripts

Has this project been abandoned? For what I gather here the owner does not care about fixing the issues :(

I don’t. I plan on archiving this repo soon, feel free to fork! 👍 https://gist.github.com/mitchellh/90029601268e59a29e64e55bab1c5bdc

mitchellh avatar Jul 22 '24 20:07 mitchellh