OrderedCollections.jl icon indicating copy to clipboard operation
OrderedCollections.jl copied to clipboard

use of rehash! in constructor

Open StephenVavasis opened this issue 7 years ago • 3 comments

I noticed the following snippet in ordered_dict.jl:

    function OrderedDict{K,V}(d::OrderedDict{K,V}) where {K,V}
        if d.ndel > 0
            rehash!(d)
        end
        @assert d.ndel == 0
        new{K,V}(copy(d.slots), copy(d.keys), copy(d.vals), 0)
    end

Isn't this a known source of bugs? See https://github.com/JuliaCollections/DataStructures.jl/issues/335

StephenVavasis avatar Sep 20 '18 19:09 StephenVavasis

Something weird related to rehash! happens when OrderedDicts are returned from a remote process. They need to be rehash!ed in order to work correctly.

@everywhere begin
    using OrderedCollections
    struct Foo
        x::Vector{Int}
    end
    function foo(i)    
        x = Foo(Int[])
        return x, OrderedDict(x => 2)
    end
end

function bar()
    x, y = pmap(foo, 1:1)[1]
    z = copy(y)
    OrderedCollections.rehash!(z)
    return haskey(y, x), haskey(z, x)
end
a, b = bar()  # false, true ?!?

odow avatar Sep 13 '20 21:09 odow

@odow In general, it would be good to open another issue for things like this (and bump it if it's important and not being responded to), or ask a question on Discourse.

In this case, the issue is that you haven't defined a hash function for Foo, so it's defaulting to using the object ID as the hash key, and that isn't the same between different processes (it's basically equivalent to the memory address of the item). Rehashing "fixes" this.

It also won't generally work (locally or remotely) if you define a hash function that depends on Foo.x, and then modify Foo.x (you'll get a different hash value, which would put this value at a different location in the hash table.).

kmsquire avatar Feb 01 '21 14:02 kmsquire

@kmsquire I don't remember opening this, or what it was related to, so I guess I figured out the reason. Thanks for the explanation.

odow avatar Feb 01 '21 18:02 odow