Memory leak when declaring a higher order function

[lbrande]

Steps to reproduce:

1. Create a new binary create.
2. Paste the code below into main.rs.

fn f<R>(_: impl Fn(Box<dyn Fn() -> R>) -> R) {}

fn main() {
    f(|_| ());
}

This causes a memory leak that makes rust-analyzer use 100 % CPU (on one core) and an ever increasing amount of memory, being completely unreachable to the VSCode plugin. I believe the leak occurs because rust-analyzer infinitely tries to determine the type of R, though the code compiles and runs just fine.

Rust-analyzer version: 2021-02-08 (newest stable version) VSCode version: 1.53.2 OS: Ubuntu 20.04

[detrumi]

Looks like a chalk issue, since the solver gets stuck in a loop.

Relevant logs with RA_LOG=hir_ty=trace:

[INFO hir_ty::traits] trait_solve_query(Normalize(<|Box<dyn Fn() -> ?1.0, Global>| -> ?0.1 as FnOnce<(Box<dyn Fn() -> ?1.2, Global>,)>>::Output => ?0.2))
[DEBUG hir_ty::traits] solve goal: UCanonical { canonical: Canonical { value: InEnvironment { environment: Env([]), goal: AliasEq(AliasTy(?) = ^0.2) }, binders: [U0 with kind type, U0 with kind type, U0 with kind type] }, universes: 1 }

No further output with CHALK_DEBUG=2, but with higher logging levels it becomes apparent that it's trying to solve bigger and bigger goals:

solve_goal goal=UCanonical { canonical: Canonical { value: InEnvironment { environment: Env([]), goal: (FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^3.0)] + 'static, ?1 := Global<[]>]>, ?1 := ^1.1]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^2.2)] + 'static, ?1 := Global<[]>]>]>]> <: ^0.3) }, binders: [U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type] }, universes: 1 }
...
solve_goal goal=UCanonical { canonical: Canonical { value: InEnvironment { environment: Env([]), goal: (FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^3.0)] + 'static, ?1 := Global<[]>]>, ?1 := ^1.1]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^5.2)] + 'static, ?1 := Global<[]>]>, ?1 := ^3.3]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^7.4)] + 'static, ?1 := Global<[]>]>, ?1 := ^5.5]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^9.6)] + 'static, ?1 := Global<[]>]>, ?1 := ^7.7]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^11.8)] + 'static, ?1 := Global<[]>]>, ?1 := ^9.9]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^10.10)] + 'static, ?1 := Global<[]>]>]>]>)] + 'static, ?1 := Global<[]>]>]>]>)] + 'static, ?1 := Global<[]>]>]>]>)] + 'static, ?1 := Global<[]>]>]>]>)] + 'static, ?1 := Global<[]>]>]>]> <: FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^3.11)] + 'static, ?1 := Global<[]>]>, ?1 := ^1.12]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^5.13)] + 'static, ?1 := Global<[]>]>, ?1 := ^3.14]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^7.15)] + 'static, ?1 := Global<[]>]>, ?1 := ^5.16]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = FnOnce::Output<[?0 := {closure:ClosureId(0)}<[?0 := () for<0> [?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^9.17)] + 'static, ?1 := Global<[]>]>, ?1 := ^7.18]]>, ?1 := 1<[?0 := Box<[?0 := dyn for<type> [for<> Implemented(^1.0: Fn<0<[]>>), for<> AliasEq(<^1.0 as FnOnce<0<[]>>>::Output = ^8.19)] + 'static, ?1 := Global<[]>]>]>]>)] + 'static, ?1 := Global<[]>]>]>]>)] + 'static, ?1 := Global<[]>]>]>]>)] + 'static, ?1 := Global<[]>]>]>]>) }, binders: [U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type, U0 with kind type] }, universes: 1 }

[lbrande]

Does this mean that the only way to resolve this issue is to wait for it to be resolved by chalk? Or could the memory leak be resolved so that at least rust-analyzer doesn't becomes unusable?

[flodiebold]

It means the way to resolve this issue is to fix it in Chalk. (The first step towards that would be writing a Chalk test to reproduce it, and making an issue there.)

[lbrande]

I believe this Chalk test corresponds to the code below (which leads to the same memory leak as in the initial example), though that Chalk test succeeds which could indicate the fault is actually in the data sent to Chalk by rust-analyzer (with reservation for the Chalk test testing the wrong thing).

#[test]
fn associated() {
    test! {
        program {
            #[lang(fn_once)]
            trait FnOnce<Args> {
                type Output;
            }

            struct Box<T> {}

            closure foo<T, U>(self, t: T) -> U {}
        }

        goal {
            forall <T, U> {
                exists<R, S> {
                    if ( Normalize(<S as FnOnce<()>>::Output -> R) ) {
                        Normalize(<foo<T, U> as FnOnce<(Box<S>,)>>::Output -> R)
                    }
                }
            }
        } yields {
            r"Unique"
        }
    }
}

fn f<R>(_: impl FnOnce(Box<dyn FnOnce() -> R>) -> R) {}

fn main() {
    f(|_| ());
}

[flodiebold]

The modeling of closures in Chalk's test framework is somewhat different from ours, and often it's not possible to reproduce issues with them, in particular if variables are involved. So it might work better to do

            struct MyClosure<F> {}
            impl<T, U> FnOnce<(T,)> for MyClosure<fn(T) -> U> {
                type Output = U;
            }

The other reason why the test doesn't reproduce the issue could be that there are some impls somewhere that are affecting what happens, though it doesn't look like that to me from the output detrumi posted.

[lbrande]

The modeling of closures in Chalk's test framework is somewhat different from ours, and often it's not possible to reproduce issues with them, in particular if variables are involved. So it might work better to do

            struct MyClosure<F> {}
            impl<T, U> FnOnce<(T,)> for MyClosure<fn(T) -> U> {
                type Output = U;
            }

It seems this just results in an error claiming the impl is not well-formed.

[lnicola]

Triage: we no longer leak memory here, we just burn CPU.

[Veykril]

Are you sure, for me this does neither it just seems to work?

[lnicola]

Yeah, pretty sure. Maybe it's non-deterministic -- affected by cache priming, for example? I'll give it another try later.

[lnicola]

And now I can't reproduce it any more 😬.