Memory overflows not caught

[UWN]

ulrich@p0:~/lftp/rusty-prolog$ ulimit -v 100000
ulrich@p0:~/lftp/rusty-prolog$ /opt/gupu/rusty-wam/target/debug/rusty-wam
prolog> a:-a,a.
prolog> ?-catch(a,error(E,_),true).
fatal runtime error: allocator memory exhausted
Illegal instruction (core dumped)

Expected (here, SICStus):

| ?- catch(a,error(E,_),true).
E = resource_error(memory) ? 
yes

Instead of memory, some other atom may be used.

(this is still a problem)

[UWN]

ulrich@p0:/opt/gupu/scryer-prolog$ /opt/gupu/scryer-prolog/target/debug/scryer-prolog 
?- [user].
a:-a,a.
?- catch(a,error(E,_),true).
memory allocation of 92274688 bytes failed
Aborted (core dumped)

[UWN]

Please do not forget this one. It always takes some time to retest this...

[mthom]

It's tricky to do this in Rust. I still don't quite know how to approach it, even though some strides have been made with custom allocators recently.. I thought that meant it would be easier to detect allocation errors, but apparently not. I will continue to look into it.

[UWN]

All interesting bugs reside behind memory overflows. And it´s important to handle these cases. After all, most programs are wrong in some initial stage

[UWN]

One thing that I do not understand is whether or not Rust can handle stackoverflows caused by recursion. If not, every routine has to be written with manual stack management.

[mthom]

Looks like there's a pending RFC to allow memory allocation errors to be caught:

https://github.com/rust-lang/rust/issues/48043

[UWN]

Currently, the error shows differently

ulrich@TU-Wien:/opt/gupu/scryer-prolog$ ulimit -v 1000000
ulrich@TU-Wien:/opt/gupu/scryer-prolog$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.14s
     Running `target/debug/scryer-prolog`
?- [user].
a:-a,a.


?- catch(a,error(E,_),true).
Segmentation fault (core dumped)

[aarroyoc]

Starting with Rust 1.57, several data structures have try_reserve available: https://blog.rust-lang.org/2021/12/02/Rust-1.57.0.html#stabilized-apis

[UWN]

Bleeding edge... but definitely fine progress.

[UWN]

And after looking at it a second time, there does not seem to be a try_reserve for the stack, right?

[aarroyoc]

After analyzing this issue a bit more, I think that we can actually catch this error right now, but it will imply a lot of internal changes.

Basically in raw_block.rs:

    pub(super) unsafe fn grow(&mut self) {
        if self.size == 0 {
            self.init_at_size(T::init_size());
        } else {
            let layout = alloc::Layout::from_size_align_unchecked(T::init_size(), T::align());
            let top_dist = self.top as usize - self.base as usize;

            self.base = alloc::realloc(self.base as *mut _, layout, self.size * 2) as *const _;
            if self.base.is_null() {
               // we can check if memory allocation failed
            }
            self.top = (self.base as usize + top_dist) as *const _;
            self.size *= 2;
        }
    }

The code inside the null check is called in the test case of this issue just before crashing. However, the real problem is using this information right in upper levels to report the problem in a nice way.

[UWN]

ulrich@p0:~/SO$ ulimit -v 50000
ulrich@p0:~/SO$ time /opt/gupu/scryer-prolog/target/release/scryer-prolog -f
?- use_module(library(lists),[length/2]).
   true.
?- length(L,N), N mod 100_000 =:= 0.
   L = [], N = 0
;  L = [_A,_B,_C,_D,_E,_F,_G,_H,_I,_J,_K,_L,_M,_N,_O,_P,_Q,_R,_S,_T,...], N = 100000
;  memory allocation of 262144 bytes failed
Aborted (core dumped)

real	0m42.284s
user	0m2.014s
sys	0m0.259s

One possibility might be to check on every Nth inference, if there is enough space. And more voracious operations need to check this manually, think of

ulrich@p0:~/SO$ ulimit -v
50000
ulrich@p0:~/SO$ time /opt/gupu/scryer-prolog/target/release/scryer-prolog -f
?- use_module(library(lists),[length/2]).
   true.
?- length(_,N), _ is 2^2^N.
   N = 0
;  N = 1
;  N = 2
;  N = 3
;  N = 4
;  N = 5
;  N = 6
;  N = 7
;  N = 8
;  N = 9
;  N = 10
;  N = 11
;  N = 12
;  N = 13
;  N = 14
;  N = 15
;  N = 16
;  N = 17
;  N = 18
;  N = 19
;  N = 20
;  N = 21
;  N = 22
;  GNU MP: Cannot allocate memory (size=1187856)
 Aborted (core dumped)

real	0m27.004s
user	0m1.419s
sys	0m0.255s

[UWN]

This needs to be fixed prior to any GC efforts. After all, you need to detect the moment when GC should be triggered.