scryer-prolog
scryer-prolog copied to clipboard
Heap memory limit
Adds a simple check for the size of the heap that is done every once in a while, as described in #2294. I haven't implemented the cli argument yet, but it should be possible with this. Example usage:
?- [user].
get_heap_limit(Lim) :- '$get_heap_limit'(Lim).
set_heap_limit(Lim) :- '$set_heap_limit'(Lim).
?- get_heap_limit(Lim).
% No limit by default.
Lim = no_limit.
?- get_heap_limit(no_limit).
true.
?- set_heap_limit(100000).
true.
?- get_heap_limit(Lim).
Lim = 100000.
?- get_heap_limit(no_limit).
false.
?- set_heap_limit(not_integer).
error(type_error(integer,'$set_heap_limit'/1),'$set_heap_limit'/1).
?- set_heap_limit(no_limit).
% I have not implemented this, so there is no way to "unset" the limit.
% Should this be implemented?
error(type_error(integer,'$set_heap_limit'/1),'$set_heap_limit'/1).
?- M is 10^6, length(L, M).
% Clean resource error. I'm not sure what to put in the second argument,
% so I made it a variable. I would appreciate suggestions.
error(resource_error(heap_limit,100000),_42).
?-
ulrich@gupu:/opt/gupu/bakaq-scryer-prolog$ target/release/scryer-prolog
?- [user].
get_heap_limit(Lim) :- '$get_heap_limit'(Lim).
set_heap_limit(Lim) :- '$set_heap_limit'(Lim).
?- set_heap_limit(100000).
thread 'main' panicked at 'index out of bounds: the len is 87987 but the index is 8935141660703064304', src/machine/dispatch.rs:545:24
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
ulrich@gupu:/opt/gupu/bakaq-scryer-prolog$ ulimit -v
unlimited
ulrich@gupu:/opt/gupu/bakaq-scryer-prolog$ target/release/scryer-prolog-v
bash: target/release/scryer-prolog-v: No such file or directory
ulrich@gupu:/opt/gupu/bakaq-scryer-prolog$ target/release/scryer-prolog -v
v0.9.3-175-g003b95df
I can't reproduce. At least not with such a high value.
.../repos/dev/scryer-prolog> ./target/release/scryer-prolog -f heap_memory_limit
?- [user].
get_heap_limit(Lim) :- '$get_heap_limit'(Lim).
set_heap_limit(Lim) :- '$set_heap_limit'(Lim).
?- set_heap_limit(100000).
true.
?- set_heap_limit(10000).
true.
?- set_heap_limit(1000).
thread 'main' panicked at src/machine/dispatch.rs:545:33:
index out of bounds: the len is 24061 but the index is 8935141660703064368
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
.../repos/dev/scryer-prolog> ./target/release/scryer-prolog -v heap_memory_limit 101
v0.9.3-175-g003b95df
Ok, I can reproduce without the -f option. It seems loading modules is causing some weird things here.
It seems to be a problem with backtracking. I think it's doing the backtracking even though there is no frame from where it could load, so I think it hits Undefined Behavior inside RawBlock and loads some instruction out of some place it shouldn't. This seems really hard to fix, and I think I didn't do anything wrong with the backtracking interface of Machine.
(One reason why you were not able to reproduce it with exactly the same values is that you invoked Scryer with a minimally different path: target/release/scryer-prolog vs. ./target/...)
The crash happens because check_heap_limit is called a second time, during the propagation of the first check_heap_limit exception, and self.machine_st.block = 0 the second time, having been reset by the first. If a hard heap limit is being set, shouldn't check_heap_limit reset the heap length to the limited amount prior to writing the exception to the heap? But even then, depending on the severity of the limit, the same problem may appear again as the heap fills during the exception propagation.
Throwing resource errors is one of the problems I'm now working on by adding a proper GC, replacing the current heap data type of Vec<HeapCellValue> with a custom block with low-level allocations. What this PR is trying to do is extremely tricky and hazardous. It may be best to wait for the GC to be completed. If @bakaq can make it work in the meantime, by all means, but as I said, it's very easy to run into such obstacles.