GUACAMOLE-663: guacd SEGVs intermittently on systems with small(er) thread stack sizes
This looks okay to me, but I'm not confident in my ability to review the configure.ac changes. @mike-jumper anything that needs to be fixed before this one goes in?
My ulimit -s returns 8192 (I think it's Kbytes, so 8 MB) and I get general protection faults in session guacds (not the parent guacd, though) somewhat intermittently. I set it to 65536 (for both hard and soft limit) and I still get intermittent GPFs. It's an "error:0 in libc-2.2.7.so" which could very well be a stack overflow, or it might be something else. Not sure.
Would setting ulimit -s 65536 before running guacd potentially fix this problem while this code is still being updated/reviewed? If so, I guess having this problem means my build either has a stack memory leak, or guacd actually needs much more than 8 MiB of stack to be stable, or ulimit -s isn't applying to guacd.
Note: I am not using a systemd start script to launch guacd. I'm just typing guacd directly as root.
OS: Ubuntu 18.04 64-bit guacamole version: 1.0.0 compiled from source (using Ubuntu's freerdp-1.1) Using RDP with xrdp/xorgxrdp (the xrdp bit works fine, directly connecting with Windows Remote Desktop is 100% stable and never crashes)
@allquixotic, please use the dev@ or user@ mailing list if you have questions:
http://guacamole.apache.org/support/#mailing-lists
There are no known issues with the behavior you describe, and it's unlikely that what you are seeing is related to this change. It's also unlikely that a bug as fundamental as complete instability in the RDP support exists, and more likely that the cause is something else:
http://guacamole.apache.org/faq/#probably-not-a-bug
There are other factors which may cause a segfault, including accidentally mixing binaries from different guacamole-server builds. If there is a bug causing what you're seeing, it's worth chasing down, but this pull request is not the place for that.