maximum size of ocall

[elichai]

Hi, I'm trying to pass a big chunk of data through an ocall, and I get a segmentation fault when I pass around ~2MB. is a known thing? Trying to debug where exactly the SIGSEGV comes from leds me to it coming from the line: __tmp = sgx_ocalloc(ocalloc_size); in Enclave_t.c with ocalloc_size = 2196985.

The C code in Enclave_t looks as follow:

sgx_status_t SGX_CDECL ocall_save_to_memory(uint64_t* retval, const uint8_t* data_ptr, size_t data_len)
{
	sgx_status_t status = SGX_SUCCESS;
	size_t _len_data_ptr = data_len * sizeof(uint8_t);

	ms_ocall_save_to_memory_t* ms = NULL;
	size_t ocalloc_size = sizeof(ms_ocall_save_to_memory_t);
	void *__tmp = NULL;


	CHECK_ENCLAVE_POINTER(data_ptr, _len_data_ptr);

	ocalloc_size += (data_ptr != NULL) ? _len_data_ptr : 0;

	__tmp = sgx_ocalloc(ocalloc_size);
	if (__tmp == NULL) {
		sgx_ocfree();
		return SGX_ERROR_UNEXPECTED;
	}
	ms = (ms_ocall_save_to_memory_t*)__tmp;
	__tmp = (void *)((size_t)__tmp + sizeof(ms_ocall_save_to_memory_t));
	ocalloc_size -= sizeof(ms_ocall_save_to_memory_t);

	if (data_ptr != NULL) {
		ms->ms_data_ptr = (const uint8_t*)__tmp;
		if (memcpy_s(__tmp, ocalloc_size, data_ptr, _len_data_ptr)) {
			sgx_ocfree();
			return SGX_ERROR_UNEXPECTED;
		}
		__tmp = (void *)((size_t)__tmp + _len_data_ptr);
		ocalloc_size -= _len_data_ptr;
	} else {
		ms->ms_data_ptr = NULL;
	}
	
	ms->ms_data_len = data_len;
	status = sgx_ocall(3, ms);

	if (status == SGX_SUCCESS) {
		if (retval) *retval = ms->ms_retval;
	}
	sgx_ocfree();
	return status;
}

In the SIGSEGV instance it is called with these parameters: #1 0x00007fff0002ae1d in ocall_save_to_memory (retval=0x7fff50c250b8, data_ptr=0x7fff02b2d718 "", data_len=2196961) at enclave/Enclave_t.c:1742

The Rust headers of the ocall is: pub unsafe extern "C" fn ocall_save_to_memory(data_ptr: *const u8, data_len: usize) -> u64 and in the EDL: uint64_t ocall_save_to_memory( [in, count=data_len] const uint8_t* data_ptr, size_t data_len);

Any ideas if there's a cap on how much data can be passed through an ocall or is it something else that I'm missing?

[dingelish]

Look at here

sgx_ocalloc allocates memory on the outside stack.

I guess you reached the maximum size of the stack?

[elichai]

Yeah I looked at that (sadly I couldn't step into trts in gdb), is that the stack of the untrusted or the trusted? maybe I'll try recompiling with something like --zstack-size=4194304 to try and increase it to 4MB

And should allocating more stack memory than is available even be a SIGSEGV?

[dingelish]

According to this thread, Ishai Nadler (Intel) said that this stack is on the untrusted side.

[elichai]

@dingelish Do you know of any way to increase the stack size? The only way I could make it work is by launching a new thread and setting the stack size

[dingelish]

is ulimit -S -s 131072 works? i haven't test it yet..

[elichai]

weirdly no, the only thing that worked for me right now is launching a new thread and setting it with a bigger stack size https://doc.rust-lang.org/std/thread/struct.Builder.html#method.stack_size

[dingelish]

hmmm on my testbed, the ulimit -S -s 131072 works. The default stack size was 8192 and I'm setting it to 131072. In the following example, I'm using sgx_ocalloc to allocate a buffer of 10MB and here is the log:

hello-rust/bin $ ./app
[+] Home dir is /home/ding
[+] Open token file success!
[+] Token file invalid, will create new token file
[+] Init Enclave Successful 2!
This is a normal world string passed into Enclave!
This is a in-Enclave Rust string!
Segmentation fault (core dumped)
hello-rust/bin $  ulimit -S -s 131072
hello-rust/bin $  ./app
[+] Home dir is /home/ding
[+] Open token file success!
[+] Token file invalid, will create new token file
[+] Init Enclave Successful 2!
This is a normal world string passed into Enclave!
This is a in-Enclave Rust string!
0x7ffde0428660
This is the ocalloc ocall
[+] say_something success...

[elichai]

Hmm so it looks like this works for you. I'll keep testing, although I prefer to have something better than using ulimit. because this means that I'll need to incorporate ulimit for the end user somehow

[mssun]

This is another example to trigger the segfault: https://github.com/sammyne/rsgx-fs-playground

Basically, std::untrusted::fs::read() will do ocall and read a large size of data on stack, resulting segmentation fault.

[volcano0dr]

Yes, because sgx_ocalloc allocates memory on the outside stack. Therefore, the size of the transfer buffer between enclave and untrusted memory using ocall is limited by the outside stack size.

I think, when using ocall file IO in encalve, checking the buffer size is to solve this problem. If the size exceeds the set constant, use ocall to allocate memory on the outside heap, otherwise use sgx_ocalloc to allocate memory on the outside stack.