sdk
sdk copied to clipboard
dart-lang
Support for capturing errno across calls
When calling low level functions, e.g. libc wrappers around syscalls, one often gets -1 back and errno will be set to the actual error message.
When using dart:ffi to to do such calls there is no atomicity between the Dart -> C call and an access of errno (if we even support that). In fact, arbitrary VM code (e.g. GC), can run in between the two. That can cause errno to be clobbered.
An option would be to save errno into a slot on Thread::last_errno_ before returning from the call. We could then add an accessor for the last_errno_.
/cc @sjindel-google @dcharkes
This makes error handling tricky and potentially flaky when dealing with Win32 API calls. Many Win32 APIs return boolean return results (or handles) instead of an error code. It's a lot more widespread for Win32 since GetLastError is a general paradigm when working with any Windows API.
This makes error handling tricky and potentially flaky when dealing with Win32 API calls. Many Win32 APIs return boolean return results (or handles) instead of an error code. It's a lot more widespread for Win32 since
GetLastErroris a general paradigm when working with any Windows API.
Encounter the same problem when debug https://github.com/Sunbreak/logic_conf.dart/blob/e4935e68399987d61262b3a7d20d0a2c38808471/lib/src/logic_conf_windows.dart#L54-L58
I am having a problem while experimenting with ffi and the C library open62541 and wanted to ask whether this could be related to this issue.
The problem appears during a network request in which a select()-syscall is executed. I tracked it down to this line (or rather to the UA_select()-call on line 99).
I found out that the problem occurs in debug mode only (about every second attempt), in both pure Dart (dart run --observe) and Flutter applications (flutter run --debug). I was not able to reproduce this in release mode yet.
My toolchain is:
~ flutter doctor -v
[✓] Flutter (Channel stable, 2.2.0, on Linux, locale de_DE.UTF-8)
• Flutter version 2.2.0 at ...
• Framework revision b22742018b (vor 6 Wochen), 2021-05-14 19:12:57 -0700
• Engine revision a9d88a4d18
• Dart version 2.13.0
[✓] Linux toolchain - develop for Linux desktop
• clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
• cmake version 3.18.4
• ninja version 1.8.2
• pkg-config version 0.29.1
If the problem is related to this issue, what would be a possible solution (apart from not using debug mode)? And why does it only occur in debug mode, is it related to the VM?
If you require further information, I would be happy to share this with you.
Thank in you in advance for any help and suggestions!
@basyskom-bmeier
If the problem is related to this issue, what would be a possible solution (apart from not using debug mode)? And why does it only occur in debug mode, is it related to the VM?
The problem is not related to this issue. See this https://github.com/flutter/flutter/issues/82488#issuecomment-842094118 for an explanation of your problem and potential workarounds.
@basyskom-bmeier
If the problem is related to this issue, what would be a possible solution (apart from not using debug mode)? And why does it only occur in debug mode, is it related to the VM?
The problem is not related to this issue. See this flutter/flutter#82488 (comment) for an explanation of your problem and potential workarounds.
Perfect, thanks a lot!
I think for desktop applications especially Windows (either dart or flutter), this would be really helpful. Are there any updates regarding this issue?
GetLastError and errno are both available on Windows, and they are set for two different sets of functions. The first one for win32 and the latter one for c library functions. So we'll likely have to provide both on Windows.
In .NET Framework. There is attribute called SetLastError and function called Marshal.GetLastPInvokeError for this purpose.
.NET is designed well.
Comment aside, just save errno when returning back from native code, and restore when calling native code. This is the method LuaJIT uses. https://luajit.org/ext_ffi_api.html#:~:text=Utility%20Functions
Is there a way to access the errno in a platform-agnostic way? In https://github.com/tvolkert/libjpeg/blob/898aaf01bbf07619cb8a9047f7b732b1b544d01f/lib/libjpeg.dart#L197, the return code will be 0 if there was an error, and errno will contain the actual error -- and I'd like to be able to access it in a platform-agnostic way.
Is there a way to access the errno in a platform-agnostic way? In https://github.com/tvolkert/libjpeg/blob/898aaf01bbf07619cb8a9047f7b732b1b544d01f/lib/libjpeg.dart#L197, the return code will be 0 if there was an error, and errno will contain the actual error -- and I'd like to be able to access it in a platform-agnostic way.
The idea with the native-assets feature is that the native code can be wrapped in C to catch the error in C and return a tagged union back to Dart. And then use FFIgen on the C wrapper function.
GetLastErroranderrnoare both available on Windows, and they are set for two different sets of functions. The first one for win32 and the latter one for c library functions. So we'll likely have to provide both on Windows.
I believe all OSes will be errno, unless libjpeg is using win32 under the hood on Windows. In that case we'd need to use some ifdef to define a platform agnostic catch.
Update on this issue: Our current thinking is that people should write wrapper C functions to capture the errno. (We should consider if we can ease that in FFIgen.)
Work on the native assets feature should enable bundling C code easily with Dart packages. (Currently in experimental.)
We have a customer that has a valid use case for this where it's not about getting the wrong error message with some low probability but the actual application behavior requires checking the error numbers (see b/323386486).
There's a few things
-
Figure out how to get the error: Apart from calling the actual C apis, it may be sufficient to load a value from a segment register (i.e. close to one instruction to obtain it). We should investigate whether this is feasible on all OSes/archs.
-
Figure out what the overhead is if we always capture the error: For leaf and non-leaf calls. It may be that the overhead of non-leaf calls is big enough that doing this extra errno load (+stash on THR) wouldn't make them much slower. So we may consider capturing it always, e.g. in non-leaf calls. ( If the overhead is too big, we may provide a flag (similar to
isLeaf) to opt into the capturing behavior.) -
The actual captured error number could be exposed as a getter in
dart:ffi. -
We could have a compile-time solution: Generate C wrappers or runtime solution: Generate trampolines at runtime (can be done in Dart code, but only works in JIT-able environments, i.e. non-iOS)
We should validate what's the bets for package:win32.
@dcharkes Could you do some investigations here?
If the overhead is too big, we may provide a flag (similar to
isLeaf) to opt into the capturing behavior. The actual captured error number could be exposed as a getter indart:ffi.
We already built that once: https://dart-review.googlesource.com/c/sdk/+/240847
@mkustermann What are your thoughts on solving this with native assets instead? (And FFIgen. But win32 has it's own code gen.)
We already built that once: https://dart-review.googlesource.com/c/sdk/+/240847
I was hoping there's a way to do this more efficiently than what that CL does (e.g. load from segment register, store into THR). If it's a high-overhead + high-complexity solution it would certainly push the favor towards codegen.
@mkustermann What are your thoughts on solving this with native assets instead? (And FFIgen. But win32 has it's own code gen.)
Certainly open to it, you can prototype changing the win32 codegen and seeing how it would look like in the end. It may push it out until native assets are non-experimental (or some end-users can use pre-release versions that use native assets).
Was both sad there wasn't an easy fix to this but appreciative that we're tracking the issue.
It is fantastic to be able to use dart:ffi directly without a build system or secondary toolchains, so I'm a tad sad that I might have to write (or generate) and compile custom C-code in order to access lower-level functions correctly - it's not the end of the world (the vast majority of code won't do this, and almost certainly no applications will do it directly) but it does feel like a hard limitation.
Was both sad there wasn't an easy fix to this but appreciative that we're tracking the issue.
It is fantastic to be able to use
dart:ffidirectly without a build system or secondary toolchains, so I'm a tad sad that I might have to write (or generate) and compile custom C-code in order to access lower-level functions correctly - it's not the end of the world (the vast majority of code won't do this, and almost certainly no applications will do it directly) but it does feel like a hard limitation.
We have a larger set of features that we could add to dart:ffi which can also be solved by adding a thin C wrapper. The problem would be that we would have to build and maintain all these features in the Dart VM. (Unfortunately I can't scale myself in that way.) Instead, we're envisioning making it easy to bundle C code (native assets work), and making it easy to generate that thin wrapper (FFIgen or something similar). This would enabled an arbitrary set of features that can be achieved by having a generated layer of C code.