ckwin icon indicating copy to clipboard operation
ckwin copied to clipboard

Loading conpty test dll in a watcom build causes a crash

Open davidrg opened this issue 3 years ago • 6 comments

conpty.dll built with Visual C++ 14, k95g.exe built with openwatcom 1.9. Loading conpty.dll causes a crash and the command, as seen by conpty, seems to be garbage. Might just be caused by the mixing of runtime libraries or it could be an actual bug. Needs investigation.

davidrg avatar Aug 01 '22 08:08 davidrg

This issue is cause different calling convention for DLL client function and DLL exported function. It requires to define same calling convention on both sides. Now it is unspecified (bad practice) and relates to compiler default calling convention which can be and really is different. Of cause if you compile both sides by same compiler then no problem, both sides use same default calling convention. But it is wrong, especially if you want to mix DLL created by various compilers with client code compiled by another compiler. Best solution is to use cdecl for all user functions in DLL and define cdecl calling convention for all function pointers imported from DLL. System DLL use system default calling convention which is defined in appropriate header files that it works always without trouble. But user defined DLL can use any calling convention therefore it must be properly defined in appropriate header file.

jmalak avatar Oct 26 '24 16:10 jmalak

It is problem with all DLLs in kermit code that it is little bit chaotic spread to multiple header files and also it uses multiple definition for same function (logical) that it introduce type mismatch on runtime. By example TCP/IP DLLs has same problem. best solution is define single header for such DLL with exported function only. There can be expressed calling convention and also can be done name mapping from DLL name to user code name etc.

jmalak avatar Oct 26 '24 16:10 jmalak

Ah, yeah that makes sense. It hadn't occurred to me Open Watcom would use a different default calling standard from Visual C++ on Windows. Really there are a bunch of places the calling standard out to be made explicit - network dlls (this), the legacy telnet crypto dll (k95crypt/k2crypt.dll), and on Windows the SSH and ZYZ-modem DLLs (on OS/2 these use _System, but nothing on Windows)

davidrg avatar Oct 27 '24 06:10 davidrg

Generaly the ckwin code is Visual studio centric. Similar problem is with POSIX functions guarded by NT macro that use Microsoft names not POSIX names. It requires renaming of all POSIX name back to POSIX from Microsoft names. For better portability only POSIX names should be used in ckwin code and create aliases or used appropriate Microsoft macros to use real POSIX names with Microsoft compilers. But it is going from history.

jmalak avatar Oct 27 '24 06:10 jmalak

Yeah, the oldest of the code in cko*.* was written for Microsoft C and whatever the IBM compiler was in the late 80s, and both compilers were supported until around 1994. ckn*.* and all of the Windows code in cko*.* was originally written for either Visual C++ 2.0 or 4.0 (both of which are still supported along with Visual C++ 1.0 32-bit for doing the RISC NT builds)

davidrg avatar Oct 27 '24 19:10 davidrg

Ah, yeah that makes sense. It hadn't occurred to me Open Watcom would use a different default calling standard from Visual C++ on Windows. Really there are a bunch of places the calling standard out to be made explicit - network dlls (this), the legacy telnet crypto dll (k95crypt/k2crypt.dll), and on Windows the SSH and ZYZ-modem DLLs (on OS/2 these use _System, but nothing on Windows)

Nothing doesn't mean "nothing" but default. Therefore it is problem with interoperability. Anyway I don't thing _System calling convention need to be used in user code. It has only sense for OS callbacks, but user code can use any calling convention, normally it uses compiler default. I don't know what other compilers use as default calling convention on OS/2, but Watcom compilers use __watcall calling convention (global default)

jmalak avatar Oct 27 '24 22:10 jmalak