[Windows] "Bad address" exception while creating socket with context from dll

[Sontik]

Hello! I've met a problem I can't deal with: There is dll with function which creates zmq-context. (void* createCtx()) I load this dll into Python, create context with it and then zmq.Context.shadow(raw_zmq_context) this context. After that I try to create socket with shadowed context: sock_ = zmq_ctx_.socket(zmq.REQ) And there happens "Bad address" exception.

pyzmq version - 19.0.0 libzmq version - 4.3.2

[minrk]

Without code to reproduce it, I'll have to guess, but I would think the most likely option is not casting the address quite correctly to a Python integer when passing it to Python.

[Sontik]

Without code to reproduce it, I'll have to guess, but I would think the most likely option is not casting the address quite correctly to a Python integer when passing it to Python.

I'll try to make a little example and add here. How can I fix possible problem with casting? As I know, there is no such problem on MacOS.

[minrk]

I can't tell you how to fix it without seeing it, since I don't know what's wrong. But a mistake I had made when testing this myself was improperly casting a pointer to a signed integer, which will often result in an incorrect value when the integer gets cast back to a pointer. If you can, double check in C and Python both, that the address is the same at every stage.

[sakurai-youhei]

FYI: Here is my code working on Windows.

from ctypes import c_void_p
from ctypes import windll
import zmq

libzmq = windll.LoadLibrary("./libzmq-v140-mt-4_3_2.dll")
libzmq.zmq_ctx_new.restype = c_void_p

addr = libzmq.zmq_ctx_new()
ctx = zmq.Context.shadow(addr)

sock1 = ctx.socket(zmq.PAIR)
sock1.bind("inproc://test")
sock2 = ctx.socket(zmq.PAIR)
sock2.connect("inproc://test")

sock1.send(b"test")
assert sock2.recv() == b"test"

Environment

• Windows: 10.0.19041
• Python: Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)]
• pyzmq: 19.0.1 zmq.zmq_version() -> 4.3.2
• MSVCCompiler: C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe
• DLLs: libzmq-v140-mt-4_3_2.dll and libsodium.dll from https://dl.bintray.com/zeromq/generic/libzmq-v140-x64-4_3_2.zip

[SotnikAP]

I did a little expamle, which represent my problem: There is dll: zmqctx.h:

#ifdef ZMQCTX_EXPORTS
#define ZMQCTX_API __declspec(dllexport)
#else
#define ZMQCTX_API __declspec(dllimport)
#endif

extern "C" ZMQCTX_API void* create_ctx();

zmqctx.cpp:

#include "pch.h"
#include "framework.h"
#include "zmqctx.h"
#include <zmq.h>
static void* current_ctx = nullptr;
void* create_ctx()
{
     void* ctx = zmq_ctx_new();
     current_ctx = ctx;
     return current_ctx;
}

And python code:

import ctypes
import zmq
import os

dll = ctypes.windll.LoadLibrary("D:\_Projects\zmqctx\Debug\ZMQCTX.dll")

create_ctx = dll.create_ctx
create_ctx.restype = ctypes.c_void_p

raw_ctx = create_ctx()
ctx = zmq.Context.shadow(raw_ctx)

socket = ctx.socket(zmq.REP)

And traceback:

Traceback (most recent call last):
  File "D:/PyProj/test_zmq/main.py", line 14, in <module>
    socket = ctx.socket(zmq.REP)
  File "C:\Python38\lib\site-packages\zmq\sugar\context.py", line 204, in socket
    s = self._socket_class(self, socket_type, **kwargs)
  File "C:\Python38\lib\site-packages\zmq\sugar\socket.py", line 59, in __init__
    super(Socket, self).__init__(*a, **kw)
  File "zmq\backend\cython\socket.pyx", line 328, in zmq.backend.cython.socket.Socket.__init__
zmq.error.ZMQError: Bad address

So dll creates context and python uses this context to create socket. Example from sakurai-youhei worked for me too, but it's a little different from my problem. Thanks for example anyway, I tried few things from it, but they didn't help.

Also I did double check in C and in Python - address is the same.

[sakurai-youhei]

@SotnikAP Yours doesn't reproduce the issue in my environment.

Environment

• Windows: 10.0.19041
• Python: Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)]
• pyzmq: 19.0.1 zmq.zmq_version() -> 4.3.2
• MSVCCompiler: C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\cl.exe
• DLLs: libzmq-v140-mt-4_3_2.dll and libsodium.dll from https://dl.bintray.com/zeromq/generic/libzmq-v140-x64-4_3_2.zip

zmqctx.h

#ifdef ZMQCTX_EXPORTS
#define ZMQCTX_API __declspec(dllexport)
#else
#define ZMQCTX_API __declspec(dllimport)
#endif

extern "C" ZMQCTX_API void* create_ctx();

zmqctx.cpp

#include "zmqctx.h"
#include <zmq.h>
static void* current_ctx = nullptr;
void* create_ctx()
{
     void* ctx = zmq_ctx_new();
     current_ctx = ctx;
     return current_ctx;
}

poc.py

# Code to compile ZMQCTX.dll
from distutils.msvccompiler import MSVCCompiler

compiler = MSVCCompiler()
compiler.initialize()

compiler.set_include_dirs(["."])
compiler.set_library_dirs(["."])
compiler.set_libraries(["libzmq-v140-mt-4_3_2"])

objects = compiler.compile(["zmqctx.cpp"])
compiler.link_shared_object(objects, output_filename="ZMQCTX.dll")

# Code to check the issue
import ctypes
import zmq
import os

dll = ctypes.windll.LoadLibrary("./ZMQCTX.dll")

create_ctx = dll.create_ctx
create_ctx.restype = ctypes.c_void_p

raw_ctx = create_ctx()
ctx = zmq.Context.shadow(raw_ctx)

socket = ctx.socket(zmq.REP)

Working directory

C:\Users\sakurai\Desktop\issue-1398>dir *.h *.cpp *.lib

2019/08/27  20:12            28,790 zmq.h
2020/06/28  11:04               165 zmqctx.h

2020/06/28  11:31               184 zmqctx.cpp

2019/08/27  20:15            20,868 libzmq-v140-mt-4_3_2.lib
2019/08/27  20:15         5,163,648 libzmq-v140-mt-s-4_3_2.lib
2020/06/28  11:30             1,700 ZMQCTX.lib