Add a way not to initialize OpenSSL

[catwell]

When used in a C program that also uses OpenSSL (or another library that uses OpenSSL), re-initializing OpenSSL like initall() does will cause issues.

There should be a way to tell the library that OpenSSL is already initialized before calling any luaopen_ function.

[wahern]

Is it the call OPENSSL_config causing problems? SSL_library_init? OpenSSL_add_all_algorithms? Initialization of the locking callbacks? Knowing the precise issues will help figuring out a proper API.

[catwell]

Sadly I don't remember exactly what the issue I encountered 3 months ago was. I should have written down the details... Given how tricky the OpenSSL API can be, I would say all of those could be issues.

I ended up not using luaossl and writing my own bindings for the functions I needed, not only because of that but also because I needed Windows support anyway.

[daurnimator]

because I needed Windows support anyway.

Spurred by this comment I spent a couple hours working on windows support. Tracking in #77

[daurnimator]

Closing until you have more details.

Note that in OpenSSL 1.1.0 we no longer initialise.

[catwell]

OK, we dropped luaossl from that project anyway (and we will switch to 1.1.0 anytime soon).

[daurnimator]

I was alerted to https://freeswitch.org/confluence/display/FREESWITCH/Lua+API+Reference#LuaAPIReference-Knownissues today.

[os11k]

Hi,

How can I help? :)

[daurnimator]

@os11k could you elaborate on the issue you hit in freeswitch? How could luaossl not break things?

[os11k]

Hi,

I was running hangup script in lua using lua ssl module and it was crashing Freeswitch. I got some input from freeswitch developers:

https://freeswitch.org/jira/browse/FS-10016

It was 8 month ago so I can't remember a lot.

I was using following function and which was causing a crash:

local params = {
    mode = "client",
    protocol = "sslv23",
    options = "all",
}

local try = socket.try
local protect = socket.protect

function create()
    local t = {c=try(socket.tcp())}

    function idx (tbl, key)
        return function (prxy, ...)
                   local c = prxy.c
                   return c[key](c,...)
               end
    end


    function t:connect(host, port)
        try(self.c:connect(host, port))
        self.c = try(ssl.wrap(self.c,params))
        try(self.c:dohandshake())
        return 1
    end

    return setmetatable(t, {__index = idx})
end

[daurnimator]

So I guess if you could do something like: openssl.set_initialized(true) that would be sufficient?

The implementation of that could set initssl: https://github.com/wahern/luaossl/blob/3c300b704fb1b36d8f1136802edefcecf8750244/src/openssl.c#L10586

[os11k]

Could you please explain a bit more? I'm not a wizard yet, I'm still learning.

[daurnimator]

Could you please explain a bit more? I'm not a wizard yet, I'm still learning.

I'm proposing an addition to luaossl (which is this project; you were actually using luasec, a different project) to allow a user to specify "I've already initialised openssl, please don't do it again". Which would make requiring luaossl work in your hangup handler.

[os11k]

Great! Please let me know if you need any more help from me. ;)

[catwell]

OpenSSL 1.1.0 makes initialization (and de-initialization) unnecessary, so a simple solution to this issue could be to migrate to the 1.1 API and just never initialize now.

[daurnimator]

a simple solution to this issue could be to migrate to the 1.1 API and just never initialize now.

Already done (https://github.com/wahern/luaossl/blob/5be1b44a6a60f32c660cc4ee09d60e676cd8c81a/src/openssl.c#L10612)... as long as the user has 1.1.0, which isn't everyone.