Connection problem

[AGG2017]

Thanks for your efforts to do a Python private server version. I installed it on Ubuntu 20.04LTS machine with Python 3.8.5 (just needed installation of the missing "construct" package). On the same machine I have a fully working version of Lumen private server on different port which is working great but I don't understand Rust and PostgreSQL and decided to try your version. From Windows 7 Pro machine I'm trying to connect with no TLS to the server by using IDA Pro 7.2/7.3/7.5. All of them are working fine with Lumen but giving me constantly with every request after the first one: lumina: connection closed (recv: Connection closed by peer). Trying to reconnect... lumina: applied metadata to 0 functions.

In debug mode I see that the metadata are accepted and stored in the json database when pushed to the server even the error message is still there. Then when requested, the right function metadata are reported by the server but IDA cannot accept the answer and populate it in the selected function.

I can provide more information if needed. I will try to compare the requests and answers between both servers. At least I have one fully working.

There was another issue storing the information in the database file. Every empty exit of the server add {} at the end of the database and next run time the file become invalid. That wasn't hard to be fixed.

[mrj0n3s]

I'm having the same issue, also tried it with TLS. On the first connection no error is being displayed but I haven't managed to get any data from the server back into an IDB yet. I'm running the server and IDA on the same machine (Windows, so the issue might be OS related).

[AGG2017]

Analyzing the network packets with Wireshark I can see invalid response to the RPC pull metadata request and then a packet to close the connection from the server. But on the screen in debug mode it shows correct response with the proper function name found. Maybe incorrect packing the data in the answer with RPC code 0x0F. It is much different with the working Lumen server where the name of the function is visible in the answer.

[mrj0n3s]

That was helpful, it works for me now :) all I did was changing the appended found values in lumina_server.py (lines 76 - 78) from metadata = self.database.pull(sig) if metadata: found.append(1)

to

metadata = self.database.pull(sig) if metadata: found.append(0)

[AGG2017]

I didn't have time to test everything else but I also found that in the result for 0x0F at least the status code is not right. In the documentation from the other working server that I have it is clearly written that the status code should be 0 if found: https://abda.nl/posts/introducing-lumen/

[Natanel-Shitrit]

Hey 😄 Found the issue that you were having (although a little late 😆)

This cause was a small mistake in the implementation of the server. Currently, the server is handling each connection as a singular request, and not expecting new RPC messages after the initial message after the RPC_HELO, so sending a second request will look like this: (colored the problematic part)

sequenceDiagram
	participant User as User
    participant IDA as IDA
    participant Server as Server
    
	User ->> IDA: Pull / Push all metadata
    IDA ->> Server: RPC_HELO
    Server -->> IDA: RPC_OK / RPC_FAIL

    alt RPC_OK received
	    IDA ->> Server: RPC_PULLMD / RPC_PUSHMD
        Server -->> IDA: RPC_PULLMD_RES / RPC_PUSHMD_RES
        IDA ->> User: Success!
    else RPC_FAIL received
         IDA ->> User: Failed 😢
    end
    rect rgb(200, 150, 255)
	Server ->> Server: I handled the connection!
	User ->> IDA: Pull / Push all metadata
    IDA ->> Server: RPC_PULLMD / RPC_PUSHMD
    Server ->> Server: WTF?? I closed that connection!
    Server ->> IDA: RST (TCP)
    IDA ->> User: connection closed ... Trying to reconnect...
    end

This is because the handle method in BaseRequestHandler represents a complete connection, from beginning to end. (which only handles one request in the current implementation)

The simple fix here is to add a loop in the handle method that keeps listening to requests after the initial connection has been made, making the process look like this: (colored the fixed part)

sequenceDiagram
	participant User as User
    participant IDA as IDA
    participant Server as Server

	User ->> IDA: Pull / Push all metadata
    IDA ->> Server: RPC_HELO
    Server -->> IDA: RPC_OK / RPC_FAIL

    alt RPC_OK received
	    rect rgb(200, 150, 255)
        loop Throughout Connection's Lifetime
	        User ->> IDA: Pull / Push all metadata
            IDA ->> Server: RPC_PULLMD / RPC_PUSHMD
            Server -->> IDA: RPC_PULLMD_RES / RPC_PUSHMD_RES
            IDA ->> User: Success!
        end
    end
    else RPC_FAIL received
        IDA ->> User: Failed 😢
    end
    Server ->> Server: I handled the connection!

In my fork I fixed this problem, but since I changed a lot of other things too I'll not PR the changes - feel free to implement a fix from my code :)