Add a debug allocator

[lhchavez]

This change allows debugging all the allocations performed by libgit2. To enable, set the environment variable GIT2GO_DEBUG_ALLOCATOR_LOG=/tmp/git2go_alloc and run the git2go program. Once the program exits, run the ./script/leak_detector.py script, and a leak summary will be printed.

In order to reduce the amount of noise due to the static allocations performed by libgit2, it is recommended to call git.Shutdown() before exiting the program.

[suhaibmujahid]

Thank you. This will be very helpful for me in debugging a memory leak issue that I'm facing.

I tried to run the following on Mac OS:

sudo python3 ./script/leak_detector.py

However, I got the following error:

Traceback (most recent call last):
  File "./script/leak_detector.py", line 132, in <module>
    main()
  File "./script/leak_detector.py", line 122, in main
    with socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) as sock:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 151, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [Errno 43] Protocol not supported

[lhchavez]

huh, socket(2) on the OS X manpages claims to support it. oh well, try applying this patch and see if that helps:

diff --git a/debug_allocator.c b/debug_allocator.c
index 56a16ec..eff9abb 100644
--- a/debug_allocator.c
+++ b/debug_allocator.c
@@ -153,7 +153,7 @@ int _go_git_setup_debug_allocator()
        struct sockaddr_un name = {};
        int error;
 
-       __alloc_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0);
+       __alloc_fd = socket(AF_UNIX, SOCK_DGRAM, 0);
        if (__alloc_fd == -1) {
                perror("socket");
                return -1;
diff --git a/script/leak_detector.py b/script/leak_detector.py
index ec8a278..72c338f 100755
--- a/script/leak_detector.py
+++ b/script/leak_detector.py
@@ -92,14 +92,18 @@ def _handle_connection(conn: socket.socket) -> None:
     """Handle a single connection."""
 
     live_allocations: Dict[int, Allocation] = {}
-    with conn:
-        for message_type, allocation in _receive_allocation_messages(conn):
-            if message_type in ('A', 'R'):
-                live_allocations[allocation.ptr] = allocation
-            elif message_type == 'D':
-                del live_allocations[allocation.ptr]
-            else:
-                raise Exception(f'Unknown message type "{message_type}"')
+    try:
+        print('Capturing allocations, press Ctrl+C to stop...')
+        with conn:
+            for message_type, allocation in _receive_allocation_messages(conn):
+                if message_type in ('A', 'R'):
+                    live_allocations[allocation.ptr] = allocation
+                elif message_type == 'D':
+                    del live_allocations[allocation.ptr]
+                else:
+                    raise Exception(f'Unknown message type "{message_type}"')
+    except KeyboardInterrupt:
+        pass
     _process_leaked_allocations(live_allocations)
 
 
@@ -119,13 +123,10 @@ def main() -> None:
     except FileNotFoundError:
         pass
 
-    with socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) as sock:
+    with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as sock:
         sock.bind(args.socket_path)
         os.chmod(args.socket_path, 0o666)
-        sock.listen(1)
-        while True:
-            conn, _ = sock.accept()
-            _handle_connection(conn)
+        _handle_connection(sock)
 
 
 if __name__ == '__main__':

[suhaibmujahid]

Thank you @lhchavez for the patch. I applied it and it seems that it fixed the unsupported protocol issue.

Now I'm getting the following error:

Traceback (most recent call last):
  File "./script/leak_detector.py", line 134, in <module>
    main()
  File "./script/leak_detector.py", line 128, in main
    sock.bind(args.socket_path)
FileNotFoundError: [Errno 2] No such file or directory

This error because the script cannot create /run/git2go_alloc.sock since /run does not exist in MacOS and it cannot be created:

mkdir: /run: Read-only file system

The socket path can be changed in the python script through the argument socket_path. However, it is hardcoded in debug_allocator.c:163. It will be nice if it can be changed, through an environment variable for example.

[lhchavez]

Thank you @lhchavez for the patch. I applied it and it seems that it fixed the unsupported protocol issue.

Now I'm getting the following error:

Traceback (most recent call last):
  File "./script/leak_detector.py", line 134, in <module>
    main()
  File "./script/leak_detector.py", line 128, in main
    sock.bind(args.socket_path)
FileNotFoundError: [Errno 2] No such file or directory

This error because the script cannot create /run/git2go_alloc.sock since /run does not exist in MacOS and it cannot be created:

mkdir: /run: Read-only file system

The socket path can be changed in the python script through the argument socket_path. However, it is hardcoded in debug_allocator.c:163. It will be nice if it can be changed, through an environment variable for example.

I don't know why I was making the code be more clever than it needed. The latest version uses a text file format that can be either printed out as a regular file, or can be slurped by a FIFO (no more overcomplicated sockets) with the Python script.

Note the change of environment variable (GIT2GO_DEBUG_ALLOCATOR_LOG=/tmp/git2go_alloc) to support changing the path.

[suhaibmujahid]

Works perfectly.

A small note: If you run ./script/leak_detector.py without the flag --pipe, it will not create a new file, and it will raise an exception if the log file does not exist.