sqlite icon indicating copy to clipboard operation
sqlite copied to clipboard

cgosqlite: add more string interns, and optimize

Open raggi opened this issue 1 year ago • 1 comments

This adds string interning to column names, table names and database names, as well as optimizing the decltype interning code path to use runtime.findnull rather than C.strlen that must cross cgo again.

We don't yet know if this is effective in practice, and should test this before integrating.

raggi avatar May 15 '24 18:05 raggi

Unclear just using storm.

Current code, storm runs:

        User time (seconds): 4758.12
        System time (seconds): 566.26
        Percent of CPU this job got: 4383%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:01.45
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 180950476
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 619
        Minor (reclaiming a frame) page faults: 47275173
        Voluntary context switches: 5860952
        Involuntary context switches: 126650
        Swaps: 0
        File system inputs: 0
        File system outputs: 283464
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
        
        User time (seconds): 5058.74
        System time (seconds): 639.58
        Percent of CPU this job got: 4470%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:07.46
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 190345548
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 610
        Minor (reclaiming a frame) page faults: 49975464
        Voluntary context switches: 5827887
        Involuntary context switches: 126963
        Swaps: 0
        File system inputs: 0
        File system outputs: 316336
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

New code:

        User time (seconds): 4948.85
        System time (seconds): 598.59
        Percent of CPU this job got: 4504%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:03.14
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 183879896
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 609
        Minor (reclaiming a frame) page faults: 48070053
        Voluntary context switches: 5578223
        Involuntary context switches: 134820
        Swaps: 0
        File system inputs: 0
        File system outputs: 314456
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
        
        User time (seconds): 4746.86
        System time (seconds): 751.61
        Percent of CPU this job got: 4382%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:05.46
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 195039308
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 573
        Minor (reclaiming a frame) page faults: 51457263
        Voluntary context switches: 6213125
        Involuntary context switches: 128425
        Swaps: 0
        File system inputs: 0
        File system outputs: 303904
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

raggi avatar May 15 '24 19:05 raggi