appbundler icon indicating copy to clipboard operation
appbundler copied to clipboard

More efficient library loading

Open adsr opened this issue 11 months ago • 0 comments

Describe the Enhancement:

Hello, I use knife regularly and notice that some commands that I'd expect to complete instantly actually run pretty slowly. For example, I'd expect a command like knife config use ... to take a handful of millis at most, but in reality it takes nearly a second to run on my setup:

$ chef version
Chef Workstation version: 23.7.1042
Test Kitchen version: 3.5.0
Cookstyle version: 7.32.2
Chef Infra Client version: 18.2.7
Chef InSpec version: 5.22.3
Chef CLI version: 5.6.12
Chef Habitat version: 1.6.652
$ knife --version
Chef Infra Client: 18.2.7
$ time knife config use default
Set default profile to default

real	0m0.816s
user	0m0.638s
sys	0m0.176s

I dug into this and found that the bootstrap process in knife (and I presume other Chef tools) is very inefficient:

$ strace -c knife config use default 2>&1 | head
Set default profile to default
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 56.32    0.062971           1     51575     47079 openat
 12.19    0.013630           5      2496           getdents64
  8.48    0.009485           0     11635     11635 readlink
  5.94    0.006646           1      4496           close
  5.65    0.006313           0      6938      4923 stat
  2.53    0.002824           0      4030           read
  1.53    0.001712           2       827           munmap
$ strace -f -e trace=openat -o out knife config use default 
Set default profile to default
$ grep -c ENOENT out 
47079
$ grep -c ENOENT -v out 
4497

Inspecting the trace file, I notice that each require '...' statement after this block checks up to ~400 non-existent filepaths before it finds the requested library. (I confirmed this by monkey-patching require.)

I don't have a lot of Ruby experience but I assume there's a more efficient way to handle library loading. Is require_relative more efficient?

(Apologies if this is the wrong repo to file under. I'm filing here because it seems appbundler is the thing that outputs the long list of gems above.)

Describe the Need:

Perf

Current Alternative

Do nothing

Can We Help You Implement This?:

If someone has an idea for how to improve this, I'd be happy to help implement.

adsr avatar Jan 10 '25 18:01 adsr