[THREESCALE-10981] - Update profiling tools

[tkan145]

What

https://issues.redhat.com/browse/THREESCALE-10981

Verification steps:

• Install Vagrant https://developer.hashicorp.com/vagrant/docs/installation
• Install QEMU
• Install vagrant-libvirt
• Start vagrant VM

vagrant up --provider=libvirt

• Test systemtap is installed correctly

stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'

The above command should run without any error.

Pass 1: parsed user script and 488 library scripts using 303940virt/97700res/17212shr/82296data k
b, in 100usr/20sys/119real ms.
Pass 2: analyzed script: 2 probes, 1 function, 5 embeds, 0 globals using 551508virt/346848res/189
36shr/329864data kb, in 1590usr/180sys/1806real ms.
Pass 3: translated to C into "/tmp/stap7kmO08/stap_5af05774dfd7657d11e5162b856c54db_2791_src.c" u
sing 551508virt/346976res/19064shr/329864data kb, in 0usr/0sys/5real ms.
Pass 4: compiled C into "stap_5af05774dfd7657d11e5162b856c54db_2791.ko" in 1500usr/390sys/1749rea
l ms.
Pass 5: starting run.
read performed
Pass 5: run completed in 10usr/20sys/1045real ms.

• Get into APIcast directory

cd /opt/app-root

• Create a apicast-config.json file with the following content

cat <<EOF >apicast-config.json
{
    "services": [
        {
            "id": "1",
            "backend_version": "1",
            "proxy": {
                "hosts": [
                    "one"
                ],
                "api_backend": "http://my-upstream:8080/response-headers",
                "backend": {
                    "endpoint": "http://127.0.0.1:8081",
                    "host": "backend"
                },
                "policy_chain": [
                    {
                        "name": "apicast.policy.apicast"
                    }
                ],
                "proxy_rules": [
                    {
                        "http_method": "GET",
                        "pattern": "/",
                        "metric_system_name": "hits",
                        "delta": 1,
                        "parameters": [],
                        "querystring_parameters": {}
                    }
                ]
            }
        }
    ]
} 
EOF

• Run apicast locally

THREESCALE_DEPLOYMENT_ENV=staging APICAST_LOG_LEVEL=debug APICAST_WORKER=1 APICAST_CONFIGURATION_LOADER=lazy APICAST_CONFIGURATION_CACHE=0 THREESCALE_CONFIG_FILE=apicast-config.json ./bin/apicast

• Open another terminal

vagrant ssh

• Get nginx worker pid

[vagrant@localhost samples]$ ps aux | grep nginx
vagrant     5250  0.0  0.6 183716  6828 pts/0    S+   07:11   0:00 nginx: master process openresty -c /tmp/lua_Ve29ck -g error_log stderr debug; -p /tmp/lua_Pwe8RF
vagrant     5255  0.0  2.1 191936 21128 pts/0    S+   07:11   0:00 nginx: worker process
vagrant     5256  0.0  0.6 175776  6192 pts/0    S+   07:11   0:00 nginx: cache manager process
vagrant     9249  0.0  0.1  12216  1116 pts/2    S+   07:40   0:00 grep --color=auto nginx

• Run stap script with pid of nginx worker

[vagrant@localhost ~]$ ngx-lua-useless-cosockets.sxx -x 5255
Tracing process 5255 (/usr/local/openresty/nginx/sbin/nginx)
Hit Ctrl-C to end.

• Open another terminal and send request

vagrant ssh
[vagrant@localhost]$ curl -H "Host: one" localhost:8080/?user_key=

In the systemtap terminal, press Ctrl-C to end the probe. You should see something like the following

Found 0 unused ngx_lua TCP cosockets.

[tkan145]

Thanks @dfennessy :smile:

[eguzki]

verification steps working, I was able to have vagrant up and running

[eguzki]

Is it possible to run those profiling tools in the existing docker compose environment? This vagrant seems like yet another environment to maintain (regarding deps). Fair enough if it is the only env where profiling tools can run.

[tkan145]

It's a bit tricky to get systemtap working inside a Docker container, last time I tried it required matching kernel version between host and container.

I'll try again tomorrow, if it still doesn't work then I'll merge this

[tkan145]

Running with Docker requires a matching kernel between the Docker image and the host. So I will merge this now and we can improve with future commit