cryostat-legacy icon indicating copy to clipboard operation
cryostat-legacy copied to clipboard

Published 20 hours ago • verified publisher icon cryostatio

Revitalize upstream documentation

Open jiekang opened this issue 3 years ago • 14 comments

The upstream documentation (README.md, website, etc.) could use another pass over to:

  • link more user-relevant information

  • review and re-organize existing information

  • standardize format among the cryostatio org repositories

  • [ ] Update cryostat developer documentation https://github.com/cryostatio/cryostat/pull/669

  • [ ] Update cryostat-web developer documentation https://github.com/cryostatio/cryostat-web/pull/269

jiekang avatar Aug 27 '21 20:08 jiekang

I've just tried to install cryostat on podman and then k8s and have to say... all documentation from cryostat or redhat is out of date and doesn't work at all.

Installation with kubeclt https://cryostat.io/get-started/ doesnt work at it has wrong URL in k apply command - and it's for 1.0.0

bash smoketest.sh in this repo fails to start pod with:

Exception in thread "main" java.lang.RuntimeException: java.nio.file.AccessDeniedException: /opt/cryostat.d/conf.d/credentials

same error for 2.0.0, latest and 1.0.0

and the same here https://developers.redhat.com/blog/2021/01/25/introduction-to-containerjfr-jdk-flight-recorder-for-containers#deploying_cryostat_on_red_hat_openshift

podman run -it --rm -p 8181 -e CRYOSTAT_WEB_HOST=0.0.0.0 quay.io/cryostat/cryostat:latest

fails with:

Caused by: java.nio.file.NoSuchFileException: /opt/cryostat.d/conf.d/credentials
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
        at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
        at java.base/java.nio.file.Files.createDirectory(Files.java:690)
        at io.cryostat.configuration.ConfigurationModule.provideCredentialsManager(ConfigurationModule.java:82)
        ... 6 more

agilob avatar Oct 26 '21 19:10 agilob

Thanks for the comment @agilob . The problem you report with bash smoketest.sh sounds like #707 - looks like that needs to be re-opened and looked at again. I'll take another look at it.

The podman run ... line you have is out of date by now from that older blog post - there should be some additional environment variables set and volumes mounted to get it to run that way.

As for the installation doc on cryostat.io - yes, we're still behind on getting some of this documentation up-to-date for the new 2.0 release. I'll make a note and prioritize getting the version numbers, URLs, etc. updated on there. Thanks for the heads up.

andrewazores avatar Oct 26 '21 19:10 andrewazores

Full log is:

Running quay.io/cryostat/cryostat:latest ...


+ '[' -z '' ']'
+ CRYOSTAT_RJMX_PORT=9091
+ '[' -z '' ']'
+ CRYOSTAT_RMI_PORT=9091
+ '[' -z '' ']'
+ CRYOSTAT_WEB_HOST=0.0.0.0
+ '[' -z '' ']'
+ CRYOSTAT_WEB_PORT=8181
+ '[' -z '' ']'
+ CRYOSTAT_EXT_WEB_PORT=8181
+ '[' -z '' ']'
+ CRYOSTAT_AUTH_MANAGER=io.cryostat.net.NoopAuthManager
+ '[' -z '' ']'
+ CRYOSTAT_REPORT_GENERATION_MAX_HEAP=200
+ '[' -z '' ']'
++ dirname /home/agilob/Projects/cryostat/run.sh
+ '[' -f /home/agilob/Projects/cryostat/certs/cryostat-keystore.p12 ']'
++ dirname /home/agilob/Projects/cryostat/run.sh
+ '[' '!' -d /home/agilob/Projects/cryostat/conf ']'
++ dirname /home/agilob/Projects/cryostat/run.sh
+ '[' '!' -d /home/agilob/Projects/cryostat/truststore ']'
++ dirname /home/agilob/Projects/cryostat/run.sh
+ '[' '!' -d /home/agilob/Projects/cryostat/clientlib ']'
+ podman pod exists cryostat
++ dirname /home/agilob/Projects/cryostat/run.sh
++ dirname /home/agilob/Projects/cryostat/run.sh
++ dirname /home/agilob/Projects/cryostat/run.sh
++ dirname /home/agilob/Projects/cryostat/run.sh
+ podman run --pod cryostat --memory 512M --mount type=tmpfs,target=/opt/cryostat.d/recordings.d --mount type=tmpfs,target=/opt/cryostat.d/templates.d --mount type=bind,source=/home/agilob/Projects/cryostat/conf,destination=/opt/cryostat.d/conf.d,relabel=shared,bind-propagation=shared --mount type=bind,source=/home/agilob/Projects/cryostat/truststore,destination=/truststore,relabel=shared,bind-propagation=shared --mount type=bind,source=/home/agilob/Projects/cryostat/certs,destination=/certs,relabel=shared,bind-propagation=shared --mount type=bind,source=/home/agilob/Projects/cryostat/clientlib,destination=/clientlib,relabel=shared,bind-propagation=shared -e CRYOSTAT_PLATFORM= -e CRYOSTAT_DISABLE_SSL= -e CRYOSTAT_DISABLE_JMX_AUTH= -e CRYOSTAT_RJMX_USER=smoketest -e CRYOSTAT_RJMX_PASS=smoketest -e CRYOSTAT_RJMX_PORT=9091 -e CRYOSTAT_RMI_PORT=9091 -e CRYOSTAT_CORS_ORIGIN= -e CRYOSTAT_WEB_HOST=0.0.0.0 -e CRYOSTAT_WEB_PORT=8181 -e CRYOSTAT_EXT_WEB_PORT=8181 -e CRYOSTAT_AUTH_MANAGER=io.cryostat.net.NoopAuthManager -e CRYOSTAT_TARGET_CACHE_SIZE= -e CRYOSTAT_TARGET_CACHE_TTL= -e CRYOSTAT_CONFIG_PATH=/opt/cryostat.d/conf.d -e CRYOSTAT_ARCHIVE_PATH=/opt/cryostat.d/recordings.d -e CRYOSTAT_TEMPLATE_PATH=/opt/cryostat.d/templates.d -e CRYOSTAT_CLIENTLIB_PATH=/clientlib -e CRYOSTAT_REPORT_GENERATION_MAX_HEAP=200 -e GRAFANA_DATASOURCE_URL=http://0.0.0.0:8080 -e GRAFANA_DASHBOARD_URL=http://0.0.0.0:3000 -e KEYSTORE_PATH= -e KEYSTORE_PASS= -e KEY_PATH= -e CERT_PATH= -e CRYOSTAT_JUL_CONFIG= --rm -it quay.io/cryostat/cryostat:latest
+------------------------------------------+
| Tue Oct 26 19:49:27 UTC 2021             |
|                                          |
| /truststore is empty; no certificates to import |
+------------------------------------------+
/tmp ~
Generating 2,048 bit RSA key pair and self-signed certificate (SHA256withRSA) with a validity of 180 days
        for: CN=cryostat, O=Cryostat, C=CA
[Storing /opt/cryostat.d/keystore.p12]
Certificate stored in file <server.cer>
Certificate was added to keystore
[Storing /opt/cryostat.d/truststore.p12]
~
+------------------------------------------+
| Tue Oct 26 19:49:34 UTC 2021             |
|                                          |
| Using self-signed SSL certificate        |
+------------------------------------------+
+ exec java -XX:+CrashOnOutOfMemoryError -Dcom.sun.management.jmxremote.autodiscovery=true -Dcom.sun.management.jmxremote.port=9091 -Dcom.sun.management.jmxremote.rmi.port=9091 -Djavax.net.ssl.trustStore=/opt/cryostat.d/truststore.p12 -Djavax.net.ssl.trustStorePassword=SgtmEEmXCqOyAup-Egt59MyOUJ7S0RDa -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.password.file=/tmp/jmxremote.password -Dcom.sun.management.jmxremote.access.file=/tmp/jmxremote.access -Dcom.sun.management.jmxremote.ssl.need.client.auth=true -Djavax.net.ssl.keyStore=/opt/cryostat.d/keystore.p12 -Djavax.net.ssl.keyStorePassword=p7_1L75MugAAD7TH7AGpbM0QYr3JPax_ -Dcom.sun.management.jmxremote.ssl=true -Dcom.sun.management.jmxremote.registry.ssl=true -cp '/app/resources:/app/classes:/app/libs/cryostat-core-2.3.1.jar:/app/libs/common-7.1.1.jar:/app/libs/encoder-1.2.2.jar:/app/libs/flightrecorder-7.1.1.jar:/app/libs/flightrecorder.rules-7.1.1.jar:/app/libs/flightrecorder.rules.jdk-7.1.1.jar:/app/libs/openshift-client-5.4.1.jar:/app/libs/openshift-model-5.4.1.jar:/app/libs/kubernetes-model-common-5.4.1.jar:/app/libs/jackson-annotations-2.11.2.jar:/app/libs/openshift-model-operator-5.4.1.jar:/app/libs/openshift-model-operatorhub-5.4.1.jar:/app/libs/openshift-model-monitoring-5.4.1.jar:/app/libs/openshift-model-console-5.4.1.jar:/app/libs/kubernetes-client-5.4.1.jar:/app/libs/kubernetes-model-core-5.4.1.jar:/app/libs/kubernetes-model-rbac-5.4.1.jar:/app/libs/kubernetes-model-admissionregistration-5.4.1.jar:/app/libs/kubernetes-model-apps-5.4.1.jar:/app/libs/kubernetes-model-autoscaling-5.4.1.jar:/app/libs/kubernetes-model-apiextensions-5.4.1.jar:/app/libs/kubernetes-model-batch-5.4.1.jar:/app/libs/kubernetes-model-certificates-5.4.1.jar:/app/libs/kubernetes-model-coordination-5.4.1.jar:/app/libs/kubernetes-model-discovery-5.4.1.jar:/app/libs/kubernetes-model-events-5.4.1.jar:/app/libs/kubernetes-model-extensions-5.4.1.jar:/app/libs/kubernetes-model-flowcontrol-5.4.1.jar:/app/libs/kubernetes-model-networking-5.4.1.jar:/app/libs/kubernetes-model-metrics-5.4.1.jar:/app/libs/kubernetes-model-policy-5.4.1.jar:/app/libs/kubernetes-model-scheduling-5.4.1.jar:/app/libs/kubernetes-model-storageclass-5.4.1.jar:/app/libs/kubernetes-model-node-5.4.1.jar:/app/libs/okhttp-3.12.12.jar:/app/libs/okio-1.15.0.jar:/app/libs/logging-interceptor-3.12.12.jar:/app/libs/slf4j-api-1.7.30.jar:/app/libs/jackson-dataformat-yaml-2.11.2.jar:/app/libs/snakeyaml-1.26.jar:/app/libs/jackson-datatype-jsr310-2.11.2.jar:/app/libs/jackson-databind-2.11.2.jar:/app/libs/jackson-core-2.11.2.jar:/app/libs/zjsonpatch-0.3.0.jar:/app/libs/generex-1.0.2.jar:/app/libs/automaton-1.11-8.jar:/app/libs/dagger-2.34.1.jar:/app/libs/javax.inject-1.jar:/app/libs/commons-lang3-3.12.0.jar:/app/libs/commons-codec-1.15.jar:/app/libs/commons-io-2.8.0.jar:/app/libs/commons-validator-1.7.jar:/app/libs/commons-beanutils-1.9.4.jar:/app/libs/commons-digester-2.1.jar:/app/libs/commons-logging-1.2.jar:/app/libs/commons-collections-3.2.2.jar:/app/libs/httpclient-4.5.13.jar:/app/libs/httpcore-4.4.13.jar:/app/libs/vertx-web-3.9.7.jar:/app/libs/vertx-web-common-3.9.7.jar:/app/libs/vertx-auth-common-3.9.7.jar:/app/libs/vertx-bridge-common-3.9.7.jar:/app/libs/vertx-core-3.9.7.jar:/app/libs/netty-common-4.1.60.Final.jar:/app/libs/netty-buffer-4.1.60.Final.jar:/app/libs/netty-transport-4.1.60.Final.jar:/app/libs/netty-handler-4.1.60.Final.jar:/app/libs/netty-codec-4.1.60.Final.jar:/app/libs/netty-handler-proxy-4.1.60.Final.jar:/app/libs/netty-codec-socks-4.1.60.Final.jar:/app/libs/netty-codec-http-4.1.60.Final.jar:/app/libs/netty-codec-http2-4.1.60.Final.jar:/app/libs/netty-resolver-4.1.60.Final.jar:/app/libs/netty-resolver-dns-4.1.60.Final.jar:/app/libs/netty-codec-dns-4.1.60.Final.jar:/app/libs/vertx-web-client-3.9.7.jar:/app/libs/slf4j-jdk14-1.7.30.jar:/app/libs/gson-2.8.6.jar:/app/libs/caffeine-3.0.1.jar:/app/libs/jsoup-1.14.2.jar:/clientlib/*' @/app/jib-main-class-file
Oct 26, 2021 7:49:37 PM io.cryostat.core.log.Logger info
INFO: cryostat started.
Oct 26, 2021 7:49:38 PM io.cryostat.core.log.Logger info
INFO: Selected SSL KeyStore strategy with keystore /opt/cryostat.d/keystore.p12
Oct 26, 2021 7:49:38 PM io.cryostat.core.log.Logger info
INFO: Local config path set as /opt/cryostat.d/conf.d
Exception in thread "main" java.lang.RuntimeException: java.nio.file.AccessDeniedException: /opt/cryostat.d/conf.d/credentials
        at io.cryostat.configuration.ConfigurationModule.provideCredentialsManager(ConfigurationModule.java:92)
        at io.cryostat.configuration.ConfigurationModule_ProvideCredentialsManagerFactory.provideCredentialsManager(ConfigurationModule_ProvideCredentialsManagerFactory.java:51)
        at io.cryostat.configuration.ConfigurationModule_ProvideCredentialsManagerFactory.get(ConfigurationModule_ProvideCredentialsManagerFactory.java:40)
        at io.cryostat.configuration.ConfigurationModule_ProvideCredentialsManagerFactory.get(ConfigurationModule_ProvideCredentialsManagerFactory.java:12)
        at dagger.internal.DoubleCheck.get(DoubleCheck.java:47)
        at io.cryostat.DaggerCryostat_Client.credentialsManager(DaggerCryostat_Client.java:595)
        at io.cryostat.Cryostat.main(Cryostat.java:74)
Caused by: java.nio.file.AccessDeniedException: /opt/cryostat.d/conf.d/credentials
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
        at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
        at java.base/java.nio.file.Files.createDirectory(Files.java:690)
        at io.cryostat.configuration.ConfigurationModule.provideCredentialsManager(ConfigurationModule.java:82)
        ... 6 more
+ cleanup

agilob avatar Oct 26 '21 20:10 agilob

@agilob could you try checking out my PR branch from #708 ? The change there is simply to the run.sh script, which smoketest.sh also invokes, so there is no need to build anything. Just checking out the branch and doing your bash smoketest.sh should be sufficient to confirm if the fix applied there works for you.

andrewazores avatar Oct 26 '21 20:10 andrewazores

It starts from your branch using bash run.sh. It opens port for https server with self-signed cert

agilob avatar Oct 26 '21 20:10 agilob

Installation with kubeclt https://cryostat.io/get-started/ doesnt work at it has wrong URL in k apply command - and it's for 1.0.0

Hi @agilob, is there a problem with: kubectl apply -k github.com/cryostatio/cryostat-operator//config/default?ref=v1.0.0 aside from it being for the 1.0.0 release?

ebaron avatar Oct 26 '21 20:10 ebaron

@ebaron oh interesting, didn't know it. This command doesn't work in zsh, but works in bash.

To make it work on zsh: kubectl apply -d 'github.com/cryostatio/cryostat-operator//config/default?ref=v1.0.0' url needs to be in quotes

agilob avatar Oct 26 '21 20:10 agilob

@andrewazores what's the username and pass in webui?

agilob avatar Oct 26 '21 20:10 agilob

@ebaron oh interesting, didn't know it. This command doesn't work in zsh, but works in bash.

To make it work on zsh: kubectl apply -d 'github.com/cryostatio/cryostat-operator//config/default?ref=v1.0.0' url needs to be in quotes

Good to know. I mainly use Bash, so I hadn't encountered this before. We'll be sure to add quotes to it.

ebaron avatar Oct 26 '21 20:10 ebaron

@andrewazores what's the username and pass in webui?

@agilob If you're using smoketest.sh or run.sh and it's asking for your Basic auth credentials, you can just leave the fields blank and submit no credentials. Some of the target JVMs you select may request their own JMX auth credentials, which will vary depending on which JVM you're trying to connect to.

andrewazores avatar Oct 26 '21 20:10 andrewazores

run.sh and it's asking for your Basic auth credentials, you can just leave the fields blank and submit no credentials

I mean this part: image

I can visit upload recording page without authentication, but then I cant browse archive of recordings

agilob avatar Oct 26 '21 20:10 agilob

haha also when I upload .jfr that's 120Mb pods crash and I have to restart them manually :dagger:

Oct 26, 2021 8:33:03 PM io.cryostat.core.log.Logger error
SEVERE: HTTP 500: Internal Server Error
io.vertx.ext.web.handler.impl.HttpStatusException: Internal Server Error
Oct 26, 2021 8:33:03 PM io.cryostat.core.log.Logger info
INFO: (10.0.2.100:38292): POST /api/v1/recordings 500 30279ms <=== look here
+ cleanup
+ podman pod kill cryostat
ea9bf41503147c4666fd08dfb70eb4072d2619190941229923a4a7eb145d8d9a
+ podman pod rm cryostat
ea9bf41503147c4666fd08dfb70eb4072d2619190941229923a4a7eb145d8d9a

agilob avatar Oct 26 '21 20:10 agilob

@agilob okay, that's asking for JMX credentials, so it depends on which target you have selected in the target dropdown.

If the target is Cryostat itself, then using run.sh the password is auto-generated and not easy to retrieve. If you're using smoketest.sh then the credentials are overrided by env vars (https://github.com/cryostatio/cryostat/blob/main/smoketest.sh#L10).

If the target is a vertx-fib-demo (es.andrewazor.demo.Main) then the credentials are defined here: https://github.com/andrewazores/vertx-fib-demo/blob/master/src/main/extras/app/resources/jmxremote.password

andrewazores avatar Oct 26 '21 20:10 andrewazores

haha also when I upload .jfr that's 120Mb pods crash and I have to restart them manually dagger

Oct 26, 2021 8:33:03 PM io.cryostat.core.log.Logger error
SEVERE: HTTP 500: Internal Server Error
io.vertx.ext.web.handler.impl.HttpStatusException: Internal Server Error
Oct 26, 2021 8:33:03 PM io.cryostat.core.log.Logger info
INFO: (10.0.2.100:38292): POST /api/v1/recordings 500 30279ms <=== look here
+ cleanup
+ podman pod kill cryostat
ea9bf41503147c4666fd08dfb70eb4072d2619190941229923a4a7eb145d8d9a
+ podman pod rm cryostat
ea9bf41503147c4666fd08dfb70eb4072d2619190941229923a4a7eb145d8d9a

run.sh by default only assigns 512MB to the Cryostat pod, so if you're dealing with very large JFR binaries you may wish to edit that script and allocate more memory.

andrewazores avatar Oct 26 '21 20:10 andrewazores