How do I get a site's TLS certificate?

[joshuaherrera]

I am trying to get the TLS certificate that a site is presenting during the TLS handshake. I looked through the documentation and the response object but did not find what I was looking for.

According to the docs, I can customize some http options by changing the default HTTP roundtripper. I tried setting custom GetCertificate and GetClientCertificate functions, assuming that these functions would be used during the TLS handshake, but the print statements are never called.

    // Instantiate default collector
    c := colly.NewCollector(
        // Visit only domains: hackerspaces.org, wiki.hackerspaces.org
        colly.AllowedDomains("pkg.go.dev"),
    )

    c.WithTransport(&http.Transport{
        TLSClientConfig: &tls.Config{
            GetCertificate: func(ch *tls.ClientHelloInfo) (*tls.Certificate, error) {
                fmt.Println("~~~GETCERT CALLED~~")
                return nil, nil
            },
            GetClientCertificate: func(cri *tls.CertificateRequestInfo) (*tls.Certificate, error) {
                fmt.Println("~~~GETCLIENTCERT CALLED~~")
                return nil, nil
            },
        },
    })

How would I get the TLS certificate using Colly?

versions: $ go list -m github.com/gocolly/colly/v2 github.com/gocolly/colly/v2 v2.1.0

$ go version go version go1.18.3 darwin/arm64

[hugokung]

I think this is because the GetCertificate property or GetClientCertificate property is only called when the server needs a certificate from the client. You can use the VerifyPeerCertificate property to try to get the server-side certificate.