zipkin-gcp icon indicating copy to clipboard operation
zipkin-gcp copied to clipboard

Label.key too long

Open JorritSalverda opened this issue 6 years ago • 14 comments

I'm running istio 0.4.0 and send all traces to the zipkin-to-stackdriver deployment running container gcr.io/stackdriver-trace-docker/zipkin-collector:v0.3.0.

When I then run the bookinfo sample the zipkin-to-stackdriver application keeps throwing errors like the following:

zipkin-to-stackdriver-74866c5cd9-s7b78 zipkin-to-stackdriver 2017-12-20 13:30:40.704  WARN 8 --- [driverStorage-1] zipkin.server.ZipkinHttpCollector        : Cannot store spans [13b58682a43a952f.13b58682a43a952f<:13b58682a43a952f, 7c0d8a04309058d1.7f5ed293f891c6e9<:5f472c38c35e1e8b, 7c0d8a04309058d1.1965565a42a9b299<:7c0d8a04309058d1, 7c0d8a04309058d1.238eb23603b1992a<:7c0d8a04309058d1, 7c0d8a04309058d1.5f472c38c35e1e8b<:7c0d8a04309058d1, 7c0d8a04309058d1.7c0d8a04309058d1<:7c0d8a04309058d1, 5f5a0c322c213e20.31d4affd593c5ceb<:7f578ce3cfe96c6c, 5f5a0c322c213e20.386db219f31a9e00<:5f5a0c322c213e20, 5f5a0c322c213e20.7db2b757b548762f<:5f5a0c322c213e20, 5f5a0c322c213e20.7f578ce3cfe96c6c<:5f5a0c322c213e20, 5f5a0c322c213e20.5f5a0c322c213e20<:5f5a0c322c213e20, 223508843e90e31a.24b34b78807b3118<:223508843e90e31a, 223508843e90e31a.223508843e90e31a<:223508843e90e31a, 07e821b6551e692a.0583947468ee72a8<:07e821b6551e692a, 07e821b6551e692a.07e821b6551e692a<:07e821b6551e692a, 47a33573140bd7a7.755cec4744707cbf<:47a33573140bd7a7, 47a33573140bd7a7.47a33573140bd7a7<:47a33573140bd7a7, 7cde1c8e41f432fc.2a403e0949d626e9<:7cde1c8e41f432fc, 7cde1c8e41f432fc.7cde1c8e41f432fc<:7cde1c8e41f432fc, 2b2e6869b685f79d.6a59b7b637487e54<:302990ef143f2473, 2b2e6869b685f79d.173250b89df0307c<:2b2e6869b685f79d, 2b2e6869b685f79d.1ef844a83abff6fc<:2b2e6869b685f79d, 2b2e6869b685f79d.302990ef143f2473<:2b2e6869b685f79d, 2b2e6869b685f79d.035b3f9d8307c66b<:165d3637ba428305, 2b2e6869b685f79d.2ba0968f11eb9c6d<:2b2e6869b685f79d, 2b2e6869b685f79d.05d2db6e1ea50b44<:2b2e6869b685f79d, 2b2e6869b685f79d.165d3637ba428305<:2b2e6869b685f79d, 2b2e6869b685f79d.0512c6e5ef74083f<:719ddc4aa227995d, 2b2e6869b685f79d.1d813bf86cb57635<:2b2e6869b685f79d, 2b2e6869b685f79d.37d7217e907ac09b<:2b2e6869b685f79d, 2b2e6869b685f79d.719ddc4aa227995d<:2b2e6869b685f79d, 2b2e6869b685f79d.012ce8036813410c<:6e780125ea917718, 2b2e6869b685f79d.2b13555068fe0c18<:2b2e6869b685f79d, 2b2e6869b685f79d.4120e994016cb628<:2b2e6869b685f79d, 2b2e6869b685f79d.6e780125ea917718<:2b2e6869b685f79d, 2b2e6869b685f79d.2b2e6869b685f79d<:2b2e6869b685f79d, 086820b0c562fa14.405214a8613b8db9<:726b01ad4eb0d4c9, 086820b0c562fa14.1333daeacda8ab6d<:086820b0c562fa14, 086820b0c562fa14.0ffef8a18531f40f<:086820b0c562fa14, 086820b0c562fa14.726b01ad4eb0d4c9<:086820b0c562fa14, 5eea61f97d7e08c4.5fd1416de1b4ccad<:16c0253cac27df00, 5eea61f97d7e08c4.0df93364cf3a9e79<:5eea61f97d7e08c4, 5eea61f97d7e08c4.5c98e42860b5a95b<:5eea61f97d7e08c4, 5eea61f97d7e08c4.16c0253cac27df00<:5eea61f97d7e08c4, 5eea61f97d7e08c4.5eea61f97d7e08c4<:5eea61f97d7e08c4, 086820b0c562fa14.086820b0c562fa14<:086820b0c562fa14, 162335cb62ad1242.55266f4d1e8a8af5<:633c323e889a6cc8, 162335cb62ad1242.2f1e490db06154a8<:162335cb62ad1242, 162335cb62ad1242.179f42f0417f28b3<:162335cb62ad1242, 162335cb62ad1242.633c323e889a6cc8<:162335cb62ad1242, 162335cb62ad1242.162335cb62ad1242<:162335cb62ad1242, 083bd2a9143e6144.1578c7f27d3241ef<:546df37fc49ff6b1, 083bd2a9143e6144.78c3c170b92d5987<:083bd2a9143e6144, 083bd2a9143e6144.1060797712da5ef6<:083bd2a9143e6144, 083bd2a9143e6144.546df37fc49ff6b1<:083bd2a9143e6144, 083bd2a9143e6144.083bd2a9143e6144<:083bd2a9143e6144, 72f3388bae2c2dc1.5ca9c1eae6f07320<:5aab96fe8d3bc1eb, 72f3388bae2c2dc1.080710a528c4006d<:72f3388bae2c2dc1, 72f3388bae2c2dc1.4ef01eb624ccb4e8<:72f3388bae2c2dc1, 72f3388bae2c2dc1.5aab96fe8d3bc1eb<:72f3388bae2c2dc1, 72f3388bae2c2dc1.72f3388bae2c2dc1<:72f3388bae2c2dc1, 6adfbda545b64620.08f3824bd7a341c2<:46c0c3ad7a374112, 2e65a5f873cb13c3.0486cfb5c813309f<:416810f450c7a736, 2e65a5f873cb13c3.1f26e91705e6dbd2<:2e65a5f873cb13c3, 2e65a5f873cb13c3.7447d0cafa651281<:2e65a5f873cb13c3, 2e65a5f873cb13c3.416810f450c7a736<:2e65a5f873cb13c3, 2e65a5f873cb13c3.2e65a5f873cb13c3<:2e65a5f873cb13c3, 6adfbda545b64620.13aa8e63caabcc0a<:6adfbda545b64620, 6adfbda545b64620.67362a4bf25d6781<:6adfbda545b64620, 6adfbda545b64620.46c0c3ad7a374112<:6adfbda545b64620, 6adfbda545b64620.6adfbda545b64620<:6adfbda545b64620, 464b31471cd8db4b.746809e45bfcf2cc<:4792cbd9690ac695, 464b31471cd8db4b.574b7d31a048ae9f<:464b31471cd8db4b, 464b31471cd8db4b.006fa0ab35926d4f<:464b31471cd8db4b, 464b31471cd8db4b.4792cbd9690ac695<:464b31471cd8db4b, 464b31471cd8db4b.464b31471cd8db4b<:464b31471cd8db4b, 3c48f39fbf75d312.4626328f960d3dc7<:4c66fe25730044ab, 3c48f39fbf75d312.7ef663915dc80c23<:3c48f39fbf75d312, 3c48f39fbf75d312.00cfce66f86a0994<:3c48f39fbf75d312, 3c48f39fbf75d312.4c66fe25730044ab<:3c48f39fbf75d312, 3c48f39fbf75d312.3c48f39fbf75d312<:3c48f39fbf75d312] due to StatusRuntimeException(INVALID_ARGUMENT: Label.key in TraceSpan.span_id 397324419836310184 with length 171 exceeds maximum permitted length.)
zipkin-to-stackdriver-74866c5cd9-s7b78 zipkin-to-stackdriver
zipkin-to-stackdriver-74866c5cd9-s7b78 zipkin-to-stackdriver io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Label.key in TraceSpan.span_id 397324419836310184 with length 171 exceeds maximum permitted length.

When sending it to zipkin instead all works fine. I this an issue in this application?

JorritSalverda avatar Dec 20 '17 13:12 JorritSalverda

seems like this sort of error should drop the tag vs kill the span?

On Wed, Dec 20, 2017 at 9:51 PM, Jorrit Salverda [email protected] wrote:

I'm running istio 0.4.0 and send all traces to the zipkin-to-stackdriver deployment running container gcr.io/stackdriver-trace- docker/zipkin-collector:v0.3.0.

When I then run the bookinfo sample the zipkin-to-stackdriver application keeps throwing errors like the following:

zipkin-to-stackdriver-74866c5cd9-s7b78 zipkin-to-stackdriver 2017-12-20 13:30:40.704 WARN 8 --- [driverStorage-1] zipkin.server.ZipkinHttpCollector : Cannot store spans [13b58682a43a952f.13b58682a43a952f<:13b58682a43a952f, 7c0d8a04309058d1.7f5ed293f891c6e9<:5f472c38c35e1e8b, 7c0d8a04309058d1.1965565a42a9b299<:7c0d8a04309058d1, 7c0d8a04309058d1.238eb23603b1992a<:7c0d8a04309058d1, 7c0d8a04309058d1.5f472c38c35e1e8b<:7c0d8a04309058d1, 7c0d8a04309058d1.7c0d8a04309058d1<:7c0d8a04309058d1, 5f5a0c322c213e20.31d4affd593c5ceb<:7f578ce3cfe96c6c, 5f5a0c322c213e20.386db219f31a9e00<:5f5a0c322c213e20, 5f5a0c322c213e20.7db2b757b548762f<:5f5a0c322c213e20, 5f5a0c322c213e20.7f578ce3cfe96c6c<:5f5a0c322c213e20, 5f5a0c322c213e20.5f5a0c322c213e20<:5f5a0c322c213e20, 223508843e90e31a.24b34b78807b3118<:223508843e90e31a, 223508843e90e31a.223508843e90e31a<:223508843e90e31a, 07e821b6551e692a.0583947468ee72a8<:07e821b6551e692a, 07e821b6551e692a.07e821b6551e692a<:07e821b6551e692a, 47a33573140bd7a7.755cec4744707cbf<:47a33573140bd7a7, 47a33573140bd7a7.47a33573140bd7a7<:47a33573140bd7a7, 7cde1c8e41f432fc.2a403e0949d626e9<:7cde1c8e41f432fc, 7cde1c8e41f432fc.7cde1c8e41f432fc<:7cde1c8e41f432fc, 2b2e6869b685f79d.6a59b7b637487e54<:302990ef143f2473, 2b2e6869b685f79d.173250b89df0307c<:2b2e6869b685f79d, 2b2e6869b685f79d.1ef844a83abff6fc<:2b2e6869b685f79d, 2b2e6869b685f79d.302990ef143f2473<:2b2e6869b685f79d, 2b2e6869b685f79d.035b3f9d8307c66b<:165d3637ba428305, 2b2e6869b685f79d.2ba0968f11eb9c6d<:2b2e6869b685f79d, 2b2e6869b685f79d.05d2db6e1ea50b44<:2b2e6869b685f79d, 2b2e6869b685f79d.165d3637ba428305<:2b2e6869b685f79d, 2b2e6869b685f79d.0512c6e5ef74083f<:719ddc4aa227995d, 2b2e6869b685f79d.1d813bf86cb57635<:2b2e6869b685f79d, 2b2e6869b685f79d.37d7217e907ac09b<:2b2e6869b685f79d, 2b2e6869b685f79d.719ddc4aa227995d<:2b2e6869b685f79d, 2b2e6869b685f79d.012ce8036813410c<:6e780125ea917718, 2b2e6869b685f79d.2b13555068fe0c18<:2b2e6869b685f79d, 2b2e6869b685f79d.4120e994016cb628<:2b2e6869b685f79d, 2b2e6869b685f79d.6e780125ea917718<:2b2e6869b685f79d, 2b2e6869b685f79d.2b2e6869b685f79d<:2b2e6869b685f79d, 086820b0c562fa14.405214a8613b8db9<:726b01ad4eb0d4c9, 086820b0c562fa14.1333daeacda8ab6d<:086820b0c562fa14, 086820b0c562fa14.0ffef8a18531f40f<:086820b0c562fa14, 086820b0c562fa14.726b01ad4eb0d4c9<:086820b0c562fa14, 5eea61f97d7e08c4.5fd1416de1b4ccad<:16c0253cac27df00, 5eea61f97d7e08c4.0df93364cf3a9e79<:5eea61f97d7e08c4, 5eea61f97d7e08c4.5c98e42860b5a95b<:5eea61f97d7e08c4, 5eea61f97d7e08c4.16c0253cac27df00<:5eea61f97d7e08c4, 5eea61f97d7e08c4.5eea61f97d7e08c4<:5eea61f97d7e08c4, 086820b0c562fa14.086820b0c562fa14<:086820b0c562fa14, 162335cb62ad1242.55266f4d1e8a8af5<:633c323e889a6cc8, 162335cb62ad1242.2f1e490db06154a8<:162335cb62ad1242, 162335cb62ad1242.179f42f0417f28b3<:162335cb62ad1242, 162335cb62ad1242.633c323e889a6cc8<:162335cb62ad1242, 162335cb62ad1242.162335cb62ad1242<:162335cb62ad1242, 083bd2a9143e6144.1578c7f27d3241ef<:546df37fc49ff6b1, 083bd2a9143e6144.78c3c170b92d5987<:083bd2a9143e6144, 083bd2a9143e6144.1060797712da5ef6<:083bd2a9143e6144, 083bd2a9143e6144.546df37fc49ff6b1<:083bd2a9143e6144, 083bd2a9143e6144.083bd2a9143e6144<:083bd2a9143e6144, 72f3388bae2c2dc1.5ca9c1eae6f07320<:5aab96fe8d3bc1eb, 72f3388bae2c2dc1.080710a528c4006d<:72f3388bae2c2dc1, 72f3388bae2c2dc1.4ef01eb624ccb4e8<:72f3388bae2c2dc1, 72f3388bae2c2dc1.5aab96fe8d3bc1eb<:72f3388bae2c2dc1, 72f3388bae2c2dc1.72f3388bae2c2dc1<:72f3388bae2c2dc1, 6adfbda545b64620.08f3824bd7a341c2<:46c0c3ad7a374112, 2e65a5f873cb13c3.0486cfb5c813309f<:416810f450c7a736, 2e65a5f873cb13c3.1f26e91705e6dbd2<:2e65a5f873cb13c3, 2e65a5f873cb13c3.7447d0cafa651281<:2e65a5f873cb13c3, 2e65a5f873cb13c3.416810f450c7a736<:2e65a5f873cb13c3, 2e65a5f873cb13c3.2e65a5f873cb13c3<:2e65a5f873cb13c3, 6adfbda545b64620.13aa8e63caabcc0a<:6adfbda545b64620, 6adfbda545b64620.67362a4bf25d6781<:6adfbda545b64620, 6adfbda545b64620.46c0c3ad7a374112<:6adfbda545b64620, 6adfbda545b64620.6adfbda545b64620<:6adfbda545b64620, 464b31471cd8db4b.746809e45bfcf2cc<:4792cbd9690ac695, 464b31471cd8db4b.574b7d31a048ae9f<:464b31471cd8db4b, 464b31471cd8db4b.006fa0ab35926d4f<:464b31471cd8db4b, 464b31471cd8db4b.4792cbd9690ac695<:464b31471cd8db4b, 464b31471cd8db4b.464b31471cd8db4b<:464b31471cd8db4b, 3c48f39fbf75d312.4626328f960d3dc7<:4c66fe25730044ab, 3c48f39fbf75d312.7ef663915dc80c23<:3c48f39fbf75d312, 3c48f39fbf75d312.00cfce66f86a0994<:3c48f39fbf75d312, 3c48f39fbf75d312.4c66fe25730044ab<:3c48f39fbf75d312, 3c48f39fbf75d312.3c48f39fbf75d312<:3c48f39fbf75d312] due to StatusRuntimeException(INVALID_ARGUMENT: Label.key in TraceSpan.span_id 397324419836310184 with length 171 exceeds maximum permitted length.) zipkin-to-stackdriver-74866c5cd9-s7b78 zipkin-to-stackdriver zipkin-to-stackdriver-74866c5cd9-s7b78 zipkin-to-stackdriver io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Label.key in TraceSpan.span_id 397324419836310184 with length 171 exceeds maximum permitted length.

When sending it to zipkin instead all works fine. I this an issue in this application?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/stackdriver-zipkin/issues/49, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD618j6FbZUG8nXJ_3_jayE7C5urRM4ks5tCRDKgaJpZM4RIal5 .

codefromthecrypt avatar Dec 21 '17 00:12 codefromthecrypt

I'm no zipkin / tracing expert, so not sure what it should actually do :)

JorritSalverda avatar Dec 21 '17 08:12 JorritSalverda

sorry this note was for the code :)

codefromthecrypt avatar Dec 21 '17 08:12 codefromthecrypt

@JorritSalverda do you know if the issue goes away if we update to gcr.io/stackdriver-trace-docker/zipkin-collector:v0.4.0 ?

douglas-reid avatar Jan 23 '18 19:01 douglas-reid

@JorritSalverda is this still a issue for you? I was able to collect tracing of sample bookinfo app through zipkin-to-stackdriver without any error logs. I ran it with istio-0.5.0 and zipkin-collector:v0.4.0.

bianpengyuan avatar Feb 09 '18 07:02 bianpengyuan

I had trouble to get istio and the bookinfo sample to work for a while, but with some modified settings in the istio configmap bookinfo itself works fine now.

However now there are no traces showing up at all in Stackdriver Trace. When I point the zipkin service to zipkin itself I see them showing up immediately, but when pointing the zipkin service to the zipkin-to-stackdriver pods nothing happens. The logs don't show anything particular either:

                                    ********
                                  **        **
                                 *            *
                                **            **
                                **            **
                                 **          **
                                  **        **
                                    ********
                                      ****
                                      ****
        ****                          ****
     ******                           ****                                 ***
  ****************************************************************************
    *******                           ****                                 ***
        ****                          ****
                                       **
                                       **


             *****      **     *****     ** **       **     **   **
               **       **     **  *     ***         **     **** **
              **        **     *****     ****        **     **  ***
             ******     **     **        **  **      **     **   **

:: Powered by Spring Boot ::         (v1.5.10.RELEASE)

2018-03-02 15:17:16.760  INFO [/] 7 --- [           main] c.g.c.t.z.StackdriverZipkinCollector     : Starting StackdriverZipkinCollector on zipkin-to-stackdriver-5dddcdbcc5-v5sw6 with PID 7 (/usr/src/app/collector/target/collector-0.6.0.jar started by root in /usr/src/app)
2018-03-02 15:17:16.763  INFO [/] 7 --- [           main] c.g.c.t.z.StackdriverZipkinCollector     : The following profiles are active: shared
2018-03-02 15:17:16.827  INFO [/] 7 --- [           main] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@3c22fc4c: startup date [Fri Mar 02 15:17:16 UTC 2018]; root of context hierarchy
2018-03-02 15:17:18.114  INFO [/] 7 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService
2018-03-02 15:17:18.115  INFO [/] 7 --- [           main] ipkinStackdriverStorageAutoConfiguration : Configured Executor for ZipkinStackDriver Storage with: Executor(corePoolSize=1, maxPoolSize=5, queueCapacity=200000)
2018-03-02 15:17:18.126  INFO [/] 7 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService  'stackdriverExecutor'
2018-03-02 15:17:19.386  INFO [/] 7 --- [           main] org.xnio                                 : XNIO version 3.3.8.Final
2018-03-02 15:17:19.402  INFO [/] 7 --- [           main] org.xnio.nio                             : XNIO NIO Implementation Version 3.3.8.Final
2018-03-02 15:17:19.474  WARN [/] 7 --- [           main] io.undertow.websockets.jsr               : UT026009: XNIO worker was not set on WebSocketDeploymentInfo, the default worker will be used
2018-03-02 15:17:19.475  WARN [/] 7 --- [           main] io.undertow.websockets.jsr               : UT026010: Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be used
2018-03-02 15:17:19.503  INFO [/] 7 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2018-03-02 15:17:19.503  INFO [/] 7 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 2679 ms
2018-03-02 15:17:19.759  INFO [/] 7 --- [           main] o.s.b.w.servlet.ServletRegistrationBean  : Mapping servlet: 'dispatcherServlet' to [/]
2018-03-02 15:17:19.764  INFO [/] 7 --- [           main] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'metricsFilter' to: [/*]
2018-03-02 15:17:19.764  INFO [/] 7 --- [           main] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'characterEncodingFilter' to: [/*]
2018-03-02 15:17:19.764  INFO [/] 7 --- [           main] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'hiddenHttpMethodFilter' to: [/*]
2018-03-02 15:17:19.765  INFO [/] 7 --- [           main] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'httpPutFormContentFilter' to: [/*]
2018-03-02 15:17:19.765  INFO [/] 7 --- [           main] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'requestContextFilter' to: [/*]
2018-03-02 15:17:19.765  INFO [/] 7 --- [           main] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'webRequestLoggingFilter' to: [/*]
2018-03-02 15:17:19.765  INFO [/] 7 --- [           main] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'applicationContextIdFilter' to: [/*]
2018-03-02 15:17:20.317  INFO [/] 7 --- [           main] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@3c22fc4c: startup date [Fri Mar 02 15:17:16 UTC 2018]; root of context hierarchy
2018-03-02 15:17:20.399  INFO [/] 7 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2018-03-02 15:17:20.400  INFO [/] 7 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
2018-03-02 15:17:20.430  INFO [/] 7 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2018-03-02 15:17:20.430  INFO [/] 7 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2018-03-02 15:17:20.828  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/auditevents || /auditevents.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public org.springframework.http.ResponseEntity<?> org.springframework.boot.actuate.endpoint.mvc.AuditEventsMvcEndpoint.findByPrincipalAndAfterAndType(java.lang.String,java.util.Date,java.lang.String)
2018-03-02 15:17:20.828  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/configprops || /configprops.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.829  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/metrics/{name:.*}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.MetricsMvcEndpoint.value(java.lang.String)
2018-03-02 15:17:20.829  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/metrics || /metrics.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.830  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/mappings || /mappings.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.830  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/heapdump || /heapdump.json],methods=[GET],produces=[application/octet-stream]}" onto public void org.springframework.boot.actuate.endpoint.mvc.HeapdumpMvcEndpoint.invoke(boolean,javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException,javax.servlet.ServletException
2018-03-02 15:17:20.831  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/env/{name:.*}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EnvironmentMvcEndpoint.value(java.lang.String)
2018-03-02 15:17:20.831  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/env || /env.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.832  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/trace || /trace.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.833  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/info || /info.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.834  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/autoconfig || /autoconfig.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.836  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/loggers/{name:.*}],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.LoggersMvcEndpoint.get(java.lang.String)
2018-03-02 15:17:20.836  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/loggers/{name:.*}],methods=[POST],consumes=[application/vnd.spring-boot.actuator.v1+json || application/json],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.LoggersMvcEndpoint.set(java.lang.String,java.util.Map<java.lang.String, java.lang.String>)
2018-03-02 15:17:20.836  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/loggers || /loggers.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.836  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/health || /health.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.invoke(javax.servlet.http.HttpServletRequest,java.security.Principal)
2018-03-02 15:17:20.837  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/dump || /dump.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.837  INFO [/] 7 --- [           main] o.s.b.a.e.mvc.EndpointHandlerMapping     : Mapped "{[/beans || /beans.json],methods=[GET],produces=[application/vnd.spring-boot.actuator.v1+json || application/json]}" onto public java.lang.Object org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke()
2018-03-02 15:17:20.958  INFO [/] 7 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2018-03-02 15:17:20.969  INFO [/] 7 --- [           main] o.s.c.support.DefaultLifecycleProcessor  : Starting beans in phase 0
2018-03-02 15:17:21.084  INFO [/] 7 --- [           main] b.c.e.u.UndertowEmbeddedServletContainer : Undertow started on port(s) 9411 (http)
2018-03-02 15:17:21.089  INFO [/] 7 --- [           main] c.g.c.t.z.StackdriverZipkinCollector     : Started StackdriverZipkinCollector in 5.123 seconds (JVM running for 5.653)

I've tried both with and without setting GOOGLE_APPLICATION_CREDENTIALS and PROJECT_ID.

JorritSalverda avatar Mar 02 '18 15:03 JorritSalverda

can you try the new docker image? https://github.com/openzipkin/docker-zipkin-gcp

codefromthecrypt avatar Mar 02 '18 15:03 codefromthecrypt

Unfortunately that doesn't make a difference. Is there an easy way to make logging more verbose so I can see whether the application is called at all?

JorritSalverda avatar Mar 02 '18 15:03 JorritSalverda

I'm not in a good timezone to troubleshoot right now, but you can hop on gitter https://gitter.im/openzipkin/zipkin

If you are using the zipkin provided image (not StackDriverCollector as was mentioned in your log), than it is easier for us to support you as all the settings are standard.

for example, at least via /metrics you should be able to see if you are receiving spans at all. /health can verify your setup /configprops what your configuration is

https://github.com/openzipkin/zipkin/tree/master/zipkin-server#metrics

At any rate github issues are not good for general troubleshooting so please try gitter.

codefromthecrypt avatar Mar 02 '18 15:03 codefromthecrypt

Probably what's happening here is StackDriver limits

https://cloud.google.com/trace/docs/quotas

Size of key per label/attribute | 128 bytes Size of value per label/attribute | 256 bytes

You can either reduce the size of your key to below that or someone can add code support for truncation. Note that the "TruncatableString" thing in the proto expects the cut to be UTF codepoint aware. This means the programming is a bit more interesting than substring.

codefromthecrypt avatar Jul 05 '19 07:07 codefromthecrypt

heres a hint

  static TruncatableString maybeTruncate(String string, int length) {
    int sizeInBytes = utf8SizeInBytes(string);
    if (sizeInBytes <= length) {
      return TruncatableString.newBuilder().setValue(string).setTruncatedByteCount(0).build();
    }
    // not precise because this doesn't truncate codepoint aware, but maybe good enough..
    return TruncatableString.newBuilder()
        .setValueBytes(ByteString.copyFrom(string.getBytes(UTF_8), 0, length))
        .setTruncatedByteCount(sizeInBytes - length).build();
  }

codefromthecrypt avatar Jul 05 '19 08:07 codefromthecrypt

And here I was hoping since opencensus doesn't truncate, maybe the server does it...

https://github.com/census-instrumentation/opencensus-java/blob/master/exporters/trace/stackdriver/src/main/java/io/opencensus/exporter/trace/stackdriver/StackdriverV2ExporterHandler.java#L364

anuraaga avatar Jul 05 '19 11:07 anuraaga

possible that it does now cc @mtwo

On Fri, Jul 5, 2019, 7:18 PM Anuraag Agrawal [email protected] wrote:

And here I was hoping since opencensus doesn't truncate, maybe the server does it...

https://github.com/census-instrumentation/opencensus-java/blob/master/exporters/trace/stackdriver/src/main/java/io/opencensus/exporter/trace/stackdriver/StackdriverV2ExporterHandler.java#L364

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-gcp/issues/49?email_source=notifications&email_token=AAAPVV37O5RBSPRNTWA6SJ3P54UXJA5CNFSM4EJBVF42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZJIW2Y#issuecomment-508726123, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPVV65K6SYROBKKM2IQZDP54UXJANCNFSM4EJBVF4Q .

codefromthecrypt avatar Jul 05 '19 11:07 codefromthecrypt

Yep, I'm pretty sure that we truncate things server side if they're too large

mtwo avatar Jul 08 '19 18:07 mtwo