monkit
monkit copied to clipboard
spacemonkeygo
How to integrate with zipkin
Geez, that'd be a neat idea.
I'm going to leave this issue open until I write up good docs but to unblock you, your main method gets set up something like this:
package main
import (
"net/http"
"gopkg.in/spacemonkeygo/monkit.v2"
"gopkg.in/spacemonkeygo/monkit.v2/environment"
"gopkg.in/spacemonkeygo/monkit.v2/present"
"gopkg.in/spacemonkeygo/monkit-zipkin.v2"
)
func main() {
environment.Register(monkit.Default)
go http.ListenAndServe("localhost:9000", present.HTTP(monkit.Default))
collector, err := zipkin.NewScribeCollector("zipkin.whatever:9042")
if err != nil {
panic(err)
}
zipkin.RegisterZipkin(monkit.Default, collector, zipkin.Options{})
}
Once you've done that, the only remaining thing is to make sure your HTTP handlers pull Zipkin Context info from the Request. That's easy to do with zipkin.ContextWrapper.
func HandleRequest(ctx context.Context, w http.ResponseWriter, r *http.Request) {
defer mon.Task()(&ctx)(nil)
... whatever
}
func DoTheThing(ctx context.Context) (err error) {
defer mon.Task()(&ctx)(&err)
return http.Serve(listener, zipkin.ContextWrapper(zipkin.ContextHTTPHandlerFunc(HandleRequest)))
}
Last, your outbound HTTP requests need to pass through Context info:
func MakeRequest(ctx context.Context) (err error) {
defer mon.Task()(&ctx)(&err)
req, err := http.NewRequest(...)
if err != nil {
return err
}
resp, err := zipkin.TraceRequest(ctx, http.DefaultClient, req)
...
}
Does that help?
Thanks! Now golang 1.7 request has a method to get its context, and that is what I am doing, it should work too, right?
func HandleRequest(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
defer mon.Task()(&ctx)(nil)
... whatever
}
Hi, I have questions. I'm new to zipkin. So I may made some dump mistakes.
Here is what I did:
docker run -d -p 9411:9411 -p 9410:9410 openzipkin/zipkin
Then in my code
environment.Register(monkit.Default)
collector, err := zipkin.NewScribeCollector("0.0.0.0:9410"); checkErr(err)
zipkin.RegisterZipkin(monkit.Default, collector, zipkin.Options{})
I guess it should be able to connect to the collector.
Then I send some request to my service but nothing shows up in zipkin http://127.0.0.1:9411 (started from docker).
Any idea what I did wrong? I already setup the mokit trace as following:
Thanks in advance.
Sorry to get back to you slowly:
First off, I wanted to apologize for my last example. This line is wrong:
return http.Serve(listener, zipkin.ContextWrapper(zipkin.ContextHTTPHandlerFunc(HandleRequest)))
It should really be:
return http.Serve(listener, zipkin.ContextWrapper(zipkin.TraceHandler(zipkin.ContextHTTPHandlerFunc(HandleRequest))))
The addition of the zipkin.TraceHandler method is key. Without it, you're right, your example using (*Request).Context is the same as what I said. But you do want to use zipkin.TraceHandler so that you can get remote trace information propagated from services that call your service.
We should definitely update our code so that it uses the new (*Request).Context stuff, that's neat.
As for why Zipkin isn't getting any traces, the default zipkin.Options struct won't send any traces unless they are opted-in using the zipkin.TraceHandler functionality (an incoming HTTP request has the appropriate zipkin X-B3-Span-Id headers and so forth). If you want all of your spans sent to Zipkin, you'll need to set different options in the zipkin.Options struct you're creating.
I updated the docs for https://github.com/spacemonkeygo/monkit-zipkin/ to suggest zipkin.Options{Fraction: 1} by default
Hi jtolds, thanks for the reply.
I've modified my code to use zipkin.Options{Fraction: 1}. And also created one middleware like following
func TraceHandlerFor17Context(c http.Handler) http.Handler {
return http.Handler(func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
trace, spanId := RequestFromHeader(r.Header).Trace()
defer httpserver.FuncNamed(r.Method).RemoteTrace(&ctx, spanId, trace)(nil)
s := monkit.SpanFromCtx(ctx)
s.Annotate("http.uri", r.RequestURI)
wrapped := &responseWriterObserver{w: w}
c.ServeHTTP(wrapped, r)
s.Annotate("http.responsecode", fmt.Sprint(wrapped.StatusCode()))
})
}
The problem is nothing showed in zipkin, but I can see some logging in tcpdump.
13:04:56.988833 IP 127.0.0.1.58538 > 127.0.0.1.9410: Flags [.], ack 703, win 12737, options [nop,nop,TS val 3027082380 ecr 3027082380], length 0
E..4..@.@.............$.<...hq....1..(.....
.m...m..
13:04:56.988897 IP 127.0.0.1.58538 > 127.0.0.1.9410: Flags [P.], seq 6739:6743, ack 703, win 12737, options [nop,nop,TS val 3027082380 ecr 3027082380], length 4
E..8{.@.@.............$.<...hq....1..,.....
.m...m......
13:04:56.988911 IP 127.0.0.1.58538 > 127.0.0.1.9410: Flags [P.], seq 6743:6984, ack 703, win 12737, options [nop,nop,TS val 3027082380 ecr 3027082380], length 241
E..%..@.@.............$.<...hq....1........
.m...m..........Log...................zipkin.......CgABKZK+s+sVQyQLAAMAAAA5Z2l0bGFiLmV2ZXJwaG90by5jbi9mYWNlYXBpL2FwaS1zZXJ2ZXIvZGF0YWJhc2UuZGF0YXN0b3JlCgAEKZK+s+sVQyQPAAYMAAAAAgoAAQAFO1cJ2VWXCwACAAAAAmNzAAoAAQAFO1cJ2V02CwACAAAAAmNyAA8ACAwAAAAAAA==..
13:04:56.988915 IP 127.0.0.1.9410 > 127.0.0.1.58538: Flags [.], ack 6743, win 12548, options [nop,nop,TS val 3027082380 ecr 3027082380], length 0
E..4c.@.@...........$...hq..<.....1..(.....
.m...m..
13:04:56.988926 IP 127.0.0.1.9410 > 127.0.0.1.58538: Flags [.], ack 6984, win 12541, options [nop,nop,TS val 3027082380 ecr 3027082380], length 0
E..4.v@.@...........$...hq..<.....0..(.....
.m...m..
13:04:56.991235 IP 127.0.0.1.9410 > 127.0.0.1.58538: Flags [P.], seq 703:730, ack 6984, win 12541, options [nop,nop,TS val 3027082382 ecr 3027082380], length 27
E..O3#@.@...........$...hq..<.....0..C.....
.m...m..............Log............
13:04:56.991259 IP 127.0.0.1.58538 > 127.0.0.1.9410: Flags [.], ack 730, win 12736, options [nop,nop,TS val 3027082382 ecr 3027082382], length 0
E..4Y+@.@.............$.<...hq.3..1..(.....
.m...m..
13:04:56.991308 IP 127.0.0.1.58538 > 127.0.0.1.9410: Flags [P.], seq 6984:6988, ack 730, win 12736, options [nop,nop,TS val 3027082382 ecr 3027082382], length 4
E..8> @.@.............$.<...hq.3..1..,.....
.m...m......
13:04:56.991318 IP 127.0.0.1.58538 > 127.0.0.1.9410: Flags [P.], seq 6988:7229, ack 730, win 12736, options [nop,nop,TS val 3027082382 ecr 3027082382], length 241
E..%..@.@.............$.<...hq.3..1........
.m...m..........Log...................zipkin.......CgABCLod6AHgl3ULAAMAAAA5Z2l0bGFiLmV2ZXJwaG90by5jbi9mYWNlYXBpL2FwaS1zZXJ2ZXIvZGF0YWJhc2UuZGF0YXN0b3JlCgAECLod6AHgl3UPAAYMAAAAAgoAAQAFO1cJ2U03CwACAAAAAmNzAAoAAQAFO1cJ2V1LCwACAAAAAmNyAA8ACAwAAAAAAA==..
What bugs me is I don't know how to trouble shoot the zipkin protocol yet. Any idea what I missed?
@shuoli84 I think the issue is the docker image. I don't think that the proper connection string to the docker container is 0.0.0.0:9410 but {docker_container_ip}:9410.
I'm trying to get zipkin setup in a docker-compose environment but I haven't had any luck
I actually got a thrift worker pool to start but still nothing in the UI
http-server_1 | 2017/01/12 03:34:56 GET / Docs 114.991µs
zipkin | 2017-01-12 03:34:56.564 INFO 7 --- [thrift-worker-0] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Started.
http-server_1 | 2017/01/12 03:35:03 GET /users List All Users 519.468µs
zipkin | 2017-01-12 03:35:13.627 INFO 7 --- [nio-9411-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring FrameworkServlet 'dispatcherServlet'
zipkin | 2017-01-12 03:35:13.627 INFO 7 --- [nio-9411-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization started
zipkin | 2017-01-12 03:35:13.642 INFO 7 --- [nio-9411-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization completed in 15 ms
Update
Looks like I'm getting a HTTP Request header error:
zipkin | 2017-01-12 03:49:51.806 INFO 6 --- [nio-9411-exec-1] o.apache.coyote.http11.Http11Processor : Error parsing HTTP request header
zipkin | Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
zipkin |
zipkin | java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
zipkin | at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:462) ~[tomcat-embed-core-8.5.6.jar!/:8.5.6]
zipkin | at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:667) ~[tomcat-embed-core-8.5.6.jar!/:8.5.6]
zipkin | at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.6.jar!/:8.5.6]
zipkin | at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:802) [tomcat-embed-core-8.5.6.jar!/:8.5.6]
zipkin | at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1410) [tomcat-embed-core-8.5.6.jar!/:8.5.6]
zipkin | at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.6.jar!/:8.5.6]
zipkin | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_112]
zipkin | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_112]
zipkin | at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.6.jar!/:8.5.6]
zipkin | at java.lang.Thread.run(Thread.java:745) [na:1.8.0_112]
zipkin |
Not sure why this isn't working
Update
I've been able to get by the errors but I still can't get anything into the zipkin-ui... I copied there example docker configuration but that still yielded nothing
