Use from other languages? (Scala, Python, Ruby, etc)

[thoward]

Would it be possible to implement a tracing client in other languages? e.g. Could I use this in a Scala/Finatra app? What would it take to implement that sort of client?

[sqs]

Yes, absolutely. The wire protocol is simple but only defined in Go right now. It basically sends JSON-encoded collectPacket structs over a TCP connection (with or without TLS).

I think the next step would be to make the wire protocol use protobufs or similar. Then it'd be super easy to see what data needs to be sent and to make libs for other languages. The actual client impl would just be something that connects to a TCP socket and sends protobufs. In Go, the Recorder is a nice wrapper around the collector that allows you to record richly typed events (HTTP client/server requests, SQL queries, etc.) instead of just raw key-value pairs, but that is easy to make in other languages and isn't really apptrace-specific.

If we make the wire protocol use protobufs, would you be interested in contributing a Scala client impl?

[thoward]

I'd be glad to help out. I'll do my best, though I'm stronger in Ruby/Python than Scala. I could also do C#/F#. We could probably code-gen the protobuf bindings for a number of languages at once, then it's just left to build the tracing client and manage the network connection.

JSON would be just as easy to implement, IMO, but protobuf is more efficient, which never hurts.

On Tue, Feb 10, 2015 at 1:05 PM, Quinn Slack [email protected] wrote:

Yes, absolutely. The wire protocol is simple but only defined in Go right now. It basically sends JSON-encoded collectPacket https://sourcegraph.com/sourcegraph/apptrace@master/.GoPackage/sourcegraph.com/sourcegraph/apptrace/.def/collectPacket structs over a TCP connection (with or without TLS).

I think the next step would be to make the wire protocol use protobufs or similar. Then it'd be super easy to see what data needs to be sent and to make libs for other languages.

If we make the wire protocol use protobufs, would you be interested in contributing a Scala client impl?

— Reply to this email directly or view it on GitHub https://github.com/sourcegraph/apptrace/issues/31#issuecomment-73783409.

[sqs]

Great. We'll add this to the TODO list and will update here on this issue.

I was thinking protobufs because it's not immediately clear to non-Go folks what

// A collectPacket is sent between a client and server collector.
type collectPacket struct {
    SpanID      SpanID
    Annotations Annotations
}

looks like in JSON. We have some other Go projects that define their schema in this way, and people from other languages have had trouble (and righly complained :).

[thoward]

That makes sense. Protobuf seems like a more natural fit anyhow. Anything I can do to help w/ the move form JSON to protobuf? (Go is completely new to me, but I'd be willing to try)

I've got a Scala/Finatra app I'm working on right now that I'd love to use this in, also a Python/Flask app. Both are internal tools that are performance sensitive. Could use Zipkin, but apptrace seems more on my level. ;)

On Tue, Feb 10, 2015 at 1:13 PM, Quinn Slack [email protected] wrote:

Great. We'll add this to the TODO list and will update here on this issue.

I was thinking protobufs because it's not immediately clear to non-Go folks what

// A collectPacket is sent between a client and server collector. type collectPacket struct { SpanID SpanID Annotations Annotations }

looks like in JSON. We have some other Go projects that define their schema in this way, and people from other languages have had trouble (and righly complained :).

— Reply to this email directly or view it on GitHub https://github.com/sourcegraph/apptrace/issues/31#issuecomment-73784774.

[sqs]

Thanks, I think we will be able to do the protobuf-ization soon. If you wanted to just write a client that sent JSON and start using it now (to work out any kinks), it should be easy to switch to protobufs when those are ready. Happy to help with any issues you run into and it is awesome to have more folks picking up apptrace (I know of a few other companies that have started looking into it beyond us and you).

[emidoots]

@thoward I've just merged a change that switches us over to protobuf (see the PR for all the good details). The protobuf portion is in a separate (internal) Go package so it should be really easy to look through, but let me know if you have any issues -- I'd be glad to help!.

Next up I'll be seeing how hard it is to get a working Python example going, as well as some docs on how to make use of it from other languages as well.

[emidoots]

I've made some more good progress towards a working Python implementation locally. I haven't merged the changes as I may or may not have to re-base them depending on #33. You can see the changes in the python_support branch though -- if you are looking for pointers on adapting other languages.

There is much more work to be done (unique span ID's, actual timespan events, a real HTTP example, integration, etc), but you can see I've already gotten a Python script delivering spans and annotations properly to a remote Go collection server:

[emidoots]

Much work has been done on python/appdash -- including lots of documentation; and it's been merged into master now.

I've also added a document describing how Appdash functions under the hood. That document, combined with the new python/appdash I think should form a really nice base for starting work on using Appdash from many other languages.

[emidoots]

@dim / @bsm has created a Ruby client for Appdash :) Check it out! https://github.com/bsm/appdash-rb

[emidoots]

With gRPC and grpc-gateway we could support all languages: https://github.com/sourcegraph/appdash/issues/125