zipkin-go
zipkin-go copied to clipboard
openzipkin
Add Google Cloud Pub Sub reporter
Add a new reporter for GCP streaming service.
The idea is to have pub sub subscriber as zipkin collector to process the spans
Coverage decreased (-2.2%) to 60.797% when pulling 5744523e47320de7584a82489ffec87a49ee2f51 on javierviera:master into b1a353815477e10673f9e4504be67b9370be9c54 on openzipkin:master.
usually we don't add reporters for transports that have no collectors. this leads to an incomplete feature story and possibly code maintained for only one user or site. can you mention what you are doing on the other side?
On Tue, Aug 6, 2019, 6:58 PM Javier Viera [email protected] wrote:
Add a new reporter for GCP streaming service.
The idea is tho have pub sub subscriber as zipkin collector to process the spans
You can view, comment on, or merge this pull request online at:
https://github.com/openzipkin/zipkin-go/pull/142 Commit Summary
- Add GCP pubsub reporter
- Merge remote-tracking branch 'upstream/master'
- Merge remote-tracking branch 'upstream/master'
- Add Pubsub dependencies
- Configure topic with env name
File Changes
- M .gitignore https://github.com/openzipkin/zipkin-go/pull/142/files#diff-0 (1)
- M README.md https://github.com/openzipkin/zipkin-go/pull/142/files#diff-1 (3)
- M go.mod https://github.com/openzipkin/zipkin-go/pull/142/files#diff-2 (14)
- M go.sum https://github.com/openzipkin/zipkin-go/pull/142/files#diff-3 (71)
- A reporter/pubsub/pubsub.go https://github.com/openzipkin/zipkin-go/pull/142/files#diff-4 (109)
- A reporter/pubsub/pubsub_test.go https://github.com/openzipkin/zipkin-go/pull/142/files#diff-5 (81)
Patch Links:
- https://github.com/openzipkin/zipkin-go/pull/142.patch
- https://github.com/openzipkin/zipkin-go/pull/142.diff
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-go/pull/142?email_source=notifications&email_token=AAAPVVY2D4SHQ5O7FK7PXMTQDE4JXA5CNFSM4IJUCARKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HDSKWZA, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPVV2NWMZODQK4VFOEFLDQDE4JXANCNFSM4IJUCARA .
@adriancole we are evaluating 2 options... most probably we will add a new collector in https://github.com/javierviera/zipkin/tree/master/zipkin-collector but we are also evaluating create a private google cloud function in GCP which pull Google PubSub message and add it to zipkin by API (https://zipkin.io/zipkin-api/#/default/post_spans)
ok probably if adding a collector, forking (and raising a pull request) on zipkin-gcp could be a nice route. the cloud function idea is also neat. I think span consumption is a neat use case of lambda though there are some gotchas the main collector code handles.
Assuming we are to merge this, let's make sure this accepts the literally same data format as other transports (list of json or proto encoded spans). If you have time, please add/update an issue on zipkin-gcp about the pub-sub so that this is less tail wagging dog
On Wed, Aug 7, 2019, 2:37 PM Javier Viera [email protected] wrote:
usually we don't add reporters for transports that have no collectors. this leads to an incomplete feature story and possibly code maintained for only one user or site. can you mention what you are doing on the other side? … <#m_3963051830818073207_m_6748561530216320011_m_-2186552932525891970_> On Tue, Aug 6, 2019, 6:58 PM Javier Viera @.***> wrote: Add a new reporter for GCP streaming service. The idea is tho have pub sub subscriber as zipkin collector to process the spans ------------------------------ You can view, comment on, or merge this pull request online at: #142 https://github.com/openzipkin/zipkin-go/pull/142 Commit Summary - Add GCP pubsub reporter - Merge remote-tracking branch 'upstream/master' - Merge remote-tracking branch 'upstream/master' - Add Pubsub dependencies - Configure topic with env name File Changes - M .gitignore https://github.com/openzipkin/zipkin-go/pull/142/files#diff-0 (1) - M README.md https://github.com/openzipkin/zipkin-go/pull/142/files#diff-1 (3) - M go.mod https://github.com/openzipkin/zipkin-go/pull/142/files#diff-2 (14) - M go.sum https://github.com/openzipkin/zipkin-go/pull/142/files#diff-3 (71)
- A reporter/pubsub/pubsub.go https://github.com/openzipkin/zipkin-go/pull/142/files#diff-4 (109) - A reporter/pubsub/pubsub_test.go https://github.com/openzipkin/zipkin-go/pull/142/files#diff-5 (81) Patch Links: - https://github.com/openzipkin/zipkin-go/pull/142.patch - https://github.com/openzipkin/zipkin-go/pull/142.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#142 https://github.com/openzipkin/zipkin-go/pull/142?email_source=notifications&email_token=AAAPVVY2D4SHQ5O7FK7PXMTQDE4JXA5CNFSM4IJUCARKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HDSKWZA>, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPVV2NWMZODQK4VFOEFLDQDE4JXANCNFSM4IJUCARA .
@adriancole https://github.com/adriancole we are evaluating 2 options... most probably we will add a new collector in https://github.com/javierviera/zipkin/tree/master/zipkin-collector but we are also evaluating create a private google cloud function in GCP which pull Google PubSub message and add it to zipkin by API ( https://zipkin.io/zipkin-api/#/default/post_spans)
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-go/pull/142?email_source=notifications&email_token=AAAPVV3MFE7BPCVVWBT37WTQDJUTPA5CNFSM4IJUCARKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3XLTLA#issuecomment-518961580, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPVV5CHM67DJS74Q5IOK3QDJUTPANCNFSM4IJUCARA .
Thanks @adriancole Why not creating a new collector in https://github.com/javierviera/zipkin/tree/master/zipkin-collector instead of updating zipkin-gcp? Seems like zipkin-gcp if for having stackdriver as storage.... My idea its just add a new transport.... Am i missing something?
@javierviera basically that. You create a goroutine when starting the reporter and listen to a channel signal and a termination signal sent by the Close method.
@javierviera I just raised this issue https://github.com/openzipkin/zipkin-gcp/issues/132
Hi @javierviera sorry for the late response. I added a couple of comments, I think we are very close. Great work!
@javierviera Any updates? If you don't have time, I can help you:)
