TheKevJames
Support pubsub push subscriptions through subscription config var
Sorry if this is already supported and I missed it in the documentation, but I don't see any support for push subscriptions with the config vars to set the subscription type to push and provide an endpoint for it to push to. It would be very convenient if this were an option.
Great suggestion! This was fairly easy to build, but I'm not loving the interface; the env var-based approach I inherited sure isn't very fun to extend.
I've made a test build accessible at thekevjames/gcloud-pubsub-emulator:pushconfig (docs/code/etc is here). Let me know if that works for you!
Thanks for implementing a feature for my request! Maybe it would be cleaner to extend the PUBSUB_PROJECT# value parser to have another level of delimiter for indicating the subscription endpoint value? Something like: PROJECT_ID,TOPIC_A:SUBSCRIPTION_1(ENDPOINT_1):SUBSCRIPTION_2,TOPIC_B:SUBCRIPTION_3(ENDPOINT_3)
While the format may be cleaner it would unfortunately complicate the parsing, since endpoints can use characters that are already used in the subscription syntax it would add a bit more complexity to the splitting code, so you'd need to pre-process the values within the parenthesis by encoding them into a safe format before splitting them, then decoding them again to get the url again. Parens should be safe to not appear for the vast majority of urls since they're reserved sub-delimiter characters, but they are used in some rare cases, like some wikipedia urls, but in the use case of pubsub endpoints it's extremely unlikely that anyone would need parens in the path, and they could always encode it if they need to.
Either way, it would be great to get this feature and thanks again!
hi @TheKevJames is the push subscription only available with this image ? thekevjames/gcloud-pubsub-emulator:pushconfig
@etler yeah, I had considered that but got worried about the parsing. Frankly, this is probably enough of an impetus to switch away from using env vars here in favour of a more structured config file, I could imagine some pretty annoying-to-deal-with edge cases no matter how this gets implemented, otherwise...
Maybe that's how I should implement this, thinking more about it: implement a config file approach to replace PUBSUB_PROJECT# (ie. and allow them side-by-side, with the env var being deprecated), then add push handling to the config file.
Thoughts? How would that impact your usage pattern? Issues that might cause?
@GZLiew that's correct -- for now it's on a branch so it's all just manual builds. If and when it gets merged into mainline, all future automatic (eg. versions/auto-updated/etc) builds would get the feature.
@TheKevJames no worries. if possible can you quick fix docker-gcloud-pubsub-emulator/pubsubc.py line 48?
it should be subscription since subscription_name is the full path
@TheKevJames I don't think switching to a config file would be a problem for how we're using this project since we're using it for dev testing and the current environment variables don't really contain any secrets. Config files are a bit harder to manage in deployments than environment variables if they need to be dynamic but we don't have that use case, and given the usage of the pubsub emulator it's probably not a problem generally to switch to a config file.
One other possible solution would be to have the structured configuration stored in a JSON string inside an environment variable that gets parsed as JSON so you can get structure from that. I don't particularly advocate for that over a config file, but thought I'd mention it as a possibility.
Since I needed the functionality, I've gone ahead and opened a PR https://github.com/TheKevJames/tools/pull/673 with the necessary changes. Temporary link to the image is in the PR
Thank you all! Sorry it's taken me so long to get back to this...
I've just opened #727 to solve this; I ended up going for the file-based approach for configuration, as I just wasn't happy with how complicated and unwieldy the environment variables felt. Just wanted to thank you all for offering your thoughts and the sample implementations!
Once I've got that branch a bit more tested out, I'll get it merged and included in the next release.