ztunnel
ztunnel copied to clipboard
istio
Fetch local workload directly, rather than based on IP
This aligns our logic with the reality of how Ztunnel is deployed today. In its initial inception, we had one Proxy, and dynamically associated requests with pods by IP.
Now, we have one Proxy instance per workload, whether that is shared/inpod or dedicated.
To align with this, this PR stops using IP as a means to associate traffic to a workload - we always use the workload the Proxy is deployed for.
For inpod, this is clear: the CNI tells us specifically what pod we should be starting the proxy for. Today, we already do a second-check that the workload we get from the IP address matches.
For dedicated, its less clear (and not yet solved, hence why its WIP). I think we should just add a new config like DEDICATED_WORKLOAD=some-ns/some-workload-name or whatever.
Aside from being more clear, less error prone, and more efficient (one lookup per pod vs per connection).
One specific bug this fixes is usage of socks5, which is currently basically broken. The source IP is always 127.0.0.1 when connecting to ztunnel over socks, so it always gets 'unknown source'
Open question:
- Should we validate the IP address at all? is there a case where some arbitrary external traffic hits our outbound listener somehow that we were protected against before but no longer are?
Blockers:
- Add a way to make it work with dedicate mode
- Ensure if the workload changes we get notified for future connections
Followups:
- Move certificates to this model as well, rather than doing the same IP lookup (note: even today, we still do the same verification against the workload, so this shouldn't be a correctness issue)
Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all
For dedicated, its less clear (and not yet solved, hence why its WIP). I think we should just add a new config like DEDICATED_WORKLOAD=some-ns/some-workload-name or whatever.
xref https://github.com/istio/ztunnel/issues/1198
- Should we validate the IP address at all? is there a case where some arbitrary external traffic hits our outbound listener somehow that we were protected against before but no longer are?
Given that by-itself IP simply isn't an identity (address != identity, for both machines and people, though it's a factor) I doubt it.
For inpod, this is clear: the CNI tells us specifically what pod we should be starting the proxy for. Today, we already do a second-check that the workload we get from the IP address matches.
Yeah I think this is fine for inpod, the secondary check doesn't hurt.
There's a couple of ways we can do this for dedicated mode tho
- just have
DEDICATED_ALLOWED_IP=whatever - have an array of env vars as mentioned in #1198 describing the singular workload identity (basically the same as the above option, with more factors + attestation from ztunnel, a little more complex, probably not worth it?)
- require outbound stuff to have a packet/connmark applied out of band by e.g. iptables before we accept it (this is more of a "you solve this in your env, not our problem" approach)
For dedicated, we could only do the IP address match and leave it at that (first option) - I think that's fine for now? I don't think we need to do a lot of work to prove workload identity for dedicated mode - something like nginx or haproxy wouldn't support much more than maybe the IP allowlist or mark approach if used in this fashion anyway, I imagine, especially at L4.
IP allowlist would work for socks5 as well for now.
Given that by-itself IP simply isn't an identity (address != identity, for both machines and people, though it's a factor) I doubt it.
My concern was more: A packet comes into the pod to pod:15001 and somehow is confused into sending outbound. It probably cannot send outbound since it would likely not have an orig_dst (unless it hit a collision?) and instead, at worst, infinite loop; our 'illegal ports' solves that?
Given that by-itself IP simply isn't an identity (address != identity, for both machines and people, though it's a factor) I doubt it.
My concern was more: A packet comes into the pod to
pod:15001and somehow is confused into sending outbound. It probably cannot send outbound since it would likely not have an orig_dst (unless it hit a collision?) and instead, at worst, infinite loop; our 'illegal ports' solves that?
We TPROXY everything coming in - so that isn't possible unless the inpod rules are defeated.
And if the inpod rules are defeated (or just potentially defeat-able) traffic can just egress without hitting ztunnel at all - so yea, I don't think it's worth doing the check here.
That is the kind of thing better handled outside of zt anyway with netrules IMO, since ztunnel checks are effectively worthless for anything that can defeat those.
- Traffic like that can't get to
outboundwithout bypassing netrules, at which point ztunnel can be completely bypassed anyway. - Inbound checking is enough on the zt side.