lxd
lxd copied to clipboard
canonical
Inconsistent network ACL behavior for nftables vs xtables
My network ACL is applied differently depending on whether I use nftables or xtables as firewall driver.
Required information
- Distribution: alpine
- Distribution version: 3.16.2
- The output of "lxc info" or if that fails:
config: {}
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- macaroon_authentication
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- devlxd_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- candid_authentication
- backup_compression
- candid_config
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- candid_config_key
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- snapshot_schedule_aliases
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- rbac
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
- daemon_storage
- instances
- image_types
- resources_disk_sata
- clustering_roles
- images_expiry
- resources_network_firmware
- backup_compression_algorithm
- ceph_data_pool_name
- container_syscall_intercept_mount
- compression_squashfs
- container_raw_mount
- container_nic_routed
- container_syscall_intercept_mount_fuse
- container_disk_ceph
- virtual-machines
- image_profiles
- clustering_architecture
- resources_disk_id
- storage_lvm_stripes
- vm_boot_priority
- unix_hotplug_devices
- api_filtering
- instance_nic_network
- clustering_sizing
- firewall_driver
- projects_limits
- container_syscall_intercept_hugetlbfs
- limits_hugepages
- container_nic_routed_gateway
- projects_restrictions
- custom_volume_snapshot_expiry
- volume_snapshot_scheduling
- trust_ca_certificates
- snapshot_disk_usage
- clustering_edit_roles
- container_nic_routed_host_address
- container_nic_ipvlan_gateway
- resources_usb_pci
- resources_cpu_threads_numa
- resources_cpu_core_die
- api_os
- container_nic_routed_host_table
- container_nic_ipvlan_host_table
- container_nic_ipvlan_mode
- resources_system
- images_push_relay
- network_dns_search
- container_nic_routed_limits
- instance_nic_bridged_vlan
- network_state_bond_bridge
- usedby_consistency
- custom_block_volumes
- clustering_failure_domains
- resources_gpu_mdev
- console_vga_type
- projects_limits_disk
- network_type_macvlan
- network_type_sriov
- container_syscall_intercept_bpf_devices
- network_type_ovn
- projects_networks
- projects_networks_restricted_uplinks
- custom_volume_backup
- backup_override_name
- storage_rsync_compression
- network_type_physical
- network_ovn_external_subnets
- network_ovn_nat
- network_ovn_external_routes_remove
- tpm_device_type
- storage_zfs_clone_copy_rebase
- gpu_mdev
- resources_pci_iommu
- resources_network_usb
- resources_disk_address
- network_physical_ovn_ingress_mode
- network_ovn_dhcp
- network_physical_routes_anycast
- projects_limits_instances
- network_state_vlan
- instance_nic_bridged_port_isolation
- instance_bulk_state_change
- network_gvrp
- instance_pool_move
- gpu_sriov
- pci_device_type
- storage_volume_state
- network_acl
- migration_stateful
- disk_state_quota
- storage_ceph_features
- projects_compression
- projects_images_remote_cache_expiry
- certificate_project
- network_ovn_acl
- projects_images_auto_update
- projects_restricted_cluster_target
- images_default_architecture
- network_ovn_acl_defaults
- gpu_mig
- project_usage
- network_bridge_acl
- warnings
- projects_restricted_backups_and_snapshots
- clustering_join_token
- clustering_description
- server_trusted_proxy
- clustering_update_cert
- storage_api_project
- server_instance_driver_operational
- server_supported_storage_drivers
- event_lifecycle_requestor_address
- resources_gpu_usb
- clustering_evacuation
- network_ovn_nat_address
- network_bgp
- network_forward
- custom_volume_refresh
- network_counters_errors_dropped
- metrics
- image_source_project
- clustering_config
- network_peer
- linux_sysctl
- network_dns
- ovn_nic_acceleration
- certificate_self_renewal
- instance_project_move
- storage_volume_project_move
- cloud_init
- network_dns_nat
- database_leader
- instance_all_projects
- clustering_groups
- ceph_rbd_du
- instance_get_full
- qemu_metrics
- gpu_mig_uuid
- event_project
- clustering_evacuation_live
- instance_allow_inconsistent_copy
- network_state_ovn
- storage_volume_api_filtering
- image_restrictions
- storage_zfs_export
- network_dns_records
- storage_zfs_reserve_space
- network_acl_log
- storage_zfs_blocksize
- metrics_cpu_seconds
- instance_snapshot_never
- certificate_token
- instance_nic_routed_neighbor_probe
- event_hub
- agent_nic_config
- projects_restricted_intercept
- metrics_authentication
- images_target_project
- cluster_migration_inconsistent_copy
- cluster_ovn_chassis
- container_syscall_intercept_sched_setscheduler
- storage_lvm_thinpool_metadata_size
- storage_volume_state_total
- instance_file_head
- resources_pci_vpd
- qemu_raw_conf
- storage_cephfs_fscache
- vsock_api
- storage_volumes_all_projects
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
environment:
addresses: []
architectures:
- aarch64
- armv7l
certificate: <redacted>
certificate_fingerprint: <redacted>
driver: lxc
driver_version: 4.0.12
firewall: xtables
kernel: Linux
kernel_architecture: aarch64
kernel_features:
idmapped_mounts: "true"
netnsid_getifaddrs: "true"
seccomp_listener: "true"
seccomp_listener_continue: "true"
shiftfs: "false"
uevent_injection: "true"
unpriv_fscaps: "true"
kernel_version: 5.15.55-0-rpi4
lxc_features:
cgroup2: "true"
core_scheduling: "true"
devpts_fd: "true"
idmapped_mounts_v2: "true"
mount_injection_file: "true"
network_gateway_device_route: "true"
network_ipvlan: "true"
network_l2proxy: "true"
network_phys_macvlan_mtu: "true"
network_veth_router: "true"
pidfd: "true"
seccomp_allow_deny_syntax: "true"
seccomp_notify: "true"
seccomp_proxy_send_notify_fd: "true"
os_name: Alpine Linux
os_version: 3.16.2
project: default
server: lxd
server_clustered: false
server_event_mode: full-mesh
server_name: hub
server_pid: 3028
server_version: 5.0.1
storage: dir
storage_version: "1"
storage_supported_drivers:
- name: lvm
version: 2.02.187(2) (2020-03-24) / 1.02.170 (2020-03-24) / 4.45.0
remote: false
- name: dir
version: "1"
remote: false
Steps to reproduce
- Setup network, ACL and containers as described below. Most importantly, there is a simple network ACL for the vpnlan network:
egress:
- action: reject
state: enabled
- Without nftables set up on the host, start both containers
- Ping container 1 (gw) from container 2 (con):
lxc exec con -- ping -c 1 gwResult: ping succeeds - Install nftables and reboot host
- Ping container 1 (gw) from container 2 (con):
lxc exec con -- ping -c 1 gwResult: Resolution of gw hostname DOES NOT succeed, neither doeslxc exec con -- ping -c 1 10.111.111.10
Notice that the jump to the ACL chain is conditioned differently in each case:
- xtables:
-A FORWARD ! -i vpnlan -o vpnlan -m comment --comment "generated for LXD network vpnlan" -j lxd_acl_vpnlan
-A FORWARD -i vpnlan ! -o vpnlan -m comment --comment "generated for LXD network vpnlan" -j lxd_acl_vpnlan
- nftables:
chain aclfwd.vpnlan {
type filter hook forward priority 0; policy accept;
iifname "vpnlan" jump acl.vpnlan
oifname "vpnlan" jump acl.vpnlan
}
Information to attach
- [ ] Dump of xtables rules:
hub:~# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N lxd_acl_vpnlan
-A INPUT -i vpnlan -p icmp -m icmp --icmp-type 12 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p icmp -m icmp --icmp-type 11 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p icmp -m icmp --icmp-type 3 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p udp -m udp --sport 68 --dport 67 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p udp -m udp --dport 53 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p tcp -m tcp --dport 53 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -m comment --comment "generated for LXD network vpnlan" -j lxd_acl_vpnlan
-A INPUT -i vpnlan -p icmp -m icmp --icmp-type 12 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p icmp -m icmp --icmp-type 11 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p icmp -m icmp --icmp-type 3 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p tcp -m tcp --dport 53 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p udp -m udp --dport 53 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i vpnlan -p udp -m udp --dport 67 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A INPUT -i lxdbr0 -p icmp -m icmp --icmp-type 12 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A INPUT -i lxdbr0 -p icmp -m icmp --icmp-type 11 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A INPUT -i lxdbr0 -p icmp -m icmp --icmp-type 3 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A INPUT -i lxdbr0 -p tcp -m tcp --dport 53 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A INPUT -i lxdbr0 -p udp -m udp --dport 53 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A INPUT -i lxdbr0 -p udp -m udp --dport 67 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A FORWARD ! -i vpnlan -o vpnlan -m comment --comment "generated for LXD network vpnlan" -j lxd_acl_vpnlan
-A FORWARD -i vpnlan ! -o vpnlan -m comment --comment "generated for LXD network vpnlan" -j lxd_acl_vpnlan
-A FORWARD -o vpnlan -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A FORWARD -i vpnlan -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A FORWARD -o lxdbr0 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A FORWARD -i lxdbr0 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A OUTPUT -o vpnlan -p icmp -m icmp --icmp-type 12 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -p icmp -m icmp --icmp-type 11 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -p icmp -m icmp --icmp-type 3 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -p udp -m udp --sport 67 --dport 68 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -m comment --comment "generated for LXD network vpnlan" -j lxd_acl_vpnlan
-A OUTPUT -o vpnlan -p icmp -m icmp --icmp-type 12 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -p icmp -m icmp --icmp-type 11 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -p icmp -m icmp --icmp-type 3 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -p tcp -m tcp --sport 53 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -p udp -m udp --sport 53 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o vpnlan -p udp -m udp --sport 67 -m comment --comment "generated for LXD network vpnlan" -j ACCEPT
-A OUTPUT -o lxdbr0 -p icmp -m icmp --icmp-type 12 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A OUTPUT -o lxdbr0 -p icmp -m icmp --icmp-type 11 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A OUTPUT -o lxdbr0 -p icmp -m icmp --icmp-type 3 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A OUTPUT -o lxdbr0 -p tcp -m tcp --sport 53 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A OUTPUT -o lxdbr0 -p udp -m udp --sport 53 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A OUTPUT -o lxdbr0 -p udp -m udp --sport 67 -m comment --comment "generated for LXD network lxdbr0" -j ACCEPT
-A lxd_acl_vpnlan -m state --state RELATED,ESTABLISHED -j ACCEPT
-A lxd_acl_vpnlan -i vpnlan -j REJECT --reject-with icmp-port-unreachable
-A lxd_acl_vpnlan -i vpnlan -j REJECT --reject-with icmp-port-unreachable
-A lxd_acl_vpnlan -o vpnlan -j REJECT --reject-with icmp-port-unreachable
hub:~# iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A POSTROUTING -s 10.66.66.0/24 ! -d 10.66.66.0/24 -m comment --comment "generated for LXD network lxdbr0" -j MASQUERADE
hub:~# iptables -S -t mangle
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A POSTROUTING -o vpnlan -p udp -m udp --dport 68 -m comment --comment "generated for LXD network vpnlan" -j CHECKSUM --checksum-fill
-A POSTROUTING -o lxdbr0 -p udp -m udp --dport 68 -m comment --comment "generated for LXD network lxdbr0" -j CHECKSUM --checksum-fill
hub:~# ebtables -L
Bridge table: filter
Bridge chain: INPUT, entries: 30, policy: ACCEPT
-s ! 00:16:3e:13:7a:d3 -i vethaed2b0ba -j DROP
-p ARP -i vethaed2b0ba --arp-mac-src ! 00:16:3e:13:7a:d3 -j DROP
-p IPv4 -s 00:16:3e:13:7a:d3 -i vethaed2b0ba --ip-src 0.0.0.0 --ip-dst 255.255.255.255 --ip-proto udp --ip-dport 67 -j ACCEPT
-p ARP -i vethaed2b0ba --arp-ip-src 10.111.111.2 -j ACCEPT
-p IPv4 -i vethaed2b0ba --ip-src 10.111.111.2 -j ACCEPT
-p ARP -i vethaed2b0ba -j DROP
-p IPv4 -i vethaed2b0ba -j DROP
-p IPv6 -s 00:16:3e:13:7a:d3 -i vethaed2b0ba --ip6-src fe80::/10 --ip6-dst ff02::1:2 --ip6-proto udp --ip6-dport 547 -j ACCEPT
-p IPv6 -s 00:16:3e:13:7a:d3 -i vethaed2b0ba --ip6-src fe80::/10 --ip6-dst ff02::2 --ip6-proto ipv6-icmp --ip6-icmp-type router-solicitation -j ACCEPT
-p IPv6 -i vethaed2b0ba --ip6-proto ipv6-icmp --ip6-icmp-type router-advertisement -j DROP
-p IPv6 -i vethaed2b0ba --ip6-src fe80:1116:1111:1111:216:3eff:fe13:7ad3 -j ACCEPT
-p IPv6 -i vethaed2b0ba -j DROP
-i vethaed2b0ba -j DROP
-s ! 00:16:3e:f2:cb:da -i vethd94f8c2c -j DROP
-p ARP -i vethd94f8c2c --arp-mac-src ! 00:16:3e:f2:cb:da -j DROP
-p IPv4 -s 00:16:3e:f2:cb:da -i vethd94f8c2c --ip-src 0.0.0.0 --ip-dst 255.255.255.255 --ip-proto udp --ip-dport 67 -j ACCEPT
-p ARP -i vethd94f8c2c --arp-ip-src 10.66.66.2 -j ACCEPT
-p IPv4 -i vethd94f8c2c --ip-src 10.66.66.2 -j ACCEPT
-p ARP -i vethd94f8c2c -j DROP
-p IPv4 -i vethd94f8c2c -j DROP
-p IPv6 -s 00:16:3e:f2:cb:da -i vethd94f8c2c --ip6-src fe80::/10 --ip6-dst ff02::1:2 --ip6-proto udp --ip6-dport 547 -j ACCEPT
-p IPv6 -s 00:16:3e:f2:cb:da -i vethd94f8c2c --ip6-src fe80::/10 --ip6-dst ff02::2 --ip6-proto ipv6-icmp --ip6-icmp-type router-solicitation -j ACCEPT
-p IPv6 -i vethd94f8c2c --ip6-proto ipv6-icmp --ip6-icmp-type router-advertisement -j DROP
-p IPv6 -i vethd94f8c2c --ip6-src fe80:6666:6666:6666:216:3eff:fef2:cbda -j ACCEPT
-p IPv6 -i vethd94f8c2c -j DROP
-i vethd94f8c2c -j DROP
-s ! 00:16:3e:4f:18:67 -i veth30a95fae -j DROP
-p IPv4 -i veth30a95fae -j ACCEPT
-p ARP -i veth30a95fae -j ACCEPT
-p IPv6 -i veth30a95fae -j ACCEPT
Bridge chain: FORWARD, entries: 24, policy: ACCEPT
-s ! 00:16:3e:13:7a:d3 -i vethaed2b0ba -j DROP
-p ARP -i vethaed2b0ba --arp-mac-src ! 00:16:3e:13:7a:d3 -j DROP
-p ARP -i vethaed2b0ba --arp-ip-src 10.111.111.2 -j ACCEPT
-p IPv4 -i vethaed2b0ba --ip-src 10.111.111.2 -j ACCEPT
-p ARP -i vethaed2b0ba -j DROP
-p IPv4 -i vethaed2b0ba -j DROP
-p IPv6 -i vethaed2b0ba --ip6-proto ipv6-icmp --ip6-icmp-type router-advertisement -j DROP
-p IPv6 -i vethaed2b0ba --ip6-src fe80:1116:1111:1111:216:3eff:fe13:7ad3 -j ACCEPT
-p IPv6 -i vethaed2b0ba -j DROP
-i vethaed2b0ba -j DROP
-s ! 00:16:3e:f2:cb:da -i vethd94f8c2c -j DROP
-p ARP -i vethd94f8c2c --arp-mac-src ! 00:16:3e:f2:cb:da -j DROP
-p ARP -i vethd94f8c2c --arp-ip-src 10.66.66.2 -j ACCEPT
-p IPv4 -i vethd94f8c2c --ip-src 10.66.66.2 -j ACCEPT
-p ARP -i vethd94f8c2c -j DROP
-p IPv4 -i vethd94f8c2c -j DROP
-p IPv6 -i vethd94f8c2c --ip6-proto ipv6-icmp --ip6-icmp-type router-advertisement -j DROP
-p IPv6 -i vethd94f8c2c --ip6-src fe80:6666:6666:6666:216:3eff:fef2:cbda -j ACCEPT
-p IPv6 -i vethd94f8c2c -j DROP
-i vethd94f8c2c -j DROP
-s ! 00:16:3e:4f:18:67 -i veth30a95fae -j DROP
-p IPv4 -i veth30a95fae -j ACCEPT
-p ARP -i veth30a95fae -j ACCEPT
-p IPv6 -i veth30a95fae -j ACCEPT
Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
- [ ] Dump of nftables rules:
hub:~# nft -n -a list tables
table inet lxd
table bridge lxd
hub:~# nft -n -a list table inet lxd
table inet lxd { # handle 5
chain pstrt.lxdbr0 { # handle 1
type nat hook postrouting priority 100; policy accept;
ip saddr 10.66.66.0/24 ip daddr != 10.66.66.0/24 masquerade # handle 2
ip6 saddr fe80:6666:6666:6666::/64 ip6 daddr != fe80:6666:6666:6666::/64 masquerade # handle 3
}
chain fwd.lxdbr0 { # handle 4
type filter hook forward priority 0; policy accept;
ip version 4 oifname "lxdbr0" accept # handle 5
ip version 4 iifname "lxdbr0" accept # handle 6
ip6 version 6 oifname "lxdbr0" accept # handle 7
ip6 version 6 iifname "lxdbr0" accept # handle 8
}
chain in.lxdbr0 { # handle 9
type filter hook input priority 0; policy accept;
iifname "lxdbr0" tcp dport 53 accept # handle 15
iifname "lxdbr0" udp dport 53 accept # handle 16
iifname "lxdbr0" icmp type { 3, 11, 12 } accept # handle 17
iifname "lxdbr0" udp dport 67 accept # handle 18
iifname "lxdbr0" icmpv6 type { 1, 2, 3, 4, 133, 135, 136, 143 } accept # handle 19
iifname "lxdbr0" udp dport 547 accept # handle 20
}
chain out.lxdbr0 { # handle 10
type filter hook output priority 0; policy accept;
oifname "lxdbr0" tcp sport 53 accept # handle 21
oifname "lxdbr0" udp sport 53 accept # handle 22
oifname "lxdbr0" icmp type { 3, 11, 12 } accept # handle 23
oifname "lxdbr0" udp sport 67 accept # handle 24
oifname "lxdbr0" icmpv6 type { 1, 2, 3, 4, 128, 134, 135, 136, 143 } accept # handle 25
oifname "lxdbr0" udp sport 547 accept # handle 26
}
chain acl.vpnlan { # handle 27
ct state 0x2,0x4 accept # handle 76
iifname "vpnlan" reject # handle 77
iifname "vpnlan" reject # handle 78
oifname "vpnlan" reject # handle 79
}
chain aclin.vpnlan { # handle 28
type filter hook input priority 0; policy accept;
iifname "vpnlan" tcp dport 53 accept # handle 35
iifname "vpnlan" udp dport 53 accept # handle 36
iifname "vpnlan" udp dport 67 accept # handle 37
iifname "vpnlan" udp dport 547 accept # handle 38
iifname "vpnlan" icmp type { 3, 11, 12 } accept # handle 39
iifname "vpnlan" icmpv6 type { 1, 2, 3, 4, 133, 135, 136, 143 } accept # handle 40
iifname "vpnlan" jump acl.vpnlan # handle 41
}
chain aclout.vpnlan { # handle 29
type filter hook output priority 0; policy accept;
oifname "vpnlan" udp sport 67 accept # handle 42
oifname "vpnlan" udp sport 547 accept # handle 43
oifname "vpnlan" icmp type { 3, 11, 12 } accept # handle 44
oifname "vpnlan" icmpv6 type { 1, 2, 3, 4, 128, 134, 135, 136, 143 } accept # handle 45
oifname "vpnlan" jump acl.vpnlan # handle 46
}
chain aclfwd.vpnlan { # handle 30
type filter hook forward priority 0; policy accept;
iifname "vpnlan" jump acl.vpnlan # handle 47
oifname "vpnlan" jump acl.vpnlan # handle 48
}
chain fwd.vpnlan { # handle 49
type filter hook forward priority 0; policy accept;
ip version 4 oifname "vpnlan" accept # handle 50
ip version 4 iifname "vpnlan" accept # handle 51
ip6 version 6 oifname "vpnlan" accept # handle 52
ip6 version 6 iifname "vpnlan" accept # handle 53
}
chain in.vpnlan { # handle 54
type filter hook input priority 0; policy accept;
iifname "vpnlan" tcp dport 53 accept # handle 60
iifname "vpnlan" udp dport 53 accept # handle 61
iifname "vpnlan" icmp type { 3, 11, 12 } accept # handle 62
iifname "vpnlan" udp dport 67 accept # handle 63
iifname "vpnlan" icmpv6 type { 1, 2, 3, 4, 133, 135, 136, 143 } accept # handle 64
iifname "vpnlan" udp dport 547 accept # handle 65
}
chain out.vpnlan { # handle 55
type filter hook output priority 0; policy accept;
oifname "vpnlan" tcp sport 53 accept # handle 66
oifname "vpnlan" udp sport 53 accept # handle 67
oifname "vpnlan" icmp type { 3, 11, 12 } accept # handle 68
oifname "vpnlan" udp sport 67 accept # handle 69
oifname "vpnlan" icmpv6 type { 1, 2, 3, 4, 128, 134, 135, 136, 143 } accept # handle 70
oifname "vpnlan" udp sport 547 accept # handle 71
}
}
hub:~# nft -n -a list table bridge lxd
table bridge lxd { # handle 6
chain in.con.eth0 { # handle 1
type filter hook input priority -200; policy accept;
iifname "veth3de13d7b" ether saddr != 00:16:3e:13:7a:d3 drop # handle 5
iifname "veth3de13d7b" arp saddr ether != 00:16:3e:13:7a:d3 drop # handle 6
iifname "veth3de13d7b" icmpv6 type 136 @nh,528,48 != 0x163e137ad3 drop # handle 7
iifname "veth3de13d7b" ip saddr 0.0.0.0 ip daddr 255.255.255.255 udp dport 67 accept # handle 8
iifname "veth3de13d7b" arp saddr ip 10.111.111.2 accept # handle 9
iifname "veth3de13d7b" ip saddr 10.111.111.2 accept # handle 10
iifname "veth3de13d7b" ether type 0x0806 drop # handle 11
iifname "veth3de13d7b" ether type 0x0800 drop # handle 12
iifname "veth3de13d7b" ip6 saddr fe80::/10 ip6 daddr ff02::1:2 udp dport 547 accept # handle 13
iifname "veth3de13d7b" ip6 saddr fe80::/10 ip6 daddr ff02::2 icmpv6 type 133 accept # handle 14
iifname "veth3de13d7b" icmpv6 type 134 drop # handle 15
iifname "veth3de13d7b" icmpv6 type 136 @nh,384,128 0xfe8011161111111102163efffe137ad3 accept # handle 16
iifname "veth3de13d7b" ip6 saddr fe80:1116:1111:1111:216:3eff:fe13:7ad3 accept # handle 17
iifname "veth3de13d7b" ether type 0x86dd drop # handle 18
iifname "veth3de13d7b" ether type != { 0x0800, 0x0806, 0x86dd } drop # handle 19
}
chain fwd.con.eth0 { # handle 2
type filter hook forward priority -200; policy accept;
iifname "veth3de13d7b" ether saddr != 00:16:3e:13:7a:d3 drop # handle 20
iifname "veth3de13d7b" arp saddr ether != 00:16:3e:13:7a:d3 drop # handle 21
iifname "veth3de13d7b" icmpv6 type 136 @nh,528,48 != 0x163e137ad3 drop # handle 22
iifname "veth3de13d7b" arp saddr ip 10.111.111.2 accept # handle 23
iifname "veth3de13d7b" ip saddr 10.111.111.2 accept # handle 24
iifname "veth3de13d7b" ether type 0x0806 drop # handle 25
iifname "veth3de13d7b" ether type 0x0800 drop # handle 26
iifname "veth3de13d7b" icmpv6 type 134 drop # handle 27
iifname "veth3de13d7b" ip6 saddr fe80:1116:1111:1111:216:3eff:fe13:7ad3 accept # handle 28
iifname "veth3de13d7b" icmpv6 type 136 @nh,384,128 0xfe8011161111111102163efffe137ad3 accept # handle 29
iifname "veth3de13d7b" ether type 0x86dd drop # handle 30
iifname "veth3de13d7b" ether type != { 0x0800, 0x0806, 0x86dd } drop # handle 31
}
chain in.gw.eth0 { # handle 32
type filter hook input priority -200; policy accept;
iifname "veth91bfe2d8" ether saddr != 00:16:3e:f2:cb:da drop # handle 36
iifname "veth91bfe2d8" arp saddr ether != 00:16:3e:f2:cb:da drop # handle 37
iifname "veth91bfe2d8" icmpv6 type 136 @nh,528,48 != 0x163ef2cbda drop # handle 38
iifname "veth91bfe2d8" ip saddr 0.0.0.0 ip daddr 255.255.255.255 udp dport 67 accept # handle 39
iifname "veth91bfe2d8" arp saddr ip 10.66.66.2 accept # handle 40
iifname "veth91bfe2d8" ip saddr 10.66.66.2 accept # handle 41
iifname "veth91bfe2d8" ether type 0x0806 drop # handle 42
iifname "veth91bfe2d8" ether type 0x0800 drop # handle 43
iifname "veth91bfe2d8" ip6 saddr fe80::/10 ip6 daddr ff02::1:2 udp dport 547 accept # handle 44
iifname "veth91bfe2d8" ip6 saddr fe80::/10 ip6 daddr ff02::2 icmpv6 type 133 accept # handle 45
iifname "veth91bfe2d8" icmpv6 type 134 drop # handle 46
iifname "veth91bfe2d8" icmpv6 type 136 @nh,384,128 0xfe8066666666666602163efffef2cbda accept # handle 47
iifname "veth91bfe2d8" ip6 saddr fe80:6666:6666:6666:216:3eff:fef2:cbda accept # handle 48
iifname "veth91bfe2d8" ether type 0x86dd drop # handle 49
iifname "veth91bfe2d8" ether type != { 0x0800, 0x0806, 0x86dd } drop # handle 50
}
chain fwd.gw.eth0 { # handle 33
type filter hook forward priority -200; policy accept;
iifname "veth91bfe2d8" ether saddr != 00:16:3e:f2:cb:da drop # handle 51
iifname "veth91bfe2d8" arp saddr ether != 00:16:3e:f2:cb:da drop # handle 52
iifname "veth91bfe2d8" icmpv6 type 136 @nh,528,48 != 0x163ef2cbda drop # handle 53
iifname "veth91bfe2d8" arp saddr ip 10.66.66.2 accept # handle 54
iifname "veth91bfe2d8" ip saddr 10.66.66.2 accept # handle 55
iifname "veth91bfe2d8" ether type 0x0806 drop # handle 56
iifname "veth91bfe2d8" ether type 0x0800 drop # handle 57
iifname "veth91bfe2d8" icmpv6 type 134 drop # handle 58
iifname "veth91bfe2d8" ip6 saddr fe80:6666:6666:6666:216:3eff:fef2:cbda accept # handle 59
iifname "veth91bfe2d8" icmpv6 type 136 @nh,384,128 0xfe8066666666666602163efffef2cbda accept # handle 60
iifname "veth91bfe2d8" ether type 0x86dd drop # handle 61
iifname "veth91bfe2d8" ether type != { 0x0800, 0x0806, 0x86dd } drop # handle 62
}
chain in.gw.eth1 { # handle 63
type filter hook input priority -200; policy accept;
iifname "vethb3cd0968" ether saddr != 00:16:3e:4f:18:67 drop # handle 65
iifname "vethb3cd0968" arp saddr ether != 00:16:3e:4f:18:67 drop # handle 66
iifname "vethb3cd0968" icmpv6 type 136 @nh,528,48 != 0x163e4f1867 drop # handle 67
}
chain fwd.gw.eth1 { # handle 64
type filter hook forward priority -200; policy accept;
iifname "vethb3cd0968" ether saddr != 00:16:3e:4f:18:67 drop # handle 68
iifname "vethb3cd0968" arp saddr ether != 00:16:3e:4f:18:67 drop # handle 69
iifname "vethb3cd0968" icmpv6 type 136 @nh,528,48 != 0x163e4f1867 drop # handle 70
}
}
- [ ] Container configuration (gw) (
lxc config show NAME --expanded)
architecture: aarch64
config:
image.architecture: arm64
image.description: Alpine 3.16 arm64 (20220916_13:01)
image.os: Alpine
image.release: "3.16"
image.requirements.secureboot: "false"
image.serial: "20220916_13:01"
image.type: squashfs
image.variant: default
raw.lxc: lxc.cap.drop=sys_boot sys_rawio sys_time sys_resource sys_tty_config wake_alarm
syslog sys_ptrace sys_pacct sys_module net_broadcast sys_chroot mknod net_bind_service
dac_override dac_read_search audit_control audit_read
security.idmap.isolated: "true"
volatile.base_image: d7a5b0fd3ab42dbbd3b1f986cd61bf782cb0e737d75f7015413917783ba20d33
volatile.eth0.host_name: veth91bfe2d8
volatile.eth0.hwaddr: 00:16:3e:f2:cb:da
volatile.eth1.host_name: vethb3cd0968
volatile.eth1.hwaddr: 00:16:3e:4f:18:67
volatile.eth1.name: eth1
volatile.idmap.base: "1065536"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.last_state.idmap: '[]'
volatile.last_state.power: RUNNING
volatile.uuid: 2a953ad5-5718-4cca-86c4-e0225ed4dd6f
devices:
eth0:
name: eth0
network: lxdbr0
security.ipv4_filtering: "true"
security.ipv6_filtering: "true"
security.mac_filtering: "true"
type: nic
eth1:
ipv4.address: 10.111.111.10
network: vpnlan
security.ipv4_filtering: "false"
security.ipv6_filtering: "false"
security.mac_filtering: "true"
type: nic
root:
path: /
pool: default
type: disk
tun:
path: /dev/net/tun
type: unix-char
vpnconf:
path: /vpns
raw.mount.options: uid=1000,gid=1000
readonly: "true"
source: /data/lxd/share
type: disk
ephemeral: false
profiles:
- gateway
stateful: false
description: ""
- [ ] Container configuration (con) (
lxc config show NAME --expanded)
architecture: aarch64
config:
image.architecture: arm64
image.description: Alpine 3.16 arm64 (20220919_13:01)
image.os: Alpine
image.release: "3.16"
image.requirements.secureboot: "false"
image.serial: "20220919_13:01"
image.type: squashfs
image.variant: default
volatile.base_image: d1e5ae542f8b69f3dbe2a9106d3667c1c773f12cbf6d111dd59a7ff30275e614
volatile.eth0.host_name: veth3de13d7b
volatile.eth0.hwaddr: 00:16:3e:13:7a:d3
volatile.eth0.name: eth0
volatile.idmap.base: "0"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":67108864},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":67108864}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":67108864},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":67108864}]'
volatile.last_state.idmap: '[]'
volatile.last_state.power: RUNNING
volatile.uuid: 5258c3d9-bb21-473b-b598-04a8ba66fa09
devices:
eth0:
network: vpnlan
security.ipv4_filtering: "true"
security.ipv6_filtering: "true"
security.mac_filtering: "true"
type: nic
root:
path: /
pool: default
type: disk
ephemeral: false
profiles:
- consumer
stateful: false
description: ""
- [ ] Network configuration:
ipv4.address: 10.111.111.1/24
ipv4.dhcp: "true"
ipv4.dhcp.gateway: 10.111.111.10
ipv4.nat: "false"
ipv4.routing: "true"
ipv6.address: fe80:1116:1111:1111::1/64
ipv6.dhcp: "true"
ipv6.nat: "false"
ipv6.routing: "true"
security.acls: vpnacl
description: VPN network
name: vpnlan
type: bridge
used_by:
- /1.0/instances/con
- /1.0/instances/gw
- /1.0/profiles/consumer
- /1.0/profiles/gateway
managed: true
status: Created
locations:
- none
- [ ] ACL configuration:
name: vpnacl
description: ""
egress:
- action: reject
state: enabled
ingress: []
config: {}
used_by:
- /1.0/networks/vpnlan
The ACL on bridge networks doesn't support intra-bridge filtering (see https://linuxcontainers.org/lxd/docs/master/howto/network_acls/#bridge-limitations).
I think you are saying that you would expect the proposed ACL to block ping traffic between instances connected to the same bridge. Which it apparently is for nftables but not for xtables.
So I think I would expect the ping between containers to work irrespective of that ACL rule.
Can you confirm that is what you meant?
