pfatt
pfatt copied to clipboard
MonkWho
wpa_supplicant unable to authorize
I am currently on pfSense 2.4.4 running on a Xenserver.
Setup: eth0 connected to ONT (interface is not assigned in pfsense)
In order to understand more on whats going wrong, I am running the script manually
Steps (on pfsense console):
- ran the following commands to clean up ngeth0 if any
/usr/sbin/ngctl shutdown waneapfilter:
/usr/sbin/ngctl shutdown laneapfilter:
/usr/sbin/ngctl shutdown eth0:
/usr/sbin/ngctl shutdown igb1:
/usr/sbin/ngctl shutdown o2m:
/usr/sbin/ngctl shutdown vlan0:
/usr/sbin/ngctl shutdown ngeth0:
/sbin/ifconfig eth0 up
/sbin/ifconfig eth0 promisc
- ran following commands to create ngeth0
/usr/sbin/ngctl mkpeer eth0: vlan lower downstream
/usr/sbin/ngctl name eth0:lower vlan0
/usr/sbin/ngctl mkpeer vlan0: eiface vlan0 ether
/usr/sbin/ngctl msg vlan0: 'addfilter { vlan=0 hook="vlan0" }'
/usr/sbin/ngctl msg ngeth0: set ec:12:34:cd:34:56
- ran the following command to start wpa_supplicant
/usr/sbin/wpa_supplicant -Dwired -ingeth0 -B -C /var/run/wpa_supplicant
wpa_cli set eapol_version 2
wpa_cli set fast_reauth 1
wpa_cli ap_scan 0
wpa_cli add_network
wpa_cli set_network 0 ca_cert \"/cf/conf/pfatt/wpa/ca.pem\"
wpa_cli set_network 0 client_cert \"/cf/conf/pfatt/wpa/client.pem\"
wpa_cli set_network 0 eap TLS
wpa_cli set_network 0 eapol_flags 0
wpa_cli identity 0 \"ec:12:34:cd:34:56\"
wpa_cli set_network 0 key_mgmt IEEE8021X
wpa_cli set_network 0 phase1 \"allow_canned_success=1\"
wpa_cli set_network 0 private_key \"/cf/conf/pfatt/wpa/private.pem\"
wpa_cli enable_network 0
- check status by running "wpa_cli status"
the status would show as "Unauthorized - Connecting" and after a minute or two it will go into "Unauthorized - Held".
Notes: I have chmod 777 for everything in /cf/conf/pfatt/wpa and /cf/conf/pfatt/bin
Different configurations: I have also tried using pci-passthrough for eth0 directly to pfsense I set eth0's mac address to my Cert's MAC
When you say that you set eth0 to your Certs MAC did you also set it in Xenserver configs? I never worked with pfSense in virtual environments or with Xenserver so I don't know how it behaves with MAC assignment.
At a first glace at your scripts, everything looks good except for "wpa_cli identity 0". In my script it's "wpa_cli set_network 0 identity". Can't test right now to confirm if that matters or not.
And you don't need to set chmod to 777 for /cf/conf/pfatt/wpa. I think all you need is 600.
I will run some tests on my end to see if I can replicate this behaviour.
At a first glace at your scripts, everything looks good except for "wpa_cli identity 0". In my script it's "wpa_cli set_network 0 identity". Can't test right now to confirm if that matters or not.
this part is an error on my part, I think i accidentally removed the set_network 0. But I just tried it now and it didnt work neither.
this is what I did
[2.4.4-RELEASE][[email protected]]/root: /usr/sbin/wpa_supplicant -Dwired -ingeth0 -B -C /var/run/wpa_supplicant
Successfully initialized wpa_supplicant
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set eapol_version 2
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set fast_reauth 1
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli ap_scan 0
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli add_network
Selected interface 'ngeth0'
0
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set_network 0 ca_cert \"/cf/conf/pfatt/wpa/ca.pem\"
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set_network 0 client_cert \"/cf/conf/pfatt/wpa/client.pem\"
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set_network 0 eap TLS
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set_network 0 eapol_flags 0
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set_network 0 identity 0 \"ec:12:34:cd:34:56\"
Selected interface 'ngeth0'
FAIL
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set_network 0 key_mgmt IEEE8021X
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set_network 0 phase1 \"allow_canned_success=1\"
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli set_network 0 private_key \"/cf/conf/pfatt/wpa/private.pem\"
Selected interface 'ngeth0'
OK
[2.4.4-RELEASE][[email protected]]/root: wpa_cli enable_network 0
Selected interface 'ngeth0'
OK
and wpa_cli status gives me
[2.4.4-RELEASE][[email protected]]/root: wpa_cli status
Selected interface 'ngeth0'
bssid=01:80:c2:00:00:03
freq=0
ssid=
id=0
mode=station
pairwise_cipher=NONE
group_cipher=NONE
key_mgmt=IEEE 802.1X (no WPA)
wpa_state=ASSOCIATED
address=ec:12:34:cd:34:56
Supplicant PAE state=CONNECTING
suppPortStatus=Unauthorized
EAP state=IDLE
uuid=1ba52f4e-aadd-54bc-b0ac-d6ef306c6787
and after a while it becomes
Selected interface 'ngeth0'
bssid=01:80:c2:00:00:03
freq=0
ssid=
id=0
mode=station
pairwise_cipher=NONE
group_cipher=NONE
key_mgmt=IEEE 802.1X (no WPA)
wpa_state=ASSOCIATED
address=ec:12:34:cd:34:56
Supplicant PAE state=HELD
suppPortStatus=Unauthorized
EAP state=FAILURE
uuid=1ba52f4e-aadd-54bc-b0ac-d6ef306c6787
That looks right.
My only other thought is that your MAC still does not match . Could it be that MAC is set on Xenserver level when it talks to ONT? This wpa_supplicant method was written and confirmed working in pfSense 2.4.4 and now in 2.4.5. Only difference when you upgrade to 2.4.5 is that you no longer need to install netgraph since it's included in the package now.
Only difference when you upgrade to 2.4.5 is that you no longer need to install netgraph since it's included in the package now.
This part doesnt really apply to us, right? cause we are using supplicant (i dont recall installing netgraph and we only use the certs and the pfatt.sh file)? I am assuming this is related to the "ng_etf.ko" file?
My only other thought is that your MAC still does not match . Could it be that MAC is set on Xenserver level when it talks to ONT?
I thought about this too and thats why I tried with pci-passthrough from xenserver to the pfsense. I will try to do pfsense barebone
Even in supplicant mode netgraph (ng_etf.ko) is still needed. In regular pfatt mode it's used to do all the work meanwhile in supplicant mode it is only used to tag traffic with VLAN0. In step two of your commands you use ngctl which is "netgraph control utility". Run "kldstat -v" to see if ng_etf is indeed loaded. You can also run "ngctl dot" and paste it in webgraphviz.com to get a graphical representation of your pathing from ngeth0 to eth0.
errm... I havent check yet, but I am using this script https://github.com/MonkWho/pfatt/blob/supplicant/bin/pfatt.sh
I see it is running kldload ng_etf.ko only when the scrip is on "bridge" mode and this command is not present in the "supplicant mode", so I thought we dont need it for supplicant mode
I don't have access to pfSense 2.4.4 at the moment so I can't test it but I think in 2.4.4 some netgraph modules are present in pfSense. Try running "kldstat -v" and see if netgraph module and a bunch of ng_* modules are present. I'm sure they are. Googling around I can tell that ng_etf.ko is responsible for filtering in the bridged node and since we are not doing any filtering then it's not needed. But as I said before we still use netgraph in supplicant mode with all those /usr/sbin/ngctl commands. When you run those commands do they work fine or do you get any errors?
Unfortunately I'm not the creator of any of these scripts but just a mere maintainer who's fork became a new parent when original author deleted or made his private. So I'm by no means a netgraph expert.
When you run those commands do they work fine or do you get any errors? No i dont get any errors,
This is what is shows when I run kldstat -v | grep ng
547 ng_socket
532 ng_ether
531 ng_eiface
546 ng_rfc1490
545 ng_pred1
530 ng_echo
544 ng_pptpgre
543 ng_pppoe
529 ng_deflate
542 ng_ppp
522 ng_UI
541 ng_pipe
528 ng_cisco
527 ng_car
540 ng_one2many
539 ng_mppc
526 ng_bridge
538 ng_lmi
537 ng_l2tp
523 ng_async
536 ng_ksocket
525 ng_bpf
535 ng_iface
534 ng_hole
533 ng_framerelay
552 ng_vlan
551 ng_vjc
550 ng_tty
549 ng_tee
548 ng_tcpmss
252 nge/miibus
251 pci/nge
ng_etf.ko is not there, but I am not sure if i need it or not
Yeah that looks right.
With everything else looking alright I think the only potential cause is virtualization on Xenserver. If you have a chance try testing pfSense on baremetal with your certs and see if it works.
when you run ifconfig, does your eth0 (physical nic connected to the ONT) has the Cert's MAC or just any MAC? I know ngeth0 should have the Cert's MAC
My ONT nic shows it's own MAC when I do ifconfig. ngeth0 has the Cert's MAC.
I'm seeing a similar issue myself on bare metal with everything (graph + MAC address) looking correct.
Selected interface 'ngeth0'
bssid=01:xx:xx:xx:xx:xx
freq=0
ssid=
id=0
mode=station
pairwise_cipher=NONE
group_cipher=NONE
key_mgmt=IEEE 802.1X (no WPA)
wpa_state=ASSOCIATED
address=74:xx:xx:xx:xx:xx (THIS IS CORRECT FOR RG)
Supplicant PAE state=HELD
suppPortStatus=Unauthorized
EAP state=FAILURE
uuid=xxxx
could this be a nic controller issue? I am assuming it is not the cert/RG issue because MonkWho uses the same RG as me. and I also have heard that it doesnt work on baremetal for some people. @doyler @MonkWho what controllers are your NICs? Mine is intel i212
device = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
device = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
That said, it looks like my EAPOL start isn't being tagged as vlan0, even with the netgraph configured.
[2.4.5-RELEASE][admin@pfSense]/root/supplicant: tcpdump -vv -s 65535 -X -n vlan 0
tcpdump: listening on ngeth0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
4 packets received by filter
0 packets dropped by kernel
[2.4.5-RELEASE][admin@pfSense]/root/supplicant: tcpdump -vv -s 65535 -X -n
tcpdump: listening on ngeth0, link-type EN10MB (Ethernet), capture size 65535 bytes
17:43:53.737022 EAPOL start (1) v2, len 0
0x0000: 0201 0000 ....
uhmmm, might need to ask @MonkWho to check that, since it is not working for me. Can you give him instructions on how to check if EAPOL has vlan0 or not
I'm having exactly the same problem, running pfSense 2.4.5-RELEASE flat on a Protectli FW6B.
I've sunk HOURS into this thing. For now I've set aside pfatt.sh, and I'm running things manually so I can see console output and errors.
netgraph seems good - ngctl shows a hook in ngeth0 and another in ONT_IF, and two hooks in VLAN0.
This command comes back in a way which suggests that 802.1 packets are going out on VLAN0 as expected:
tcpdump -i ngeth0 -e vlan
-------------------------------------
05:20:52.486546 f8:2d:c0:xx:xx:xx (oui Unknown) > 01:80:c2:00:00:03 (oui Unknown), ethertype 802.1Q (0x8100), length 22: vlan 0, p 0, ethertype EAPOL, EAPOL start (1) v2, len 0
For wpa_supplicant, I'm running this command using this conf. The output from this execution is verbose: https://pastebin.com/smD8n2sk -- and it looks just like the problem reported in the OP for this issue.
/usr/sbin/wpa_supplicant -dd -t -K -Dwired -ingeth0 -c /conf/pfatt/wpa_supplicant.conf
-------------------------------------
eapol_version=2
ap_scan=0
fast_reauth=1
network={
ca_cert="/conf/pfatt/wpa/ca.pem"
client_cert="/conf/pfatt/wpa/client.pem"
eap=TLS
eapol_flags=0
identity="F8:2D:C0:xx:xx:xx"
key_mgmt=IEEE8021X
phase1="allow_canned_success=1"
private_key="/conf/pfatt/wpa/private.pem"
}
-------------------------------------
I've restarted the ONT a couple of times. No joy.
I'm starting to wonder if the certs I pulled off of my brand-new RG are somehow bad. The decoder didn't toss any errors:
mfg_dat_decode.exe
-------------------------------------
802.1x Credential Extraction Tool
Copyright (c) 2018-2019 devicelocksmith.com
Version: 1.04 windows 386
Found client certificate for Serial Number: 001E46-272875392xxxxxx
Found certificates with following Subjects:
F8:2D:C0:xx:xx:xx
expires 2038-04-16 11:28:26 -0500 CDT
ARRIS Group, Inc. Device Intermediate CA ATTCPE1
expires 2038-10-24 15:22:56 -0500 CDT
ARRIS Group, Inc. Device Root CA ATTCPE1
expires 2043-10-24 15:17:26 -0500 CDT
Verifying certificates.. success!
Validating private key.. success!
Found valid AAA server root CA certificates:
System Infrastructure Root CA (SHA256)
expires 2044-07-10 15:12:33 -0500 CDT
ATT Services Inc Root CA
expires 2031-02-23 17:59:59 -0600 CST
Frontier-RootCA
expires 2024-05-01 08:20:27 -0500 CDT
Successfully saved EAP-TLS credentials to EAP-TLS_8021x_001E46-272875392174800.tar.gz
I even imported the certs into pf just to see if it would work, and they came in fine (I've since removed them). I can't grab them again because I managed to brick the RG and I'm waiting on a replacement. After I receive the new RG, I'm going to run in bridge mode until the wrinkles are ironed out of supplicant mode.
Looking at some of my output, it would appear that netgraph isn't working at all now for some reason? I'm unable to see/send any traffice across re0, and it is showing as link down
[2.4.5-RELEASE][[email protected]]/root: tcpdump -i re0 -e vlan
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on re0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
Are you seeing any actual traffic on the physical interface @timtrace ?
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether xx:xx:xx:xx:xx:xx
hwaddr 00:xx:xx:xx:xx:xx
inet6 fe80::201:2eff:fe6e:5a8c%re0 prefixlen 64 scopeid 0x1
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (none)
status: no carrier
@doyler - as promised, I've already torn down my supplicant mode and I've set up bridge mode again. I'm just awaiting delivery of my new RG in the morning.
But to answer your question - when I had wpa_supplicant up and running (although not authorizing) I could see packets coming back and forth between my ngeth0 and something with a MAC of 01:80:c2:00:00:03, although that vendor doesn't show up on any of the lists I've looked at.
Ah yea, I think my issue is causing neither bridge NOR supplicant mode to work.
Yea, ngeth0 isn't the physical interface, so that won't have them tagged as VLAN0, re0 is the one that will have them tagged. That's actually the LLDP broadcast address that you use for the EAPOL authentication.
You guys know so much more about this stuff than I do :)
So my tcpdump had vlan0 in the results -- I guess that doesn't do the trick.
I should've run it through a span port and grabbed the outbound packets off of my em5 interface.
does anyone know of an intel chipset this definitely works with? i've got a pfsense machine i built a machine and used a "Dell K9CR1 I350-T4 Quad Port Gigabit Ethernet Adapter" which does not seem to work with supplicant mode
does anyone know of an intel chipset this definitely works with? i've got a pfsense machine i built a machine and used a "Dell K9CR1 I350-T4 Quad Port Gigabit Ethernet Adapter" which does not seem to work with supplicant mode
i think anything works on ESXi as long as the hardware works on ESXi, I am not sure about baremetal
@eric62451 @timtrace I was having the same issue with a FW6B Protectli device for the supplicant mode. The script was hanging on waiting for EAP_authentication. The fix for me was to boot into pfsense normally, drop into shell and and create ngeth0 manually:
/usr/sbin/ngctl mkpeer em0: vlan lower downstream
/usr/sbin/ngctl name em0:lower vlan0
/usr/sbin/ngctl mkpeer vlan0: eiface vlan0 ether
/usr/sbin/ngctl msg vlan0: 'addfilter { vlan=0 hook="vlan0" }'
/usr/sbin/ngctl msg ngeth0: set xx:xx:xx:xx:xx:xx
/sbin/ifconfig em0 ether xx:xx:xx:xx:xx:xx
/sbin/ifconfig em0 up
/sbin/ifconfig em0 promisc
Then manually assign ngeth0 as the wan port in shell.
Then calling wpa_supplicant, changing the -i flag to -ingeth0 from -i$ONT_IF (which is whatever you specified at the beginning of the script):
/usr/sbin/wpa_supplicant -Dwired -ingeth0 -B -C /var/run/wpa_supplicant
Give all the necessary wpa_params via wpa_cli.
Then make the interface change on the wpa_supplicant call in the script and it runs successfully upon reboot.
@M-Pepper I'm attempting to create ngeth0 manually at the shell but I'm getting
ngctl: send msg: no such file or directory
I did hack up the base pfatt.sh supplicant script to remove the waits for EAP authorization and an IP address being assigned. I killed the wpa_supplicant and ran it manually entering parameters with wpa_cli. I still saw it CONNECTING then FAILED. Your solution of creating ngeth0 manually points to an issue with the interface creation in the script but nothing is standing out.
Can you post the exact section of the script that's causing the error and what your sending before that?
Here's a thread that has someone that had the same error message; didn't have a chance to read through all of the thread though.
https://forum.netgate.com/topic/99190/att-uverse-rg-bypass-0-2-btc/77?lang=en-US
@M-Pepper I'm running what you posted above, plus the shutdowns ahead of it in case anything is carrying over from a reboot. It seems I can only create ngeth0 before pfSense is up and running (via earlyshellcmd). When did you run yours?
/usr/sbin/ngctl shutdown waneapfilter: >/dev/null 2>&1
/usr/sbin/ngctl shutdown laneapfilter: >/dev/null 2>&1
/usr/sbin/ngctl shutdown em0: >/dev/null 2>&1
/usr/sbin/ngctl shutdown o2m: >/dev/null 2>&1
/usr/sbin/ngctl shutdown vlan0: >/dev/null 2>&1
/usr/sbin/ngctl shutdown ngeth0: >/dev/null 2>&1
/usr/sbin/ngctl mkpeer em0: vlan lower downstream
/usr/sbin/ngctl name emo:lower vlan0
/usr/sbin/ngctl mkpeer vlan0: eiface vlan0 ether
/usr/sbin/ngctl msg vlan0: 'addfilter { vlan=0 hook="vlan0" }'
/usr/sbin/ngctl msg ngeth0: set XXXXXXXXXXXX
/sbin/ifconfig em0 ether XXXXXXXXXXXX
/sbin/ifconfig em0 up
/sbin/ifconfig em0 promisc
I’m so confused. There are so many methods.
From as simple as using a powered dumb switch to fool the RG to authenticating and using the switch to keep the connection alive. RG is disconnected until power failure requires reauthentication.
Then there is this Pfatt method which is basically the same just doesn’t need a switch to be constantly powered.
The last method is simply root and grab the certificate and generate the EAP file. Now how to use that file in Pfsense or OPNsense?
I’m leaning towards the last method. The first method is so simple I could swing by a Walmart get a switch and do it immediately. Put the switch on a UPS.
With the first method nothing is tagging VLAN0. So why is that necessary after the auth?
