I've been playing around getting a 4G modem (Sierra AirPrime MC7455 3G/4G LTE Modem) working on VyOS with GoogleFi
It all seemed easy, just set the APN and DHCP/DHCPv6
This led me to this forum post talking about this exact issue: https://forum.vyos.io/t/wwan-not-getting-an-ip/7757
It seems that this card does not support a link layer protocol of 802-3 (ethernet), but raw-ip only
We can see this with
root@vyos:# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-get-data-format [/dev/cdc-wdm0] Successfully got data format QoS flow header: no Link layer protocol: 'raw-ip' Uplink data aggregation protocol: 'disabled' Downlink data aggregation protocol: 'disabled' NDP signature: '0' Downlink data aggregation max datagrams: '0' Downlink data aggregation max size: '0'
I tried to set 802-3, via qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-set-data-format=802-3
root@vyos:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-set-data-format=802-3 [/dev/cdc-wdm0] Successfully set data format QoS flow header: no Link layer protocol: 'raw-ip' Uplink data aggregation protocol: 'disabled' Downlink data aggregation protocol: 'disabled' NDP signature: '0' Downlink data aggregation max datagrams: '0' Downlink data aggregation max size: '0' Uplink data aggregation max datagrams: '1' Uplink data aggregation max size: '4294967295'
The protocol doesn't change unfortunately, because it's not supported.
Looking more into it, the reason why DHCP doesn't work is because dhclient and dhcp6c doesn't support raw-ip, there seems to be a PR in the works for dhclient but I believe it's stalled and likely won't ever be picked up https://gitlab.isc.org/isc-projects/dhcp/-/merge_requests/67
Jan 21 09:31:14 vyos systemd[1]: Stopped dhclient@wwan0.service - DHCP client on wwan0. Jan 21 09:31:14 vyos systemd[1]: Starting dhclient@wwan0.service - DHCP client on wwan0... Jan 21 09:31:14 vyos systemd[1]: Started dhclient@wwan0.service - DHCP client on wwan0. Jan 21 09:31:14 vyos dhclient[3030]: Internet Systems Consortium DHCP Client 4.4.3-P1 Jan 21 09:31:14 vyos dhclient[3030]: Internet Systems Consortium DHCP Client 4.4.3-P1 Jan 21 09:31:14 vyos dhclient[3030]: Copyright 2004-2022 Internet Systems Consortium. Jan 21 09:31:14 vyos dhclient[3030]: All rights reserved. Jan 21 09:31:14 vyos dhclient[3030]: For info, please visit https://www.isc.org/software/dhcp/ Jan 21 09:31:14 vyos dhclient[3030]: Copyright 2004-2022 Internet Systems Consortium. Jan 21 09:31:14 vyos dhclient[3030]: All rights reserved. Jan 21 09:31:14 vyos dhclient[3030]: For info, please visit https://www.isc.org/software/dhcp/ Jan 21 09:31:14 vyos dhclient[3030]: Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: Current dhclient PID: 3030, Parent PID: 1, IP version: 4, All dhclients for interface wwan0: 3030 Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: Passing command to /usr/sbin/ip: "link set dev wwan0 up" Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: No changes to apply via vyos-hostsd-client Jan 21 09:31:14 vyos dhclient[3030]: Unsupported device type 65534 for "wwan0" Jan 21 09:31:15 vyos dhclient[3030]: Unsupported device type 65534 for "wwan0" Jan 21 09:31:15 vyos dhclient[3030]: If you think you have received this message due to a bug rather Jan 21 09:31:15 vyos dhclient[3030]: than a configuration issue please read the section on submitting Jan 21 09:31:15 vyos dhclient[3030]: bugs on either our web page at www.isc.org or in the README file Jan 21 09:31:15 vyos dhclient[3030]: before submitting a bug. These pages explain the proper Jan 21 09:31:15 vyos dhclient[3030]: process and the information we find helpful for debugging. Jan 21 09:31:15 vyos dhclient[3030]: exiting. Jan 21 09:31:14 vyos dhclient[3030]: Jan 21 09:31:14 vyos dhclient[3030]: If you think you have received this message due to a bug rather Jan 21 09:31:14 vyos dhclient[3030]: than a configuration issue please read the section on submitting Jan 21 09:31:14 vyos dhclient[3030]: bugs on either our web page at www.isc.org or in the README file Jan 21 09:31:14 vyos dhclient[3030]: before submitting a bug. These pages explain the proper Jan 21 09:31:14 vyos dhclient[3030]: process and the information we find helpful for debugging. Jan 21 09:31:14 vyos dhclient[3030]: Jan 21 09:31:15 vyos dhclient[3030]: exiting. Jan 21 09:31:15 vyos systemd[1]: dhclient@wwan0.service: Main process exited, code=exited, status=1/FAILURE Jan 21 09:31:15 vyos systemd[1]: dhclient@wwan0.service: Failed with result 'exit-code'.
Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set client ID (len 18) Jan 21 09:46:03 vyos dhcp6c[6091]: copyout_option: set identity association Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set elapsed time (len 2) Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set option request (len 4) Jan 21 09:46:03 vyos dhcp6c[6091]: client6_send: transmit failed: Cannot assign requested address Jan 21 09:46:03 vyos dhcp6c[6091]: dhcp6_reset_timer: reset a timer on wwan0, state=SOLICIT, timeo=6, retrans=68413
This is also noted by ModemManager (which is what VyOS uses to handle starting/stopping the connection) - https://modemmanager.org/docs/modemmanager/ip-connectivity-setup-in-lte-modems/
In particular, QMI modems running in raw-ip mode (without ethernet headers in the link layer) will by default be requested to use static IP addressing; not because the DHCP server in the modem doesn’t work, but because not all DHCP clients in Linux can work with network interfaces without ethernet headers (e.g. dhclient can’t).
The modem does have an IP address as mentioned above
vyos@vyos# sudo mmcli -m 0 -b 1 ------------------------------------ General | path: /org/freedesktop/ModemManager1/Bearer/2 | type: default ------------------------------------ Status | connected: yes | suspended: no | multiplexed: no | interface: wwan0 | ip timeout: 20 ------------------------------------ Properties | apn: h2g2 | roaming: allowed | ip type: ipv4v6 ------------------------------------ IPv4 configuration | method: static | address: 26.112.194.94 | prefix: 30 | gateway: 26.112.194.93 | dns: 10.177.0.34, 106.151.0.253 | mtu: 1500 ------------------------------------ IPv6 configuration | method: static | address: 2607:fb91:ee1:9e39:f488:eb51:81eb:2902 | prefix: 64 | gateway: 2607:fb91:ee1:9e39:a91c:c31f:4e25:a5e2 | dns: fd00:976a::9, fd00:976a::10 | mtu: 1500 ------------------------------------ Statistics | start date: 2024-01-21T09:44:53Z | duration: 240 | attempts: 1 | total-duration: 240 vyos@vyos# sudo qmicli -d /dev/cdc-wdm0 --device-open-proxy --wds-get-current-settings [/dev/cdc-wdm0] Current settings retrieved: IP Family: IPv4 IPv4 address: 26.112.194.94 IPv4 subnet mask: 255.255.255.252 IPv4 gateway address: 26.112.194.93 IPv4 primary DNS: 10.177.0.34 IPv4 secondary DNS: 106.151.0.253 MTU: 1500 Domains: none
Another note on the ModemManager page is
ModemManager does not perform the network interface configuration in the host, as it is exclusively in charge of communicating in the signaling plane between host and modem. The network manager process in use in the system will collect the IP settings exposed in the ModemManager Bearer object, and use them to configure the network interface accordingly.
Which leads me to believe NetworkManager will read that IP information (can get JSON output with --output-json) then manually configured the IP/default route. I did statically set the IP/route myself and was able to reach the internet via the 4G connection
I was hoping there is a way to use DHCP to do this still and I stumbled on an article (https://forums.raspberrypi.com/viewtopic.php?t=325044) that mentioning udhcpc would work with raw-ip, so I tried it and can confirm it works!
root@vyos# udhcpc -i wwan0 udhcpc: started, v1.35.0 udhcpc: broadcasting discover udhcpc: broadcasting select for 26.112.68.182, server 26.112.68.181 udhcpc: lease of 26.112.68.182 obtained from 26.112.68.181, lease time 7200
Alternatively, I tried with systemd-networkd but did not get anywhere, and noticed this comment on an issue (https://github.com/systemd/systemd/issues/27219#issuecomment-1505640023) that seems to say that systemd-networkd does not support raw-ip either
vyos@vyos:~$ show version Version: VyOS 1.5-rolling-202401161743 Release train: current Built by: autobuild@vyos.net Built on: Tue 16 Jan 2024 19:41 UTC Build UUID: f7976e02-b740-4027-8c96-6335bbea4315 Build commit ID: 365f10340ec2f1 Architecture: x86_64 Boot via: installed image System type: bare metal Hardware vendor: Lanner Electronics Inc. Hardware model: NCA-1515B-VS1 Hardware S/N: LR202112012772 Hardware UUID: 03000200-0400-0500-0006-000700080009 Copyright: VyOS maintainers and contributors
I believe this issue is not seen on modules/modems that support 802-3 for their link layer protocol as DHCP would work normally as expected in that case. It seems to be specifically related to raw-ip