I've been playing around getting a 4G modem (Sierra AirPrime MC7455 3G/4G LTE Modem) working on VyOS with GoogleFi
It all seemed easy, just set the APN and DHCP/DHCPv6
This led me to this forum post talking about this exact issue: https://forum.vyos.io/t/wwan-not-getting-an-ip/7757
It seems that this card does not support a link layer protocol of `802-3` (ethernet), but `raw-ip` only
```
wwan0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1430
inet 26.112.194.94 netmask 255.255.255.252 destination 26.112.194.94
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 1000 (UNSPEC)
RX packets 3 bytes 652 (652.0 B)
RX errors 0 dropped 2 overruns 0 frame 0
TX packets 4 bytes 808 (808.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
```
We can see this with
```
root@vyos:# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-get-data-format
[/dev/cdc-wdm0] Successfully got data format
QoS flow header: no
Link layer protocol: 'raw-ip'
Uplink data aggregation protocol: 'disabled'
Downlink data aggregation protocol: 'disabled'
NDP signature: '0'
Downlink data aggregation max datagrams: '0'
Downlink data aggregation max size: '0'
```
I tried to set `802-3`, via `qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-set-data-format=802-3`
```
root@vyos:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-set-data-format=802-3
[/dev/cdc-wdm0] Successfully set data format
QoS flow header: no
Link layer protocol: 'raw-ip'
Uplink data aggregation protocol: 'disabled'
Downlink data aggregation protocol: 'disabled'
NDP signature: '0'
Downlink data aggregation max datagrams: '0'
Downlink data aggregation max size: '0'
Uplink data aggregation max datagrams: '1'
Uplink data aggregation max size: '4294967295'
```
The protocol doesn't change unfortunately, because it's not supported.
Looking more into it, the reason why DHCP doesn't work is because `dhclient` and `dhcp6c` doesn't support raw-ip, there seems to be a PR in the works for dhclient but I believe it's stalled and likely won't ever be picked up https://gitlab.isc.org/isc-projects/dhcp/-/merge_requests/67
```
Jan 21 09:31:14 vyos systemd[1]: Stopped
[email protected] - DHCP client on wwan0.
Jan 21 09:31:14 vyos systemd[1]: Starting
[email protected] - DHCP client on wwan0...
Jan 21 09:31:14 vyos systemd[1]: Started
[email protected] - DHCP client on wwan0.
Jan 21 09:31:14 vyos dhclient[3030]: Internet Systems Consortium DHCP Client 4.4.3-P1
Jan 21 09:31:14 vyos dhclient[3030]: Internet Systems Consortium DHCP Client 4.4.3-P1
Jan 21 09:31:14 vyos dhclient[3030]: Copyright 2004-2022 Internet Systems Consortium.
Jan 21 09:31:14 vyos dhclient[3030]: All rights reserved.
Jan 21 09:31:14 vyos dhclient[3030]: For info, please visit https://www.isc.org/software/dhcp/
Jan 21 09:31:14 vyos dhclient[3030]: Copyright 2004-2022 Internet Systems Consortium.
Jan 21 09:31:14 vyos dhclient[3030]: All rights reserved.
Jan 21 09:31:14 vyos dhclient[3030]: For info, please visit https://www.isc.org/software/dhcp/
Jan 21 09:31:14 vyos dhclient[3030]:
Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: Current dhclient PID: 3030, Parent PID: 1, IP version: 4, All dhclients for interface wwan0: 3030
Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: Passing command to /usr/sbin/ip: "link set dev wwan0 up"
Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: No changes to apply via vyos-hostsd-client
Jan 21 09:31:14 vyos dhclient[3030]: Unsupported device type 65534 for "wwan0"
Jan 21 09:31:15 vyos dhclient[3030]: Unsupported device type 65534 for "wwan0"
Jan 21 09:31:15 vyos dhclient[3030]: If you think you have received this message due to a bug rather
Jan 21 09:31:15 vyos dhclient[3030]: than a configuration issue please read the section on submitting
Jan 21 09:31:15 vyos dhclient[3030]: bugs on either our web page at www.isc.org or in the README file
Jan 21 09:31:15 vyos dhclient[3030]: before submitting a bug. These pages explain the proper
Jan 21 09:31:15 vyos dhclient[3030]: process and the information we find helpful for debugging.
Jan 21 09:31:15 vyos dhclient[3030]: exiting.
Jan 21 09:31:14 vyos dhclient[3030]:
Jan 21 09:31:14 vyos dhclient[3030]: If you think you have received this message due to a bug rather
Jan 21 09:31:14 vyos dhclient[3030]: than a configuration issue please read the section on submitting
Jan 21 09:31:14 vyos dhclient[3030]: bugs on either our web page at www.isc.org or in the README file
Jan 21 09:31:14 vyos dhclient[3030]: before submitting a bug. These pages explain the proper
Jan 21 09:31:14 vyos dhclient[3030]: process and the information we find helpful for debugging.
Jan 21 09:31:14 vyos dhclient[3030]:
Jan 21 09:31:15 vyos dhclient[3030]: exiting.
Jan 21 09:31:15 vyos systemd[1]:
[email protected]: Main process exited, code=exited, status=1/FAILURE
Jan 21 09:31:15 vyos systemd[1]:
[email protected]: Failed with result 'exit-code'.
```
```
Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set client ID (len 18)
Jan 21 09:46:03 vyos dhcp6c[6091]: copyout_option: set identity association
Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set elapsed time (len 2)
Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set option request (len 4)
Jan 21 09:46:03 vyos dhcp6c[6091]: client6_send: transmit failed: Cannot assign requested address
Jan 21 09:46:03 vyos dhcp6c[6091]: dhcp6_reset_timer: reset a timer on wwan0, state=SOLICIT, timeo=6, retrans=68413
```
This is also noted by ModemManager (which is what VyOS uses to handle starting/stopping the connection) - https://modemmanager.org/docs/modemmanager/ip-connectivity-setup-in-lte-modems/
```
In particular, QMI modems running in raw-ip mode (without ethernet headers in the link layer) will by default be requested to use static IP addressing; not because the DHCP server in the modem doesn’t work, but because not all DHCP clients in Linux can work with network interfaces without ethernet headers (e.g. dhclient can’t).
```
The modem does have an IP address as mentioned above
```
vyos@vyos# sudo mmcli -m 0 -b 1
------------------------------------
General | path: /org/freedesktop/ModemManager1/Bearer/2
| type: default
------------------------------------
Status | connected: yes
| suspended: no
| multiplexed: no
| interface: wwan0
| ip timeout: 20
------------------------------------
Properties | apn: h2g2
| roaming: allowed
| ip type: ipv4v6
------------------------------------
IPv4 configuration | method: static
| address: 26.112.194.94
| prefix: 30
| gateway: 26.112.194.93
| dns: 10.177.0.34, 106.151.0.253
| mtu: 1500
------------------------------------
IPv6 configuration | method: static
| address: 2607:fb91:ee1:9e39:f488:eb51:81eb:2902
| prefix: 64
| gateway: 2607:fb91:ee1:9e39:a91c:c31f:4e25:a5e2
| dns: fd00:976a::9, fd00:976a::10
| mtu: 1500
------------------------------------
Statistics | start date: 2024-01-21T09:44:53Z
| duration: 240
| attempts: 1
| total-duration: 240
vyos@vyos# sudo qmicli -d /dev/cdc-wdm0 --device-open-proxy --wds-get-current-settings
[/dev/cdc-wdm0] Current settings retrieved:
IP Family: IPv4
IPv4 address: 26.112.194.94
IPv4 subnet mask: 255.255.255.252
IPv4 gateway address: 26.112.194.93
IPv4 primary DNS: 10.177.0.34
IPv4 secondary DNS: 106.151.0.253
MTU: 1500
Domains: none
```
Another note on the ModemManager page is
```
ModemManager does not perform the network interface configuration in the host, as it is exclusively in charge of communicating in the signaling plane between host and modem. The network manager process in use in the system will collect the IP settings exposed in the ModemManager Bearer object, and use them to configure the network interface accordingly.
```
Which leads me to believe NetworkManager will read that IP information (can get JSON output with `--output-json`) then manually configured the IP/default route. I did statically set the IP/route myself and was able to reach the internet via the 4G connection
I was hoping there is a way to use DHCP to do this still and I stumbled on an article (https://forums.raspberrypi.com/viewtopic.php?t=325044) that mentioning udhcpc would work with `raw-ip`, so I tried it and can confirm it works!
```
root@vyos# udhcpc -i wwan0
udhcpc: started, v1.35.0
udhcpc: broadcasting discover
udhcpc: broadcasting select for 26.112.68.182, server 26.112.68.181
udhcpc: lease of 26.112.68.182 obtained from 26.112.68.181, lease time 7200
```
Alternatively, I tried with `systemd-networkd` but did not get anywhere, and noticed this comment on an issue (https://github.com/systemd/systemd/issues/27219#issuecomment-1505640023) that seems to say that systemd-networkd does not support `raw-ip` either
```
vyos@vyos:~$ show version
Version: VyOS 1.5-rolling-202401161743
Release train: current
Built by:
[email protected]
Built on: Tue 16 Jan 2024 19:41 UTC
Build UUID: f7976e02-b740-4027-8c96-6335bbea4315
Build commit ID: 365f10340ec2f1
Architecture: x86_64
Boot via: installed image
System type: bare metal
Hardware vendor: Lanner Electronics Inc.
Hardware model: NCA-1515B-VS1
Hardware S/N: LR202112012772
Hardware UUID: 03000200-0400-0500-0006-000700080009
Copyright: VyOS maintainers and contributors
```
I believe this issue is not seen on modules/modems that support `802-3` for their link layer protocol as DHCP would work normally as expected in that case. It seems to be specifically related to `raw-ip`