Linksys E8450

From Leo's Notes
Last edited on 30 September 2022, at 05:28.

The Linksys E8450 is a Wi-Fi 6 (802.11ax) router. This router is based on MediaTek MT7622 SOC and a MT7531 switch. As of 2022, it is supported by OpenWRT 22.03.

I purchased this switch on sale from Amazon for $109 and later $90 CAD in mid 2022.

OpenWRT

Installing OpenWRT

If you are installing OpenWRT from the stock Linksys firmware, please refer to the instructions outlined by Daniel at https://github.com/dangowrt/owrt-ubi-installer.

Linksys E8450 Wizard
Linksys E8450 Wizard

Login to stock firmware

The steps that I used to install OpenWRT are as follows:

  1. Power on the device, navigate to http://192.168.1.1.
  2. Click on the radio button and click 'Cancel' to quit the setup wizard.
  3. Login using 'admin' as the password.
Linksys E8450 Flash Firmware
Linksys E8450 Flash Firmware

Backup flash data

  1. Ensure that you are using version 1.0 of the Linksys firmware. You might want to consider downgrading if you are on a newer version.
  2. Go to Configuration -> Administration -> Firmware Upgrade
  3. Upload linksys_e8450-ubi-initramfs-recovery.itb (Note: Not the installer, we want the initramfs one to do a flash backup only.)
  4. The firmware update page will reload once the device reboots. You should be able to see OpenWRT at 192.168.1.1
  5. Backup the flash contents in case you need to restore to the stock firmware. (Note: You'll likely need serial access to restore, which requires soldering wires onto the board). SSH to root@192.168.1.1 and dump the flash:
# cd /dev
# for part in mtd[0123] ; do dd if=$part of=/tmp/$part; done

Copy the resulting mtd files to your computer. There should be 4 files each being 524288, 1310720, 1048576, and 131072000 bytes large.

If you didn't do this step, a copy of the first 3 and a partial of the last mtd block are saved on the ubi0_3 device by the OpenWRT installer / conversion script (in the next step below). You can get these by mounting ubi0_3 with: mount -t ubifs -o ro /dev/ubi0_3 /mnt/.

Installing OpenWRT for real

  1. Reboot the router 3 times until it boots back into the original Linksys firmware. You may need to power cycle the device multiple times. Do not try anything else other than rebooting / powercycling.
  2. On the original Linksys firmware, go to Configuration -> Administration -> Firmware Upgrade
  3. Upload the UBI based installer openwrt-mediatek-mt7622-linksys_e8450-ubi-initramfs-recovery-installer.itb. This installer will set up the UBI based filesystem by executing a conversion script.
  4. The firmware update page will reload once the device reboots. You should be able to see OpenWRT at 192.168.1.1
  5. Upgrade the OpenWRT firmware using the UBI sysupgrade file. Go to System -> Flash Firmware, under "Flash new firmware image", upload openwrt-mediatek-mt7622-linksys_e8450-ubi-squashfs-sysupgrade.itb. No need to save any configs or package lists.
  6. Once the device reboots, you should see OpenWRT again. This time, when you log in, it should not say it is running in initramfs / recovery mode.

Configuring OpenWRT

Log in to OpenWRT and navigate to System -> Startup -> Local Startup. Edit /etc/rc.local with the following contents. These changes are to:

  1. Prevent the system from locking up when the iwinfo library tries to read the now invalid /dev/mtdblock2 device
  2. To make sure the device doesn't hang on boot due to the CPU clock being too low (defaults to 300MHz), resulting in "the CPU supply voltage [set] to 0.9V which causes DDR RAM calibration in ARM Trusted Firmware bl2 to never complete and hang forever". [1] 437.5MHz apparently is what you need.
  3. Change the CPU governor to either ondemand or schedutil to allow CPU to clock down when idle (to slightly reduce power usage)
# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

# libiwinfo will try to read /dev/mtdblock*, but mtdblock2 is invalid and can hang up everything.
mv /dev/mtdblock2 /dev/mtdblock2a

# Defaults to 300000
echo 437500 > /sys/devices/system/cpu/cpufreq/policy0/scaling_min_freq
# Defaults to userspace
echo schedutil > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor

exit 0

Upgrading OpenWRT

After you have installed OpenWRT with the steps above, you can upgrade to newer versions of OpenWRT by uploading a UBI-compatible sysupgrade file from:

Make sure you select the ubi-squashfs-upgrade.itb file. Eg. openwrt-22.03-snapshot-r19451-2726c8c315-mediatek-mt7622-linksys_e8450-ubi-squashfs-sysupgrade.itb.

Note that you'll lose any installed packages once you do this upgrade. If you have any specific kernel modules or programs installed, you'll have to reinstall them again. The configs however should remain persistent.

Troubleshooting

blk_update_request I/O errors on mtdblock2

The Linksys E8450 running OpenWRT started stalling out and reporting lots of blk_update_request I/O errors on /dev/mtdblock2. This started to happen recently, perhaps after adding more devices to the network. When the stall happens, WiFi connections just stop working entirely on both 2.4 and 5GHz. Luci, the web control panel, also doesn't respond.

Solution / Update: Move the /dev/mtdblock2 device to /dev/mtdblock2a (or some other name) so that the iwinfo library won't try reading it and hang up the system.

My original troubleshooting log follows below..

Checking on the kernel dmesg logs show that some sectors are failing to write:

[355545.032552] blk_update_request: I/O error, dev mtdblock2, sector 256 op 0x0:(READ) flags 0x84700 phys_seg 32 prio class 0
[355545.045135] blk_update_request: I/O error, dev mtdblock2, sector 264 op 0x0:(READ) flags 0x84700 phys_seg 31 prio class 0
[355545.056711] blk_update_request: I/O error, dev mtdblock2, sector 272 op 0x0:(READ) flags 0x84700 phys_seg 30 prio class 0
[355545.068156] blk_update_request: I/O error, dev mtdblock2, sector 280 op 0x0:(READ) flags 0x84700 phys_seg 29 prio class 0
[355545.079640] blk_update_request: I/O error, dev mtdblock2, sector 288 op 0x0:(READ) flags 0x84700 phys_seg 28 prio class 0
[355545.091038] blk_update_request: I/O error, dev mtdblock2, sector 296 op 0x0:(READ) flags 0x84700 phys_seg 27 prio class 0
[355545.102418] blk_update_request: I/O error, dev mtdblock2, sector 304 op 0x0:(READ) flags 0x84700 phys_seg 26 prio class 0
[355545.113821] blk_update_request: I/O error, dev mtdblock2, sector 312 op 0x0:(READ) flags 0x84700 phys_seg 25 prio class 0
[355545.125211] blk_update_request: I/O error, dev mtdblock2, sector 320 op 0x0:(READ) flags 0x84700 phys_seg 24 prio class 0
[355545.136592] blk_update_request: I/O error, dev mtdblock2, sector 328 op 0x0:(READ) flags 0x84700 phys_seg 23 prio class 0
[355545.155734] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.164051] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.172324] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.180307] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.188286] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.196255] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.204253] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.213419] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.221699] Buffer I/O error on dev mtdblock2, logical block 32, async page read
[355545.231766] Buffer I/O error on dev mtdblock2, logical block 32, async page read

And the kernel logs just continue erroring with the Buffer I/O error message.

For some reason, /sbin/rpcd -s /var/run/ubus/ubus.sock -t 30 was using 25% CPU. Killing it brought back Luci?

/dev/mtdblock2 really can't be read:

root@linksys:/# dd if=/dev/mtdblock2  of=/tmp/x conv=noerror
dd: /dev/mtdblock2: I/O error
dd: /dev/mtdblock2: I/O error
dd: /dev/mtdblock2: I/O error
dd: /dev/mtdblock2: I/O error
dd: /dev/mtdblock2: I/O error
...

root@linksys:/# dmesg
[149324.472780] print_req_error: 3434 callbacks suppressed
[149324.472788] blk_update_request: I/O error, dev mtdblock2, sector 256 op 0x0:(READ) flags 0x84700 phys_seg 26 prio class 0
[149324.489773] blk_update_request: I/O error, dev mtdblock2, sector 264 op 0x0:(READ) flags 0x84700 phys_seg 25 prio class 0
[149324.501309] blk_update_request: I/O error, dev mtdblock2, sector 272 op 0x0:(READ) flags 0x84700 phys_seg 24 prio class 0
[149324.513110] blk_update_request: I/O error, dev mtdblock2, sector 280 op 0x0:(READ) flags 0x84700 phys_seg 23 prio class 0
[149324.524642] blk_update_request: I/O error, dev mtdblock2, sector 288 op 0x0:(READ) flags 0x84700 phys_seg 22 prio class 0
[149324.536136] blk_update_request: I/O error, dev mtdblock2, sector 296 op 0x0:(READ) flags 0x84700 phys_seg 21 prio class 0
[149324.547439] blk_update_request: I/O error, dev mtdblock2, sector 304 op 0x0:(READ) flags 0x84700 phys_seg 20 prio class 0
[149324.558870] blk_update_request: I/O error, dev mtdblock2, sector 312 op 0x0:(READ) flags 0x84700 phys_seg 19 prio class 0
[149324.570155] blk_update_request: I/O error, dev mtdblock2, sector 320 op 0x0:(READ) flags 0x84700 phys_seg 18 prio class 0
[149324.581596] blk_update_request: I/O error, dev mtdblock2, sector 328 op 0x0:(READ) flags 0x84700 phys_seg 17 prio class 0
[149324.604720] buffer_io_error: 3425 callbacks suppressed
[149324.604728] Buffer I/O error on dev mtdblock2, logical block 32, async page read

Updated from OpenWRT 22.03-SNAPSHOT to 22.03.0 but still can't read mtdblock2. No USB devices are mounted, in case that's relevant.

Looking through this very unwieldy discussion https://forum.openwrt.org/t/belkin-rt3200-linksys-e8450-wifi-ax-discussion/94302, the developer (daniel) says "This error can show when trying to access areas in the flash with broken OOB/ECC data." and that mtdblock2 is the factory partition containing calibration data.

Restarting the 5GHz WiFi will trigger this error, despite me trying to make /dev/mtdblock2 chmod 000.

lsof shows the rpcd process having /dev/mtdblock2 open:

root@linksys:~# lsof -p 3422
COMMAND  PID USER   FD      TYPE             DEVICE SIZE/OFF                NODE NAME
rpcd    3422 root  cwd       DIR               0,18      352                  67 /
rpcd    3422 root  rtd       DIR               0,18      352                  67 /
rpcd    3422 root  txt       REG               0,18    66139 9223372036854776300 /sbin/rpcd
rpcd    3422 root  mem       REG              259,1                          492 /sbin/rpcd (path dev=0,18, inode=9223372036854776300)
rpcd    3422 root  mem       BLK               31,8                           54 /dev/mtdblock2
rpcd    3422 root  mem       REG              259,1                         1122 /usr/lib/rpcd/rrdns.so (path dev=0,18, inode=9223372036854776930)
rpcd    3422 root  mem       REG              259,1                         1121 /usr/lib/rpcd/luci.so (path dev=0,18, inode=9223372036854776929)
rpcd    3422 root  mem       REG              259,1                          599 /usr/lib/libnl-tiny.so (path dev=0,18, inode=9223372036854776407)
rpcd    3422 root  mem       REG              259,1                          583 /usr/lib/libiwinfo.so.20210430 (path dev=0,18, inode=9223372036854776391)
rpcd    3422 root  mem       REG              259,1                         1120 /usr/lib/rpcd/iwinfo.so (path dev=0,18, inode=9223372036854776928)
rpcd    3422 root  mem       REG              259,1                         1119 /usr/lib/rpcd/file.so (path dev=0,18, inode=9223372036854776927)
rpcd    3422 root  mem       REG              259,1                          318 /lib/libgcc_s.so.1 (path dev=0,18, inode=9223372036854776126)
rpcd    3422 root  mem       REG              259,1                          588 /usr/lib/libjson-c.so.5.1.0 (path dev=0,18, inode=9223372036854776396)
rpcd    3422 root  mem       REG              259,1                          315 /lib/libblobmsg_json.so.20220515 (path dev=0,18, inode=9223372036854776123)
rpcd    3422 root  mem       REG              259,1                          325 /lib/libuci.so (path dev=0,18, inode=9223372036854776133)
rpcd    3422 root  mem       REG              259,1                          324 /lib/libubus.so.20220601 (path dev=0,18, inode=9223372036854776132)
rpcd    3422 root  mem       REG              259,1                          323 /lib/libubox.so.20220515 (path dev=0,18, inode=9223372036854776131)
rpcd    3422 root  mem       REG              259,1                          316 /lib/libc.so (path dev=0,18, inode=9223372036854776124)
rpcd    3422 root    0r      CHR                1,3      0t0                  26 /dev/null
rpcd    3422 root    1w      CHR                1,3      0t0                  26 /dev/null
rpcd    3422 root    2w      CHR                1,3      0t0                  26 /dev/null
rpcd    3422 root    3u  a_inode                0,8        0                  74 [eventpoll:4,6]
rpcd    3422 root    4r     FIFO                0,7      0t0               11701 pipe
rpcd    3422 root    5w     FIFO                0,7      0t0               11701 pipe
rpcd    3422 root    6u     unix 0x000000000aec3471      0t0               11702 type=STREAM
rpcd    3422 root    7u  netlink                         0t0               15712 GENERIC
rpcd    3422 root    8r      BLK               31,8      0t0                  54 /dev/mtdblock2

This is likely what's causing things to hang since rpcd is used for handling other things like WiFi connections or luci.

Searching through the entire filesystem for mtdblock2 returned nothing. The discussion mentioned that iwinfo might be reading mtdblock2. There is a string 'mtdblock%d' in the libiwinfo.so.20210430 library. The source for this does indeed show it try to read /dev/mtdblockN as seen here: https://git.openwrt.org/?p=project/iwinfo.git;a=blob_plain;f=iwinfo_utils.c;hb=3d38d1d17eba37b0985dd2857b4539410824a3e6 in order to determine the hardware information.

Perhaps a simple fix here would be to rename /dev/mtdblock2 to /dev/mtdblock2a so it can't try reading from the device? Basically, mv /dev/mtdblock2 /dev/mtdblock2a

See also