APM X-C1 (Mustang): Difference between revisions

From Leo's Notes
This page was last edited on 14 July 2017, at 18:32.
Line 615: Line 615:
}}
}}


The kernel output is probably going to the wrong place. Set the {{code|console=ttyS0,115200}} as part of the kernel arguments.
The kernel output is probably going to the wrong place. Set the {{code|1=console=ttyS0,115200}} as part of the kernel arguments.




[[Category:Linux]]
[[Category:Linux]]

Revision as of 18:32, 14 July 2017

The APM X-C1 (code name Mustang) is a server chip under their X-GENE family of products.

First Look

The APM unit is a compact box. Because it has no graphical output, the sole means to interface with it out of the box is via serial.

The box talks at 19200 baud (ie: screen /dev/ttyS0 19200)

U-Boot

The board came with U-Boot loaded. When the system is first turned on, you will see the following in the console:

U-Boot 2013.04-mustang_sw_1.12.09-beta (Jun 12 2014 - 10:48:45)

CPU0: APM ARM 64-bit Potenza Rev A3 2400MHz PCP 2400MHz
     32 KB ICACHE, 32 KB DCACHE
     SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz
Boot from SPI-NOR
Board: Mustang - AppliedMicro APM883208-xNA24SPT Reference Board
I2C:   ready
DRAM:  ECC 16 GiB @ 1600MHz
SF: Detected N25Q256 with page size 256 Bytes, total 32 MiB
MMC:   X-Gene SD/SDIO/eMMC: 0
PCIE0: (RC) link down
In:    serial
Out:   serial
Err:   serial
CPUs:  11111111
Net:   eth0

Here are the environment variables that are set in the U-Boot shell out of the box:

Mustang# printenv
addip=setenv bootargs ${bootargs} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}:${netdev}:off panic=1
addmisc=setenv bootargs ${bootargs} earlyprintk=uart8250-32bit,0x1c020000 debug maxcpus=${num_cores} swiotlb=65536
addtty=setenv bootargs ${bootargs} console=ttyS0,${baudrate}
baudrate=115200
bootcmd=run sata_self
bootdelay=5
bootfile=uImage
dnsip=10.66.18.10
enet=8
enet_mode=x:n
eth1addr=00:01:73:02:21:35
eth2addr=00:01:73:02:21:36
eth3addr=00:01:73:02:21:37
ethact=eth0
ethaddr=00:01:73:02:21:34
fdt_addr_f=0x1000000
fdt_addr_r=0x4003000000
fdt_file=apm-mustang.dtb
fdt_size=7285
flash_nfs=run nfsargs addip addtty addmisc; bootm ${kern_addr_f}
flash_self=run ramargs addip addtty addmisc; bootm ${kern_addr_f} ${ramdisk_addr_f} ${fdt_addr_f}
fw_addr_r=0x4003800000
gatewayip=10.66.19.1
hostname=dhcp-10-66-19-173
initrd_high=0x43e0000000
ipaddr=10.66.19.173
kern_addr_f=0x100000
kern_addr_r=0x4002000000
kern_size=a9a050
kernel_addr_r=0x4002000000
load=tftp ${uboot_addr_r} ${user_dir}/${u-boot}
load_fdt=tftp ${fdt_addr_r} ${user_dir}/${fdt_file}
load_kern=tftp ${kern_addr_r} ${user_dir}/${bootfile}
load_ramdisk=tftp ${ramdisk_addr_r} ${user_dir}/${ramdisk_file}
loadaddr=0x100000000
media_addr_r=0x4001000000
media_img=mustang_media.img
mgmt_enet=e
mslim_bootfile=uImage-mslim
mslim_fdt_addr_r=0x43f0600000
mslim_fdt_file=mslim-mustang.dtb
mslim_kern_addr_r=0x43f1000000
mslim_net_self=run net_load_mslim; startmslim
mslim_ramdisk_addr_r=0x43f2000000
mslim_ramdisk_file=uRamdisk-mslim
mslim_sd_self=run sd_load_mslim; startmslim
mslim_uboot_addr_r=0x43f0000000
mslim_uboot_file=u-boot-mslim.bin
mslim_user_dir=mslim
net_load_mslim=run nload_mslim_uboot nload_mslim_kern nload_mslim_ramdisk nload_mslim_fdt
net_nfs=run load_kern load_fdt; run nfsargs addip addtty addmisc; bootm ${kern_addr_r} - ${fdt_addr_r}
net_sata=run load_kern load_fdt; run sataargs addip addtty addmisc; bootm ${kern_addr_r} - ${fdt_addr_r}
net_self=run net_self_load; run ramargs addip addtty addmisc; bootm ${kern_addr_r} ${ramdisk_addr_r} ${fdt_addr_r}
net_self_load=run load_kern load_ramdisk load_fdt
netdev=eth0
netmask=255.255.255.0
nfsargs=setenv bootargs root=/dev/nfs rw nfsroot=${serverip}:${rootpath}
nload_mslim_fdt=tftp ${mslim_fdt_addr_r} ${mslim_user_dir}/${mslim_fdt_file}
nload_mslim_kern=tftp ${mslim_kern_addr_r} ${mslim_user_dir}/${mslim_bootfile}
nload_mslim_ramdisk=tftp ${mslim_ramdisk_addr_r} ${mslim_user_dir}/${mslim_ramdisk_file}
nload_mslim_uboot=tftp ${mslim_uboot_addr_r} ${mslim_user_dir}/${mslim_uboot_file}
num_cores=8
pcie_gen=3:3:3:3:3
pcie_mode=RP:NA:NA:NA:NA
pcie_reset=y:n:n:n:n
pxefile_addr_r=0x4001000000
ram_nfs=run nfsargs addip addtty addmisc; bootm ${kern_addr_r} - ${fdt_addr_r}
ram_self=run ramargs addip addtty addmisc; bootm ${kern_addr_r} ${ramdisk_addr_r} ${fdt_addr_r}
ramargs=setenv bootargs root=/dev/ram rw
ramdisk_addr_f=0x1100000
ramdisk_addr_r=0x4004000000
ramdisk_file=uRamdisk_basic
ramdisk_size=344abe
rgmii_phyid=3
rootpath=/opt/nfsrootfs/arm64/release
sata=0x3c
sata_enet=e:e:n:n
sata_load_fdt=ext4load scsi 0:1 0x4003000000 apm-mustang.dtb
sata_load_kern=ext4load scsi 0:1 0x4002000000 uImage
sata_self=scsi init; run sata_load_kern sata_load_fdt; run sataargs addip addtty addmisc; bootm 0x4002000000 - 0x4003000000
sataargs=setenv bootargs root=/dev/sda1 rw
script_addr_r=0x4004000000
sd_load_mslim=mmc dev 0; run sdload_mslim_uboot sdload_mslim_kern sdload_mslim_ramdisk sdload_mslim_fdt
sd_self=mmc dev 0; ext2load mmc 0  ${kern_addr_r} ${bootfile}; ext2load mmc 0  ${fdt_addr_r} ${fdt_file}; ext2load mmc 0  ${ramdisk_addr_r} ${ramdisk_file}; run ram_self
sdload_mslim_fdt=ext2load mmc 0 ${mslim_fdt_addr_r} ${mslim_fdt_file}
sdload_mslim_kern=ext2load mmc 0 ${mslim_kern_addr_r} ${mslim_bootfile}
sdload_mslim_ramdisk=ext2load mmc 0 ${mslim_ramdisk_addr_r} ${mslim_ramdisk_file}
sdload_mslim_uboot=ext2load mmc 0 ${mslim_uboot_addr_r} ${mslim_uboot_file}
sdphy=1
serverip=10.66.19.133
sgmii_phyid=4:5::
spi_fdt_upd=run load_fdt; setenv fdt_size ${filesize}; save; run update_fdt
spi_fw_img=mustang_slimpro_ext_spi.bin
spi_fw_load=tftp ${fw_addr_r} ${user_dir}/${spi_fw_img}
spi_fw_upd=run spi_fw_load;run spi_fw_update
spi_fw_update=bmc stop; i2c dev 1; eeprom write ${fw_addr_r} 0x0 ${filesize}
spi_kern_upd=run load_kern; setenv kern_size ${filesize}; save; run update_kern
spi_load=tftp ${media_addr_r} ${user_dir}/${media_img}
spi_ramdisk_upd=run load_ramdisk; setenv ramdisk_size ${filesize}; save; run update_ramdisk
spi_self=sf probe 0; sf read ${kern_addr_r} ${kern_addr_f} ${kern_size}; sf read ${fdt_addr_r} ${fdt_addr_f} ${fdt_size}; sf read ${ramdisk_addr_r} ${ramdisk_addr_f} ${ramdisk_size}; run ram_self
spi_upd=run spi_load;run spi_update
spi_update=sf probe 0; sf erase 0x0 ${filesize}; sf write ${media_addr_r} 0x0 ${filesize}
stderr=serial
stdin=serial
stdout=serial
u-boot=u-boot.bin
uboot_addr_f=0x8100000000
uboot_addr_r=0x4001000000
uboot_size=0x80000
upd=run load; setenv uboot_size ${filesize};run update
updall=run spi_kern_upd; run spi_ramdisk_upd; run spi_fdt_upd
update=protect off ${uboot_addr_f} +${uboot_size}; erase ${uboot_addr_f} +${uboot_size}; cp.b ${uboot_addr_r} ${uboot_addr_f} ${uboot_size}
update_fdt=sf probe 0; sf erase ${fdt_addr_f} ${filesize}; sf write ${fdt_addr_r} ${fdt_addr_f} ${filesize}
update_kern=sf probe 0; sf erase ${kern_addr_f} ${filesize}; sf write ${kern_addr_r} ${kern_addr_f} ${filesize}
update_ramdisk=sf probe 0; sf erase ${ramdisk_addr_f} ${filesize}; sf write ${ramdisk_addr_r} ${ramdisk_addr_f} ${filesize}
user_dir=/1.12.09-Beta/BUILDS/potenza
xgmii_phyid=:

Environment size: 5832/32764 bytes

The system comes with their version of Linux installed. It's using a custom version of the 3.12 Linux Kernel. Interestingly, the hostname does not reflect the actual IP address, so most likely it's the IP from a network install from the factory.

     POWERED BY:

     X     X             GGGG
      X   X             G    G
       X X             G         EEEEE   N NNNN    EEEEE
        X     #######  G   GGG  E     E  NN    N  E     E
       X X             G     G  EEEEEEE  N     N  EEEEEEE
      X   X             G    G  E        N     N  E
     X     X             GGGGG   EEEEE   N     N   EEEEE   (TM)

     Welcome to Applied Micro X-Gene 64-bit ARM Server Platform

Applied Micro Linux 3.12.0-mustang_sw_1.12.09-beta (aarch64) (ttyS0)
dhcp-10-66-19-173 login: root
Password:
[root@dhcp-10-66-19-173 ~]# cat /proc/cpuinfo
Processor       : AArch64 Processor rev 0 (aarch64)
processor       : 0
model name      : APM X-Gene Mustang board @ 2.40GHz
cpu MHz         : 2400.00
fpu             : yes

processor       : 1
model name      : APM X-Gene Mustang board @ 2.40GHz
cpu MHz         : 2400.00
fpu             : yes

processor       : 2
model name      : APM X-Gene Mustang board @ 2.40GHz
cpu MHz         : 2400.00
fpu             : yes

processor       : 3
model name      : APM X-Gene Mustang board @ 2.40GHz
cpu MHz         : 2400.00
fpu             : yes

processor       : 4
model name      : APM X-Gene Mustang board @ 2.40GHz
cpu MHz         : 2400.00
fpu             : yes

processor       : 5
model name      : APM X-Gene Mustang board @ 2.40GHz
cpu MHz         : 2400.00
fpu             : yes

processor       : 6
model name      : APM X-Gene Mustang board @ 2.40GHz
cpu MHz         : 2400.00
fpu             : yes

processor       : 7
model name      : APM X-Gene Mustang board @ 2.40GHz
cpu MHz         : 2400.00
fpu             : yes

Features        : fp asimd
CPU implementer : 0x50
CPU architecture: AArch64
CPU variant     : 0x0
CPU part        : 0x000
CPU revision    : 0

Hardware        : APM X-Gene Mustang board

Installing Fedora

The steps outlined below should let you successfully install Fedora 21 and higher.

Before you begin, read through the guide on the FedoraProject wiki: https://fedoraproject.org/wiki/Architectures/AArch64/Hardware/Mustang?rd=Architectures/ARM/AArch64/Mustang

To install Fedora on the system, we need to get the UEFI loader running first. You can either chainload this from U-Boot by having U-Boot run the UEFI loader from the SD card on start-up, or flash it directly to the system and replace the U-Boot loader altogether. Since I don't want to replace U-Boot, I will chainload it during startup.

Once the UEFI loader is installed and is made to start automatically, you will need to install Fedora from either USB or PXE. USB will be easier to do unless you already have a PXE server setup to handle aarch64 and UEFI.

Find a suitable mirror with aarch64 at https://admin.fedoraproject.org/mirrormanager/mirrors/Fedora/23/aarch64

Preparing the Net Install USB Stick

The FedoraProject wiki has instructions on setting up the USB stick in addition to a USB image that you dd to the drive. Unfortunately the instructions are pretty sparse and the USB net install image exist only for Fedora 21.

We can make use of the Fedora 21 USB net image to boot the Fedora 22 and 23 by altering the initrd image on the USB net install image.

First, download and extract the USB net install image at https://pwhalen.fedorapeople.org/Fedora/21/aarch64/

# wget https://pwhalen.fedorapeople.org/Fedora/21/aarch64/Fedora-Server-netinst-aarch64-21.img.xz
# unxz Fedora-Server-netinst-aarch64-21.img.xz

Second, determine and mount the data partition from the USB image in order ot update the kernel and initrd image.

# fdisk -l Fedora-Server-netinst-aarch64-21.img
Disk Fedora-Server-netinst-aarch64-21.img: 110 MiB, 115343360 bytes, 225280 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 70AE0993-389E-4DB6-9455-DB9A75729515

Device                                       Start          End Size Type
Fedora-Server-netinst-aarch64-21.img1         2048        22527  10M EFI System
Fedora-Server-netinst-aarch64-21.img2        22528       225246  99M Linux filesystem

# mount -o loop,offset=$((22528*512)) /tmp/Fedora-Server-netinst-aarch64-21.img /mnt/usb

Then replace vmlinuz and initrd.img under /images/pxeboot with the newer Fedora release kernel/initrd.

Optionally, update the grub config so that the installer will point to the proper repository. If you skip this step, you have the option to edit it manually in the grub menu later, but you'll have to do it every time you boot the image. Do not update the search --no-floppy --set=root -l 'Fedora-S-21-aarc' line in the grub config unless you also update the USB label. Otherwise, grub will not find the files and give you error: file `/images/pxeboot/vmlinuz' not found..

# mount -o loop,offset=$((2048*512)) /tmp/Fedora-Server-netinst-aarch64-21.img /mnt/usb
# cd /mnt/usb/EFI/BOOT
# vi grub.cfg

Lastly, unmount the filesystem and dd the image to the USB drive.

# umount /mnt/usb
# dd if=Fedora-Server-netinst-aarch64-21.img of=/dev/sdb

Chain-Loading UEFI

There are issues with doing this!
This method of chainloading UEFI from u-boot will cause the kernel to fail at starting up any additional CPUs probably because the device tree binary blob isn't provided.

You should really just flash the box with APM's UEFI tianocore bios instead described in the next section.


Download mustang_uefi_upgrade.tar.gz and dump the contents onto a 2GB FAT16 formatted SD card partition. U-Boot doesn't appear to see the partition if it is any larger than 2GB. Insert the SD card into the X-C1 board and turn it on. Enter the U-Boot shell and run the following:

U-Boot 2013.04-mustang_sw_1.12.09-beta (Jun 12 2014 - 10:48:45)

CPU0: APM ARM 64-bit Potenza Rev A3 2400MHz PCP 2400MHz
     32 KB ICACHE, 32 KB DCACHE
     SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz
Boot from SPI-NOR
Board: Mustang - AppliedMicro APM883208-xNA24SPT Reference Board
I2C:   ready
DRAM:  ECC 16 GiB @ 1600MHz
SF: Detected N25Q256 with page size 256 Bytes, total 32 MiB
MMC:   X-Gene SD/SDIO/eMMC: 0
PCIE0: (RC) link down
In:    serial
Out:   serial
Err:   serial
CPUs:  11111111
Net:   eth0
Hit any key to stop autoboot:  0
Mustang# setenv rh_sd_tiano_sec_ubt fatload mmc 0:1 0x1d000000 /mustang/mustang_tianocore_sec_ubt.fd
Mustang# setenv rh_sd_tiano_ubt fatload mmc 0:1 0x4002000000 /mustang/mustang_tianocore_ubt.fd
Mustang# setenv rh_run_tiano_ubt go 0x1d000000
Mustang# setenv bootcmd run rh_sd_tiano_ubt rh_sd_tiano_sec_ubt rh_run_tiano_ubt
Mustang# saveenv
Saving Environment to SPI Flash...
SF: Detected N25Q256 with page size 256 Bytes, total 32 MiB
.Erasing SPI flash.....SF: STMicro: Successfully erased 262144 bytes @ 0xc0000
Writing to SPI flash.......done

The commands above create three new boot variables, rh_sd_tiano_ubt, rh_sd_tiano_sec_ubt, and rh_run_tiano_ubt defining the file and location to execute. It also updates the automatic boot command to these variables.

From now on, U-Boot will now run run rh_sd_tiano_ubt rh_sd_tiano_sec_ubt rh_run_tiano_ubt every time it boots which loads the UEFI loader from the SD card.

You may either reset the machine with the reset command, or run the bootcmd command to continue with the installation.

Mustang# reset
resetting ...
cold reset ...▒

Flashing Tianocore UEFI Bios

Flashing the BIOS involves booting the Mustang box into the UEFI BIOS (if it's still using uboot) and then running the EFI update program via the UEFI shell.

If you are still using uboot, you can force the board to boot into the UEFI shell by loading the UEFI BIOS onto an SD card and forcing it to boot from the SD card by setting two jumpers on the board.

Copy your MAC address
Your Mustang box will lose its configured MAC address during the flashing process. Make sure you copy the MAC address *before* you do anything.


The Tianocore UEFI bios can be obtained from MyAPM's website. The latest version as of this writing is version 3.06.25b released in 2016.

To update the BIOS on an already UEFI BIOS

## Download and extract mustang_tianocore_binaries_3.06.25.tar.xz
## Copy the files to the root directory of a FAT formatted USB drive.
# cp mustang_binaries/* /mnt/usb
# ls -al /mnt/usb
total 6.2M
drwxr-x---. 2 root root  16K Jul 14 10:47 ./
drwxr-xr-x. 3 root root 4.0K Jul 14 10:46 ../
-rwxr-x---. 1 root root   64 Jul 14 10:39 apm_upgrade_mustang_tianocore.cmd*
-rwxr-x---. 1 root root 2.1M Jul 14 10:39 mustang_firmware_capsule.Cap*
-rwxr-x---. 1 root root 2.1M Jul 14 10:39 mustang_tianocore_media.img*
-rwxr-x---. 1 root root 256K Jul 14 10:39 mustang_tianocore_sec_spi.fd*
-rwxr-x---. 1 root root 1.8M Jul 14 10:39 mustang_tianocore_spi.fd*
-rwxr-x---. 1 root root  58K Jul 14 10:39 UpgradeFirmware.efi*
# umount /mnt/usb

## Plug the USB to the APM board and restart it.
## On the serial console, enter the EFI Shell, then run:
FS0:\> FS2:
FS2:\> dir
Directory of: FS2:\
07/14/2017  10:39                  64  apm_upgrade_mustang_tianocore.cmd
07/14/2017  10:39           2,129,952  mustang_firmware_capsule.Cap
07/14/2017  10:39           2,129,920  mustang_tianocore_media.img
07/14/2017  10:39             262,144  mustang_tianocore_sec_spi.fd
07/14/2017  10:39           1,835,008  mustang_tianocore_spi.fd
07/14/2017  10:39              58,944  UpgradeFirmware.efi
          6 File(s)   6,416,032 bytes

FS2:\> UpgradeFirmware.efi

Firmware Upgrader
Update Firmware via USB/SD/SATA
Enter command file path: apm_upgrade_mustang_tianocore.cmd

Writing Tianocore firmware from the file: mustang_tianocore_media.img
Writing the firmware. Please wait...

Upgrade the firmware successfully

**************************

## Reboot the machine

To update the BIOS from U-BOOT

  1. Create the SD card that will contain the UEFI bios. Copy the tianocore firmware at BUILDS/potenza/tianocore_media.img to a SD card dd if=BUILDS/potenza/tianocore_media.img of=/dev/sdx.
  2. Power off the machine
  3. Configure the machine to boot from the SD card by closing jumpers 11-12 and 17-18 on the motherboard.
  4. Insert the SD card
  5. Follow the steps above to create a USB drive containing the BIOS update and apply the update normally.
  6. Revert the jumpers in the step above. Remove the SD card and USB drive.

When the machine powers up again, it should boot into the Tianocore UEFI bios. The MAC address out of the box will be set to 00:00:00:00:00:00 and may cause issues with the OS trying to bring the network up. Fix this by going back to the UEFI shell and run:

# set MAC0 00:01:73:02:1e:10

For more information on flashing the APM Mustang's firmware, see https://marcin.juszkiewicz.com.pl/2015/11/18/unbricking-apm-mustang/.

Installing Fedora

Insert the Fedora USB stick to the first USB port (the one nearest the board under the Ethernet jack). When the UEFI loader starts, enter the shell.

## Starting application at 0x1D000000 ...
X-Gene Mustang Board
Boot firmware (version 1.1.0 built at 12:03:43 on Mar 23 2015)
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3021001 I0
TianoCore 1.1.0 UEFI 2.4.0 Mar 23 2015 12:03:18
CPU: APM ARM 64-bit Potenza Rev A3 2400MHz PCP 2400MHz
     32 KB ICACHE, 32 KB DCACHE
     SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz
Board: X-Gene Mustang Board
Slimpro FW:
        Ver: 2.1
The default boot selection will start in   4 seconds
[1] Shell
[2] Boot Manager
[3] Reboot
[4] Shutdown
Start: 1

In the UEFI shell, execute:

# FS0:\EFI\BOOT\BOOTAA64.EFI

FS0 should be the SD card. If you aren't sure, enter the drive by running FS0: and list the directories using dir.

The BOOTAA64.EFI program will eventually start grub and from there the Fedora installer.

.
      Install Fedora-Server 23
      Test this media & install Fedora-Server 23
      Troubleshooting -->














      Use the  and  keys to change the selection.
      Press 'e' to edit the selected item, or 'c' for a command prompt.

Continue the Fedora installation via VNC when prompted by Anaconda. You should use auto partitioning since it will automatically create the necessary EFI partition for the UEFI loader to work properly.

Tip: You can Alt-Tab in the Anaconda console using Esc-Tab key combination.

See Also

Troubleshooting

Bad Flash

When starting the box up, you get:

X-Gene Mustang Board
Boot firmware (version 1.15.23 built at 13:21:52 on Nov 24 2015)
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3020002 I0
PROGRESS CODE: V3020003 I0
PROGRESS CODE: V3021001 I0


Synchronous Exception at 0x00000043FFA1D024
0: FP 0x00000043FFA78AE0 SP 0x00000043FFA78AE0 PC 0x00000043FFA1D6B0
1: FP 0x00000043FFA78AF0 SP 0x00000043FFA78AF0 PC 0x00000043FFA1D6AC
2: FP 0x00000043FFA78B50 SP 0x00000043FFA78B00 PC 0x00000043FFA1DDC4
3: FP 0x00000043FFA78BD0 SP 0x00000043FFA78B60 PC 0x00000043FF9C4560
4: FP 0x00000043FFA78D10 SP 0x00000043FFA78BE0 PC 0x00000043FFA82D4C
5: FP 0x00000043FFA78D50 SP 0x00000043FFA78D20 PC 0x00000043FFA82914
6: FP 0x00000043FFA78D90 SP 0x00000043FFA78D60 PC 0x00000043FFA831B8
7: FP 0x00000043FFA78DB0 SP 0x00000043FFA78DA0 PC 0x00000043FF9C47E4
8: FP 0x00000043FFA78DE0 SP 0x00000043FFA78DC0 PC 0x00000043FF9C4318
9: FP 0x00000043FFA78E20 SP 0x00000043FFA78DF0 PC 0x00000043FFA7BBD0
10: FP 0x00000043FFA78E70 SP 0x00000043FFA78E30 PC 0x00000043FFA83D60
11: FP 0x00000043FFA78EF0 SP 0x00000043FFA78E80 PC 0x00000043FFA79748
12: FP 0x00000043FFA78FE0 SP 0x00000043FFA78F00 PC 0x00000043FFA79270

Possibly a bad flash?

Grub Fails to Start Anaconda Installer

If Grub doesn't seem to start the installer, make sure that your USB labels match that in grub.cfg.

For example, if grub.cfg contains the line:

search --no-floppy --set=root -l 'Fedora-S-21-aarc'

The USB label should be set to 'Fedora-S-21-aarc':

# lsblk -o name,label
NAME   LABEL
sdb
├─sdb1
└─sdb2 Fedora-S-21-aarc

If the label names are mismatched, grub will not be able to find where root is.


Secondary CPU Cores Are Disabled

The stock Fedora kernels (tested with 4.0.4-303.fc22.aarch64, 4.2.3-300.fc23.aarch64) is unable to bring the other CPUs online on startup:

[    0.018839] ------------[ cut here ]------------
[    0.018853] WARNING: CPU: 0 PID: 1 at drivers/firmware/dmi_scan.c:649 dmi_matches+0x120/0x128()
[    0.018861] dmi check: not initialized yet.
[    0.018866] Modules linked in:
[    0.018874] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.0.4-303.fc22.aarch64 #1
[    0.018882] Hardware name: APM X-Gene Mustang board (DT)
[    0.018887] Call trace:
[    0.018894] [<fffffe0000097738>] dump_backtrace+0x0/0x150
[    0.018901] [<fffffe00000978a8>] show_stack+0x20/0x30
[    0.018909] [<fffffe000076e9b4>] dump_stack+0x78/0x94
[    0.018915] [<fffffe00000c0888>] warn_slowpath_common+0xa0/0xe0
[    0.018921] [<fffffe00000c0928>] warn_slowpath_fmt+0x60/0x78
[    0.018927] [<fffffe00005de86c>] dmi_matches+0x11c/0x128
[    0.018933] [<fffffe00005de8a4>] dmi_check_system+0x2c/0x70
[    0.018940] [<fffffe0000c4f5e0>] lockup_detector_init+0x10/0x54
[    0.018946] [<fffffe0000c40abc>] kernel_init_freeable+0x108/0x260
[    0.018953] [<fffffe000076b76c>] kernel_init+0x1c/0xe8
[    0.018967] ---[ end trace 514f64f4445b269e ]---
[    0.018973] ------------[ cut here ]------------
[    0.018979] WARNING: CPU: 0 PID: 1 at drivers/firmware/dmi_scan.c:649 dmi_matches+0x120/0x128()
[    0.018987] dmi check: not initialized yet.
[    0.018991] Modules linked in:
[    0.018998] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G        W       4.0.4-303.fc22.aarch64 #1
[    0.019005] Hardware name: APM X-Gene Mustang board (DT)
[    0.019010] Call trace:
[    0.019015] [<fffffe0000097738>] dump_backtrace+0x0/0x150
[    0.019021] [<fffffe00000978a8>] show_stack+0x20/0x30
[    0.019027] [<fffffe000076e9b4>] dump_stack+0x78/0x94
[    0.019033] [<fffffe00000c0888>] warn_slowpath_common+0xa0/0xe0
[    0.019038] [<fffffe00000c0928>] warn_slowpath_fmt+0x60/0x78
[    0.019044] [<fffffe00005de86c>] dmi_matches+0x11c/0x128
[    0.019051] [<fffffe00005de8a4>] dmi_check_system+0x2c/0x70
[    0.019057] [<fffffe0000c4f5e0>] lockup_detector_init+0x10/0x54
[    0.019063] [<fffffe0000c40abc>] kernel_init_freeable+0x108/0x260
[    0.019068] [<fffffe000076b76c>] kernel_init+0x1c/0xe8
[    0.019073] ---[ end trace 514f64f4445b269f ]---
[    1.018060] CPU1: failed to come online
[    2.018179] CPU2: failed to come online
[    3.018300] CPU3: failed to come online
[    4.018418] CPU4: failed to come online
[    5.018544] CPU5: failed to come online
[    6.018662] CPU6: failed to come online
[    7.018782] CPU7: failed to come online
[    7.018797] Brought up 1 CPUs
[    7.018803] SMP: Total of 1 processors activated.

The CPUs will not come online post-startup either:

# echo 1 > /sys/devices/system/cpu/cpu1/online
-bash: echo: write error: Input/output error
# lscpu
Architecture:          aarch64
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0
Off-line CPU(s) list:  1-7
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             1

You can fix this by flashing the Tianocore UEFI bios instead of chainloading it from a SD card.


Error! No DIMMs Detected

After updating the BIOS to 3.06.25, booting the device now shows

Boot firmware (version 3.06.25 built at 13:52:18 on Oct 17 2016)
Error! No DIMM detected!!!
DRAM ver. 2016_0808
DRAM: 2GB DDR3 1600 ECC
X-Gene Mustang Board

Booting on version 1.1.0 seems to work, so this is most likely an issue with the BIOS.

No Output / Kernel Hangs After Booting

If you don't see any output after these lines:

EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
L3c Cache: 8MB

The kernel output is probably going to the wrong place. Set the console=ttyS0,115200 as part of the kernel arguments.