hansdegoede | Entries tagged with grub

Now that FESCo has decided that Fedora will keep supporting BIOS booting, the people working on Fedora's bootloader stack will need help from the Fedora community to keep Fedora booting on systems which require Legacy BIOS to boot.

To help with this the Fedora BIOS boot SIG (special interest group) has been formed. The main goal of this SIG are to help the Fedora bootloader people by:

Doing regular testing of nightly Fedora N + 1 composes on hardware
which only supports BIOS booting
Triaging and/or fixing BIOS boot related bugs.

A biosboot-sig@lists.fedoraproject.org mailinglist and bugzilla account has been created, which will be used to discuss testing result and as assignee / Cc for bootloader bugzillas which are related to BIOS booting.

If you are interested in helping with Fedora BIOS boot support please:

Subscribe to the email-list
Add yourself to the Members section of the SIG's wiki page

As you may know I've been doing a lot of hw-enablement work on Bay- and Cherry-Trail tablets as a side-project for the last couple of years.

Some of these tablets have one interesting feature intended to "flash" Android on them. When turned on with both the volume-up and the volume-down buttons pressed at the same time they enter something called DNX mode, which it will then also print to the LCD panel, this is really just a variant of the android fastboot protocol built into the BIOS. Quite a few models support this, although on Bay Trail it sometimes seems to be supported (it gets shown on the screen) but it does not work since many models which only shipped with Windows lack the external device/gadget-mode phy which the Bay Trail SoC needs to be able to work in device/gadget mode (on Cherry Trail the gadget phy has been integrated into the SoC).

So on to the topic of this blog-post, I recently used DNX mode to unbrick a tablet which was dead due to the BIOS settings get corrupted in a way where it would not boot and it was also impossible to enter the BIOS setup. After some duckduckgo-ing I found a thread about how in DNX mode you can upload a replacement for the efilinux.efi bootloader normally used for "fastboot boot" and how you can use this to upload a binary to flash the BIOS. I did not have a BIOS image of this tablet, so that approach did not work for me. But it did point me in the direction of a different, safer (no BIOS flashing involved) solution to unbrick the tablet.

If you run the following 2 commands on a PC with a Bay- or Cherry-Trail connected in DNX mode:

fastboot flash osloader some-efi-binary.efi
fastboot boot some-android-boot.img

Then the tablet will execute the some-efi-binary.efi. At first I tried getting an EFI shell this way, but this failed because the EFI binary gets passed some arguments about where in RAM it can find the some-android-boot.img. Then I tried booting a grubx64.efi file and that result in a grub commandline. But I had not way to interact with it and replacing the USB connection to the PC with a OTG / USB-host cable with a keyboard attached to it did not result in working input.

So my next step was to build a new grubx64.efi with "-c grub.cfg" added to the commandline for the final grub2-mkimage step, embedding a grub.cfg with a single line in there: "fwsetup". This will cause the tablet to reboot into its BIOS setup menu. Note on some tablets you still will not have keyboard input if you just let the tablet sit there while it is rebooting. But during the reboot there is enough time to swap the USB cable for an OTG adapter with a keyboard attached before the reboot completes and then you will have working keyboard input. At this point you can select "load setup defaults" and then "save and exit" and voila the tablet works again.

For your convenience I've uploaded a grubia32.efi and a grubx64.efi with the necessary "fwsetup" grub.cfg here. This is build from this branch at this commit (this was just a random branch which I had checked out while working on this).

Note the image used for the "fastboot boot some-android-boot.img" command does not matter much, but it must be a valid android boot.img format file otherwise fastboot will refuse to try to boot it.

Recently I acquired an Acer Aspire Switch 10 E SW3-016, this device was the main reason for writing my blog post about the shim boot loop. The EFI firmware of this is bad in a number of ways:

It considers its eMMC unbootable unless its ESP contains an EFI/Microsoft/Boot/bootmgfw.efi file.

But it will actually boot EFI/Boot/bootx64.efi ! (wait what? yes really)

It will only boot from an USB disk connected to its micro-USB connector, not from the USB-A connector on the keyboard-dock.

You must first set a BIOS admin password before you can disable secure-boot (which is necessary to boot home-build kernels without doing your own signing)

Last but not least it has one more nasty "feature", it detect if the OS being booted is Windows, Android or unknown and it updates the ACPI DSDT based in this!

Some more details on the OS detection mis feature. The ACPI "Device (SDHB) node for the MMC controller connected to the SDIO wifi module contains:

Name (WHID, "80860F14")
Name (AHID, "INT33BB")

Depending on what OS the BIOS thinks it is booting it renames one of these 2 to _HID. This is weird given that it will only boot if EFI/Microsoft/Boot/bootmgfw.efi exists, but it still does this. Worse it looks at the actual contents of EFI/Boot/bootx64.efi for this. It seems that that file must be signed, otherwise it goes in OS unknown mode and keeps the 2 above DSDT bits as is, so there is no _HID defined for the wifi's mmc controller and thus no wifi. I hit this issue when I replaced EFI/Boot/bootx64.efi with grubx64.efi to break the bootloop. grubx64.efi is not signed so the DSDT as Linux saw it contained the above AML code. Using the proper workaround for the bootloop from my previous blog post this bit of the DSDT morphes into:

Name (_HID, "80860F14")
Name (AHID, "INT33BB")

And the wifi works.

The Acer Aspire Switch 10 E SW3-016's firmware also triggers an actual bug / issue in Linux' ACPI implementation, causing the bluetooth to not work. This is discussed in much detail here. I have a patch series fixing this here.

And the older Acer Aspire Switch 10 SW5-012's and S1002's firmware has some similar issues:

It considers its eMMC unbootable unless its ESP contains an EFI/Microsoft/Boot/bootmgfw.efi file

These models will actually always boot the EFI/Microsoft/Boot/bootmgfw.efi file, so that is somewhat more sensible.

On the SW5-012 you must first set a BIOS admin password before you can disable secure-boot.

The SW5-012 is missing an ACPI device node for the PWM controller used for controlling the backlight brightness. I guess that the Windows i915 gfx driver just directly pokes the registers (which are in a whole other IP block), rather then relying on a separate PWM driver as Linux does. Unfortunately there is no way to fix this, other then using a DSDT overlay. I have a DSDT overlay for the V1.20 BIOS and only for the v1.20 BIOS available for this here.

Because of 1. and 2. you need to take the following steps to get Linux to boot on the Acer Aspire Switch 10 SW5-012 or the S1002:

Rename the original bootmgfw.efi (so that you can chainload it in the multi-boot case)

Replace bootmgfw.efi with shimia32.efi

Copy EFI/fedora/grubia32.efi to EFI/Microsoft/Boot

This assumes that you have the files from a 32 bit Windows install in your ESP already.

How to fix the Linux EFI secure-boot shim bootloop issue seen on some systems.

Quite a few Bay- and Cherry-Trail based systems have bad firmware which completely ignores any efibootmgr set boot options. They basically completely reset the boot order doing some sort of auto-detection at boot. Some of these even will given an error about their eMMC not being bootable unless the ESP has a EFI/Microsoft/Boot/bootmgfw.efi file!

Many of these end up booting EFI/Boot/bootx64.efi unconditionally every boot. This will cause a boot loop since when Linux is installed EFI/Boot/bootx64.efi is now shim. When shim is started with a path of EFI/Boot/bootx64.efi, shim will add a new efibootmgr entry pointing to EFI/fedora/shimx64.efi and then reset. The goal of this is so that the firmware's F12 bootmenu can be used to easily switch between Windows and Linux (without chainloading which breaks bitlocker). But since these bad EFI implementations ignore efibootmgr stuff, EFI/Boot/bootx64.efi shim will run again after the reset and we have a loop.

There are 2 ways to fix this loop:

1. The right way: Stop shim from trying to add a bootentry pointing to EFI/fedora/shimx64.efi:

rm EFI/Boot/fbx64.efi
cp EFI/fedora/grubx64.efi EFI/Boot

The first command will stop shim from trying to add a new efibootmgr entry (it calls fbx64.efi to do that for it) instead it will try to execute grubx64.efi from the from which it was executed, so we must put a grubx64.efi in the EFI/Boot dir, which the second command does. Do not use the livecd EFI/Boot/grubx64.efi file for this as I did at first, that searches for its config and env under EFI/Boot which is not what we want.

Note that upgrading shim will restore EFI/Boot/fbx64.efi. To avoid this you may want to backup EFI/Boot/bootx64.efi, then do "sudo rpm -e shim-x64" and then restore the backup.

2. The wrong way: Replace EFI/Boot/bootx64.efi with a copy of EFI/fedora/grubx64.efi

This is how I used to do this until hitting the scenario which caused me to write this blog post. There are 2 problems with this:

2a) This requires disabling secure-boot (which I could live with sofar)
2b) Some firmwares change how they behave, exporting a different DSDT to the OS dependending on if EFI/Boot/bootx64.efi is signed or not (even with secure boot disabled) and their behavior is totally broken when it is not signed. I will post another rant ^W blogpost about this soon. For now lets just say that you should use workaround 1. from above since it simply is a better workaround.

Note for better readability the above text uses bootx64, shimx64, fbx64 and grubx64 throughout. When using a 32 bit EFI (which is typical on Bay Trail systems) you should replace these with bootia32, shimia32, fbia32 and grubia32. Note 32 bit EFI Bay Trail systems should still use a 64 bit Linux distro, the firmware being 32 bit is a weird Windows related thing.

Also note that your system may use another key then F12 to show the firmware's bootmenu.

There have been questions about the Fedora 30 Flicker Free Boot Change in various places, here is a FAQ which hopefully answers most questions:

1) I get a black screen for a couple of seconds during boot?

1.1) If you have an AMD or Nvidia GPU driving your screen, then this is normal. The graphics drivers for AMD and Nvidia GPUs reset the hardware when loading, this will cause the display to temporarily go black. There is nothing which can be done about this.

1b) If you have a somewhat older Intel GPU (your CPU is pre Skylake) then the i915 driver's support to skip the mode-reset is disabled by default (for now) to fix this add "i915.fastboot=1" to your kernel commandline. For more info on modifying the kernel cmdline, see question 7.

1c) Do "ls /sys/firmware/efi/efivars" if you get a "No such file or directory" error then your system is booting in classic BIOS mode instead of UEFI mode, to fix this you need to re-install and boot the livecd/installer in UEFI mode when installing. Alternatively you can try to convert your existing install, note this is quite tricky, make backups first!

1d) Your system may be using the classic VGA BIOS during boot despite running in UEFI mode. Often you can select BIOS mode compatility in your BIOS settings aka the CSM setting. If you can select this on a per component level, set the VIDEO/VGA option to "UEFI only" or "UEFI first", alternatively you can try completely disabling the CSM mode. On some systems you can disable the classic VGA BIOS by disabling / unselecting the "Legacy Option ROMs" option.

2) The vendor-logo/firmware-splash looks squashed or has the wrong size?

Your system may be using the classic VGA BIOS during boot despite running in UEFI mode, see answer 1d.

3) I get a black background instead of the firmware splash while Fedora is booting?

Do "ls /sys/firmware/acpi/bgrt" if you get a "No such file or directory" error then try answers 1c and 1d. If you do have a /sys/firmware/acpi/bgrt directory, but you are still getting the Fedora logo + spinner on a black background instead of on top of the firmware-splash, please file a bug about this and drop me a mail with a link to the bug.

4) Getting rid of the vendor-logo/firmware-splash being shown while Fedora is booting?

If you don't want the firmware-splash to be used as background during boot, you can switch plymouth to the spinner theme, which is identical to the new bgrt theme, except that it does not use the firmware-splash as background, to do this execute the following command from a terminal: "sudo plymouth-set-default-theme -R spinner"

Note that the kernel will restore the vendor-logo early on at boot in case it got damaged by e.g. option ROM messages. If you are switching to the spinner theme you may also want to add "video=efifb:nobgrt" to your kernel commandline. See 7 below for how to edit the kernel commandline.

5) Keeping the firmware-splash as background while unlocking the disk?

If you prefer this, it is possible to keep the firmware-splash as background while the diskcrypt password is shown. To do this do the following:

"sudo mkdir /usr/share/plymouth/themes/mybgrt"

"sudo cp /usr/share/plymouth/themes/bgrt/bgrt.plymouth /usr/share/plymouth/themes/mybgrt/mybgrt.plymouth"

edit /usr/share/plymouth/themes/mybgrt/mybgrt.plymouth, change DialogClearsFirmwareBackground=true to DialogClearsFirmwareBackground=false, change DialogVerticalAlignment=.382 to DialogVerticalAlignment=.6

"sudo plymouth-set-default-theme -R mybgrt"

Note if you do this the disk-passphrase entry dialog may be partially drawn over the vendor-logo part of the firmware-splash, if this happens then try increasing DialogVerticalAlignment to e.g. 0.7 .

6) Get detailed boot progress instead of the boot-splash ?

To get detailed boot progress info press ESC during boot.

7) Always get detailed boot progress instead of the boot-splash ?

To always get detailed boot progress instead of the boot-splash, remove "rhgb" from your kernel commandline:

Edit /etc/default/grub and remove rhgb from GRUB_CMDLINE_LINUX and then if you are booting using UEFI (see 1c) run:
"grub2-mkconfig -o /etc/grub2-efi.cfg"
else (if you are booting using classic BIOS boot) run:
"grub2-mkconfig -o /etc/grub2.cfg".

As discussed in my previous blog post one of my TODO list items for plymouth is creating a new plymouth theme.

Since the transition to plymouth is not entirely smooth plymouth by default will wait 5 seconds (counted from starting the kernel) before showing itself so that on systems which boot under 5 seconds it never shows. As can be seen in this video, this leads to a very non-smooth experience when the boot takes say 7 seconds as plymouth then only shows briefly, leading to a kinda "flash" effect while it briefly shows.

Another problem with the 5 second wait, is now that we do not show GRUB the user is looking at the firmware's bootsplash for not only the often long firmware initialization time, but also for the 5 seconds plymouth waits on top, making it look as if nothing is happening.

To fix this I've been working on a new plymouth theme which draws a spinner over the firmware boot splash, eliminating the ugly transition from the firmware boot splash to plymouth. This also allows removing the show-delay, so that we provide feedback that something is happening as soon as plymouth starts.

Firmware being firmware getting this done right was somewhat harder then I expected, but I've a first "draft" of a new theme doing this now. I've created some videos showing 2 different systems booting the new theme:

My workstation booting, plain version of the theme

My workstation booting, Fedora branded version of the theme

My laptop with diskcrypt booting, plain version of the theme

My laptop with diskcrypt booting, Fedora branded version of the theme

Note the videos with diskcrypt where paused when I entered my passphrase. So there is a bit of a jump in them because of this.

I've built a test version of plymouth for Fedora 29, to give this a try download all rpm files from here except the .src.rpm and -devel files and then from a directory with all those files in it, run:

sudo rpm -Uvh plymouth*.rpm

Since plymouth is part of your initrd, you also need to regenerate your initrd:

sudo dracut -f /boot/initramfs-$(uname -r).img $(uname -r)

This regenerates the initrd for the kernel you are currently running, so if you've installed a kernel update and have not rebooted since then you may not get the new theme when rebooting. In this case rerun the dracut command after rebooting.

Note if you've previously followed my instructions to test flickerfree boot, then you need to remove "plymouth.splash_delay=20" from your kernel commandline, since we now no longer want to have a splash-delay.

Now reboot and you should get the new spinner on firmware-boot-splash theme, with Fedora branding.

If you give this a try and the new theme somehow does not look correct, please mail at hdegoede@redhat.com. If you mail me about the theme not displaying correctly please attach the /run/plymouth.log file which this test-build generates to the email and a video of how the theme misbehaves would be great too.

I still need to discuss the idea of using a new theme incorporating the firmware boot splash with the GNOME design team so this is all subject to change.

A big project I've been working on recently for Fedora Workstation is what we call flickerfree boot. The idea here is that the firmware lights up the display in its native mode and no further modesets are done after that. Likewise there are also no unnecessary jarring graphical transitions.

Basically the machine boots up in UEFI mode, shows its vendor logo and then the screen keeps showing the vendor logo all the way to a smooth fade into the gdm screen. Here is a video of my main workstation booting this way.

Part of this effort is the hidden grub menu change for Fedora 29. I'm happy to announce that most of the other flickerfree changes have also landed for Fedora 29:

There have been changes to shim and grub to not mess with the EFI framebuffer, leaving the vendor logo intact, when they don't have anything to display (so when grub is hidden)

There have been changes to the kernel to properly inherit the EFI framebuffer when using Intel integrated graphics, and to delay switching the display to the framebuffer-console until the first kernel message is printed. Together with changes to make "quiet" really quiet (except for oopses/panics) this means that the kernel now also leaves the EFI framebuffer with the logo intact if quiet is used.

There have been changes to plymouth to allow pressing ESC as soon as plymouth loads to get detailed boot messages.

With all these changes in place it is possible to get a fully flickerfree boot today, as the video of my workstation shows. This video is made with a stock Fedora 29 with 2 small kernel commandline tweaks:

Add "i915.fastboot=1" to the kernel commandline, this removes the first and last modeset during the boot when using the i915 driver.

Add "plymouth.splash-delay=20" to the kernel commandline. Normally plymouth waits 5 seconds before showing the charging Fedora logo so that on systems which boot in less then 5 seconds the system simply immediately transitions to gdm. On systems which take slightly longer to boot this makes the charging Fedora logo show up, which IMHO makes the boot less fluid. This option increases the time plymouth waits with showing the splash to 20 seconds.

So if you have a machine with Intel integrated graphics and booting in UEFI mode, you can give flickerfree boot support a spin with Fedora 29 by just adding these 2 commandline options. Note this requires the new grub hidden menu feature to be enabled, see the FAQ on this.

The need for these 2 commandline options shows that the work on this is not yet entirely complete, here is my current TODO list for finishing this feature:

Work with the upstream i915 driver devs to make i915.fastboot the default. If you try i915.fastboot=1 and it causes problems for you please let me know.

Write a new plymouth theme based on the spinner theme which used the vendor logo as background and draws the spinner beneath it. Since this keeps the logo and black background as is and just draws the spinner on top this avoids the current visually jarring transition from logo screen to plymouth, allowing us to set plymouth.splash-delay to 0. This also has the advantage that the spinner will provide visual feedback that something is actually happening as soon as plymouth loads.

Look into making this work with AMD and NVIDIA graphics.

Please give the new flickerfree support a spin and let me know if you have any issues with it.

There have questions about the new GRUB hidden menu Change in various places, here is a FAQ which hopefully answers most questions:

1. What is the GRUB hidden menu change?

See the Detailed Description on the change page. The main motivation for adding this is to get to a fully flickerfree boot.

2. How to enable hidden GRUB menu?

On new Fedora 29 Workstation installs this will be enabled by default. If your system has been upgraded to F29 from an older release, you can enable it by running these commands:

On a system using UEFI booting ("ls /sys/firmware/efi/efivars" returns a bunch of files):

sudo grub2-editenv - set menu_auto_hide=1
sudo grub2-mkconfig -o /etc/grub2-efi.cfg

On a system using legacy BIOS boot:

sudo grub2-editenv - set menu_auto_hide=1
sudo grub2-mkconfig -o /etc/grub2.cfg

Note the grub2-mkconfig will overwrite any manual changes you've made to your grub.cfg (normally no manually changes are done to this file).

If your system has Windows on it, but you boot it only once a year so you would still like to hide the GRUB menu, you can tell GRUB to ignore the presence of Windows by running:

sudo grub2-editenv - set menu_auto_hide=2

3. How to disable hidden GRUB menu

To permanently disable the auto-hide feature run:

sudo grub2-editenv - unset menu_auto_hide

That is it.

4.How to access the GRUB menu when hidden

If for some reason you need to access the GRUB menu while it is hidden there are multiple ways to get to it:

If you can get to gdm, access the top-right menu (the system menu) and click on the power [⏻] icon. Then keep ALT pressed to change the "Restart" option into "Boot Options" and click "Boot Options".

While booting keep SHIFT pressed, usually you need to first press SHIFT when the vendor logo is shown by the firmware / when the firmware says e.g. "Press F2 to enter setup" if you press it earlier it may not be seen. Note this may not work on some machines.

During boot press ESC or F8 while GRUB loads (simply press the key repeatedly directly after power on until you are in the menu).

Force the previous boot to be considered failed:
1. Press CTRL + ALT + DEL while booting so that the system reboots before hitting gdm
2. Press CTRL + ALT + F6 to switch away from gdm, followed by CTRL + ALT + DEL.
3. Press the power-button for 4 seconds to force the machine off.
Either of these will cause the boot_success grub_env flag to not get set and
the menu will show the next boot.

Manually set the menu show once flag by running: "grub-set-bootflag menu_show_once" This will cause the menu to show for 60 seconds before continuing with the default boot-option.

5. When is a boot considered successful ?

The boot_success grub_env flag gets set when you login as a normal user and your session lasts at least 2 minutes; or when you shutdown or restart the system from the GNOME system (top-right) menu.

So if you e.g. login, do something and then within 30 seconds type reboot in a terminal (instead of doing the reboot from the menu) then this will not count as a successful boot and the menu will show the next boot.