hansdegoede | Entries tagged with wayland

A while ago as a spin-off of my project to improve support for Logitech wireless keyboards and mice I have also done some work on improving support for (Gaming) keyboards with a builtin LCD panel.

Specifically if you have a Logitech MX5000, G15, G15 v2 or G510 and you want the LCD panel to show something somewhat useful then on Fedora 31 you can now install the lcdproc package and it will automatically recognize the keyboard and show "top" like information on it. No need to manually write an LCDd.conf or anything, this works fully plug and play:

sudo dnf install lcdproc
sudo udevadm trigger

If you have a MX5000 and you do not want the LCD panel to show "top" like info, you may still want to install the mx5000tools package, this will automatically send the system time to the keyboard, after which it will display the time.

Once the 5.5 kernel becomes available as an update for Fedora you will also be able to use the keys surrounding the LCD panel to control the lcdproc menu-s on the LCD panel. The 5.5 kernel will also export key backlight brightness control through the standardized /sys/class/leds API, so that you can control it from e.g. the GNOME control-center's power-settings and you get a nice OSD when toggling the brightnesslevel using the key on the keyboard.

The 5.5 kernel will also make the "G" keys send standard input-events (evdev events), once userspace support for the new key-codes these send has landed, this will allow e.g. binding them to actions in GNOME control-center's keyboard-settings. But only under Wayland as the new keycodes are > 255 and X11 does not support this.

As mentioned in an earlier blogpost, I have been working on fixing many games showing a small image centered on a black background when they are run fullscreen under Wayland. In that blogpost I was moslty looking at how to solve this for native Wayland games. But for various reasons almost all games still use X11, so instead I've ended up focussing on fixing this for games using Xwayland.

Xwayland now has support for emulating resolution changes requested by an app through the randr or vidmode extensions. If a client makes a resolution change requests this is remembered and if the client then creates a window located at the monitor's origin and sized to exactly that resolution, then Xwayland will ask the compositor to scale it to fill the entire monitor.

For apps which use _NET_WM_FULLLSCREEN (e.g. SDL2, SFML or OGRE based apps) to go fullscreen some help from the compositor is necessary. This is currently implemented in mutter. If you are a developer of another compositor and have questions about this, please drop me an email.

I failed to get this all upstream in time for Fedora 31 final. But now it is all upstream, so 've backported the changes and created an update with the changes. This update is currently in updates-testing, to install this update run the following command:

sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2019-103a594d07

First of all I do not want people to get their hopes up about $subject of this blogpost. Improving gaming support is a subjects which holds my personal interest and it is an issue I plan to spend time on trying to improve. But this will take a lot of time (think months for simple things, years for more complex things).

As I see it there are currently 2 big issues when running games under Wayland:

1. Many games show as a smal centered image with a black border (letterbox) around the image when running fullscreen.

For 2D games this is fixed by switching to SDL2 which will transparently scale the pixmap the game renders to the desktop resolution. This assumes that 2D games in general do not demand a lot of performance and thus will not run into performance issues when introducing an extra scaling step. A problem here is that many games still use SDL1.2 (and some games do not use SDL at all).

I plan to look into the recently announced SDL1.2 compatibility wrapper around SDL2. If this works well this should fix this issue for all SDL1.2 2D games, by making them use SDL2 under the hood.

For 3D games this can be fixed by rendering at the desktop resolution, but this might be slow and rendering at a lower resolution leads to the letterbox issue.

Recently mutter has has grown support for the WPviewport extension, which allows Wayland apps to tell the compositor to scale the pixmap the app gives to the compositor before presenting it. If we add support to SDL2's Wayland backend for this then, this can be used to allow rendering 3D apps at a lower resolution and still have them fill the entire screen.

Unfortunately there are 2 problems with this plan:

SDL2 by default uses its x11 backend, not its wayland backend. I'm not sure what fixes need to be done to change this, at a minimum we need a fix at either the SDL or mutter side for this issue, which is going to be tricky.

This only helps for SDL2 apps, again hopefully the SDL1.2 compatibility wrapper for SDL2 can help here, at least for games using SDL.

2. Fullscreen performance is bad with many games.

Since under Wayland games cannot change the monitor resolution, they need to either render at the full desktop resolution, which can be very slow; or they render at a lower resolution and then need to do an extra scaling step each frame.

If we manage to make SDL2's Wayland backend the default and then add WPviewport support to it then this should help by reducing an extra memcpy/blit of a desktop-sized pixmap. Currently what apps which use scaling do is:

render lower-res-pixmap;

scale lower-res-pixmap to desktop-res-pixmap

give desktop-res-pixmap to the compositor;

compositor does a hardware blit of the desktop-res-pixmap to the framebuffer.

With viewport support this becomes:

render lower-res-pixmap;

give low-res-pixmap to the compositor;

compositor uses hardware to do a scaling blit from the low-res-pixmap to the desktop-res framebuffer

Also with viewport support, the compositor could in the case of there only being the one fullscreen app even keep the framebuffer in lowres and use a hardware scaling drm-plane to send the low-res framebuffer scaled to desktop-res to the output while only reading the low-res framebuffer from memory saving a ton of memory bandwidth. But this optimization is going to be a challenge to pull off.

Thank you all for the large amount of feedback I have received after my previous Wayland Itches blog post. I've received over 40 mails, below is an attempt at summarizing all the mails.

Highlights

1. Middle click on title / header bar to lower the Window does not work for native apps. Multiple people have reported this issue to me. A similar issue was fixed for not being able to raise Windows. It should be easy to apply a similar fix for the lowering problem. There are bugs open for this here, here and here.

2. Running graphical apps via sudo or pxexec does not work. There are numerous examples of apps breaking because of this, such as lshw-gui and usbivew. At least for X11 apps this is not that hard to fix. But sofar this has deliberately not been fixed. The reasoning behind this is described in this bug. I agree with the reasoning behind this, but I think it is not pragmatic to immediately disallow all GUI apps to connect when run as root starting today.

We need some sort of transition period. So when I find some time for this, I plan to submit a merge-requests which optionally makes gnome-shell/mutter start Xwayland with an xauth file, like how it is done when running in GNOME on Xorg mode. This will be controlled by a gsettings option, which will probably default to off upstream and then distros can choice to override this for now, giving us a transition period

Requests for features implemented as external programs on X11

There are various features which can be implemented as external programs
on X11, but because of the tighter security need to be integrated into the
compositor with Wayland:

Hiding of the mouse-cursor when not used à la unclutter-xfixes, xbanish.

Rotating screen 90 / 270 degrees à la "xrandr -o [left|right]" mostly used through custom hotkeys, possible fix is defining bindable actions for this in gsd-media-keys.

Mapping actions to mouse buttons à la easystroke

Some touchscreen's, e.g. so called smart-screens for education, need manual calibration. Under X11 there are some tools to get the callibration matrix for the touchscreen, after which this can be manually applied through xinput. Even under X11 this currently is far from ideal but at least it is possible there.

Keys Indicator gnome-shell extension. This still works when using Wayland, but only works for apps using Xwayland, it does not work for native apps.

Some sort of xkill and xdotool utility equivalents would be nice

The GNOME on screen keyboard is not really suitable for use with apps which are not touch-enabled, as it lacks a way to send ctrl + key, etc. Because of this some users have reported that it is impossible to use alternative on screen keyboards with Wayland. Not being able to use alternative on screen keyboards is by design and IMHO the proper fix here is to improve GNOME's on screen keyboard.

App specific problems

Citrix ICA Client does not work well with Xwayland

Eclipse does not work well with Xwayland

Teamviewer does not work with Wayland. It needs to be updated to use pipewire for screencapturing and the RemoteDesktop portal to inject keyboard and mouse events.

Various apps lack screenrecording / capture support due to the app not having support for pipewire: gImageReader, green-recorder, OBS studio, peek, screenrecorder, slack

For apps which do support pipewire, there is not an option to share the contents of a window, other then the window making the request. On Xorg it is possible to share a random window and since pipewire allows sharing the whole desktop I see no security reason why we would not allow sharing another window.

guake window has incorrect size when using HiDPI scaling, see this issue

Miscellaneous problems

Mouse cursor is slow / lags

Drag and drop sometimes does not work, e.g. dragging files into file roller to compress or out of file roller to extract.

Per keyboard layouts. On X11 after plugging in a keyboard, the layout/keymap for just that one keyboard can be updated manually using xinput, allowing different keyboard layouts for different keyboards when multiple keyboards are connected

No-title-bar shell extension, X button can be hit unintentionally, see this issue

Various issues with keyboard layout switching

Hard to fix issues

Alt-F2, r equivalent (restart the gnome-shell)

X11 apps running on top of Xwayland do not work well on HiDPI screens

Push-to-talk (passive key grab on space) does not work in Mumble when using native Wayland apps, see this issue

Problems with other compositors then GNOME3 / mutter

I've also received several reports about issues when using another Wayland compositor as GNOME / mutter (Weston, KDE, Sway). I'm sorry but I have not looked very closely into these reports. I believe that it is great that Linux users have multiple Desktop Environments to choose from and I wish for the other DEs to thrive. But there are only so many hours in a day so I've chosen to mainly focus on GNOME.

Now that GNOME3 on Wayland is the default in Fedora I've been trying to use this as my default desktop, but until recently I've kept falling back to GNOME3 on Xorg because of various small issues.

To fix this I've switched to using GNOME3 on Wayland as day to day desktop now and I'm working on fixing any issues which this causes as I hit them, aka "The Wayland Itches project". So far I've hit and fixed the following issues:

1. TopIcons

The TopIcons extension, which I depend on for some of my workflow, was not working well under Wayland with GNOME-3.30, only the top row of icons was clickable. This was fixed in GNOME-3.32, but with GNOME-3.32 using TopIcons was causing gnome-shell to go into a loop leading to a very high CPU load. The day I wanted to start looking into fixing this I was chatting to Carlos Garnacho and he pointed out to me that this was fixed a couple of days ago in gnome-shell. The fix for this is in gnome-shell 3.32.2 .

2. Hotkeys/desktop shortcuts not working in VirtualBox Virtual Machines

When running a VirtualBox VM under GNOME3 on Wayland, hotkeys such as alt+tab go to the GNOME3 desktop, rather then being forwarded to the VM as happens under Xorg. This can be fixed by changing 2 settings:

gsettings set org.gnome.mutter.wayland xwayland-allow-grabs true
gsettings set org.gnome.mutter.wayland xwayland-grab-access-rules "['VirtualBox Machine']"

This is a decent workaround, but we want things to "just work" of course, so we have been working on some changes to make this just work in the next GNOME version.

3. firefox-wayland

I've been also trying to use firefox-wayland as my day to day browser, this has lead to me filing three firefox bugs and I've switched back to regular
firefox (x11) for now.

If you have any Wayland Itches yourself, please drop me an email at hdegoede@redhat.com explaining them in as much detail as you can and I will see what I can do. Note that I typically get a lot of emails when asking for feedback like this, so I cannot promise that I will reply to every email; but I will be reading them all.