Split out from wayland_backend. So far no unit test as there is no proper way to simulate input events.