Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config-libkrunfw_x86_64: Chop chop #85

Open
wants to merge 21 commits into
base: main
Choose a base branch
from

Conversation

sbrivio-rh
Copy link
Contributor

I'm trying to get a binary entirely executed by muvm in less than 200ms. I couldn't quite achieve that, yet, but this got me substantially closer.

All tests are done on top of AsahiLinux/muvm#111 and AsahiLinux/muvm#134 on a system powered by an Intel Xeon E5-2667 v4 at 3.2GHz, 8 cores, no SMT, and enough memory. Builds were done on current Fedora Rawhide, host kernel is a 6.13-ish.

Altogether, with these changes

  • the uncompressed kernel size decreases by approximately 3 MB, from 29.4 MB to 26.4 MB
  • the total execution time for muvm --mem=64 --vram=0 -c 0,1 -- true decreases by approximately 21 ms, from 241 ms to 220 ms (average over 1000 runs)

I doubt any modern containerised application would ever use mqueue,
it's just for legacy Solaris applications nowadays, and applications
in a container couldn't access it anyway.

This saves 3280 bytes (29433904 -> 29430624) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
UNWINDER_GUESS should be good enough for any typical usage (unrelated
to kernel debugging), and 16 KiB of kernel logs is also more than
enough.

This saves 291624 bytes (29430624 -> 29139000) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
…eatures

These might be useful for libkrun and libkrunfw development, but not
really for typical users, and single options can be easily enabled as
needed during development anyway.

This saves:

- 166976 bytes (29139000 -> 28972024) in the uncompressed kernel image

- 11 ms (241 -> 230, average of 1000 runs) for:
     muvm --mem=64 --vram=0 -c 0,1 -- true

Signed-off-by: Stefano Brivio <[email protected]>
Even if we had a PC speaker, we couldn't access it anyway.

This saves 176 bytes (28972024 -> 28971848) in the uncompressed kernel
image.

Signed-off-by: Stefano Brivio <[email protected]>
…'t need

We typically don't need access to local APIC modes via MSRs in a
guest. MTRRs are not relevant either.

Disable all processor and memory features and quirks we'll never need
in a virtual machine.

This saves 76904 bytes (28971848 -> 28894944) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
@MatiasVara
Copy link
Collaborator

I am only wondering if these are changes that we would like in any x86-64 deployment or there could be different x86-64 flavors.

@sbrivio-rh
Copy link
Contributor Author

I am only wondering if these are changes that we would like in any x86-64 deployment or there could be different x86-64 flavors.

I think I left out anything that's possibly controversial, that is, anything that could be realistically used on top of libkrun.

But if I got it wrong, I think it would be easier to drop single patches (if somebody needs, say, firmware loading, or beep the PC speaker) rather than having yet another flavour.

@tylerfanelli
Copy link
Member

I am only wondering if these are changes that we would like in any x86-64 deployment or there could be different x86-64 flavors.

I think I left out anything that's possibly controversial, that is, anything that could be realistically used on top of libkrun.

But if I got it wrong, I think it would be easier to drop single patches (if somebody needs, say, firmware loading, or beep the PC speaker) rather than having yet another flavour.

We usually only bring new flavors up if it's enabling new features on top of certain architectures. For instance, SEV-SNP on AMD CPUs, TDX on Intel CPUs, EFI for macOS, etc.

Upon first glance, there's nothing too out of the ordinary here. If we find something that we want to keep, I think we can just drop the patch rather than introduce a new flavor.

MatiasVara
MatiasVara previously approved these changes Jan 24, 2025
config-libkrunfw_x86_64 Show resolved Hide resolved
@slp
Copy link
Contributor

slp commented Jan 27, 2025

This work is very valuable, thanks @sbrivio-rh !

It's, of course, kind of risky, but also worth the risk. This trimming is something we should probably try to do once in a while.

@@ -139,7 +139,7 @@ CONFIG_TASK_IO_ACCOUNTING=y
# CONFIG_PSI is not set
# end of CPU/Task time and stats accounting

CONFIG_CPU_ISOLATION=y
# CONFIG_CPU_ISOLATION is not set
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to be able to pin VCPUs, shouldn't we drop this one too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not directly related to CPU pinning, it's about isolcpus only (kernel isolates those CPUs and doesn't schedule workqueues on them, or other stuff, depending on parameters).

Besides, it's deprecated: cpusets replace that option.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I mixed things. I though tuned and tuned-profiles-realtime profile was still using isolcpus parameter.

The host takes care of that for us.

Keep CPU_IDLE, CPU_IDLE_GOV_HALTPOLL, and HALTPOLL_CPUIDLE, as they
might be used when pinning vCPUs.

This saves 2189032 bytes (28894944 -> 26705912) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
x32 won't work anyway, and I doubt that any post-modern application
outside x32 would use any of these.

This saves 80280 bytes (26705912 -> 26625632) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
I doubt a guest would ever need them.

This saves:

- 7896 bytes (26625632 -> 26617736) in the uncompressed kernel image

- 5 ms (230 -> 225, average of 1000 runs) for:
     muvm --mem=64 --vram=0 -c 0,1 -- true

Signed-off-by: Stefano Brivio <[email protected]>
...such as ZONE_DMA, and low-level statistics and counters that don't
really tell us anything in a guest anyway.

This saves 32968 bytes (26617736 -> 26584768) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
We don't need:

- SYN cookies: we'll never switch them on with TSI or passt
- TCP MD5 signatures: same here
- ethtool support: of little use with virtio-net, no use with TSI
- XFRM (IPsec), I doubt typical users would set up IPsec endpoints
- TCP advanced congestion features: we only have CUBIC enabled anyway
- netlabel: CIPSO, CALIPSO and friends are pretty much obsolete

This saves 128208 bytes (26584768 -> 26456560) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
VirtIO devices don't need any firmware to work.

This saves 7192 bytes (26456560 -> 26449368) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
…ices

The host would be in charge of them anyway, we would only see
abstracted input devices I guess.

This saves 944 bytes (26449368 -> 26448424) in the uncompressed kernel
image.

Signed-off-by: Stefano Brivio <[email protected]>
virtio-serial does all that for us.

This saves 36480 bytes (26448424 -> 26411944) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
…ver see

...such as DMA engines, x86 platform devices (laptop stuff), Microsoft
Surface platforms, clock subsystem. Disable staging drivers (we don't
have any selected).

This saves 32848 bytes (26411944 -> 26379096) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
… devices

This saves 888 bytes (26379096 -> 26378208) in the uncompressed kernel
image.

Signed-off-by: Stefano Brivio <[email protected]>
It doesn't make a lot of sense for virtual CPUs, I suppose.

This saves 1104 bytes (26378208 - 26377104) in the uncompressed kernel
image.

Signed-off-by: Stefano Brivio <[email protected]>
We don't support WiFi adapters and debugfs either.

This saves 2216 bytes (26377104 -> 26374888) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
This saves 760 bytes (26374888 -> 26374128) in the uncompressed kernel
image.

Signed-off-by: Stefano Brivio <[email protected]>
In typical libkrunfw usage, a small, nimble kernel is probably more
important than oversized lookup tables and suchlike.

This saves 80 bytes (26374128 -> 26374048) in the uncompressed kernel
image, which is not significant. However, memory savings might be
substantial depending on the usage.

Signed-off-by: Stefano Brivio <[email protected]>
Our users don't configure it anyway.

This saves 1632 bytes (26374048 -> 26372416) in the uncompressed
kernel image.

Signed-off-by: Stefano Brivio <[email protected]>
This is what I've been using to measure size and execution times for
a test command (based on muvm) for the previous patches. It's made to
be used in a rebase script, for example:

  x ./utils/kernel_size_time.sh
  pick 12181bf config-libkrunfw_x86_64: Disable processor quirks and features we don't need
  x ./utils/kernel_size_time.sh
  pick 5c5badc config-libkrunfw_x86_64: Drop power management features and cpufreq
  x ./utils/kernel_size_time.sh

it's a bit bigger than what I wanted because of the time -p trick:
nowadays most distributions don't ship /usr/bin/time (at least by
default), but many shells skip support for the POSIX compatibility
mode (-p, which simplifies calculations) in their 'time' built-in,
so, if we need it, we need to re-execute under Bash (assuming it's
not the default non-interactive shell).

Signed-off-by: Stefano Brivio <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants