Shipping a Linux Kernel with Windows
Beginning with Windows Insiders builds this Summer, we will include an in-house custom-built Linux kernel to underpin the newest version of the Windows Subsystem for Linux (WSL). This marks the first time that the Linux kernel will be included as a component in Windows. This is an exciting day for all of us on the Linux team at Microsoft and we are thrilled to be able to tell you a little bit about it.
Tuned for WSL
The term “Linux” is often used to refer both to the Linux kernel as well as the GNU userspace. As with WSL1, WSL2 will not provide any userspace binaries. Instead, the Microsoft kernel will interface with a userspace selected by the user. This will generally come through installation via the Windows store but can also be “sideloaded” through the creation of a custom distribution package. The only exception to this rule is a small init script that is injected to bootstrap the startup process, forming the connections between Windows and Linux that make WSL so magical. The kernel itself will initially be based on version 4.19, the latest long-term stable release of Linux. The kernel will be rebased at the designation of new long-term stable releases to ensure that the WSL kernel always has the latest Linux goodness. In addition to the LTS source from Kernel.org, a number of local patches are being applied. These patches tune the resulting binary for use in WSL2 by improving launch times, reducing the memory footprint and curating a minimal set of supported devices. The result is a small, lightweight kernel that is purpose built for WSL2 to be a drop-in replacement for the emulation architecture featured in the design of WSL1.
Code goes upstream
Microsoft employs a growing number of Linux contributors who have brought industry leading Linux knowhow into the company. For years now, these Linux developers have enabled Microsoft to support new platform features in the wide number of distributions provided in the Azure Marketplace. An important philosophy of Linux at Microsoft is that all changes go upstream. Maintaining downstream patches adds complexity and is not standard practice in the open source community. In leveraging Linux, we are making a commitment to be good citizens and contribute back the changes that we make. However, during development it is necessary to work with local patches that enable new features or address issues in upstream. In these cases, we either create, or find patches that fulfill our product requirements and then work with the community to get that code integrated as soon as possible. To protect the stability of the LTS branches, some patches – such as for new features – might only be included in future versions of the kernel, and not be back-ported to the current LTS version. When the WSL kernel source becomes available it will consist of links to a set of patches in addition to the long-term stable source. Over time, we hope this list will shrink as patches make it upstream and grow as we add new local patches to support new WSL features.
Security
The WSL kernel will be built using Microsoft’s world-class CI/CD systems and serviced through Windows Update in an operation transparent to the user. The kernel will stay up to date with the newest features and fixes in the latest stable branch of Linux. To ensure the provenance of our sources we mirror repositories locally. We continually monitor Linux security mailing lists and partner with several CVE database companies to help ensure that our kernel has the most recent fixes and mitigations. One of the great things about Linux is its stable and backwards compatible system call interface. This will enable us to ship the latest stable branch of the Linux kernel to all versions of WSL2. We will rebase the kernel when a new LTS is established and when we have sufficiently validated it.
Open Source
The kernel provided for WSL2 will be fully open source! When WSL2 is released in Windows Insider builds, instructions for creating your own WSL kernel will be made available on Github. We will work with developers interested in contributing to help get changes upstream. Check back in a few weeks for more information.
Thanks!
This is the culmination of years of effort from the Linux Systems Group as well as multiple other teams across Microsoft. We are excited to be able to share the result and look forward to the new and interesting ways in which you will use WSL. If you are interested in positions at Microsoft working with Linux check out this job listing.
How to Compile The Linux Kernel for WSL
Compiling the Linux kernel is a milestone for any user, even though it is easier them compiling must packages depending on if your sticking with the defaults or enabling some feature. The same should holds true for those that are run Linux via wsl like your or me.
When I first got stuck back on Windows (against my will) and the only way for me to maintain any from of Linux develop is working inside wsl; I though I was stuck with whatever kernel version Mircosoft released. Thankfully, the kernel used is the lts version and is open sourced and is available on GitHub.
That is why I’m here producing a guide on how to compile the Linux kernel, but unlike the multitude of guides out there, this one is the current stable/mainline release. Not the lts version.
Installing Required Build Packages
Depending on what distro you are determines what packages need to be installed. Some distro’s like Arch install most build related packages when base-devel is installed, Ubuntu however does not.
sudo pacman -S base-devel bc openssl ncurses sudo dnf install fedpkg fedora-packager rpmdevtools ncurses-devel pesign grubby sudo build-dep linux linux-image-$(uname -r) libncurses-dev gawk flex bison openssl libssl-dev dkms libelf-dev libudev-dev libpci-dev libiberty-dev autoconf That is one of the reason I like Arch over most other distro’s. When you install the base-devel almost every build system and compiler is installed and most general packages include header libraries. Once you have the required packages installed it is time to get the kernel.
Getting the Kernel a Different Way
Most guide either go about downloading and extract the tarball or clone a repository. Which includes running multiple commands and doing something wrong. Believe me I’ve been there. Instead of doing something wrong with multiple command, why not just do one command wrong.
curl -s https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/linux-version.tar.gz | tar --transform 's/^linux-version/linux-kernel/' -xvz At the time of this writting the current stable version was 5.15.14 and the current mainline version was 5.16. To find out what the current version is go to the Linux kernel main page. Once you do that, just make sure the you replace linux-version with linux- and the current version for both the link and tar extraction.
Please note that the above command will not work for tar.xz tarball files you well get an gzip: stdin: not in gzip format error. That is because the tarball is not being download directly, but instead is being fed as a stdin. This also means that there will be no tarball to remove later.
To add the support for wsl if you are on a x86_64 system just apply a patch to the kernel source.
curl -s https://raw.githubusercontent.com/MichaelSchaecher/MichaelSchaecher/main/patchs/wsl-support.patch | patch -p1 Once that is done a config file is placed in arch/x86/config titled “wsl_x8664_defconfig,” and the arch/x86Makefile ie edited allowing for _wsl detection.
If you are on an arm/arm64 system you will have to copy the config from Microsoft’s Github page for their kernel source.
wget -c https://raw.githubusercontent.com/microsoft/WSL2-Linux-Kernel/linux-msft-wsl-5.10.y/Microsoft/config-wsl-arm64 -O .config Configuring and Compiling
The source tree needs to be configured before the kernel can be compiled. If you want to enable some features the best option is to use make menuconfig . Plus if you applied the patch for compiling on a x8664 system the config file for building on _wsl well be used.
To just compile with the defaults from the config file.
The above command well set any defaults “y” from the Kconfig scripts to be build configuration in the .config. This command well also work if you copied the the .config from Microsoft’s linux kernel source.
Once the source tree is configured it is time to compile the kernel. Just note that depending on what is enable and the system hardware this can take time; you maybe looking at up to a hour before the compilation is finished.
Setting the localversion to an empty value well cause the + sign to not be at the end of version if you run uname -r . You can however set a value if you wish. When compilation is finished the location of the “bzImage” file well be displayed, you well need to keep the image to a location outside of wsl.
The best practice is to not follow the quides that tell to create or place anything wsl related in the root of the C:\ drive. There are two reason for this. One, wsl isntalled or imported distro’s are unable to support multiuser login and second a restore of a Windows’s back can wipeout the directory. Hell even a Windows update can cause problems.
In you Windows profile home folder create a directory labeled wsl, this can be done from the terminal. Just be sure to use your actual profilename.
mdir -p /mnt/c/Users/profilename/wsl/kernel cp arch/x86/boot/bzImage /mnt/c/Users/profilename/wsl/kernel/$(file arch/x86/boot/bzImage | awk -F' ' '') The image well be copied to the desired location and renamed to the build version AKA, if mainline the it would something like 5.16.0-wsl-mainline .
Enable the Kernel
In order to use the new kernel you need to tell wsl to boot with it, otherwise the default kernel well be used. This is done from a “.wslconfig” file loacted it the root of you Windows profile home folder. The file must be UTF-8 with LF line-ending, this means that Windows Notepad is good edit to use.
User VSCode or your favorite cli text editor copy the following.
Note the double “\\” for path to the kernel in-between directories.
[wsl2] # Following if set will apply # Use stable kernel instead of the default lts from Microsoft. # An absolute Windows path to a custom Linux kernel. kernel = c:\\Users\\profilename\\wsl\\kernel # How many processors to assign to the WSL2 VM. If left unset # WSL2 well use all available cores/threads. # processors = 2 # Specify if ports bound to wildcard or localhost in the WSL2 # VM should be connectable from the host via localhost:port localhostForwarding = true # 50% of total memory on Windows or 8GB, whichever is less; on # builds before 20175: 80% of your total memory on Windows. # 80% is a bit to much for a VM that is commandline only. memory = 8GB # How much swap space to add to the WSL2 VM, 0 for no swap file. # Swap storage is disk-based RAM used when memory demand exceeds # limit on hardware device. # Some build environments require SWAP, setting to equal that of # the RAM or no more then 16GB. swap = 8GB # Disable page reporting so WSL retains all allocated memory claimed # from Windows and releases none back when free. pageReporting = false Restarting WSL
As you can see from the archey3 info that my Arch Linux install is using the mainline kernel.
Once you have you have done all that go to your exit your distro and open up the _PowerShell and enter wsl —shutdown , then restart you distro. If everything went without any problem you should be running with the kernel you just compiled. One thing I learned is to double check the .wslconfig file first if something doesn’t work and to make sure that that the file is labeled right.