Skip to content

Commit

Permalink
UBUNTU: [Packaging] Add base packaging
Browse files Browse the repository at this point in the history
Signed-off-by: Emil Renner Berthing <[email protected]>
  • Loading branch information
esmil committed Sep 24, 2024
1 parent a8028be commit f53cf31
Show file tree
Hide file tree
Showing 44 changed files with 18,876 additions and 0 deletions.
97 changes: 97 additions & 0 deletions debian/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Building debian packages in a PPA

## Preparation

### Install packages
```
apt-get install devscripts debhelper build-essential
```

### Get the orig tarball
```
curl -fLO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.6.21.tar.gz
mv linux-6.6.21.tar.gz ../linux-premier_6.6.21.orig.tar.gz
```


## Create new release

### Update kernel config

After changing Kconfig files or updating the configuration in debian/config/annotations
you'll need to make sure the configuration is in sync with the kernel's Kconfig system.
```
debian/rules clean
debian/rules updateconfigs
git add debian/config/annotations
git commit -sm 'UBUNTU: [Config] Update for ..'
```

### Update debian/reconstruct

The source package is split into the .orig.tar.gz tarball plus a big patch on
top. Unfortunately the patch doesn't preserve file permissions of added files,
so the debian/reconstruct script is needed to fix those after applying the
patch. If any new files with execute permission (scripts) are added we need update
the debian/reconstruct file like so:
```
debian/rules autoreconstruct
git add debian/reconstruct
```

### Update debian/changelog

The source package name and version is all controlled by the debian/changelog
file. It has a certain format and is most easily updated with the following
command:
```
dch -i
```

The version number is usually of the form 6.6.21-42.1 eg.
`<upstream version>-<abi>.<upload>`. Different kernels can be installed
simultaneously as long as they have different upstream version and/or ABI
numbers. The upload number is used if fx. one release failed to build in the
PPA. Then you can fix it and bump just the upload number without bumping the
ABI release number.

Make sure to change the release codename from `UNRELEASED` and commit the
updated changelog (and possibly debian/reconstruct). Usually the commit
message should contain the new version number in the changelog. Eg.:
```
git add debian/changelog
git commit -sm 'UBUNTU: linux-premier-6.6.21-42.1'
```

### Build source package

Make sure you have a clean git repository
```
git clean -fdx
```
and then build the source package with
```
debian/rules build-sources
```

This builds a source package that just refers to the .orig.tar.gz, but doesn't
include it. If you update the base kernel version or upload to a new PPA that
doesn't already have the .orig.tar.gz you'll need to include it in the source
package like so:
```
debian/rules build-sources-with-orig
```

### Push the source package to a PPA

For the PPA to accept the upload the source package needs to be signed by
a user that has push access to the PPA:
```
debsign ../linux-premier_6.6.21-42.1_source.changes
```

Finally you can upload it to your PPA with
```
dput <ppa> ../linux-premier_6.6.21-42.1_source.changes
```
Here `<ppa>` of course needs to be defined in your `~/.dput.cf`.
6 changes: 6 additions & 0 deletions debian/changelog
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
linux-premier (6.6.21-1.1) noble; urgency=medium

First build of the SiFive kernel for the HiFive Premier P550 board featuring
the ESWIN EIC7700 SoC.

-- Emil Renner Berthing <[email protected]> Mon, 02 Sep 2024 12:36:52 +0200
27 changes: 27 additions & 0 deletions debian/config.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
arch = riscv64
human_arch = RISC-V
build_arch = riscv
defconfig = defconfig
build_image = Image
kernel_file = arch/$(build_arch)/boot/Image
install_file = vmlinuz

supported = ESWIN EIC7700
target = Geared toward desktop and server systems.
desc = the HiFive Premier P550
bootloader = grub-pc [amd64] | grub-efi-amd64 [amd64] | grub-efi-ia32 [amd64] | grub [amd64] | lilo [amd64] | flash-kernel [armhf arm64] | grub-efi-arm64 [arm64] | grub-efi-arm [armhf] | grub-ieee1275 [ppc64el]

gcc = gcc-13
vdso = vdso_install
no_dumpfile = true

do_libc_dev_package = false
do_metas = true
do_tools_usbip = true
do_tools_cpupower = true
do_tools_perf = true
do_tools_perf_jvmti = true
do_tools_perf_python = true
do_tools_bpftool = true
do_tools_rtla = false
do_dtbs = true
185 changes: 185 additions & 0 deletions debian/config/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
==================
Config Annotations
==================

:Author: Andrea Righi

Overview
========

Each Ubuntu kernel needs to maintain its own .config for each supported
architecture and each flavour.

Every time a new patch is applied or a kernel is rebased on top of a new
one, we need to update the .config's accordingly (config options can be
added, removed and also renamed).

So, we need to make sure that some critical config options are always
matching the desired value in order to have a functional kernel.

State of the art
================

At the moment configs are maintained as a set of Kconfig chunks (inside
`debian.<kernel>/config/`): a global one, plus per-arch / per-flavour
chunks.

In addition to that, we need to maintain also a file called
'annotations'; the purpose of this file is to make sure that some
critical config options are not silently removed or changed when the
real .config is re-generated (for example after a rebase or after
applying a new set of patches).

The main problem with this approach is that, often, we have duplicate
information that is stored both in the Kconfig chunks *and* in the
annotations files and, at the same time, the whole .config's information
is distributed between Kconfig chunks and annotations, making it hard to
maintain, review and manage in general.

Proposed solution
=================

The proposed solution is to store all the config information into the
"annotations" format and get rid of the config chunks (basically the
real .config's can be produced "compiling" annotations).

Implementation
==============

To help the management of the annotations an helper script is provided
(`debian/scripts/misc/annotations`):

```
usage: annotations [-h] [--version] [--file FILE] [--arch ARCH] [--flavour FLAVOUR] [--config CONFIG]
(--query | --export | --import FILE | --update FILE | --check FILE)

Manage Ubuntu kernel .config and annotations

options:
-h, --help show this help message and exit
--version, -v show program's version number and exit
--file FILE, -f FILE Pass annotations or .config file to be parsed
--arch ARCH, -a ARCH Select architecture
--flavour FLAVOUR, -l FLAVOUR
Select flavour (default is "generic")
--config CONFIG, -c CONFIG
Select a specific config option

Action:
--query, -q Query annotations
--export, -e Convert annotations to .config format
--import FILE, -i FILE
Import a full .config for a specific arch and flavour into annotations
--update FILE, -u FILE
Import a partial .config into annotations (only resync configs specified in FILE)
--check FILE, -k FILE
Validate kernel .config with annotations
```

This script allows to query config settings (per arch/flavour/config),
export them into the Kconfig format (generating the real .config files)
and check if the final .config matches the rules defined in the
annotations.

Examples (annotations is defined as an alias to `debian/scripts/annotations`):

- Show settings for `CONFIG_DEBUG_INFO_BTF` for master kernel across all the
supported architectures and flavours:

```
$ annotations --query --config CONFIG_DEBUG_INFO_BTF
{
"policy": {
"amd64": "y",
"arm64": "y",
"armhf": "n",
"ppc64el": "y",
"riscv64": "y",
"s390x": "y"
},
"note": "'Needs newer pahole for armhf'"
}
```

- Dump kernel .config for arm64 and flavour generic-64k:

```
$ annotations --arch arm64 --flavour generic-64k --export
CONFIG_DEBUG_FS=y
CONFIG_DEBUG_KERNEL=y
CONFIG_COMPAT=y
...
```

- Update annotations file with a new kernel .config for amd64 flavour
generic:

```
$ annotations --arch amd64 --flavour generic --import build/.config
```

Moreover, an additional kernelconfig commands are provided
(via debian/rules targets):
- `migrateconfigs`: automatically merge all the previous configs into
annotations (local changes still need to be committed)

Annotations headers
===================

The main annotations file should contain a header to define the architectures
and flavours that are supported.

Here is the format of the header for the generic kernel:
```
# Menu: HEADER
# FORMAT: 4
# ARCH: amd64 arm64 armhf ppc64el riscv64 s390x
# FLAVOUR: amd64-generic arm64-generic arm64-generic-64k armhf-generic armhf-generic-lpae ppc64el-generic riscv64-generic s390x-generic

```

Example header of a derivative (linux-aws):
```
# Menu: HEADER
# FORMAT: 4
# ARCH: amd64 arm64
# FLAVOUR: amd64-aws arm64-aws
# FLAVOUR_DEP: {'amd64-aws': 'amd64-generic', 'arm64-aws': 'arm64-generic'}

include "../../debian.master/config/annotations"

# Below you can define only the specific linux-aws configs that differ from linux generic

```

Pros and Cons
=============

Pros:
- avoid duplicate information in .config's and annotations
- allow to easily define groups of config settings (for a specific
environment or feature, such as annotations.clouds, annotations.ubuntu,
annotations.snapd, etc.)
- config options are more accessible, easy to change and review
- we can easily document how config options are managed (and external
contributors won't be discouraged anymore when they need to to change a
config option)

Cons:
- potential regressions: the new tool/scripts can have potential bugs,
so we could experience regressions due to some missed config changes
- kernel team need to understand the new process (even if everything
is transparent, kernel cranking process is the same, there might be
corner cases that need to be addressed and resolved manually)

TODO
====

- Migrate all flavour and arch definitions into annotations (rather
than having this information defined in multiple places inside
debian/scripts); right now this information is "partially" migrated,
meaning that we need to define arches and flavours in the headers
section of annotations (so that the annotations tool can figure out
the list of supported arches and flavours), but arches and flavours
are still defined elsewhere, ideally we would like to have arches and
flavours defined only in one place: annotations.
Loading

0 comments on commit f53cf31

Please sign in to comment.