Skip to content

Commit 001d31c

Browse files
mcgrofmcgrof
committed
docs: declare kdevops a software 3.0 enabler
For a long time we've worked hard to make it easier to extend kdevops with strict rules and clear declarative language. We've been so strict on some of these policies I'm quite sure it discouraged some developers. The projections and hope was that these strict goals would reduce the barrier to the use of generative AI for extending kdevops. Although we've been experimenting with different AI agents on kdevops for a while now, advanced AI agent tools such as Claude Code bring this closer to home, enabling prompts to complete about 70-95% of the tasks. In a short span of about one week we've been able to cut through a huge back log of work which would have otherwise taken us considerable amount of time. This, and our automated tests put in place gives us confidence to declare that the time is here and now and it should be easy for others to also extend kdevops for their needs with generative AI. Extend our documentation with the rationale of the implicit long term goals we had, make these explicit, and also provide guidance for Claude Code. We also provide some recent prompt examples and respective commits merged which should be useful for users wishing to look to extend kdevops. CLAUDE.md [0] is a special file that Claude automatically pulls into context when starting a conversation. This makes it an ideal place for documenting: - Common commands folks should be aware of that we run - Core files and utility functions - Code style guidelines - Testing instructions - Repository etiquette (e.g., branch naming, merge vs. rebase, etc.) - Developer environment setup - Any unexpected behaviors or warnings particular to the project - Other information you want Claude to remember We extend this to ensure the DCO is respected and also give it a few prompts examples and target historical commits with respective grades. Also, extend the video documentation links with the recent kdevops talk in Open Source Summit North America [1] where I mentioned for the first time publicly that kdevops was Software 3.0 enabler for Linux kernel development. Link: https://www.anthropic.com/engineering/claude-code-best-practices # [0] Link: https://www.youtube.com/watch?v=VF-jr_ZE-9Y&list=PLjaT52x3HVboZtdwZnONSHQHM8217ELcc # [1] Acked-by: Chuck Lever <[email protected]> Acked-by: Daniel Gomez <[email protected]> Tested-by: Daniel Gomez <[email protected]> Signed-off-by: Luis Chamberlain <[email protected]>
1 parent 191ae56 commit 001d31c

File tree

3 files changed

+395
-0
lines changed

3 files changed

+395
-0
lines changed

‎CLAUDE.md‎

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with
4+
code in this repository.
5+
6+
## Project Overview
7+
8+
kdevops is a comprehensive DevOps framework for Linux kernel development and
9+
testing. It provides automation for setting up kernel development environments
10+
and complex testing laboratories for kernel subsystems.
11+
12+
**Version**: 5.0.2
13+
**Main Repository**: https://github.com/linux-kdevops/kdevops
14+
**License**: copyleft-next-0.3.1
15+
16+
## Core Architecture
17+
18+
### Build System
19+
- **Configuration**: Uses Linux kernel Kconfig system (`make menuconfig`,
20+
`make nconfig`, `make dynconfig`)
21+
- **Build**: Make-based with modular Makefiles (`Makefile.*`)
22+
- **Automation**: Ansible playbooks in `playbooks/` directory
23+
- **Infrastructure**: Supports virtualization (libguestfs/libvirt),
24+
cloud providers (AWS, Azure, GCE, OCI), and bare metal
25+
26+
### Key Components
27+
- `workflows/` - Testing workflows (fstests, blktests, selftests, CXL,
28+
mmtests, NFS, etc.)
29+
- `playbooks/` - Ansible automation with 40+ specialized roles
30+
- `kconfigs/` - Modular configuration system. Provides modeling variabiilty
31+
support. This is essentially support for kconfig taken from
32+
the Linux kernel, with support for special "output yaml"
33+
support to allow the project to leverage kconfig symbols on
34+
ansible through the `extra_vars.yaml` file which is always
35+
present.
36+
- `defconfigs/` - Pre-built configurations for common setups. These are
37+
extremely useful for supporting not only easy quick setups
38+
but are also instrumental for our continuous integration
39+
support.
40+
- `scripts/` - Workflow automation and helper scripts
41+
42+
## Common Development Commands
43+
44+
### Configuration and Setup
45+
```bash
46+
make menuconfig # Interactive configuration
47+
make dynconfig # Supports dynamically generated kconfig files
48+
make defconfig-<name> # Use predefined configuration (see defconfigs/)
49+
make # Build dependencies and setup
50+
make bringup # Provision and configure systems
51+
make destroy # Destroy provisioned systems
52+
```
53+
54+
### Kernel Development
55+
```bash
56+
make linux # Build and install kernel
57+
make linux HOSTS="host1 host2" # Target specific hosts
58+
make linux-uninstall KVER="6.6.0-rc2" # Uninstall specific kernel version
59+
```
60+
61+
### Testing Workflows
62+
```bash
63+
# Filesystem testing
64+
make fstests # Run filesystem tests
65+
make fstests-baseline # Establish baseline
66+
make fstests-results # View results
67+
68+
# Block layer testing
69+
make blktests # Run block layer tests
70+
make blktests-baseline # Establish baseline
71+
make blktests-results # View results
72+
73+
# Linux kernel selftests
74+
make selftests # Run all selftests
75+
make selftests-firmware # Run specific test suite
76+
make selftests-kmod # Run kernel module tests
77+
78+
# Other workflows
79+
make pynfs # NFS testing
80+
make gitr # Git regression testing
81+
make ltp # Linux Test Project
82+
make sysbench # Database performance testing
83+
make mmtests # Memory management tests from mmtests
84+
```
85+
86+
### Development Utilities
87+
```bash
88+
make help # Show available targets
89+
make V=1 [target] # Verbose build output
90+
make AV=1-6 [target] # Ansible verbose output (levels 0-6)
91+
make dynconfig # Generate dynamic configuration
92+
make mrproper # Clean everything and restart from scratch
93+
```
94+
95+
## Key Workflows
96+
97+
### fstests (Filesystem Testing)
98+
- **Purpose**: Comprehensive filesystem testing for XFS, Btrfs, EXT4, CIFS, NFS, tmpfs
99+
- **Features**: Expunge list management, baseline tracking, regression detection
100+
- **Location**: `workflows/fstests/`
101+
- **Config**: Enable fstests workflow in menuconfig
102+
103+
### blktests (Block Layer Testing)
104+
- **Purpose**: Block layer subsystem testing
105+
- **Supports**: NVMe, SCSI, loop devices, NBD, ZBD
106+
- **Location**: `workflows/blktests/`
107+
- **Features**: Similar baseline/regression tracking as fstests
108+
109+
### Linux Kernel Building
110+
- **Source Management**: Multiple git trees (Linus, stable, next, subsystem trees)
111+
- **Features**: 9P filesystem for host-guest development, mirror support
112+
- **Location**: `workflows/linux/`
113+
114+
### selftests (Kernel Selftests)
115+
- **Purpose**: Parallel execution of Linux kernel selftests
116+
- **Supports**: firmware, kmod, sysctl, and other kernel subsystem tests
117+
- **Location**: `workflows/selftests/`
118+
119+
## Architecture Highlights
120+
121+
### Configuration System
122+
- Uses Linux kernel Kconfig for consistent configuration management
123+
- Modular configuration files in `kconfigs/` for different subsystems
124+
- Dynamic configuration generation with `make dynconfig`
125+
- Pre-built configurations in `defconfigs/` directory
126+
127+
### Workflow System
128+
- Each workflow has dedicated Kconfig and Makefile
129+
- Workflow-specific scripts in `scripts/workflows/`
130+
- Ansible roles for automation in `playbooks/roles/`
131+
- Result collection and baseline management
132+
133+
### Infrastructure Support
134+
- **Virtualization**: libguestfs with libvirt (recommended), legacy Vagrant
135+
- **Cloud**: AWS, Azure, GCE, OCI, OpenStack support via Terraform
136+
- **PCIe Passthrough**: Real hardware testing in VMs with dynamic device assignment
137+
- **Mirror Support**: For air-gapped environments
138+
139+
### Kernel CI Features
140+
- Built-in continuous integration support
141+
- Baseline management for known test failures
142+
- Regression detection with git-style diff output
143+
- Watchdog systems for automated recovery from hung tests
144+
145+
## Important File Locations
146+
147+
- `Kconfig` - Main configuration entry point
148+
- `workflows/*/Kconfig` - Workflow-specific configuration options
149+
- `workflows/*/Makefile` - Workflow automation targets
150+
- `playbooks/roles/` - Reusable Ansible automation components
151+
- `scripts/workflows/` - Workflow-specific helper scripts
152+
- `docs/` - Comprehensive documentation
153+
154+
## Development Patterns
155+
156+
1. **Configuration-Driven**: Everything configurable through Kconfig
157+
2. **Modular Design**: Workflow-specific components included conditionally
158+
3. **Ansible Automation**: Role-based infrastructure and testing automation
159+
4. **Baseline Management**: Comprehensive tracking of known failures and regressions
160+
5. **Template Generation**: Dynamic file generation based on configuration
161+
162+
## Quick Setup Examples
163+
164+
### XFS Filesystem Testing
165+
```bash
166+
make defconfig-xfs # Configure for XFS testing
167+
make bringup # Setup test environment
168+
make fstests # Run filesystem tests
169+
```
170+
171+
### Kernel Development Environment
172+
```bash
173+
make menuconfig # Configure kernel workflow
174+
make bringup # Setup development VMs
175+
make linux # Build and install kernel
176+
```
177+
178+
### Block Layer Testing with NVMe
179+
```bash
180+
make defconfig-blktests_nvme
181+
make bringup
182+
make blktests
183+
```
184+
185+
## Testing and Quality Assurance
186+
187+
- Expunge lists track known test failures in `workflows/*/expunges/`
188+
- Baseline commands establish expected test results
189+
- Results commands show test outcomes and regressions
190+
- Watchdog scripts provide automated monitoring for long-running tests
191+
- Integration with kernel development workflows and patchwork systems
192+
193+
This framework is designed by kernel developers for kernel developers,
194+
providing production-ready automation for kernel testing and development
195+
workflows.
196+
197+
## One commit per change
198+
199+
As with the Linux kernel, this project prefers commits to be atomic and to
200+
the point. We don't want spell fixes to be blended in with code changes.
201+
Spell fixes should go into separate commits. When in doubt, just don't do
202+
any spell fixes unless asked explicitly to do that.
203+
204+
## Use the Signed-off-by tag
205+
206+
We want to use the Signed-off-by tag which embodies the application of the
207+
Developer Certificate or Origin.
208+
209+
## Use Generated-by: Claude AI
210+
211+
Use this tag for code generated by Claude code AI. Put this before the
212+
Signed-off-by tag.
213+
214+
## Prompt Examples
215+
216+
Refer to PROMPTS.md for example set of prompts used to generate code on
217+
kdevops using different AI agents and their respective commits and gradings.
218+
This can be useful for humans but also for generative AI so it can improve.

‎PROMPTS.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# kdevops PROMPTS.md
2+
3+
This file can be used by generative AI agents to learn previous prompts
4+
and example commits and their outcomes, and notes by users of the AI agent
5+
grading. It is also instructive for humans to learn how to use generative
6+
AI to easily extend kdevops for their own needs.
7+
8+
## Extending existing Linux kernel selftests
9+
10+
Below are a set of example prompts / result commits of extending existing
11+
kdevops selftests support to add new selftests. We bundle together "selftests"
12+
with a few custom tests which don't fit into the "selftetsts" category but
13+
due to efficiency we simply augment its support to cover them. The special
14+
unicorn workloads are:
15+
16+
- xarray - has kernel module and userspace tests
17+
- maple tree - has kernel module and userspace tests
18+
- vma - only has userspace tests
19+
20+
### Adding a new custom Linux kernel userspace test under kdevops selftests
21+
22+
**Prompt:**
23+
We already support testing the Linux kernel maple tree and radix tree on the
24+
selftests workflow. These are custom tests and each of these are tested on
25+
kdevops in-kernel and then in userspace. The in-kernel testing done requires
26+
using a loadable kernel module for each.
27+
28+
We have support for bundling these two tests under the
29+
defconfigs/seltests-radix-tree defconfig and we use the
30+
SELFTESTS_TEST_BUNDLE_RADIX_TREE to help us bundle all radix-tree related tests.
31+
We want to extend this with a new userspace only test. Its the vma tests on the
32+
Linux kernel. So it won't have a corresponding Linux kernel module load,
33+
contrary to test_xarray and test_maple_tree. The only thing that needs to be
34+
done to test it is to change directory on the linux kernel directory sources
35+
into tools/testing/vma and as a user run make -j$(nproc) and then just run
36+
sudo ./vma. So add support for this and augment SELFTESTS_TEST_BUNDLE_RADIX_TREE
37+
to select this.
38+
39+
**AI:** Claude Code
40+
**Commit:** [`191ae5678688`](https://github.com/linux-kdevops/kdevops/commit/191ae5678688)
41+
**Result:** Almost perfect.
42+
**Grading:** 95%
43+
44+
**Notes:**
45+
46+
The only issue caught was a complex bug which any human would have also run
47+
into. It created a separate task for running the userspace tests and registered
48+
the output to the same variable name as the task which runs the userspace
49+
tests on the maple tree and xarray. This confuses ansible. And so the
50+
selftests task "Run userspace selftests" needed to be manually fixed to
51+
check if the test was either xarray or maple tree or vma, and use that same
52+
task with variables for the directory where we change into and the command
53+
we run.
54+
55+
56+
## Adding completely new workflows to kdevops
57+
58+
Below are a list of successful prompts used with Claude Code to generate
59+
commits for kdevops. They range from fixing bugs, to adding new workflows
60+
to extending existing selftests support. These are intended to be useful
61+
for humans and Claude Code. Please extend them with more examples. You
62+
can add new types of fields. It just needs to make coherent sense.
63+
64+
### mmtests integration
65+
66+
**Prompt:**
67+
mmtests by Mel Gorman is used to help force memory fragmentation. I had
68+
extended mmtests so to ensure it works with debian-testing. And so given
69+
kdevops default are to work with debian-testing adding support for mmtests to
70+
kdevops should be straight forward, it should consist of just looking at its
71+
README and for each distribution adding requirements on dependencies just as we
72+
do for the sysbench workflow on kdevops. The kdevops sysbench workflow also
73+
intelligently used intelligently the kconfig KDEVOPS_BASELINE_AND_DEV option so
74+
to enable A/B testing. So extend kdevops to add support for mmtests. Add
75+
mirror support so that we have it as one of the mirrored git trees we take.
76+
Then add a new mmtests workflow with support for it, and only initially focus
77+
on thpchallenge-fio and thpcompact as possible mmtests workflow targets we can
78+
work. Ensure to use the new output yaml feature so to enable ansible tasks for
79+
mmtests to leverage the kconfig logic passed down automatically.
80+
81+
**AI:** Claude Code
82+
**Commit:** [`0b829e6a1fb8`](https://github.com/linux-kdevops/kdevops/commit/0b829e6a1fb8)
83+
**Result:** Comprehensive tests generated, with minor manual fixups.
84+
**Grading:** 70%
85+
86+
**Notes:**
87+
Most of the nuggets were properly implemented except it lacked insight to
88+
review the build component and that:
89+
90+
a) Upon build the script called may expect user input. This required us to
91+
modify the call to use 'yes yes | ./run-mmtests.sh -b' and that meant
92+
also using and leveraging the ansible shell module. The ansible shell
93+
module is not ideal and not recommended due to the fact that pipes can
94+
easily lead to non deteriministic behaviour. We prefer long term to
95+
use the ansible command module and split task as needed. In this case
96+
in the future to enhance determinism we want to instead add an option
97+
upstream mmtest to the ./run-mmtests.sh command line options to let us
98+
not have to enter any input from users and let it just install depdenencies
99+
for us.
100+
101+
b) The script ./run-mmtests.sh asks us for user input to install dependencies.
102+
The lack of insight was to realize that other than the README it should have
103+
looked at the script ./run-mmtests.sh to help review if it did try to
104+
install dependencies on its own. What we can do later is instead as a
105+
secondary step use Caude Code to later ask it to analyze the latest
106+
upstream mmtests run-mmtests.sh script and ask it to install augment
107+
the dependency list we have on kdevops for each distribution.
108+
109+
c) The prompt the user gave lacked sufficient hints to help it understand
110+
where to get more sensible configurations from to be used as templates
111+
for ninja2 conversion. So for example ./bin/autogen-configs generates
112+
configurations for us and these should have been used for the base
113+
templates.
114+
115+
d) I manually had to enhance styling for variables for the different mmtests
116+
test types. This consisted of for example using variable names like
117+
MMTESTS_THPCOMPACT_THREADS_MIN for thpcompact related knobs. Likewise I
118+
preferred to split configurations for the different mmtests test types
119+
into their own kconfig file. So for example we ended up with:
120+
121+
source "workflows/mmtests/Kconfig.thpcompact"
122+
source "workflows/mmtests/Kconfig.thpchallenge"
123+
source "workflows/mmtests/Kconfig.fs"
124+
125+
This separation is preferred as it helps us scale.

0 commit comments

Comments
 (0)