Skip to content

Commit e09409d

Browse files
committed
fio-tests: add multi-filesystem testing support
This merges the long-pending fio-tests filesystem support patch that adds comprehensive filesystem-specific performance testing capabilities to kdevops. The implementation allows testing filesystem optimizations, block size configurations, and I/O patterns against actual mounted filesystems rather than just raw block devices. The implementation follows the proven mmtests architecture patterns with modular Kconfig files and tag-based ansible task organization, avoiding the proliferation of separate playbook files that would make maintenance more complex. Key filesystem testing features include XFS support with configurable block sizes from 4K to 64K with various sector sizes and modern features like reflink and rmapbt. The ext4 support provides both standard and bigalloc configurations with different cluster sizes. For btrfs, modern features including no-holes, free-space-tree, and compression options are available. The multi-filesystem section-based testing enables comprehensive performance comparison across different filesystem configurations by creating separate VMs for each configuration. This includes support for XFS block size comparisons, comprehensive XFS block size analysis, and cross-filesystem comparisons between XFS, ext4, and btrfs. Node generation for multi-filesystem testing uses dynamic detection based on enabled sections, creating separate VM nodes for each enabled section with proper Ansible groups for each filesystem configuration. A/B testing support is included across all configurations. Results collection and analysis is handled through specialized tooling with performance overview across filesystems, block size performance heatmaps, IO depth scaling analysis, and statistical summaries with CSV exports. The patch has been updated to work with the current codebase which now uses workflow-specific template includes for host file generation rather than embedding all workflow templates in a single hosts.j2 file. The fio-tests specific template has been enhanced with multi-filesystem support while maintaining backward compatibility with single filesystem testing. Generated-by: Claude AI Signed-off-by: Luis Chamberlain <[email protected]>
1 parent d55d3dc commit e09409d

33 files changed

+4336
-286
lines changed

.github/workflows/fio-tests.yml

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
name: Run fio-tests on self-hosted runner
2+
3+
on:
4+
push:
5+
branches:
6+
- '**'
7+
pull_request:
8+
branches:
9+
- '**'
10+
workflow_dispatch: # Add this for manual triggering of the workflow
11+
12+
jobs:
13+
run-fio-tests:
14+
name: Run fio-tests CI
15+
runs-on: [self-hosted, Linux, X64]
16+
steps:
17+
- name: Checkout repository
18+
uses: actions/checkout@v4
19+
20+
- name: Set CI metadata for kdevops-results-archive
21+
run: |
22+
echo "$(basename ${{ github.repository }})" > ci.trigger
23+
git log -1 --pretty=format:"%s" > ci.subject
24+
# Start out pessimistic
25+
echo "not ok" > ci.result
26+
echo "Nothing to write home about." > ci.commit_extra
27+
28+
- name: Set kdevops path
29+
run: echo "KDEVOPS_PATH=$GITHUB_WORKSPACE" >> $GITHUB_ENV
30+
31+
- name: Configure git
32+
run: |
33+
git config --global --add safe.directory '*'
34+
git config --global user.name "kdevops"
35+
git config --global user.email "[email protected]"
36+
37+
- name: Run kdevops make defconfig for quick fio-tests
38+
run: |
39+
KDEVOPS_TREE_REF="${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}"
40+
SHORT_PREFIX="$(echo ${KDEVOPS_TREE_REF:0:12})"
41+
FIO_TESTS_QUICK_TEST=y make KDEVOPS_HOSTS_PREFIX="$SHORT_PREFIX" \
42+
ANSIBLE_CFG_CALLBACK_PLUGIN="debug" \
43+
defconfig-fio-tests-quick
44+
45+
- name: Run kdevops make
46+
run: |
47+
make -j$(nproc)
48+
49+
- name: Run kdevops make bringup
50+
run: |
51+
make bringup
52+
53+
- name: Run quick fio-tests to verify functionality
54+
run: |
55+
make fio-tests
56+
echo "ok" > ci.result
57+
# Collect basic test completion info
58+
find workflows/fio-tests/results -name "*.json" -type f | head -5 > ci.commit_extra || echo "No JSON results found" > ci.commit_extra
59+
if find workflows/fio-tests/results -name "*.json" -type f | grep -q .; then
60+
echo "ok" > ci.result
61+
else
62+
echo "No fio-tests results found" > ci.commit_extra
63+
fi
64+
65+
- name: Generate fio-tests graphs if results exist
66+
run: |
67+
if [ -d workflows/fio-tests/results ] && find workflows/fio-tests/results -name "*.json" -type f | grep -q .; then
68+
make fio-tests-graph || echo "Graph generation failed" >> ci.commit_extra
69+
fi
70+
71+
- name: Get systemd journal files
72+
if: always() # This ensures the step runs even if previous steps failed
73+
run: |
74+
make journal-dump
75+
76+
- name: Start SSH Agent
77+
if: always() # Ensure this step runs even if previous steps failed
78+
uses: webfactory/[email protected]
79+
with:
80+
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
81+
82+
- name: Build our kdevops archive results
83+
if: always() # This ensures the step runs even if previous steps failed
84+
run: |
85+
make ci-archive
86+
87+
- name: Upload our kdevops results archive
88+
if: always() # This ensures the step runs even if previous steps failed
89+
uses: actions/upload-artifact@v4
90+
with:
91+
name: kdevops-fio-tests-results
92+
path: ${{ env.KDEVOPS_PATH }}/archive/*.zip
93+
94+
# Ensure make destroy always runs, even on failure
95+
- name: Run kdevops make destroy
96+
if: always() # This ensures the step runs even if previous steps failed
97+
run: |
98+
make destroy

0 commit comments

Comments
 (0)