Skip to content

export-db/pop-db: extend functionality to return more data, allow import of any table in database#534

Merged
mergify[bot] merged 2 commits intoflux-framework:masterfrom
cmoussa1:restructure.db-info
Feb 13, 2026
Merged

export-db/pop-db: extend functionality to return more data, allow import of any table in database#534
mergify[bot] merged 2 commits intoflux-framework:masterfrom
cmoussa1:restructure.db-info

Conversation

@cmoussa1
Copy link
Member

Problem

The export-db and pop-db commands only handle a couple of columns from just the association_table and bank_table. These commands would be more robust if they included export and import support from all of the tables.


This PR restructures the export_db_info() function to return data from all of the tables in the flux-accounting DB into separate .csv files, labeled with their table names in the database. It also restructures the populate_db() function to handle populating any of the tables in the flux-accounting DB with a corresponding .csv file. It adds an optional argument to the command which allows the user to specify which columns to include from the file when populating the table.

As a result of the improvements made to both functions, I've also restructured the tests for both of these commands slightly, mostly just:

  • updated the filenames to match the table names in the flux-accounting DB
  • include the column names in the first line of the .csv files
  • drop the use of the --banks and --users optional arguments

@cmoussa1 cmoussa1 added improvement upgrades to an already existing feature low priority items that can be worked on at a later date labels Nov 12, 2024
@cmoussa1 cmoussa1 force-pushed the restructure.db-info branch from 14c8cbd to b80bcbf Compare December 30, 2024 19:06
@cmoussa1 cmoussa1 force-pushed the restructure.db-info branch from b80bcbf to 01b7c81 Compare January 18, 2025 01:09
@cmoussa1 cmoussa1 force-pushed the restructure.db-info branch 2 times, most recently from 9d8fa64 to 0065312 Compare May 13, 2025 20:36
@cmoussa1 cmoussa1 force-pushed the restructure.db-info branch from 0065312 to b63bc10 Compare October 14, 2025 22:25
@cmoussa1 cmoussa1 force-pushed the restructure.db-info branch from b63bc10 to fea4acd Compare December 23, 2025 23:29
Problem: The export-db and pop-db commands only handle a couple of
columns from just the association_table and bank_table. These commands
would be more robust if they included export and import support from all
of the tables.

Restructure the export_db_info() function to return data from *all* of
the tables in the flux-accounting DB into separate .csv files, labeled
with their table names in the database.

Restructure the populate_db() function to handle populating any of the
tables in the flux-accounting DB with a corresponding .csv file. Add an
optional argument to the function to allow the user to specify which
columns to include from the file when populating the table.
Problem: The sharness tests for the pop-db and export-db commands are
out-of-date in a couple of ways:

- their filenames are not those of the actual table names in the
flux-accounting database
- they do not include the column names in the first line of the .csv
file
- the pop-db command no longer has a "--users" optional argument to
specify a file containing data for the association_table

Update the tests accordingly to account for the improvements made to the
export-db and pop-db commands.
Copy link
Member

@jameshcorbett jameshcorbett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me!

@cmoussa1
Copy link
Member Author

Thanks @jameshcorbett! Setting MWP here

@mergify mergify bot added the queued label Feb 13, 2026
@mergify mergify bot merged commit be0faa5 into flux-framework:master Feb 13, 2026
15 of 16 checks passed
@mergify
Copy link
Contributor

mergify bot commented Feb 13, 2026

Merge Queue Status

Rule: default


  • Entered queue2026-02-13 16:28 UTC
  • Checks passed · in-place
  • Merged2026-02-13 16:28 UTC · at f571b96babc954ed2c0e0a7b1174901acf400ba4

This pull request spent 5 seconds in the queue, with no time running CI.

Required conditions to merge
  • #approved-reviews-by >= 1 [🛡 GitHub branch protection]
  • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
  • any of [🛡 GitHub branch protection]:
    • check-success = coverage
    • check-neutral = coverage
    • check-skipped = coverage
  • any of [🛡 GitHub branch protection]:
    • check-success = el8 - distcheck
    • check-neutral = el8 - distcheck
    • check-skipped = el8 - distcheck
  • any of [🛡 GitHub branch protection]:
    • check-success = el8 - py3.6
    • check-neutral = el8 - py3.6
    • check-skipped = el8 - py3.6
  • any of [🛡 GitHub branch protection]:
    • check-success = jammy - py3.6
    • check-neutral = jammy - py3.6
    • check-skipped = jammy - py3.6
  • any of [🛡 GitHub branch protection]:
    • check-success = spelling
    • check-neutral = spelling
    • check-skipped = spelling
  • any of [🛡 GitHub branch protection]:
    • check-success = python format
    • check-neutral = python format
    • check-skipped = python format
  • any of [🛡 GitHub branch protection]:
    • check-success = python lint
    • check-neutral = python lint
    • check-skipped = python lint
  • any of [🛡 GitHub branch protection]:
    • check-success = validate commits
    • check-neutral = validate commits
    • check-skipped = validate commits

@mergify mergify bot removed the queued label Feb 13, 2026
@codecov
Copy link

codecov bot commented Feb 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.93%. Comparing base (f5e6cc5) to head (f571b96).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #534   +/-   ##
=======================================
  Coverage   82.93%   82.93%           
=======================================
  Files          27       27           
  Lines        2479     2479           
=======================================
  Hits         2056     2056           
  Misses        423      423           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement upgrades to an already existing feature low priority items that can be worked on at a later date merge-when-passing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants