Skip to content

Commit

Permalink
feat: Add --filter-jq option (#1372)
Browse files Browse the repository at this point in the history 
Adds filtering using `jq` syntax. You can do something like `--filter-jq
'.summary.files_new > 51'` - works similar to what you'd expect from
`jq`.

Also puts the `rhai` and `jq` filtering into features which are enabled
by default.

---------

Signed-off-by: simonsan <[email protected]>
Co-authored-by: simonsan <[email protected]>
  • Loading branch information
@aawsome @simonsan
aawsome and simonsan authored Nov 30, 2024
1 parent 5b02cd2 commit c571279
Show file tree
Hide file tree
Showing 8 changed files with 225 additions and 33 deletions.
68 changes: 68 additions & 0 deletions Cargo.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 11 additions & 2 deletions Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ rustic - fast, encrypted, deduplicated backups powered by Rust
"""

[features]
default = ["tui", "webdav"]
default = ["tui", "webdav", "rhai", "jq"]
release = ["default", "self-update"]

# Allocators
Expand All @@ -37,6 +37,10 @@ webdav = [
]
mount = ["dep:fuse_mt"]

# Filtering
rhai = ["dep:rhai"]
jq = ["dep:jaq-core", "dep:jaq-std", "dep:jaq-json"]

[[bin]]
name = "rustic"
path = "src/bin/rustic.rs"
Expand Down Expand Up @@ -87,7 +91,6 @@ serde_with = { version = "3", features = ["base64"] }
aho-corasick = "1"
chrono = { version = "0.4", default-features = false, features = ["clock", "serde"] }
comfy-table = "7"
rhai = { version = "1", features = ["sync", "serde", "no_optimize", "no_module", "no_custom_syntax", "only_i64"] }
scopeguard = "1"
semver = { version = "1", optional = true }
simplelog = "0.12"
Expand Down Expand Up @@ -117,6 +120,12 @@ self_update = { version = "=0.39.0", default-features = false, optional = true,
tar = "0.4.43"
toml = "0.8"

# filtering
jaq-core = { version = "2", optional = true }
jaq-json = { version = "1", features = ["serde_json"], optional = true }
jaq-std = { version = "2", optional = true }
rhai = { version = "1", features = ["sync", "serde", "no_optimize", "no_module", "no_custom_syntax", "only_i64"], optional = true }

[dev-dependencies]
abscissa_core = { version = "0.8.1", default-features = false, features = ["testing"] }
assert_cmd = "2.0.16"
Expand Down
31 changes: 16 additions & 15 deletions config/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -159,21 +159,22 @@ See [Global Hooks](#global-hooks-globalhooks).

### Snapshot-Filter Options `[snapshot-filter]`

| Attribute | Description | Default Value | Example Value | CLI Option |
| ------------------ | ---------------------------------------------------------------------- | ------------- | ------------------------ | -------------------- |
| filter-hosts | Array of hosts to filter snapshots. | Not set | ["myhost", "host2"] | --filter-host |
| filter-labels | Array of labels to filter snapshots. | Not set | ["mylabal"] | --filter-label |
| filter-paths | Array of pathlists to filter snapshots. | Not set | ["/home,/root"] | --filter-paths |
| filter-paths-exact | Array or string of paths to filter snapshots. Exact match. | Not set | ["path1,path2", "path3"] | --filter-paths-exact |
| filter-tags | Array of taglists to filter snapshots. | Not set | ["tag1,tag2"] | --filter-tags |
| filter-tags-exact | Array or string of tags to filter snapshots. Exact match. | Not set | ["tag1,tag2", "tag3"] | --filter-tags-exact |
| filter-before | Filter snapshots before the given date/time | Not set | "2024-01-01" | --filter-before |
| filter-after | Filter snapshots after the given date/time | Not set | "2023-01-01 11:15:23" | --filter-after |
| filter-size | Filter snapshots for a total size in the size range. | Not set | "1MB..1GB" | --filter-size |
| | If a single value is given, this is taken as lower bound. | | "500 k" | |
| filter-size-added | Filter snapshots for a size added to the repository in the size range. | Not set | "1MB..1GB" | --filter-size-added |
| | If a single value is given, this is taken as lower bound. | | "500 k" | |
| filter-fn | Custom filter function for snapshots. | Not set | | --filter-fn |
| Attribute | Description | Default Value | Example Value | CLI Option |
| ------------------ | ---------------------------------------------------------------------- | ------------- | -------------------------- | -------------------- |
| filter-hosts | Array of hosts to filter snapshots. | Not set | ["myhost", "host2"] | --filter-host |
| filter-labels | Array of labels to filter snapshots. | Not set | ["mylabal"] | --filter-label |
| filter-paths | Array of pathlists to filter snapshots. | Not set | ["/home,/root"] | --filter-paths |
| filter-paths-exact | Array or string of paths to filter snapshots. Exact match. | Not set | ["path1,path2", "path3"] | --filter-paths-exact |
| filter-tags | Array of taglists to filter snapshots. | Not set | ["tag1,tag2"] | --filter-tags |
| filter-tags-exact | Array or string of tags to filter snapshots. Exact match. | Not set | ["tag1,tag2", "tag3"] | --filter-tags-exact |
| filter-before | Filter snapshots before the given date/time | Not set | "2024-01-01" | --filter-before |
| filter-after | Filter snapshots after the given date/time | Not set | "2023-01-01 11:15:23" | --filter-after |
| filter-size | Filter snapshots for a total size in the size range. | Not set | "1MB..1GB" | --filter-size |
| | If a single value is given, this is taken as lower bound. | | "500 k" | |
| filter-size-added | Filter snapshots for a size added to the repository in the size range. | Not set | "1MB..1GB" | --filter-size-added |
| | If a single value is given, this is taken as lower bound. | | "500 k" | |
| filter-fn | Custom filter function for snapshots. | Not set | | --filter-fn |
| filter-jq | Custom filter jq function for snapshots. Should return bool | Not set | ".summary.files_added > 1" | --filter-jq |

### Backup Options `[backup]`

Expand Down
2 changes: 2 additions & 0 deletions config/full.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ filter-before = "2024-02-05 12:15" # Default: not set
filter-size = "200MiB" # Default: not set
filter-size-added = "1 MB..10MB" # Default: not set
filter-fn = '|sn| {sn.host == "host1" || sn.description.contains("test")}' # Default: no filter function
filter-jq = '.description | contains ("test")' # Default: no jq filter function

# Backup options: These options are used for all sources when calling the backup command.
# They can be overwritten by source-specific options (see below) or command line options.
Expand Down Expand Up @@ -174,6 +175,7 @@ filter-before = "2024-02-05 12:15" # Default: not set
filter-size = "200MiB" # Default: not set
filter-size-added = "1 MB..10MB" # Default: not set
filter-fn = '|sn| {sn.host == "host1" || sn.description.contains("test")}' # Default: no filter function
filter-jq = '.description | contains ("test")' # Default: no jq filter function
# The retention options follow. All of these are not set by default.
keep-tags = ["tag1", "tag2,tag3"] # Default: not set
keep-ids = [
Expand Down
2 changes: 2 additions & 0 deletions src/error.rs
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
//! Error types
use abscissa_core::error::{BoxError, Context};
#[cfg(feature = "rhai")]
use rhai::EvalAltResult;
use std::{
fmt::{self, Display},
Expand All @@ -18,6 +19,7 @@ pub(crate) enum ErrorKind {
}

/// Kinds of [`rhai`] errors
#[cfg(feature = "rhai")]
#[derive(Debug, Error)]
pub(crate) enum RhaiErrorKinds {
#[error(transparent)]
Expand Down
Loading

0 comments on commit c571279

Please sign in to comment.