rspamd selector syntax not supported

[pipera]

conf-files using selector syntax (https://rspamd.com/doc/configuration/selectors.html) seem not to be supported. For example, I see no way to create 'ratelimit.conf' (https://rspamd.com/doc/modules/ratelimit.html) with the following syntax:

rates {
# Selector based ratelimit
some_limit = {
selector = 'user.lower';
# You can define more than one bucket, however, you need to use array syntax only
bucket = [
{
burst = 100;
rate = "10 / 1min";
},
{
burst = 10;
rate = "100 / 1min";
}]
}
}

[oxc]

Can you be a bit more explicit about what part exactly of the syntax is not supported? That would make it easier to find a solution. Thanks!

[pipera]

The problem is the nested structure of blocks (defined by '{..}') inside of an array (defined by '[..]' as seen in the 'bucket'-example above. I don't see a way to define this, I can only define blocks inside blocks (defined by '{...{...}{...}...}').

[oxc]

It should be easy to support this in the custom functions, but do you have any suggestions how to support this in hiera?

[SimonHoenscheid]

I ran into this as well, trying to work with dkim, but it has primarily to do with the way the module handles data types:

RSPAMD exspects the selector to be a string value. The only way to force the module to quote the value, is to add a special caracter to it, in the case "_".
A way to go around this, would be maps, but maps don't seem to be supported by the module.

domain {
  example.com {
    path = "/var/lib/rspamd/dkim/$domain.$selector.key";
    selector = "date_20200822";
  }
  example.net {
    path = "/var/lib/rspamd/dkim/$domain.$selector.key";
    selector = 20200822;
  }
}

[oxc]

I ran into this as well, trying to work with dkim, but it has primarily to do with the way the module handles data types:

RSPAMD exspects the selector to be a string value. The only way to force the module to quote the value, is to add a special caracter to it, in the case "_".
A way to go around this, would be maps, but maps don't seem to be supported by the module.

domain {
  example.com {
    path = "/var/lib/rspamd/dkim/$domain.$selector.key";
    selector = "date_20200822";
  }
  example.net {
    path = "/var/lib/rspamd/dkim/$domain.$selector.key";
    selector = 20200822;
  }
}

The rspamd::config resource already supports specifying the type, e.g. $type = 'string'. By default the type is 'auto', which coerces strings that look like numbers into numbers, which is what you are describing here.

Please open a different issue for this. If you can, please provide an idea on how to solve this.

[oxc]

@pipera, I tried to model your example from the first post:

    rates:
      some_limit:
        selector: 'user.lower'
        bucket:
          - burst: 100
            rate: "10 / 1min"
          - burst: 10
            rate: "100 / 1min"

This results in the following UCL:

rates {
  some_limit {
    selector = "user.lower";
    bucket {
      burst = 100;
      rate = "10 / 1min";
    }
    bucket {
      burst = 10;
      rate = "100 / 1min";
    }
  }
}

According to the UCL documentation, "Non-unique keys in an object are allowed and are automatically converted to the arrays internally". Unless this is wrong, the generated code should be fully equivalent to what you have pasted (at least as long as you stay within one file, I vaguely remember encountering issues when automatic arrays were merged).

Does this work for you?

[pipera]

Hello and thanks for this suggestion, if I implement this solution, my 'ratelimit.conf' contains (as suggested): rates { from { selector = "from:addr"; bucket { burst = 100; rate = "10 / 1min"; } bucket { burst = 10; rate = "100 / 1min"; } } to { bucket { burst = 100; rate = "10 / 1min"; } bucket { burst = 10; rate = "100 / 1min"; } } } but 'rspamadm configtest' fails with: 2020-09-24 12:08:57 #0(main) <c8tgm5>; lua; ratelimit.lua:317: cannot parse bucket for to: field "burst": expected type "number", or type "string"; original value: {[burst] = {[1] = 100, [2] = 10}, [rate] = {[1] = 10 / 1min, [2] = 100 / 1min}} 2020-09-24 12:08:57 #0(main) <c8tgm5>; lua; ratelimit.lua:749: bad ratelimit bucket for to: "{[burst] = {[1] = 100, [2] = 10}, [rate] = {[1] = 10 / 1min, [2] = 100 / 1min}}" 2020-09-24 12:08:57 #0(main) <c8tgm5>; lua; ratelimit.lua:317: cannot parse bucket for from: field "burst": expected type "number", or type "string"; original value: {[burst] = {[1] = 100, [2] = 10}, [rate] = {[1] = 10 / 1min, [2] = 100 / 1min}} 2020-09-24 12:08:57 #0(main) <c8tgm5>; lua; ratelimit.lua:749: bad ratelimit bucket for from: "{[burst] = {[1] = 100, [2] = 10}, [rate] = {[1] = 10 / 1min, [2] = 100 / 1min}}" I therefore assume, that https://rspamd.com/doc/configuration/ucl.html#automatic-arrays-creation does not work with hashes as values of the non-unique keys. Regards, Andreas Piper Am 23.09.20 um 18:01 schrieb Bernhard Frauendienst:

…

@pipera <https://github.com/pipera>, I tried to model your example from the first post: rates: some_limit: selector:'user.lower' bucket: -burst:100 rate:"10 / 1min" -burst:10 rate:"100 / 1min" This results in the following UCL: |rates { some_limit { selector = "user.lower"; bucket { burst = 100; rate = "10 / 1min"; } bucket { burst = 10; rate = "100 / 1min"; } } } | According to the UCL documentation, "Non-unique keys in an object are allowed and are automatically converted to the arrays internally". Unless this is wrong, the generated code should be fully equivalent to what you have pasted (at least as long as you stay within one file, I vaguely remember encountering issues when automatic arrays were merged). Does this work for you? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABUQIHUJBT52RMTZRDIIDSTSHILVXANCNFSM4I2MGVMA>.

[smortex]

I am also hitting this today trying to have multiple DKIM signatures on outgoing messages. The expected configuration is:

domain {
  blogreen.org {
    selectors [
      {
        path = "/usr/local/etc/rspamd/dkim-keys/20251021-rsa2048.key";
        selector = "20251021-rsa2048";
      },
      {
        path = "/usr/local/etc/rspamd/dkim-keys/20251021-ed25519.key";
        selector = "20251021-ed25519";
      }
    ]
  }
}

but as stated above, the module cannot produce this verbatim and the suggestion to make it output the following:

domain {
  blogreen.org {
    selectors {
      path = "/usr/local/etc/rspamd/dkim-keys/20251021-rsa2048.key";
      selector = "20251021-rsa2048";
    }
    selectors {
      path = "/usr/local/etc/rspamd/dkim-keys/20251021-ed25519.key";
      selector = "20251021-ed25519";
    }
  }
}

is not working as expected as reported by rspamadm configdump dkim_signing:

[...]
domain {
    blogreen.org {
        selectors {
            path = "/usr/local/etc/rspamd/dkim-keys/20251021-rsa2048.key";
            path = "/usr/local/etc/rspamd/dkim-keys/20251021-ed25519.key";
            selector = "20251021-rsa2048";
            selector = "20251021-ed25519";
        }
    }
}

If you can, please provide an idea on how to solve this.

According to the rspamd documentation, UCL is fully compatible with JSON format, so maybe switching form outputting UCL to outputting JSON could help?

I am new to rspamd so maybe missing some obvious issues with such a switch. Before investing time working on this, may I have your opinion @oxc, @SimonHoenscheid and @pipera ?

[smortex]

According to the rspamd documentation, UCL is fully compatible with JSON format, so maybe switching form outputting UCL to outputting JSON could help?

I gave it a try in this branch, unfortunately it is currently blocked by a bug in libucl that prevent it from including a well-formed JSON file: vstakhov/libucl#274.

I proposed a fix in vstakhov/libucl#342 but that mean that this approach will not be usable until libucl is fixed, released, rspamd update it, and cut a new release including the fix 😓.