Configure RSEs in Rucio

[egazzarr]

CRIC can be used as a library for RSEs configuration details (ref. https://github.com/cern-vre/project/issues/15). With a script running in a cronjob, this information can then be pulled from CRIC and imported into rucio.

Tasks

Preview Give feedback

1. Implement this script as a cronjob: https://github.com/ESCAPE-WP2/Utilities-and-Operations-Scripts/tree/master/cric-rucio-sync
2. Investigate current RSEs and request new endpoints #76
   cern-escape-infra cern-vre-infra component/rucio +3
   

[goseind]

@garciagenrique maybe for testing we deploy this: https://github.com/goseind/k8s-tutorial/blob/master/xrd.yaml

[goseind]

Update

The following commands have been executed already:

rucio-admin rse add XRD1
rucio-admin rse add XRD2
rucio-admin rse add XRD3
rucio-admin rse add-protocol --hostname xrd1 --scheme root --prefix //rucio --port 1094 --impl rucio.rse.protocols.gfal.Default --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' XRD1
rucio-admin rse add-protocol --hostname xrd2 --scheme root --prefix //rucio --port 1094 --impl rucio.rse.protocols.gfal.Default --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' XRD2
rucio-admin rse add-protocol --hostname xrd3 --scheme root --prefix //rucio --port 1094 --impl rucio.rse.protocols.gfal.Default --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}, "lan": {"read": 1, "write": 1, "delete": 1}}' XRD3
rucio-admin rse set-attribute --rse XRD1 --key fts --value https://fts3-pilot.cern.ch:8446
rucio-admin rse set-attribute --rse XRD2 --key fts --value https://fts3-pilot.cern.ch:8446
rucio-admin rse set-attribute --rse XRD3 --key fts --value https://fts3-pilot.cern.ch:8446
rucio-admin rse add-distance --distance 1 --ranking 1 XRD1 XRD2
rucio-admin rse add-distance --distance 1 --ranking 1 XRD1 XRD3
rucio-admin rse add-distance --distance 1 --ranking 1 XRD2 XRD1
rucio-admin rse add-distance --distance 1 --ranking 1 XRD2 XRD3
rucio-admin rse add-distance --distance 1 --ranking 1 XRD3 XRD1
rucio-admin rse add-distance --distance 1 --ranking 1 XRD3 XRD2
rucio-admin account set-limits root XRD1 -1
rucio-admin account set-limits root XRD2 -1
rucio-admin account set-limits root XRD3 -1
rucio-admin scope add --account root --scope test
dd if=/dev/urandom of=file1 bs=10M count=1
dd if=/dev/urandom of=file2 bs=10M count=1
dd if=/dev/urandom of=file3 bs=10M count=1
dd if=/dev/urandom of=file4 bs=10M count=1

⚠️ Creating the xrd pods and secrets with our certificate and key but pods crash loop for yet unknown reason. Pods start when using the dummy certificates.

[garciagenrique]

Two RSEs added to the CRE-VRE cluster: EULAKE-1 and LAPP-DCACHE.

Followed this documentation: https://github.com/cern-vre/cern-vre/wiki/Components#adding-rses-to-the-rucio-instance

[garciagenrique]

Uploads to EULAKE-1 are working fine, we receive a warning that we don't know why it's appearing. RUCIO team contacted.

/root/envs-virtualenv/rucio-cli/lib/python3.6/site-packages/urllib3/connectionpool.py:1050: InsecureRequestWarning: Unverified HTTPS request is being made to host 'vre-rucio.cern.ch'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,

[garciagenrique]

It seems that the above Warning was also appearing in the escape-gitops cluster.

[garciagenrique]

• @egazzarr configured CNAF-STROM in the VRE.

• CESNET-S3 gives the following problem after the DB was upgraded:

$ rucio-admin rse info CESNET-S3
Database exception.
Details: An unknown Database Exception has ocurred.
Rucio exited with an unexpected/unknown error, please provide the traceback below to the developers.
Traceback (most recent call last):
  File "/root/envs-virtualenv/rucio-cli/bin/rucio-admin", line 97, in new_funct
    return function(*args, **kwargs)
  File "/root/envs-virtualenv/rucio-cli/bin/rucio-admin", line 530, in info_rse
    rseinfo = client.get_rse(rse=args.rse)
  File "/root/envs-virtualenv/rucio-cli/lib/python3.6/site-packages/rucio/client/rseclient.py", line 50, in get_rse
    raise exc_cls(exc_msg)
rucio.common.exception.DatabaseException: Database exception.
Details: An unknown Database Exception has ocurred.

• Erasing all replicas and files within the RSE:

$ rucio list-rules --account=<ACCOUNT_NAME>
$ rucio delete-rule <RULE_ID>

• However, some files are still there

# Connect to the PSQL DB
rucio=> SELECT * FROM replicas WHERE rse_id = '<RSE_ID>';

# To get the rse-id: (f.ex:)
$ rucio list-rse-usage <RSE_NAME>

Issue being investigated

[goseind]

Current status:

• Investigation of participation and functioning of current/old RSEs: https://docs.google.com/spreadsheets/d/1tp3osag7GrH_8TNd3R_LkfPOdAe71LwZd82L-UHM7WI/edit?usp=sharing
• Request of new endpoints for these RSEs to use with the new Rucio instance

[garciagenrique]

IMO, I will take out this script because it might lead to wrong RSE configuration.
If everybody agrees we could just add a section on the Wiki explaining the main attributes and points to be configured, but without proposing a template.

[goseind]

CRIC will not be used atm. Ne RSE endpoints will be added as they come in.