Install and configure Rucio helm charts in the cern-vre cluster #8

goseind · 2022-10-19T15:57:50Z

Rucio should be installed using helm charts deployed by terraform. Therefore the following tasks need to be completed:

Tasks

Give feedback

Create k8s secrets via sealed-secrets for the rucio server, auth and UI and commit them to the repo
Test the secrets
Create a secret for DB access and specific it in the values for the chart via terraform
Configure and install rucio server chart
Configure and install rucio daemons chart
Configure IAM ESCAPE authentication service for Rucio #32

cern-vre-infra component/other component/rucio
Create additional hermes certificate secrets for file transfers
Configure storage endpoints for rucio files (RSEs) (eos and s3 storage for first testing) Configure RSEs in Rucio #16
Test FTS transfer between RSEs Connect RUCIO to FTS Pilot at CERN #24
Postponed: Exchange CERN host certificates with Sectigo one Create required certificates for RUCIO and create secrets for them #14 - is this necessary except for the UI which (needs) to be accessible outside?
Postponed: Configure and install rucio UI chart
IAM-Rucio sync cern-vre/cern-vre#46

cern-vre-infra component/rucio
Options

goseind · 2023-01-16T17:44:02Z

Rucio Server helm chart is now deployed and the database was initialized with the init container thus it is now possible to log in with the default admin user.
Rucio daemons are currently being installed but failing due to problems with FTS, refer to documentation: More work on documentation Fix #86 rucio/rucio#24

goseind · 2023-01-27T11:05:11Z

Rucio UI helm chart can be deployed but runs into the following SSL errors when calling the URL:

[Fri Jan 27 11:01:25.649399 2023] [ssl:info] [pid 21:tid 139908013438720] [client 10.100.96.192:37102] AH02008: SSL library error 1 in handshake (server 10.100.37.168:443)
[Fri Jan 27 11:01:25.649561 2023] [ssl:info] [pid 21:tid 139908013438720] SSL Library Error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
[Fri Jan 27 11:01:25.649567 2023] [ssl:info] [pid 21:tid 139908013438720] [client 10.100.96.192:37102] AH01998: Connection closed to child 384 with abortive shutdown (server 10.100.37.168:443)
[Fri Jan 27 11:01:25.650464 2023] [ssl:info] [pid 21:tid 139908005046016] [client 10.100.96.192:37104] AH01964: Connection to child 385 established (server 10.100.37.168:443)
[Fri Jan 27 11:01:25.650577 2023] [ssl:info] [pid 21:tid 139908005046016] [client 10.100.96.192:37104] AH02008: SSL library error 1 in handshake (server 10.100.37.168:443)
[Fri Jan 27 11:01:25.650590 2023] [ssl:info] [pid 21:tid 139908005046016] SSL Library Error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
[Fri Jan 27 11:01:25.650594 2023] [ssl:info] [pid 21:tid 139908005046016] [client 10.100.96.192:37104] AH01998: Connection closed to child 385 with abortive shutdown (server 10.100.37.168:443)
[Fri Jan 27 11:01:25.651335 2023] [ssl:info] [pid 21:tid 139907996653312] [client 10.100.96.192:37106] AH01964: Connection to child 386 established (server 10.100.37.168:443)
[Fri Jan 27 11:01:25.651451 2023] [ssl:info] [pid 21:tid 139907996653312] [client 10.100.96.192:37106] AH02008: SSL library error 1 in handshake (server 10.100.37.168:443)
[Fri Jan 27 11:01:25.651477 2023] [ssl:info] [pid 21:tid 139907996653312] SSL Library Error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
[Fri Jan 27 11:01:25.651481 2023] [ssl:info] [pid 21:tid 139907996653312] [client 10.100.96.192:37106] AH01998: Connection closed to child 386 with abortive shutdown (server 10.100.37.168:443)

Cause unknown atm. --> Possibly debug the installation in the minikube env.

goseind · 2023-01-30T14:21:37Z

Previously service type LoadBalancer could not have been used due to issues with OpenStack. A workaround is to create a load balancer manually for that service e.g. openstack loadbalancer create --vip-network-id public --name kube_service_8d85602e-7816-4153-b85b-ea86586e47c1_rucio_rucio-ui-cvre.

However when trying to log in with userpass the service throws the following error now:

[Mon Jan 30 12:05:51.860445 2023] [ssl:info] [pid 19:tid 139907996653312] [client 10.100.160.64:26629] AH01964: Connection to child 130 established (server 10.100.37.168:443)
[Mon Jan 30 12:05:51.960288 2023] [wsgi:info] [pid 19:tid 139907996653312] [client 10.100.160.64:26629] mod_wsgi (pid=19, process='', application='10.100.37.168|'): Loading Python script file '/usr/local/lib/python3.6/site-packages/rucio/web/ui/main.py'., referer: https://vre-rucio-ui.cern.ch/login?account=&vo=
[Mon Jan 30 12:05:58.707361 2023] [ssl:info] [pid 19:tid 139907996653312] (70007)The timeout specified has expired: [client 10.100.160.64:26629] AH01991: SSL input filter read failed.
[Mon Jan 30 12:56:21.537152 2023] [ssl:info] [pid 21:tid 139907813041920] [client 10.100.135.128:28615] AH01964: Connection to child 399 established (server 10.100.37.168:443)
[Mon Jan 30 12:56:26.610992 2023] [ssl:info] [pid 21:tid 139907813041920] (70007)The timeout specified has expired: [client 10.100.135.128:28615] AH01991: SSL input filter read failed.
[Mon Jan 30 12:57:06.818449 2023] [ssl:info] [pid 18:tid 139907863398144] [client 188.184.72.100:59538] AH01964: Connection to child 9 established (server 10.100.37.168:443)

Debugging has led nowhere so far, this remains to be fixed.

* mod rucio ui values and replace network-id #8 * rm new rucio webui chart #8 * replace network-id in rucio srv and daemons #8

goseind · 2023-01-31T17:07:50Z

Service type LoadBalancer is now working with the correct network-id set in the service's annotations. However, web server errors on the userpass login continue:

[Tue Jan 31 17:00:33.098378 2023] [ssl:info] [pid 19:tid 140151248946944] [client 10.100.135.128:7773] AH01964: Connection to child 0 established (server 10.100.99.156:443)
[Tue Jan 31 17:00:33.570844 2023] [wsgi:info] [pid 19:tid 140151248946944] [client 10.100.135.128:7773] mod_wsgi (pid=19, process='', application='10.100.99.156|'): Loading Python script file '/usr/local/lib/python3.6/site-packages/rucio/web/ui/main.py'.
[Tue Jan 31 17:00:52.127880 2023] [ssl:info] [pid 20:tid 140151240554240] [client 10.100.20.192:9320] AH01964: Connection to child 129 established (server 10.100.99.156:443)
[Tue Jan 31 17:00:55.269670 2023] [wsgi:info] [pid 20:tid 140151240554240] [client 10.100.20.192:9320] mod_wsgi (pid=20, process='', application='10.100.99.156|'): Loading Python script file '/usr/local/lib/python3.6/site-packages/rucio/web/ui/main.py'., referer: https://vre-rucio-ui.cern.ch/
[Tue Jan 31 17:01:15.071819 2023] [ssl:info] [pid 21:tid 140151248946944] [client 10.100.253.128:21893] AH01964: Connection to child 256 established (server 10.100.99.156:443)
[Tue Jan 31 17:01:15.076146 2023] [ssl:info] [pid 20:tid 140151158208256] [client 10.100.53.0:16837] AH01964: Connection to child 130 established (server 10.100.99.156:443)
[Tue Jan 31 17:01:20.079807 2023] [ssl:info] [pid 20:tid 140151240554240] (70007)The timeout specified has expired: [client 10.100.20.192:9320] AH01991: SSL input filter read failed.
[Tue Jan 31 17:01:20.270969 2023] [ssl:info] [pid 21:tid 140151248946944] (70007)The timeout specified has expired: [client 10.100.253.128:21893] AH01991: SSL input filter read failed.
[Tue Jan 31 17:01:32.291420 2023] [ssl:info] [pid 22:tid 140151248946944] [client 10.100.161.128:53781] AH01964: Connection to child 384 established (server 10.100.99.156:443)
[Tue Jan 31 17:01:32.664823 2023] [wsgi:info] [pid 22:tid 140151248946944] [client 10.100.161.128:53781] mod_wsgi (pid=22, process='', application='10.100.99.156|'): Loading Python script file '/usr/local/lib/python3.6/site-packages/rucio/web/ui/main.py'., referer: https://vre-rucio-ui.cern.ch/login?account=&vo=
[Tue Jan 31 17:01:57.900773 2023] [ssl:info] [pid 22:tid 140151248946944] (70007)The timeout specified has expired: [client 10.100.161.128:53781] AH01991: SSL input filter read failed.
[Tue Jan 31 17:02:23.220254 2023] [ssl:info] [pid 22:tid 140151240554240] [client 10.100.132.64:63513] AH01964: Connection to child 385 established (server 10.100.99.156:443)
[Tue Jan 31 17:02:32.889826 2023] [ssl:info] [pid 22:tid 140151240554240] (70007)The timeout specified has expired: [client 10.100.132.64:63513] AH01991: SSL input filter read failed.
[Tue Jan 31 17:03:00.864457 2023] [ssl:info] [pid 22:tid 140151158208256] [client 10.100.214.64:13691] AH01964: Connection to child 386 established (server 10.100.99.156:443)
[Tue Jan 31 17:03:03.146518 2023] [ssl:info] [pid 21:tid 140151158208256] [client 10.100.244.128:57799] AH01964: Connection to child 258 established (server 10.100.99.156:443)
[Tue Jan 31 17:03:03.172330 2023] [ssl:info] [pid 21:tid 140151158208256] [client 10.100.244.128:57799] AH01998: Connection closed to child 258 with abortive shutdown (server 10.100.99.156:443)
[Tue Jan 31 17:03:12.128240 2023] [ssl:info] [pid 22:tid 140151158208256] (70007)The timeout specified has expired: [client 10.100.214.64:13691] AH01991: SSL input filter read failed.
[Tue Jan 31 17:03:15.860412 2023] [ssl:info] [pid 22:tid 140151149815552] [client 10.100.2.128:48370] AH01964: Connection to child 387 established (server 10.100.99.156:443)
[Tue Jan 31 17:04:19.729226 2023] [ssl:info] [pid 554:tid 140151248946944] [client 10.100.96.192:14640] AH01964: Connection to child 0 established (server 10.100.99.156:443)
[Tue Jan 31 17:04:20.071306 2023] [wsgi:info] [pid 554:tid 140151248946944] [client 10.100.96.192:14640] mod_wsgi (pid=554, process='', application='10.100.99.156|'): Loading Python script file '/usr/local/lib/python3.6/site-packages/rucio/web/ui/main.py'., referer: https://vre-rucio-ui.cern.ch/login
[Tue Jan 31 17:04:45.273059 2023] [ssl:info] [pid 554:tid 140151248946944] (70007)The timeout specified has expired: [client 10.100.96.192:14640] AH01991: SSL input filter read failed.

Putting the UI implementation on hold for now and rather implement third-party auth with IAM ESCAPE.

goseind · 2023-02-02T14:58:15Z

This might be helpful: https://github.com/cfenell/rucio-1/blob/install-doc/doc/rucio-server.md also subscribed to this: rucio/documentation#178

goseind · 2023-02-02T15:20:07Z

Upload example to EULAKE-1:

(rucio-cli) [root@kike-dev ~]# rucio  -vvv upload --rse EULAKE-1 --scope test rtest_k_20220124-01.txt
2023-01-26 12:18:17,526 DEBUG   baseclient.py   No trace_host passed. Using rucio_host instead
2023-01-26 12:18:17,526 DEBUG   baseclient.py   No creds passed. Trying to get it from the config file.
2023-01-26 12:18:17,526 DEBUG   baseclient.py   HTTPS is required, but no ca_cert was passed. Trying to get it from X509_CERT_DIR.
2023-01-26 12:18:17,527 DEBUG   baseclient.py   HTTPS is required, but no ca_cert was passed and X509_CERT_DIR is not defined. Trying to get it from the config file.
2023-01-26 12:18:17,527 DEBUG   baseclient.py   No account passed. Trying to get it from the RUCIO_ACCOUNT environment variable or the config file.
2023-01-26 12:18:17,527 DEBUG   baseclient.py   No VO passed. Trying to get it from environment variable RUCIO_VO.
2023-01-26 12:18:17,527 DEBUG   baseclient.py   No VO found. Trying to get it from the config file.
2023-01-26 12:18:17,527 DEBUG   baseclient.py   No VO found. Using default VO.
2023-01-26 12:18:17,527 DEBUG   baseclient.py   got token from file
2023-01-26 12:18:17,548 DEBUG   uploadclient.py Num. of files that upload client is processing: 1
2023-01-26 12:18:17,548 DEBUG   baseclient.py   HTTP request: GET https://vre-rucio.cern.ch:443/rses/?expression=EULAKE-1
2023-01-26 12:18:17,548 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2023-01-26 12:18:17,548 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2023-01-26 12:18:17,548 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2023-01-26 12:18:17,548 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/1.30.1
2023-01-26 12:18:17,548 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-vvv
2023-01-26 12:18:17,548 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2023-01-26 12:18:17,565 DEBUG   baseclient.py   HTTP Response: 200 OK
2023-01-26 12:18:17,612 DEBUG   uploadclient.py Input validation done.
2023-01-26 12:18:17,613 INFO    Preparing upload for file rtest_k_20220124-01.txt
2023-01-26 12:18:17,613 DEBUG   baseclient.py   HTTP request: GET https://vre-rucio.cern.ch:443/rses/EULAKE-1/attr/
2023-01-26 12:18:17,613 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2023-01-26 12:18:17,613 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2023-01-26 12:18:17,613 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2023-01-26 12:18:17,613 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/1.30.1
2023-01-26 12:18:17,613 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-vvv
2023-01-26 12:18:17,613 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2023-01-26 12:18:17,636 DEBUG   baseclient.py   HTTP Response: 200 OK
2023-01-26 12:18:17,636 DEBUG   uploadclient.py wan domain is used for the upload
2023-01-26 12:18:17,637 DEBUG   uploadclient.py Registering file
2023-01-26 12:18:17,637 DEBUG   baseclient.py   HTTP request: GET https://vre-rucio.cern.ch:443/accounts/root/scopes/
2023-01-26 12:18:17,637 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2023-01-26 12:18:17,637 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2023-01-26 12:18:17,637 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2023-01-26 12:18:17,637 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/1.30.1
2023-01-26 12:18:17,637 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-vvv
2023-01-26 12:18:17,637 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2023-01-26 12:18:17,699 DEBUG   baseclient.py   HTTP Response: 200 OK
2023-01-26 12:18:17,699 DEBUG   uploadclient.py Skipping dataset registration
2023-01-26 12:18:17,699 DEBUG   baseclient.py   HTTP request: GET https://vre-rucio.cern.ch:443/dids/test/rtest_k_20220124-01.txt/meta
2023-01-26 12:18:17,699 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2023-01-26 12:18:17,699 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2023-01-26 12:18:17,699 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2023-01-26 12:18:17,699 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/1.30.1
2023-01-26 12:18:17,699 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-vvv
2023-01-26 12:18:17,699 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2023-01-26 12:18:17,722 DEBUG   baseclient.py   HTTP Response: 404 NOT FOUND
2023-01-26 12:18:17,725 DEBUG   baseclient.py   Response text (length=124): [{"ExceptionClass": "DataIdentifierNotFound", "ExceptionMessage": "Data identifier 'test:rtest_k_20220124-01.txt' not found"}]
2023-01-26 12:18:17,725 DEBUG   uploadclient.py File DID does not exist
2023-01-26 12:18:17,725 DEBUG   baseclient.py   HTTP request: POST https://vre-rucio.cern.ch:443/replicas
2023-01-26 12:18:17,725 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2023-01-26 12:18:17,725 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2023-01-26 12:18:17,725 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2023-01-26 12:18:17,725 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/1.30.1
2023-01-26 12:18:17,725 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-vvv
2023-01-26 12:18:17,725 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2023-01-26 12:18:17,725 DEBUG   baseclient.py   Request data (length=260): [{"rse": "EULAKE-1", "files": [{"scope": "test", "name": "rtest_k_20220124-01.txt", "bytes": 54, "adler32": "fc0d110b" ... ty": true}]
2023-01-26 12:18:17,825 DEBUG   baseclient.py   HTTP Response: 201 CREATED
2023-01-26 12:18:17,825 INFO    Successfully added replica in Rucio catalogue at EULAKE-1
2023-01-26 12:18:17,826 DEBUG   baseclient.py   HTTP request: POST https://vre-rucio.cern.ch:443/rules/
2023-01-26 12:18:17,826 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2023-01-26 12:18:17,826 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2023-01-26 12:18:17,826 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2023-01-26 12:18:17,826 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/1.30.1
2023-01-26 12:18:17,826 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-vvv
2023-01-26 12:18:17,826 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2023-01-26 12:18:17,826 DEBUG   baseclient.py   Request data (length=438): [{"dids": [{"scope": "test", "name": "rtest_k_20220124-01.txt"}], "copies": 1, "rse_expression": "EULAKE-1", "weight": ... ta": null}]
2023-01-26 12:18:18,008 DEBUG   baseclient.py   HTTP Response: 201 CREATED
2023-01-26 12:18:18,008 INFO    Successfully added replication rule at EULAKE-1
2023-01-26 12:18:18,026 DEBUG   logging.py      gfal.Default: connecting to storage
2023-01-26 12:18:18,056 DEBUG   logging.py      gfal.Default: checking if file exists None
2023-01-26 12:18:18,056 DEBUG   rsemanager.py   Checking if root://eoseulake.cern.ch:1094//eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt exists
2023-01-26 12:18:18,057 DEBUG   logging.py      gfal.Default: checking if file exists root://eoseulake.cern.ch:1094//eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt
2023-01-26 12:18:18,125 DEBUG   logging.py      gfal.Default: closing protocol connection
2023-01-26 12:18:18,125 DEBUG   uploadclient.py [{'hostname': 'eoseulake.cern.ch', 'scheme': 'davs', 'port': 443, 'prefix': '/eos/eulake/tests/rucio_test/eulake_1', 'impl': 'rucio.rse.protocols.gfal.Default', 'domains': {'lan': {'read': 3, 'write': 3, 'delete': 3}, 'wan': {'read': 3, 'write': 3, 'delete': 3, 'third_party_copy_read': 3, 'third_party_copy_write': 3}}, 'extended_attributes': None}, {'hostname': 'eoseulake.cern.ch', 'scheme': 'root', 'port': 1094, 'prefix': '//eos/eulake/tests/rucio_test/eulake_1', 'impl': 'rucio.rse.protocols.gfal.Default', 'domains': {'lan': {'read': 1, 'write': 2, 'delete': 1}, 'wan': {'read': 1, 'write': 2, 'delete': 1, 'third_party_copy_read': 2, 'third_party_copy_write': 2}}, 'extended_attributes': None}, {'hostname': 'eulakeftp.cern.ch', 'scheme': 'gsiftp', 'port': 2811, 'prefix': '/eos/eulake/tests/rucio_test/eulake_1', 'impl': 'rucio.rse.protocols.gfal.Default', 'domains': {'lan': {'read': 2, 'write': 1, 'delete': 2}, 'wan': {'read': 2, 'write': 1, 'delete': 2, 'third_party_copy_read': 1, 'third_party_copy_write': 1}}, 'extended_attributes': None}]
2023-01-26 12:18:18,125 INFO    Trying upload with gsiftp to EULAKE-1
2023-01-26 12:18:18,125 DEBUG   uploadclient.py Processing upload with the domain: wan
2023-01-26 12:18:18,126 DEBUG   logging.py      gfal.Default: connecting to storage
2023-01-26 12:18:18,126 DEBUG   uploadclient.py The PFN created from the LFN: gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt
2023-01-26 12:18:18,126 DEBUG   logging.py      gfal.Default: checking if file exists gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt
2023-01-26 12:18:18,348 DEBUG   logging.py      gfal.Default: checking if file exists gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt.rucio.upload
2023-01-26 12:18:18,350 DEBUG   utils.py        put: Attempt 1
2023-01-26 12:18:18,351 DEBUG   logging.py      gfal.Default: uploading file from rtest_k_20220124-01.txt to gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt.rucio.upload
2023-01-26 12:18:18,381 INFO    Successful upload of temporary file. gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt.rucio.upload
2023-01-26 12:18:18,381 DEBUG   uploadclient.py skip_upload_stat=False
2023-01-26 12:18:18,381 DEBUG   uploadclient.py stat: pfn=gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt.rucio.upload
2023-01-26 12:18:18,381 DEBUG   logging.py      gfal.Default: getting stats of file gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt.rucio.upload
2023-01-26 12:18:18,385 DEBUG   uploadclient.py Filesize: Expected=54 Found=54
2023-01-26 12:18:18,385 DEBUG   uploadclient.py Checksum: Expected=fc0d110b Found=fc0d110b
2023-01-26 12:18:18,385 DEBUG   uploadclient.py Renaming file gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt.rucio.upload to gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt
2023-01-26 12:18:18,385 DEBUG   logging.py      gfal.Default: renaming file from gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt.rucio.upload to gsiftp://eulakeftp.cern.ch:2811/eos/eulake/tests/rucio_test/eulake_1/test/99/27/rtest_k_20220124-01.txt
2023-01-26 12:18:18,391 DEBUG   logging.py      gfal.Default: closing protocol connection
2023-01-26 12:18:18,391 DEBUG   uploadclient.py Upload done.
2023-01-26 12:18:18,392 INFO    Successfully uploaded file rtest_k_20220124-01.txt
/root/envs-virtualenv/rucio-cli/lib/python3.6/site-packages/urllib3/connectionpool.py:1050: InsecureRequestWarning: Unverified HTTPS request is being made to host 'vre-rucio.cern.ch'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
2023-01-26 12:18:18,411 DEBUG   baseclient.py   HTTP request: PUT https://vre-rucio.cern.ch:443/replicas
2023-01-26 12:18:18,411 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2023-01-26 12:18:18,411 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2023-01-26 12:18:18,411 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2023-01-26 12:18:18,411 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/1.30.1
2023-01-26 12:18:18,411 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-vvv
2023-01-26 12:18:18,411 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2023-01-26 12:18:18,411 DEBUG   baseclient.py   Request data (length=231): [{"rse": "EULAKE-1", "files": [{"scope": "test", "name": "rtest_k_20220124-01.txt", "bytes": 54, "adler32": "fc0d110b" ... e": "A"}]}]
2023-01-26 12:18:18,500 DEBUG   baseclient.py   HTTP Response: 200 OK
Completed in 0.9749 sec.

egazzarr · 2023-04-14T10:22:40Z

I think we can mark it as closed and add only Rucio UI

goseind · 2023-04-19T09:44:03Z

Rucio is fully operational.

goseind added the help wanted Extra attention is needed label Oct 19, 2022

goseind changed the title ~~Install and configure rucio~~ Install and configure RUCIO in the escape-future cluster Nov 15, 2022

goseind changed the title ~~Install and configure RUCIO in the escape-future cluster~~ Install and configure RUCIO in the eosc-future cluster Nov 15, 2022

goseind added this to the Full-scale RUCIO cluster on CERN resources milestone Nov 15, 2022

goseind assigned egazzarr and goseind Nov 15, 2022

goseind mentioned this issue Jan 16, 2023

Install RUCIO with terraform helm provider #17

Closed

goseind added component/rucio priority/critical Needs to be done very soon and removed help wanted Extra attention is needed labels Jan 20, 2023

goseind changed the title ~~Install and configure RUCIO in the eosc-future cluster~~ Install and configure RUCIO in the cern-vre cluster Jan 25, 2023

goseind added a commit that referenced this issue Jan 30, 2023

update rucio ui values #8

4b16f1a

goseind changed the title ~~Install and configure RUCIO in the cern-vre cluster~~ Install and configure Rucio helm charts in the cern-vre cluster Jan 30, 2023

goseind added a commit that referenced this issue Jan 31, 2023

mod rucio ui values and replace network-id #8

421223e

goseind added a commit that referenced this issue Jan 31, 2023

rm new rucio webui chart #8

673fc69

goseind added a commit that referenced this issue Jan 31, 2023

replace network-id in rucio srv and daemons #8

6bca7bc

goseind added a commit that referenced this issue Jan 31, 2023

Replace LBaaS network-id and mod rucio ui chart (#29)

458012a

* mod rucio ui values and replace network-id #8 * rm new rucio webui chart #8 * replace network-id in rucio srv and daemons #8

goseind pinned this issue Feb 7, 2023

goseind closed this as completed Apr 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Install and configure Rucio helm charts in the cern-vre cluster #8

Install and configure Rucio helm charts in the cern-vre cluster #8

goseind commented Oct 19, 2022 •

edited

Loading

Tasks

goseind commented Jan 16, 2023

goseind commented Jan 27, 2023 •

edited

Loading

goseind commented Jan 30, 2023

goseind commented Jan 31, 2023

goseind commented Feb 2, 2023

goseind commented Feb 2, 2023

egazzarr commented Apr 14, 2023

goseind commented Apr 19, 2023

Install and configure Rucio helm charts in the cern-vre cluster #8

Install and configure Rucio helm charts in the cern-vre cluster #8

Comments

goseind commented Oct 19, 2022 • edited Loading

Tasks

goseind commented Jan 16, 2023

goseind commented Jan 27, 2023 • edited Loading

goseind commented Jan 30, 2023

goseind commented Jan 31, 2023

goseind commented Feb 2, 2023

goseind commented Feb 2, 2023

egazzarr commented Apr 14, 2023

goseind commented Apr 19, 2023

goseind commented Oct 19, 2022 •

edited

Loading

goseind commented Jan 27, 2023 •

edited

Loading