Enable Plugin to Select Cache(s) to Use

[aowen-uwmad]

Pelican Service:

• Client
• Plugin
• Registry
• Director
• Origin
• Cache
• Other (please give the detail)

Is your feature request related to a problem? Please describe.

Being able to specify which cache is used for downloading the object is useful for testing the cache within the particular Pelican / HTCSS infrastructure the cache supports.

Further, for a site like CHTC with a cache in the same datacenter as the destination, preferring a particular cache can help minimize external network traffic for the datacenter.

Describe the solution you'd like

1. Enable the plugin to accept a list of cache(s) that restricts the client to only download objects from said caches.

2. Enable the plugin to accept a list of cache(s) that makes the client prefer to download objects from said caches.

Additional context

Item (1) seems straightforward.
Item (2) introduces the question of what the strength of the preference is. Ideally, if a preferred cache is overloaded (by some definition), it won't be used. Otherwise, the preferred cache is used. Exactly what scaler should be used in the weighting probably needs to be tested to identify a good default.

Ideally, items (1) and (2) are interoperable. Propose that list for (1) is the ultimate authority on what caches can be used. If a cache is listed for (2) that is not in the list for (1), then it won't be used. (Could consider doing data validation and throwing an error if there is an inconsistency.)

Coordination with HTCSS will be required for

A. The process of how the list(s) of cache(s) are passed to the plugin at the EP at the start of the job execution.

B. Whether a regular user can access this functionality, or if its configured by the AP/EP admin(s).