Objectives

The general objective of the outlined plan is to enhance the REANA system by replacing the existing Kubernetes job submission with Kueue, addressing challenges related to local deployment, implementing FAIR sharing mechanisms, creating an API abstraction layer between REANA and Kueue, and improving the REANA job controller API. 

Specific Objectives: 

1. Investigation of Integration Challenges:

   • Conduct an in-depth analysis to identify and understand the multifaceted challenges and intricacies inherent in integrating Kueue into the REANA platform. Special emphasis will be placed on the nuanced requirements and considerations associated with optimizing runtime user jobs.
   • Explore innovative solutions for the creation of high-level abstractions or Python wrappers. This exploration aims to cultivate a more intuitive and efficient interaction between REANA and Kueue's low-level Kubernetes API. The goal is to establish a framework that not only overcomes integration hurdles but also enhances code readability and maintainability.

2. Performance Evaluation:

   • Leverage open data from CERN CMS experiments to simulate real-world computational workloads. This simulation will provide a stringent and realistic test environment, facilitating an in-depth analysis of Kueue's efficiency under diverse and demanding conditions.

3. Viability Analysis:

   • Evaluate the impact of Kueue on job scheduling efficiency within the REANA ecosystem. This assessment will gauge the system's ability to handle runtime user jobs more effectively, taking into account factors such as job submission speed, task distribution, and overall responsiveness.