Add Arguments for Distributed Mode in Qualification Tool CLI

[parthosa]

This PR add arguments to enable distributed tools for qualification tool.

Changes

• Extended RapidsJob: Introduced two subclasses—RapidsDistributedJob and RapidsLocalJob and a concrete class for the OnPrem platform.
• Created a JarCmdArgs class to encapsulate all arguments needed to construct the JAR command.
• Implemented the DistributedToolsConfig class, allowing configurations for distributed tools (like Spark properties) to be specified via the --tools_config_file option.

Example

CMD:

spark_rapids qualification --platform onprem --eventlogs /path/to/eventlogs  --verbose --filter_apps all \
 --distributed --tools_config_file /path/to/custom_conf_file.yaml

Sample Config File:

api_version: '1.0'
distributed_tools:
  spark_properties:
    - name: 'spark.executor.memory'
      value: '20g'

Details:

• user_tools/src/spark_rapids_pytools/cloud_api/onprem.py: Added a new class OnPremDistributedRapidsJob and a method create_distributed_submission_job to support distributed RAPIDS jobs. [1] [2]
• user_tools/src/spark_rapids_pytools/rapids/rapids_job.py: Introduced RapidsDistributedJob class and updated methods to handle distributed tool configurations. [1] [2] [3] [4]
• user_tools/src/spark_rapids_pytools/rapids/rapids_tool.py: Added methods to get distributed tools configurations and submit distributed jobs. [1] [2]

Enhancements to argument processing:

• user_tools/src/spark_rapids_pytools/rapids/qualification.py: Added methods to process distributed tools arguments. [1] [2]
• user_tools/src/spark_rapids_tools/cmdli/argprocessor.py: Updated QualifyUserArgModel and build_tools_args to include distributed_tools_enabled. [1] [2]

Platform class updates:

• user_tools/src/spark_rapids_pytools/cloud_api/databricks_aws.py, databricks_azure.py, dataproc.py, dataproc_gke.py, emr.py: Disabled pylint warnings for abstract methods. [1] [2] [3] [4] [5]

Other improvements:

• user_tools/src/spark_rapids_pytools/rapids/qualification.py: Added a check to ensure the DataFrame is not empty before accessing it.
• user_tools/src/spark_rapids_tools/cmdli/tools_cli.py: Added a new parameter distributed to the qualification function.