[DPE-5016] Hierarchical configuration handling

[Batalex]

Changes:

• Service account configuration values take precedence over integration hub's.
• Fixed a few typos in readme

The environment

We added spark.app.another-name=something_in_the_way to the integration hub (you can thank Pedro for the inspiration), and spark.app.another-name=something_in_the_air to the test service account using spark-client.

Before

spark-client.service-account-registry get-config --username test --namespace test | grep another-name
# spark.app.another-name=something_in_the_way

spark-client.service-account-registry get-config --username test --namespace test --ignore-integration-hub | grep another-name
# spark.app.another-name=something_in_the_air

spark-client.pyspark --username test --namespace test

Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /__ / .__/\_,_/_/ /_/\_\   version 3.4.2
      /_/

Using Python version 3.10.12 (main, Sep 11 2024 15:47:36)
Spark context Web UI available at http://192.168.1.103:4040
Spark context available as 'sc' (master = k8s://https://192.168.1.103:16443, app id = spark-6537ceeacf4346dd9640acf23c9cc4e5).
SparkSession available as 'spark'.
>>>
>>> spark.sparkContext._conf.get("spark.app.another-name")
'something_in_the_way'

After

python ./spark8t/cli/service_account_registry.py get-config --username test --namespace test | grep another-name
# spark.app.another-name=something_in_the_air

python ./spark8t/cli/service_account_registry.py get-config --username test --namespace test --ignore-integration-hub | grep another-name
# spark.app.another-name=something_in_the_air

python ./spark8t/cli/pyspark.py --username test --namespace test --ignore-integration-hub   

Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /__ / .__/\_,_/_/ /_/\_\   version 3.4.2
      /_/

Using Python version 3.10.15 (main, Oct 16 2024 04:37:23)
Spark context Web UI available at http://192.168.1.103:4040
Spark context available as 'sc' (master = k8s://https://192.168.1.103:16443, app id = spark-ef4e4299efcd473399912745a66bc878).
SparkSession available as 'spark'.
>>> spark.sparkContext._conf.get("spark.app.another-name")
'something_in_the_air'

[welpaolo]

LGTM

[deusebio]

LGTM!