Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't initialize DSS latest/edge, is MLFlow not deployed with --trust? #191

Open
motjuste opened this issue Dec 3, 2024 · 3 comments
Open
Labels
bug Something isn't working

Comments

@motjuste
Copy link
Contributor

motjuste commented Dec 3, 2024

Bug Description

There seems to be some issue with the latest/edge update version 0.1-2237f5211c19... release on 2-Dec-2024 where DSS cannot be initialized. This was seen during the Checkbox validation run for DSS and all validations failed for latest/edge with the same error, for example here.

To Reproduce

  1. Install microk8s 1.28/stable or 1.31/stable, classic snap, and enable hostpath-storage dns rbac.
  2. Install data-science-stack snap from channel latest/edge.
  3. Run dss initialize --kubeconfig="$(sudo microk8s config)"

Environment

  • microk8s, either 1.28/stable or 1.31/stable, classic snaps
  • data-science-stack, latest/edge

Relevant Log Output

Executing initialize command
  Storing provided kubeconfig to /home/ubuntu/snap/data-science-stack/39/.dss/config
  Error:  Failed to initialize dss: Cannot apply required resources. Charm may be missing `--trust`.

Additional Context

No response

@motjuste motjuste added the bug Something isn't working label Dec 3, 2024
Copy link

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6635.

This message was autogenerated

@motjuste
Copy link
Contributor Author

motjuste commented Dec 3, 2024

Checking the dss status I get:

~$ dss status
[ERROR] Failed to retrieve status: deployments.apps "mlflow" is forbidden: User "system:anonymous" cannot get resource "deployments" in API group "apps" in the namespace "dss".
~$ sudo dss status
Traceback (most recent call last):
  File "/snap/data-science-stack/39/bin/dss", line 5, in <module>
    from dss.main import main
  File "/snap/data-science-stack/39/lib/python3.10/site-packages/dss/main.py", line 4, in <module>
    from dss.create_notebook import create_notebook
  File "/snap/data-science-stack/39/lib/python3.10/site-packages/dss/create_notebook.py", line 19, in <module>
    from dss.remove_notebook import remove_notebook
  File "/snap/data-science-stack/39/lib/python3.10/site-packages/dss/remove_notebook.py", line 8, in <module>
    from dss.utils import does_notebook_exist
  File "/snap/data-science-stack/39/lib/python3.10/site-packages/dss/utils.py", line 21, in <module>
    logger = setup_logger("logs/dss.log")
  File "/snap/data-science-stack/39/lib/python3.10/site-packages/dss/logger.py", line 52, in setup_logger
    os.makedirs(os.path.dirname(log_file_path))
  File "/usr/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: 'logs'

@kimwnasptd
Copy link
Collaborator

kimwnasptd commented Dec 5, 2024

Looks like it's an underlying issue with gtsystem/lightkube#78 and we can also see it in

EDIT: For the solution we'll need to bump lightkube to 0.15.6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants