Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanly split or remove component and pipeline SDKs / extras #877

Open
RobbeSneyders opened this issue Feb 26, 2024 · 0 comments
Open

Cleanly split or remove component and pipeline SDKs / extras #877

RobbeSneyders opened this issue Feb 26, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@RobbeSneyders
Copy link
Member

With the introduction of the lightweight components, our component and pipeline SDKs are no longer cleanly split. We should either split them again, or include the optional installs for both in the default installation.

This is the error I currently get when trying to run a pipeline without any extras installed. The pipeline does not use a lightweight component.

(base) jupyter@download-images-small-eu:~/src$ fondant run local pipeline.py
Traceback (most recent call last):
File "/opt/conda/bin/fondant", line 5, in
from fondant.cli import entrypoint
File "/opt/conda/lib/python3.10/site-packages/fondant/cli.py", line 32, in
from fondant.pipeline import Pipeline
File "/opt/conda/lib/python3.10/site-packages/fondant/pipeline/init.py", line 1, in
from .lightweight_component import ( # noqa
File "/opt/conda/lib/python3.10/site-packages/fondant/pipeline/lightweight_component.py", line 13, in
from fondant.component import BaseComponent, Component
File "/opt/conda/lib/python3.10/site-packages/fondant/component/init.py", line 12, in
from .component import ( # noqa
File "/opt/conda/lib/python3.10/site-packages/fondant/component/component.py", line 7, in
import dask
ModuleNotFoundError: No module named 'dask'

@RobbeSneyders RobbeSneyders added the bug Something isn't working label Feb 29, 2024
RobbeSneyders pushed a commit that referenced this issue Mar 6, 2024
Fixes:
- #891

 Fixes (maybe needs more work):
- #877
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Backlog
Development

No branches or pull requests

1 participant