do the packages in the rye workspace need to use a specific build system

[mmerickel]

I'm looking at converting a monorepo using pip-tools to a rye workspace and one issue I ran into immediately is that each of my packages is using setuptools right now - with some needing it because to build themselves they have custom build commands.

# myrepo/pyproject.toml

[tool.rye]
virtual = true

[tool.rye.workspace]
members = ["src/*"]

❯ rye sync
Initializing new virtualenv in /Users/luser/work/myrepo/.venv
Python version: [email protected]
Generating production lockfile: /Users/luser/work/myrepo/requirements.lock
error: could not write production lockfile for workspace

Caused by:
    project from '/Users/luser/work/myrepo/src/example-app' has no name

Of course src/example-app fully supports PEP517 builds with setuptools. It has a pyproject.toml containing a [build-system] but no it isn't using PEP-621 metadata right now (no [project] section).

I'm asking how far off base I am as I'm just getting started.

Thanks!

[mitsuhiko]

Rye requires a name and version for a workspace, even if you are using it as a virtual container only.

[mmerickel]

@mitsuhiko The error message here is not about my workspace but rather one of the projects in the src folder. I tried adding a [project] setting to my virtual workspace but the error about src/example-app persists.

[mmerickel]

If I add [project] with name/version keys to the projects in the src folder then the error disappears but that's the point of my question - is it expected that rye requires you to use PEP-621 metadata? These projects build fine and are PEP-517 compliant but have the data defined in setup.cfg or in setup.py instead of moving it to [project].

[mmerickel]

How deep does the rabbit hole go with requiring PEP-621? Do I need to move all package metadata including dependencies into [project] config such that Rye is not intended to work with packages in the workspace that use other build systems?

[bluss]

The unstated assumption in your question is that workspaces should work with any python packages. But I think it could be the case that rye (again, without documenting this explicitly) so far has designed workspaces to work only for rye-managed packages or projects, and other configurations have not been tested(?). It's likely both design and implementation work is needed here.

[bluss]

Let's say "rye managed" or so because rye doesn't have its own build system, it uses hatchling to build wheels by default. It's probably not a lot missing for this to work, some metadata bits? Should this be a bug report / feature request?

Rye does support setuptools, so things I'm wondering about here - maybe we expect this to work if you use rye init (to import) on all your setuptools packages?

[mmerickel]

I've dived into this further and here's my findings:

• If I add a pyproject.toml to each package in my workspace with a name and version it is not enough. It does not pick up the rest of the metadata from setup.py and setup.cfg. For example, I must define the dependencies in [project] dependencies =.
• If I fully convert to PEP621 [project] config, which is supported by setuptools, then things actually do work.
• Rye does not support finding any [build-system] requires = requirements in the workspace.

1. This last point probably I can open as a bug.

[build-system]
requires = [
    "setuptools",
    "wheel",
    "my-custom-package-in-the-workspace",  # <--- rye chokes on this one
]
build-backend = "setuptools.build_meta"

3. This first point about having to fully convert to PEP621 I'm less clear about, hence this discussion topic. How much of it is a bug versus intentional in the design?

Thanks for the help in understanding!

[mmerickel]

To extend the issue one step further, I can't get to this point yet because of point 1 above (build-system requirements), but I do have some packages that want to define custom cmdclass definitions in setup.py that use the package from the build-system requirements (as that's the point of them) and I'm not having a lot of confidence that Rye will use it correctly, given that it didn't pull any metadata from setup.cfg/setup.py until I moved it into pyproject.toml - which I can't do for the cmdclass that has to be in setup.py.

[mmerickel]

I opened #813 related to the build-system issue.

[mmerickel]

After fully converting to PEP621 metadata and removing setup.cfg my projects work with rye aside from the issue I opened in #813. This means at the very least defining [project] with name, version, and dependencies keys. Rye does not seem to be capable of sourcing that information from a PEP517 build backend.