Support poetry, flit, pipenv, or ...? #3270
Comments
|
The main issue I have seen in my experience using pipenv is that it doesn't mix very cleanly with a [[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[requires]
python_version = "3.6.5"
[packages]
myapp = {path = ".", editable = true}
[dev-packages]
myapp = {path = ".", extras = ["testing"], editable = true}The workflow here is to basically edit The main reasons why we use |
|
What would be a better way to define entry points? Is the PyPA working on this? This does not bode well: |
|
Entry points is an issue that pypa is starting to talk about on distutils-sig as the next likely thing to get split out from setuptools... but very slowly. The more practical option I think will be to rip |
|
I've updated and added to the questions above. Regarding making a distributable Python package, do you mean you want to remove the these two pages?
If so, I'd like to at least mention why someone would want to make a dist and provide references to resources for how to do so, instead of simply deleting those pages from the two wiki tutorials. Or do you mean something else? |
|
Maybe https://github.com/takluyver/entrypoints could be used instead of setuptools/pkg_resources for entry points? |
|
@piotr-dobrogost Unfortunately the issue here is how to declare entry points, not how to consume them. That project only looks to consume existing |
|
Ok, how about switching from setuptools to flit which does have support for entry points (https://flit.readthedocs.io/en/latest/pyproject_toml.html#entry-points-sections)? |
|
I like flit and would be open to promoting it more. Unfortunately for the purposes of integrating with pipenv it's basically the same as setup.py - it's making your project a redistributable package and thus provides a place to define dependencies which conflicts with the The alternative that integrates with pipenv more cleanly is the flask approach of punting on this and not advocating patterns that use setup.py. You're basically responsible yourself for putting your project on the PYTHONPATH however you want - usually just running it out of the CWD. The real benefits to having an installable package are:
pipenv's approach (in my experience using it) has been to directly compete with point 2 and doesn't provide a solution for points 1 and 3. It seems mostly useful for pulling together third party projects into a lockfile and less about managing the link between the project-under-development and the virtualenv. Please correct me if anyone disagrees, but this is what I've seen, and yes I have used it for a real project in this way. |
|
https://github.com/sdispater/poetry is by far the best solution I have seen to managing a project yet. I think it blows pipenv's approach out of the water from what I have seen so far and solves all the issues I've mentioned above. |
|
Compared to flit for example, it seems like poetry is made by one person who does not follow Python packaging discussions, so I’m not sure about the degree of integration (i.e. implementation of standards) that this tool provides. For example, the author doesn’t see the point of develop installs, and doesn’t accept the uses cases brought up in the ticket. |
|
@merwok there are a variety of projects that don't use develop installs at all, for example warehouse doesn't instead it expects warehouse to be in it's PYTHONPATH automatically. Others like Flask and I think Django too, punt on having installable packages, unless you manually do a bunch of work (its not documented, and not the default recommendation) and thus they don't have a need for editable installs. While we have promoted I personally really like the idea of a single file that contains package pins/Information rather than a "Pipfile" vs "setup.py" (or one of it's replacements) where the data is duplicated, or you end up having on rely on the other. Using |
|
Right... poetry solves the editable install issue by basically requiring you to use Finally, the sdist/wheel it outputs if you do choose to On top of that, I guess that as @bertjwregeer said, poetry is adding some editable install support in a future version anyway. |
stevepiercy
changed the title
|
After reading this thread I decided to try poetry, this is the project I created: https://github.com/josuemontano/API-platform. The development configuration works as expected, but in production running |
|
@josuemontano |
|
Got it. Thanks @bertjwregeer! https://github.com/josuemontano/API-platform/blob/master/scripts/predeploy#L3-L4. Hopefully I can make a cookiecutter out of this soon. |
|
FWIW to get rid of pyramid's requirement for an entry point it's possible to replace |
|
Was any progress ever made of having config optionally come from something other than an ini file? |
I guess the ironies are in the fire, given that entry points were designed for applications, rather than libraries. |
@cjw296 yes, see pyramid's usage of plaster in newer releases
The irony is not lost on me... but it is lost on the pipenv developers that have closed issues opened about adding entry point support to it with significant hostility. The main thing to point out is that your link references "installed distributions" and "packages" and pipenv explicitly wants nothing to do with these things. It only wants to manage a virtualenv of stuff, and is not interested in helping you develop a package. That's between you and your build system (setuptools/flit/etc). Since entry points are currently defined as package metadata, pipenv has no intention to provide any specific support for them above what I described in #3270 (comment) (installing your package in editable mode with a setup.py or pyproject.toml next to it). |
|
Given the hostile maintainers and lack of functionality, I'd advocate just giving up on pipenv and putting weight behind poetry. I've found it to be a much more pleasant tool to work with, it features a real dependency solver, and the maintainer is polite, pragmatic and practical. |
|
Is anyone using poetry on a decently sized project? I've tried to use it twice in the past year on a 300+ deps project and it exploded. I didn't really have time to investigate and decided to wait a bit until it matures. Has it matured yet? |
FTR, this was done in Poetry 0.10.0 on May 28, 2018.
Hard to say without specifics for your use case, but it has been in active development with recent bug fix and 1.0.0a releases. It looks like they're 2/3 of the way there to 1.0.0. |
Of course, that's why I am asking is anyone actually using it in production on a sizeable project and it works great for them. I'm looking for a trigger to try it again :) |
|
@zupo I am happily using it on a project right now for a customer. The main issue I've found is when people upload broken packages to PyPi with bad metadata, which I don't fault poetry for, because pip just blindly downloads and executes code, whereas poetry attempts to avoid that to be able to do proper dependency resolution. For building an application it is working really well for me right now, and I really enjoy using it. I haven't used it for building a library. |
|
Great to hear, thanks! |
|
@zupo - in passing, 300 deps seems quite extreme for one app, how come so many deps? |
|
@cjw296: it's a hosting automation project. Pyramid is used to manage 500+ servers deployed on 20+ hosting providers, dns records on 15 +providers, email services, DNS propagation services, etc. Lots of API clients that we depend on. |
|
Here’s my return on experience FWIW: flit Pros:
Cons:
poetry Pros:
Cons:
pipenv Tries to integrate existing tools together (pip, virtualenv, pip-tools, safety) and the result sometimes doesn’t feel well integrated. Maintainer hostility was mentioned previously in this ticket (discussions shut down abruptly, cowboy development, etc). Last I used it, adding one dependency caused updates to existing ones, which is really not wanted and inconvenient. Have switched to poetry for these projects. See also notes by Nick Coghlan in https://discuss.python.org/t/developing-a-single-tool-for-building-developing-projects/2584/17 Conclusion The classic distutils-based packaging tools have improved a lot; we can build wheel+sdist with setuptools, develop install with pip, manage pinned requirements with pip-tools, check metadata and upload securely with twine, detect reST issues or use Markdown in metadata long description. That said, there is a lot of old or contradictory doc out there, and it’s five tools with different configs and UX. flit is easy to use and avoids some classes of problems entrirely. The Pyramid documentation could change setuptools instructions to flit, if it provides all features required by the examples [edit: and if it’s acceptable to have sub-par dependency management (either all open without any conflict/compat check, or all manually pinned and updated)]. I like how it’s one tool for packaging tasks (init, build, upload), and lets you use other tools for other things (pytest, tox, bump2version, etc). That said, I could see people disliking the requirement for VCS setup before you can use flit. Poetry in my opinion has a couple rough edges but is overall rather great, and could become the community-standard way of managing a web app (except maybe for Django, where typically your app is one or many packages imported from current dir, without install step). For me, flit is great to package a pure-Python library, and poetry is really nice for an app. A project can also start with flit and move to poetry without a ton of effort (see also pypa/flit#115 (comment)). I’m personally happy using both, but if Pyramid docs want to show only one tool, I think poetry wins. To package Pyramid itself, I think flit would be enough, but poetry may work too if you wanted only one tool overall. |
|
At my new place of employment almost all projects are run directly in a container or lambda functions and there is no Recently a co-worker started with the example from
There's no real "quick start"... except that we have 2, maybe 3 of them, which caused a bunch of confusion:
but the quick tutorial doesn't cover the basics of "how do I deal with various HTTP verbs" nor does it deal with "how do I deal with JSON posted data or sending JSON back out". I couldn't point my co-worker at one place in the documentation where he could easily see an example and copy and paste (and yes, people learn that way). Instead the project is now a flask project :-/ I went off on a tangent (unrelated to what this ticket is about), and I'll likely create more tickets at a later point in time (although they are likely to get lost anyway), but the point is that focusing on how to create a python package to ship Pyramid is something many people don't care about. Look at warehouse or hypothesis (thanks for the examples @mmerickel) and notice that they too don't care about packaging their Pyramid applications, however a lot of our narrative documentation currently depends on someone learning all of the packaging stuff just to get going with Pyramid. I hate to suggest maintaining more documentation, but this is an instance where maybe we should consider simplifying and removing any discussion about packaging your Pyramid application, or at least leaving that as an optional in case you want to take advantage of certain Pyramid features (like easy overrides and all that fun stuff). |
I love me a good hijacking! I would suggest you direct your cow-orkers to have a look at pyramid_openapi3, thanks to Nejc, Domen, and other Niteo folks. There are two demos, and a third full-blown RealWorld.io app. After looking at that, are there any gaps that need to be covered? |
|
Yes, pyramid_openapi3 is a project that allows you to implement what used to be known as i.e. there are no examples that I can easily find that have: Along with an explanation of how that works. The example |
|
The example in https://trypyramid.com links to the following page, where further down are the predicate arguments: I get your point, though. The content in the narrative docs is presented through technical terms of "predicate" and so on, instead of a "show me a JSON REST API example". It would definitely make a good cookbook recipe or an additional step in the Quick Tutorial. |
|
There's two practical things I could pull out of this discussion:
I'd like to find a middle-ground where Pyramid stops making decisions here for the user so that they can adapt the cookiecutter to their own needs. The source of these issues is not specifically dependency management. I think it's actually how the ini file connects to the code through entry points. So I see two possible changes, in the following order:
First, modify the Second, converting a We'd keep all the virtualenv stuff we do now, but we'd stop defining the project as a publishable package and instead focus just on installing dependencies with pip and |
|
In our docs and tutorials, what would we suggest to users to manage their If we want to avoid endorsing a specific tool in the docs, then we probably need to resort to a statement like, "manage your |
|
|
I have found pip-tools mostly convenient to generate full list of pinned requirements from a file containing loose requirements (i.e. only direct app dependencies, with no version constraints or range constraints, edited by devs). Generating a (or many) requirement file(s) ensures broad compatibility with update checkers (like dependabot, now part of github), Heroku deployments, tools like |
|
@merwok would you please write one or two sentences for :term:`my-requirements-tool`and call it soup. |
|
The goal of I do not agree that Pyramid should document how to use it with pip-tools, pipenv, poetry, etc in the core docs. |
|
I agree we should not document how to use those tools in any way. That would be duplicating the tools' documentation. I think it is fine to provide links to a few of those tools, and a one or two sentence description for when to use each tool ("distributing", "installing dependencies", etc.) in the Glossary. There are precedents for distribute, distutils, pip, and setuptools. While we are talking about packaging stuff and whether to document it, do we want to delete the distributing pages from the tutorials and anything from Creating a Pyramid Project? |
|
A link to https://packaging.python.org/tutorials/packaging-projects/ should be enough. One could argue that the instructions there are not the most modern (e.g. could be using static metadata in setup.cfg, or use flit instead of setuptools), but that’s actually the point: let packaging people update that packaging tutorial, then people coming from Pyramid docs will benefit from updates. |
|
In case someone wanders here from the Internets, I've created a Poetry-based scaffold of the Pyramid's "SQLAlchemy + URL dispatch" wiki tutorial: https://github.com/zupo/tutorial |
|
|
Although I understand the decision I do not like it:
|
|
The author of You can use locally installed Python as mentioned in the documentation:
|
This is a placeholder for supporting whatever becomes the successor to
pipand managing virtual environments in Pyramid, includingpoetry,flit, andpipenv. Please add to it.Questions
SUCCESSOR?SUCCESSORfit into our current installation documentation?SUCCESSORthat we would use beside installation of packagesand distribution of an app?$VENVin most of our commands?setup.pyonly, specifically entry points ofconsole_scriptsandpaster.app_factory?pcreate?SUCCESSOR, do we retain or remove Windows commands for everything except installation and setup of an environment, with a disclaimer that it is up to the user to know how to translate Unix commands to their platform of choice? Refs: Tutorial instructions for windows may not work correctly in Powershell #3260 (comment)Things to update
pipis used.setup.py.References
The text was updated successfully, but these errors were encountered: