-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MSRV policy for libc crate #72
Comments
@rfcbot merge |
Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members: Concerns:
Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See this document for info about what commands tagged team members can give me. |
This is such an unreasonably short support window for a foundational crate like libc. |
Yeah it's a bit tough, since this impacts a lot of crates that have libc in their dep tree. I think being conservative here is worthwhile, unfortunately. One danger here is if a lot of crates end up pinning libc versions (which they shouldn't1, but in practice is the tool many people reach for), those crates become incompatible to use with eachother. Footnotes
|
@petrochenkov Assume, for the moment, that we aren't looking to support the use case of "run brand new crates from crates.io on a multi-year-old snapshot of rustc/cargo in an enterprise distribution". Are there any other use cases for which "couple of years" is actually a specific end-user need? Not something where crate-author asks crate-author asks crate-author based on hypotheticals, but actual end users who have an old rustc, and a need for new crates, and no better options? |
My instinctual response to this is also, frankly, that
I basically don't have a great concrete answer to this. And I will say, I've personally been looking for one for a while. Any time someone tells me about wanting crates to compile with older versions of Rust, I always try to ask: why? From memory, I've basically gotten these answers:
I think (1) is basically some combination of an awareness/workflow/perception problem. I think there's probably something to (2), but I just don't know enough concrete details about any specific use case to really know for sure. And in particular, I suspect (2) is kind of also a perception problem. Both (1) and (2) seem somewhat rooted in the idea that Rust and its ecosystem move so much faster than the things it tends to supplant that it tends to be really noticeable. Now I do think perception problems are real problems, but I'm not convinced they reflect concrete use cases that argue strongly for a conservative MSRV policy. (By "conservative," I'm thinking that there is roughly support for
OK, so I think this is really where my real concern with this change is. Since
Maybe I'm overplaying it, but that's my sense. So I'll register a concern for that: @rfcbot concern opportunity for ecosystem to give feedback Otherwise, for me personally, I don't think I feel strongly for or against this change. I've long wanted good reasons for a conservative MSRV policy, and I've had a hard time finding them. Maybe this issue will force out some good ones that I've not heard of before. But I can definitely empathize with the annoyance of having to wait so long to use new Rust stuff, so I get where this is coming from. |
I was attempting to narrow the question to libc, not because I think it likely that other official crates can or should realistically be more conservative than libc, but rather, I wanted to separate the questions so that if someone wanted to argue "we should support X" they didn't have to simultaneously argue whether that meant X for libc or for everything. If we do agree on N-2 for libc, then our realistic choices of policies for other official crates are N-2, N-1, or N. I'd tend to suggest N-2 for "foundational" crates like libc or libz-sys. |
I more or less meant this scenario. Crate versions still can be updated more often because it doesn't require similar coordination. |
Not necessarily, but only possibly:
It just means that libc could bump the MSRV if needed. As you said...
|
@petrochenkov Can you say more please? Here are the things I'm personally interested in:
I think you're technically correct, but I think the color you're adding here might give the wrong impression. There's a burden to supporting older MSRVs, and while @joshtriplett's post does include the language "Only upgrade if there's a feature we actually need," I think the word "need" there is doing a lot of work. For example, "need" might mean, "we want to use a new std API and don't feel like writing version sniffing code in For example, if "need" is interpreted broadly, then I think it's very likely that |
Someone else I'd like to hear from is @dtolnay. As I understand it, you maintain a Rust 1.13 MSRV for serde. This is despite requiring at least some non-trivial effort to do so. I don't think I've seen you opine on why exactly you do this. Could you share your perspective here? |
(Also, I have not actually broadcast this issue to a wider audience yet. Are people okay if I do that? Or do folks think we should wait on that for a bit or go about it a different way? As a former mod, I always feel a little weird shining spotlights on GitHub issue threads, since it can go sideways pretty easily.) |
Because it's not that much effort in practice.
It's acceptable, and the issue is not a showstopper, but newer crate versions are not purely cosmetic and do fix relevant bugs sometimes (or add relevant Linux constants or whatever, in case of libc). |
Eh. The It's doubly annoying because finding real tangible use cases for conservative MSRV has proven difficult. So it just feels like a lot of hand wringing for... not much in exchange.
Certainly not! It's extra maintenance for everyone who tries to maintain a conservative MSRV. If |
I was not attempting to add color, or even opinions, just surface the wording concerns which you have now expanded upon. For SNAFU, I maintain a Rust 1.34 MSRV, although I don't use a build.rs for newer versions. Instead, I use feature flags for each version that I need and use default features to enable a "new enough" version of Rust. This places the burden on people who want to use SNAFU with an "old enough" version of Rust, while still allowing a wide range of Rust versions to use SNAFU. This works for me because I can write the majority of my code targeting old Rust versions. This does mean I'm locked to edition 2018, among other things, but that isn't amazingly painful for me in practice. I have received feedback from Linux distro users that are using old versions of Rust so I know that there are real people out there affected by this. Is there a list of what "new Rust" features the libc crate would like to make use of? |
I totally understand that folks in the Rust ecosystem generally don't want to support traditional distro toolchain timelines, especially in enterprise where that can stretch 10 years or more. However, I do want distro Rust toolchains to be a viable option, and for RHEL we've tried to make that happen by keeping a more aggressive "rolling stream". That doesn't mean we ship every 6-week Rust release, but we do update the Rust toolchain in each RHEL minor release, every 6 months. (I also maintain Fedora's Rust, and we do ship every update there on all active releases.) Now, with our own internal development pipeline, the version we put out in a RHEL minor release will already be 3-4 months old when it comes out. That's our latest version for the next 6 months, so it will be 9-10 months old by the time the next one comes around. For example, in May 2022, RHEL 8.6 and 9.0 shipped Rust 1.58, and those will be in use until 8.7 and 9.1 around November. People also don't upgrade RHEL releases right away. Conservatively, padding that out to a year for an MSRV update policy is probably good enough, and if they still haven't updated they'll have some impetus. And FWIW, we did choose a one-year MSRV policy for I know many people won't care about RHEL, but we're trying to find a good compromise there.
And in some cases, this sort of case is also "outsourced" to the distro update process. |
As someone who delivers software written in Rust to RHEL environments, I would really like to hear from users who are developing on RHEL using the distro |
I don't have hard customer data to offer here, but I think there's also a chicken-and-egg problem, that Rust won't even be a viable option for those customers at all if the ecosystem won't support this kind of use. |
I mean for our usage, the rest of RHEL packages are an obstacle, so we never even considered using a Rust toolchain from the distro. So I wonder if this is a more severe chicken-egg scenario. |
Fair enough. :) |
I'm one of the maintainers of pyca/cryptography, a popular Python package (top 20 downloads from PyPI most days) with a risk component. I can provide some limited stats on versions of Rust we see. Our current MSRV is 1.41, and our next release intends to increase our MSRV to 1.48. Looking at downloads of our source artifacts (i.e., users who need to compile themselves, excluding pre-compiled artifacts) we have rustc version data for about 1.6% of those downloads (because tracking this information requires a relatively recent pip). Here is the % of such downloads where the rustc version is >= to each target, from the past 30 days:
Hopefully this presentation of the data makes sense. |
If/once cargo can use the |
Two scenarios for "My company pins its Rust version in CI and it takes a lot of resources to upgrade." I've seen personally:
The problem with "just using older versions of crates" is that sometimes you want to upgrade something for unrelated reason. Like, you use "latest-async-framework-my-company-fork-5.0.0.beta.8", and you want to switch to "latest-async-framework-5.0.0" b/c that has much fewer bugs, and, while doing that work, you discover you need to upgrade rustc as well. For both of these scenarios as I've encountered them, |
A minor thing is build caching. When using a shared Rust compilation cache, it's nice to be able to stay on a specific toolchain for longer, because then you get fewer rebuilds. This is maybe a less important reason for libc, though, as upgrading libc means recompiling much of the world anyway. (And as a sidenote: it's naive to say that this policy is just for libc. libc's msrv policy basically is the baseline msrv policy for the entire ecosystem... at least for msrv at maximal dependency versions.) It's definitely worth polling major ecosystem crates for what their msrv policy is and for what users they hold that policy. E.g. tokio has committed to a 6 month msrv through 2024 at a minimum. Plus it's worth restating: a rust-version aware version resolution will make msrv bumping much smoother. msrv musingsAt least from my viewpoint, the realistic msrvs are
But also, it's imho silly to try to enforce msrv without an msrv supporting dependency resolution. Right now, the best we have is |
Yeah I think @joshtriplett and I discussed this briefly above. I think that's the idea behind considering the MSRV policy for |
@alex Thank you, that's useful information. Would it be possible to get, rather than the last 30 days, three chunks of this information? Specifically: 2022-04-07 to 2022-05-19 (while 1.60 was newest), 2022-05-20 to 2022-06-30 (while 1.61 was newest), and 2022-07-01 to today (while 1.62 was newest)? It would help to see how much correlation that has. (Which in turn will depend on how many of your users get rust from rustup vs a distribution package; I would guess many do, but this will at least give us a conservative bound to consider.) Also, what does
mean here? How old is that version of pip? (We'd like to take into account the possible bias towards recent versions there.) |
One downside to an as-need basis is if you upgrade too infrequently, your users might build up an expectation that diverges from what is documented. clap has always had an N-2 policy but went so long without updating the MSRV that we got backlash when updating our MSRV to something less than a year old. |
One effect here is that we might weaken ecosystem-wide "live at HEAD" property. Today, the world mostly looks like this:
No idea what's the size of the effect here. |
This PR rust-lang/cargo#12043 will help us bumb msrv for libc crate with less efforts next time |
That PR doesn't actually do anything besides add a no-op |
I seem to recall that the root problem was specifically with the newly-introduced distinction between In my ideal formulation, standard C types are all re-exported from libcore, then all the OS- and target-specific types are relocated from libc to their own highly-specific crates (which, unlike libcore, should be allowed to make breaking changes; see the ongoing dilemma with musl), and then the libc crate just becomes a facade over those. |
This is a bit of a tricky topic, as the Rust ecosystem is waiting on better Cargo support to make usability better, and there is a lot of differing opinions and theories on how best to handle an MSRV. Not a lot of concrete data out there. The Rust compiler itself explicitly only supports the latest stable version, so any MSRV policy is essentially how long you're willing to wait for dependents of your library to upgrade. This can always be changed, but I wanted to have something in place to make sure that it's covered. See: - rust-lang/libs-team#72 - time-rs/time#535
This is a bit of a tricky topic, as the Rust ecosystem is waiting on better Cargo support to make usability better, and there are a lot of differing opinions and theories on how best to handle an MSRV. Not a lot of concrete data out there. The Rust compiler itself explicitly only supports the latest stable version, so any MSRV policy is essentially how long you're willing to wait for dependents of your library to upgrade before you adopt new features. Artificially holding this back might also mean missing out on security fixes, and I'm a big fan of staying current when possible. This can always be changed, but I wanted to have something in place to make sure that it's covered. See: - rust-lang/libs-team#72 - time-rs/time#535
This is a bit of a tricky topic, as the Rust ecosystem is waiting on better Cargo support to make usability better, and there are a lot of differing opinions and theories on how best to handle an MSRV. Not a lot of concrete data out there. The Rust compiler itself explicitly only supports the latest stable version, so any MSRV policy is essentially how long you're willing to wait for dependents of your library to upgrade before you adopt new features. Artificially holding this back might also mean missing out on security fixes, and I'm a big fan of staying current when possible. This can always be changed, but I wanted to have something in place to make sure that it's covered. See: - rust-lang/libs-team#72 - time-rs/time#535
This is a bit of a tricky topic, as the Rust ecosystem is waiting on better Cargo support to make usability better, and there are a lot of differing opinions and theories on how best to handle an MSRV. Not a lot of concrete data out there. The Rust compiler itself explicitly only supports the latest stable version, so any MSRV policy is essentially how long you're willing to wait for dependents of your library to upgrade before you adopt new features. Artificially holding this back might also mean missing out on security fixes, and I'm a big fan of staying current when possible. This can always be changed, but I wanted to have something in place to make sure that it's covered. See: - rust-lang/libs-team#72 - time-rs/time#535
fix: Change the defaults to always check-in `Cargo.lock` ### What does this PR try to resolve? Having libraries leave `Cargo.lock` "on the float" has been serving the ecosystem well, including - Encouraging people to validate that latest dependencies work - Encouraging ecosystem-wide health by acting as a "distributed crater" These benefits are limited though. The policy is inconsistent between workspaces with or without `[[bin]]`s, reducing the affect of testing the latest. This is also subject to when CI last ran; for passively maintained projects, there is little coverage of new dependencies. There are also costs associated with this policy - `git bisect` is using an unpredictable set of dependencies, affecting the ability to identify root cause - This is another potential cause for Red CI / broken local development if version is yanked or a bug is introduced - Impacting the perceived level of quality for a project - Confusing to new contributors who might not recognize why CI failed and assume its their fault - Requiring context switching from maintainers to get fixes in In particular, since this policy was decided, there has been an increased interest in supporting an MSRV (as recently as v1.56.0, cargo gained support for specifying a package's MSRV). This has led to long discussions on *what* MSRV a package should use (e.g. rust-lang/libs-team#72,. time-rs/time#535). Worst, there has been a growing trend for people to set an non-semver upper bound on dependencies, making it so packages can't work well with other packages (see #12323). Tooling support would help with this (#9930) but the sooner we address this, the less entrenched bad practices will be. On the positive side, since the policy was decided - In general, CI became easier to setup and maintain with Github Actions compared to TravisCI - Dependabot went GA on Github in 2021 (https://github.blog/changelog/2021-03-31-dependabot-version-updates-are-now-generally-available) - I believe Dependabot will post security update PRs even when Dependabot is not more generally enabled So to get some of the benefit from not checking in `Cargo.lock`, we can recommend either automatically applying updates or having CI check the latest dependencies in a way to get this out of the critical path of PRs and releases. Since there is no one right answer on how to solve all of these problems, we're documenting these trade offs so people can make the choice that is most appropriate for them. However, we are changing the default to a consistent "always check it in" as the answer for those who don't want to think about it. Prior art - [Yarn (Javascript)](https://yarnpkg.com/getting-started/qa#should-lockfiles-be-committed-to-the-repository) - [Poetry (Python)](https://python-poetry.org/docs/basic-usage/#committing-your-poetrylock-file-to-version-control) - [Bundler (Ruby)](https://bundler.io/guides/faq.html#using-gemfiles-inside-gems) Fixes #8728 ### How should we test and review this PR? Please review per-commit. I tried to minimize changes I made to the structure of the CI document In #8728, I brought up having a CI reference page. I keep going back and forth on whether this is guide-level material or reference-level material. Obviously, right now I'm leaning towards it being guide-level. ### Additional information This changes cargo from telling people what to do to giving them a starting point, or default, and giving them the information to make their own choice, if needed. So the question for defaults is who are we targeting? For a default path in documentation, it would be for new to intermediate users. As for `cargo new`, we've been prioritizing new users over those that run it frequently (boiler plate comment, bin is default, etc). See #8728 for the FCP on this policy change
Some quick updates on improving the MSRV experience since my last comment at #72 (comment)
|
Since the last comment the libc crate has increased MSRV to 1.71 in rust-lang/libc#3525. This is a bit awkard for us on the https://github.com/fish-shell/fish-shell side because we just decided on 1.70 to facilitate the next Debian Stable building fish with their package, which appears to be 1.70 (in fact we increased MSRV from 1.67). There are a bunch of distributions with 1.70, see repology for details. I'm not sure if I should file an issue on libc to ask them to reduce MSRV to 1.70 - I can't find any discussion on the change, so I don't know what led to 1.71 specifically (the PR also doesn't show any specifics for 1.70 vs 1.71, I think?), but I'm also not very "plugged in" to the rust ecosystem or the libc crate. |
As I just commented in that PR, it's surprising to me that the MSRV for such a foundational crate is bumped in a PR that doesn't reference any discussion on what the new MSRV should be. (Personally I also think this issue should have had a pointer to such a discussion prior to the PR being merged.) |
So basically this RFC went through (effectively), even though according to #72 (comment) there is no consensus on it? I completely understand the want to bump the MSRV, as a maintainer myself I've the MSRV issue a couple of times and it's a pain in the behind. But I don't think this is the way to do it to be honest. We have entire discussion only to go around it do it any way... Not going to lie, it a little disappointing to see. Especially for a crate that is so fundamental. |
rust-lang/libc#3525 only applies to the v1.0 branch (main) which is not released yet. The v0.2 branch still has the old MSRV. |
This comment was marked as duplicate.
This comment was marked as duplicate.
Ah, cool, I must have misinterpreted what branch I was on (we don't really do long-lived branches so I sometimes forget to check). That also explains why the version in Cargo.toml threw me off. Anyway, consider me someone who'd like to see a pretty conservative MSRV policy (but then there's a massive culture shock in that I'm used to using C++11 in 2020 because asking users to upgrade compilers is too much) |
In a previous libs team meeting, we discussed MSRV policy for the libc crate, and we debated between an N-1 policy and a slightly more conservative policy such as N-3 or N-4. We ended the meeting without having settled on a specific consensus, though it seemed like we were roughly on the same page.
Rather than make ad-hoc changes based on a non-specific MSRV policy, I think we should go ahead and nail down an MSRV policy for present and future changes.
I'd like to propose the following concrete policy:
Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates. Nonetheless, libc MSRV policy seems likely to serve as a defacto lower-bound for supported Rust versions among other libs team crates, and among many crates in the ecosystem.
For further information on the benefits of this: the PR dropping support and version detection for older versions of rustc was able to remove ~1600 lines of code, and that's not counting subsequent work that may be able to drop some of the wrapper macros. This represents a substantial improvement to maintainability, and may make it easier in the future to move to more automation to keep libc updated with the latest platform APIs.
The text was updated successfully, but these errors were encountered: