Add transactional semantics to `rustfix` #14747

saites · 2024-10-30T04:57:58Z

What does this PR try to resolve?

This PR adds transactional semantics to rustfix::Data, enabling rustfix::CodeFix to apply Suggestions as atomic units and rollback partially-applied changes when they conflict with existing ones. The basic approach and goals are discussed in a comment on issue 14699. In that comment, I proposed a solution which extended the existing State enumeration, but described an alternative that simplifies the overall tracking of incoming changes; this PR implements the latter.

The PR is broken up into 5 commits:

The first commit is more-or-less the minimal changes necessary for this approach. It adds the commit and restore methods, and it changes how replacements are tracked to make this easier.
The second commit adds a little extra information to Error::AlreadyReplaced in order to remove the special-case handling for duplicate replacements.
The third commit passes through a lifetime parameter so that the data can be borrowed, since it's not edited directly.
The fourth commit generalizes the &[u8] using AsRef to make the interface easier to use (since this code is primarily interested in &str).
The fifth commit adds a to_string method and moves the logic from lib.rs for the same reason as above.

How should we test and review this PR?

I've added an additional test and updated the existing ones, but it's a good idea to experiment with cargo clippy --fix on repos that match the cases described in these open issues.

Additional information

This removes special handling for rust-lang/rust#51211.

Fixes #14699
Fixes #13027
Fixes rust-lang/rust-clippy/issues/13549

rustbot · 2024-10-30T04:58:02Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @weihanglo (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

@rustbot author: the review is finished, PR author should check the comments and take action accordingly
@rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

weihanglo

This is awesome! Thank you for this patch!

weihanglo · 2024-10-31T03:05:39Z

crates/rustfix/src/lib.rs

+pub fn ignore_identical<T: Default>(err: Error) -> Result<T, Error> {
+    match err {
+        Error::AlreadyReplaced {
+            is_identical: true, ..
+        } => Ok(Default::default()),
+        _ => Err(err),
+    }
+}


Probably unnecessary pub and generics.

Suggested change

pub fn ignore_identical<T: Default>(err: Error) -> Result<T, Error> {

match err {

Error::AlreadyReplaced {

is_identical: true, ..

} => Ok(Default::default()),

_ => Err(err),

}

}

fn ignore_identical(err: Error) -> Result<(), Error> {

match err {

Error::AlreadyReplaced {

is_identical: true, ..

} => Ok(()),

_ => Err(err),

}

}

Feels like we can just inline this in apply_suggestions, but that doesn't really matter :)

weihanglo · 2024-10-31T03:27:22Z

crates/rustfix/src/lib.rs

@@ -216,34 +216,34 @@ pub fn collect_suggestions<S: ::std::hash::BuildHasher>(
 /// 2. Calls [`CodeFix::apply`] to apply suggestions to the source code.
 /// 3. Calls [`CodeFix::finish`] to get the "fixed" code.
 #[derive(Clone)]
-pub struct CodeFix {
-    data: replace::Data,
+pub struct CodeFix<'orig> {


nit: I would probably just call it 'a. Less visual noise and there is no ambiguity.

weihanglo · 2024-10-31T03:32:06Z

crates/rustfix/src/replace.rs

 #[derive(Debug, Clone, Default)]
 pub struct Data {
    /// Original data.
    original: Vec<u8>,
    /// [`Span`]s covering the full range of the original data.
+    /// Important: it's expected that the underlying implementation maintains this in order,


Although the code looks pretty valid, should we add some debug_assertion for this in to_vec()?

weihanglo · 2024-10-31T03:34:45Z

crates/rustfix/src/replace.rs

+    ///
+    /// See the module-level documentation for more information on why uncommitted changes are included.
+    pub fn to_vec(&self) -> Vec<u8> {
+        let mut prev_end = 0usize;


nit: This should be sufficient.

Suggested change

let mut prev_end = 0usize;

let mut prev_end = 0;

weihanglo · 2024-10-31T04:27:54Z

src/cargo/ops/fix.rs

            match fixed.apply(suggestion) {
                Ok(()) => fixed_file.fixes_applied += 1,
+                Err(rustfix::Error::AlreadyReplaced {


While I don't think the compiler will get this fixed in any near future, we may want to restore this comment in case there is such a future.

(probably ditto in apply_suggestions()?)

// If this replacement matches exactly the part that we would // otherwise split then we ignore this for now. This means that you // can replace the exact same range with the exact same content // multiple times and we'll process and allow it. // // This is currently done to alleviate issues like // rust-lang/rust#51211 although this clause likely wants to be // removed if that's fixed deeper in the compiler.

weihanglo · 2024-10-31T04:34:45Z

crates/rustfix/src/replace.rs

+    fn insert_same_twice() {
+        let mut d = Data::new("foo");
+        d.replace_range(1..1, "b").unwrap();
+        assert_eq!("fboo", str(&d.to_vec()));
+        assert!(matches!(
+            d.replace_range(1..1, "b").unwrap_err(),
+            Error::AlreadyReplaced {
+                is_identical: true,
+                ..
+            },
+        ));
+        assert_eq!("fboo", str(&d.to_vec()));


This test change should move to the second commit maybe.

While this has already covered the case of #13027, I feel we should generate a snapshot test from the exact macro diagnostics, like https://github.com/rust-lang/cargo/blob/de5832fe8fe1a05e278eee82bbc801d544ed4bd1/crates/rustfix/tests/everything/E0178.rs, if we aim to close it.

I can help with that if it is too much a burden.

weihanglo · 2024-10-31T04:43:18Z

crates/rustfix/src/replace.rs

+        // Keep sorted by start position, with pure inserts before replacements.
+        let ins_point = self.parts.partition_point(|span| {
+            span.range.start < range.start
+                || (span.range.start == range.start && span.range.end < range.end)


Could you explain a bit why we need span.range.end < range.end?

with pure inserts before replacements.

I think got it :)

weihanglo · 2024-10-31T04:46:32Z

crates/rustfix/src/lib.rs

Looks like the last three commits are mostly refactoring. To make things clearer and more traceable, could we move them to a separate or a follow-up PR?

weihanglo · 2024-10-31T05:01:30Z

crates/rustfix/src/replace.rs

-                range.end,
-                self.parts,
-            );
-            return Err(Error::MaybeAlreadyReplaced(range));


If this is no longer needed, we should remove it (and maybe add #[mon_exhaustive] to the error enum as a follow-up).

add transactional semantics to rustfix

42bba5f

rustbot assigned weihanglo Oct 30, 2024

rustbot added Command-fix S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 30, 2024

saites added 4 commits October 29, 2024 22:04

replace special-case duplicate handling with error

66728d7

borrow original/replacement data instead of cloning

3362a66

accept both &[u8] and &str in rustfix methods

84d83ca

make to_string a proper method of replace::Data

c738b3e

saites force-pushed the rustfix-transactional branch from c34580f to c738b3e Compare October 30, 2024 05:08

saites changed the title ~~Rustfix transactional~~ Add transactional semantics to rustfix Oct 30, 2024

weihanglo reviewed Oct 31, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add transactional semantics to `rustfix` #14747

Add transactional semantics to `rustfix` #14747

saites commented Oct 30, 2024 •

edited

Loading

rustbot commented Oct 30, 2024

weihanglo left a comment

weihanglo Oct 31, 2024

weihanglo Oct 31, 2024

weihanglo Oct 31, 2024 •

edited

Loading

weihanglo Oct 31, 2024

weihanglo Oct 31, 2024

weihanglo Oct 31, 2024

weihanglo Oct 31, 2024 •

edited

Loading

weihanglo Oct 31, 2024

weihanglo Oct 31, 2024

weihanglo Oct 31, 2024

weihanglo Oct 31, 2024

Add transactional semantics to rustfix #14747

Are you sure you want to change the base?

Add transactional semantics to rustfix #14747

Conversation

saites commented Oct 30, 2024 • edited Loading

What does this PR try to resolve?

How should we test and review this PR?

Additional information

rustbot commented Oct 30, 2024

weihanglo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

weihanglo Oct 31, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

weihanglo Oct 31, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add transactional semantics to `rustfix` #14747

Add transactional semantics to `rustfix` #14747

saites commented Oct 30, 2024 •

edited

Loading

weihanglo Oct 31, 2024 •

edited

Loading

weihanglo Oct 31, 2024 •

edited

Loading