New custom type system

Implemented a new custom type system described in the new docs.  This
system is easier to use and also allows us to do some extra tricks
behind the scene.  For example, I hope to add a flag that will toggle
between implementing `FfiConverter` for the local tag vs a blanket impl.
It's not possible to do that with the current system and adding support
would be awkward.

I wanted to keep backwards compatibility for a bit, but couldn't figure
out a good way to do it.  The main issue is that for the old system, if
a custom type is declared in the UDL then we generate the `FfiConverter`
implementation, while the new system expects the user to call
`custom_type!` to create the impl.  I tried to get things working by
creating a blanket impl of `FfiConverter` for types that implemented
`UniffiCustomTypeConverter`, but ran into issues -- the first blocker I
found was that there's no way to generate `TYPE_ID_META` since we don't
know the name of the custom type.

Removed the nested-module-import fixture.  The custom type code will no
longer test it once we remove the old code, since it's not using
`UniffiCustomTypeConverter`.  I couldn't think of a way to make it work
again.

CHANGELOG.md

@@ -6,6 +6,13 @@
 
 ## [[UnreleasedUniFFIVersion]] (backend crates: [[UnreleasedBackendVersion]]) - (_[[ReleaseDate]]_)
 
+### ⚠️ Breaking Changes ⚠️
+
+- The Rust side of the custom type system has changed and users will need to update their code.
+  The `UniffiCustomTypeConverter` trait is no longer used,  use the `custom_type!` macro instead.
+  We did this to help fix some edge-cases with custom types wrapping types from other crates (eg, Url).
+  See https://mozilla.github.io/uniffi-rs/next/Upgrading.html for help upgrading and https://mozilla.github.io/uniffi-rs/next/udl/custom_types.html for details.
+
 [All changes in [[UnreleasedUniFFIVersion]]](https://github.com/mozilla/uniffi-rs/compare/v0.28.2...HEAD).
 
 ## v0.28.2 (backend crates: v0.28.2) - (_2024-10-08_)

Cargo.toml

@@ -53,7 +53,6 @@ members = [
   "fixtures/regressions/swift-callbacks-omit-labels",
   "fixtures/regressions/swift-dictionary-nesting",
   "fixtures/regressions/unary-result-alias",
-  "fixtures/regressions/nested-module-import",
   "fixtures/regressions/wrong-lower-check",
   "fixtures/trait-methods",
   "fixtures/uitests",

docs/adr/0009-remote-types-interfaces-and-proc-macros.md

@@ -158,16 +158,16 @@ pub enum LogLevel {
     Trace = 5,
 }
 
-/// Define a custom type conversion from `log::Level` to the above type.
+/// Define a type conversion from `log::Level` to the custom [`LogLevel`] enum.
 uniffi::custom_type!(log::Level, LogLevel, {
-  from_custom: |l| match l {
+  lower: |l| match l {
     log::Level::Error => LogLevel::Error,
     log::Level::Warn => LogLevel::Warn,
     log::Level::Info => LogLevel::Info,
     log::Level::Debug => LogLevel::Debug,
     log::Level::Trace => LogLevel::Trace,
   },
-  try_into_custom: |l| Ok(match l ({
+  try_lift: |l| Ok(match l ({
     LogLevel::Error => log::Level::Error,
     LogLevel::Warn => log::Level::Warn,
     LogLevel::Info => log::Level::Info,

docs/manual/src/Upgrading.md

@@ -0,0 +1,46 @@
+# v0.28.x -> v0.29.x
+
+## Custom types
+
+Custom types are now implemented using a macro, rather than implementing the `UniffiCustomTypeConverter` trait.
+We did this to help fix some edge-cases with custom types wrapping types from other crates (eg, Url).
+
+Before:
+
+```rust
+impl UniffiCustomTypeConverter for NewCustomType {
+    type Builtin = BridgeType;
+
+    fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
+        ...
+    }
+
+    fn from_custom(obj: Self) -> Self::Builtin {
+        ...
+    }
+}
+```
+
+After:
+
+```
+uniffi::custom_type!(NewCustomType, BridgeType, {
+    try_lift: |val| { Ok(...) },
+    lower: |obj| { ... },
+})
+```
+
+Changes:
+
+* The term "bridge type" replaces "builtin type".  The reason for this is that this type does not actually need to be a builtin type.
+* `into_custom/from_custom` are now named `try_lift/lower`.  This simplifies/improves the docs,
+  since we can leverage the existing concepts of lifting/lowering rather than introduce a new
+  converter concept.
+* Records, Enums, and Objects are also supported.
+* The `custom_type!` macro is more flexible than the old system.
+  For example, the `try_lift` and `lower` can be omitted in many cases.
+  If `lower` is omitted, then `Into<BridgeType>` will be used instead.
+  If `try_lift` is omitted, then `TryInto<NewCustomType>` will be used instead.
+  The non-symmetry is slightly awkward, but `TryInto` is better to use than `TryFrom` because of the blanket impls in the standard library and we decided that writing `try_lift` would be more natural for users invoking the macro.
+
+See the [Custom Types](./udl/custom_types.md) for details and other features of the new macro.

docs/manual/src/proc_macro/index.md

@@ -381,44 +381,8 @@ impl Foo {
 
 ## The `uniffi::custom_type` and `uniffi::custom_newtype` macros
 
-There are 2 macros available which allow procmacros to support "custom types" as described in the
-[UDL documentation for Custom Types](../udl/custom_types.md)
-
-The `uniffi::custom_type!` macro requires you to specify the name of the custom type, and the name of the
-builtin which implements this type. Use of this macro requires you to manually implement the
-`UniffiCustomTypeConverter` trait for for your type, as shown below.
-```rust
-pub struct Uuid {
-    val: String,
-}
-
-// Use `Uuid` as a custom type, with `String` as the Builtin
-uniffi::custom_type!(Uuid, String);
-
-impl UniffiCustomTypeConverter for Uuid {
-    type Builtin = String;
-
-    fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
-        Ok(Uuid { val })
-    }
-
-    fn from_custom(obj: Self) -> Self::Builtin {
-        obj.val
-    }
-}
-```
-
-There's also a `uniffi::custom_newtype!` macro, designed for custom types which use the
-"new type" idiom. You still need to specify the type name and builtin type, but because UniFFI
-is able to make assumptions about how the type is laid out, `UniffiCustomTypeConverter`
-is implemented automatically.
-
-```rust
-uniffi::custom_newtype!(NewTypeHandle, i64);
-pub struct NewtypeHandle(i64);
-```
-
-and that's it!
+See the [UDL documentation for Custom Types](../udl/custom_types.md).  It works exactly the same for
+proc-macros.
 
 ## The `uniffi::Error` derive
 

docs/manual/src/udl/custom_types.md

@@ -1,75 +1,139 @@
 # Custom types
 
-Custom types allow you to extend the UniFFI type system to support types from your Rust crate or 3rd
-party libraries. This works by converting to and from some other UniFFI type to move data across the
-FFI. You must provide a `UniffiCustomTypeConverter` implementation to convert the types.
+Custom types allow users to extend the UniFFI type system to support types that normally could not be used in an interface.
+
+As described in the [Lifting and Lowering](../internals/lifting_and_lowering.html) section, UniFFI passes types across the FFI by *lifting* and *lowering* them.
+For example, when the foreign code wants to make a Rust call:
+
+* The generated foreign code *lowers* all argument values to an primitive FFI type.
+* The generated foreign code calls the FFI function with the lowered values
+* The generated Rust code *lifts* the arguments to a Rust type
+* The generated Rust code calls the exported function.
+* The generated Rust code *lowers* the return value of the function and returns that back to the foreign side.
+* The generated foreign code *lifts* the return value and returns that to the consumer code.
+
+UniFFI supports many [builtin types](../udl/builtin_types.html) as well as user-defined [structs](..,udl/structs.html), [enumerations](..,udl/enumerations.html), [objects](..,udl/interfaces.html), etc.
+
+Custom types allow library authors to extend the type system even further.
+Instead of lifted from/lowered into a primitive FFI type, custom types are lifted from/lowered into a builtin or user-defined types, called a "bridge type".
+This creates a 2-step lifting/lowering process: the custom type is lifted/lowered, then the result is lifted/lowered using the normal logic for the bridge type.
+The foreign bindings will treat these types as the bridge type.
+
+For example, suppose a library uses the `url::Url` type, which can not be lifted/lowered directly by UniFFI.
+The library can define `url::Url` as a custom type, with `String` as the bridge type.
+The generated Rust code will then lower `Url` in 2 steps:
+
+* The `Url` value is lowered to a `String`
+* The `String` value is lowered to a `RustBuffer` (a UniFFI type that stores a utf8 bytes)
+
+The generated Rust code will then lift `Url` using the reverse of those steps.
+
+By default, this type will appear as a string to foreign consumers.
+However, each foreign language can be configured to execute a process, for example by converting the string to a `java.net.URL` in Kotlin.
+This would mean that `Url` would be:
+
+* Represented by the `url::Url` type in Rust
+* Passed across the FFI as a string
+* Represented by the `java.net.URL` type in Kotlin
 
 ## Custom types in the scaffolding code
 
-Consider the following trivial Rust abstraction for a "handle" which wraps an integer:
+### custom_type!
+
+Use the `custom_type!` macro to define a new custom type.
 
 ```rust
-pub struct Handle(i64);
+
+// Some complex struct that can be serialized/deserialized to a string.
+// This example assumes that Into/TryInto are implemented using the
+// serialisation code.
+use some_mod::SerializableStruct;
+
+// When passing `SerializableStruct` objects to the foreign side, they will
+// be converted to a `String`, then `String` will be used as the bridge type
+// to pass the value across the FFI. Conversely, when objects are passed to
+// Rust, they will be passed across the FFI as a String then converted to
+// `SerializableStruct`.
+uniffi::custom_type!(SerializableStruct, String);
 ```
 
-In this trivial example, the simplest way to expose this is with a macro.
+Default conversions to the bridge type:
+
+- Values passed to the foreign code will be converted using `Into<String>` then lowered as a `String` value.
+- Values passed to the Rust code will lifted as a `String` then converted using `<String as TryInto<SerializableStruct>>`.
+- The `TryInto::Error` type can be anything that implements `Into<anyhow::Error>`.
+- `TryFrom<String>` and `From<SerializableStruct>` will also work, using the blanket impl from the core library. 
+ue
+### custom_type! with manual conversions
 
+You can also manually specify the conversions by passing extra params to the
+macro.   Use this when the trait implementations do not exist, or they aren't
+desirable for some reason.
+
+```rust
+uniffi::custom_type!(SerializableStruct, String, {
+    lower: |s| s.serialize(),
+    try_lift: |s| s.deserialize(),
+});
 ```
+
+### custom_newtype!
+
+The custom_newtype! macro is able to handle Rust newtype-style structs which wrap a UniFFI type.
+
+```rust
+/// handle which wraps an integer
+pub struct Handle(i64);
+
+/// `Handle` objects will be passed across the FFI the same way `i64` values are.
 uniffi::custom_newtype!(Handle, i64);
 ```
 
-Or you can define this in UDL via a `typedef` with a `Custom` attribute,
-defining the builtin type that it's based on.
+### UDL
+
+Define custom types in UDL via a `typedef` with a `Custom` attribute, specifying the UniFFI type
+followed by the custom type.
 
 ```idl
 [Custom]
 typedef i64 Handle;
 ```
 
-For this to work, your Rust code must also implement a special trait named
-`UniffiCustomTypeConverter`.
+**note**: UDL users still need to call the `custom_type!` or `custom_newtype!` macro in their Rust
+code.
 
-An implementation is provided if you used the `uniffi::custom_newtype!()` macro.
-But if you use UDL or otherwise need to implement your own:
+## User-defined types
 
-This trait is generated by UniFFI and can be found in the generated
-Rust scaffolding - it is defined as:
+All examples so far in this section convert the custom type to a builtin type.
+It's also possible to convert them to a user-defined type (Record, Enum, interface, etc.).
+For example you might want to convert `log::Record` class into a UniFFI record:
 
-```Rust
-trait UniffiCustomTypeConverter {
-    type Builtin;
-
-    fn into_custom(val: Self::Builtin) -> uniffi::Result<Self>
-    where
-        Self: Sized;
-    fn from_custom(obj: Self) -> Self::Builtin;
-}
-```
+```rust
 
-where `Builtin` is the Rust type corresponding to the UniFFI builtin-type - `i64` in the example above. Thus, the trait
-implementation for `Handle` would look something like:
+pub type LogRecord = log::Record;
 
-```rust
-impl UniffiCustomTypeConverter for Handle {
-    type Builtin = i64;
+#[derive(uniffi::Record)]
+pub type LogRecordData {
+    level: LogLevel,
+    message: String,
+}
 
-    fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
-        Ok(Handle(val))
+uniffi::custom_type!(LogRecord, LogRecordData, {
+    lower: |r| LogRecordData {
+        level: r.level(),
+        message: r.to_string(),
     }
+    try_lift: |r| LogRecord::builder()
+        .level(r.level)
+        .args(format_args!("{}", r.message))
+        .build()
+});
 
-    fn from_custom(obj: Self) -> Self::Builtin {
-        obj.0
-    }
-}
 ```
 
-Because `UniffiCustomTypeConverter` is defined in each crate, this means you can use custom types even
-if they are not defined in your crate - see the 'custom_types' example which demonstrates
-`url::Url` as a custom type.
-
 ## Error handling during conversion
 
-You might have noticed that the `into_custom` function returns a `uniffi::Result<Self>` (which is an
+You might have noticed that the `try_lift` function returns a `uniffi::Result<Self>` (which is an
 alias for `anyhow::Result`) and might be wondering what happens if you return an `Err`.
 
 It depends on the context. In short:
@@ -112,20 +176,14 @@ pub enum ExampleError {
     InvalidHandle,
 }
 
-impl UniffiCustomTypeConverter for ExampleHandle {
-    type Builtin = i64;
-
-    fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
-        if val == 0 {
-            Err(ExampleErrors::InvalidHandle.into())
-        } else if val == -1 {
-            Err(SomeOtherError.into()) // SomeOtherError decl. not shown.
-        } else {
-            Ok(Handle(val))
-        }
+uniffi::custom_type!(ExampleHandle, Builtin, {
+    lower: |handle| handle.0,
+    try_lift: |value| match value {
+        0 => Err(ExampleErrors::InvalidHandle.into()),
+        -1 => Err(SomeOtherError.into()), // SomeOtherError decl. not shown.
+        n => Ok(Handle(n)),
     }
-    // ...
-}
+})
 ```
 
 The behavior of the generated scaffolding will be:
@@ -152,9 +210,9 @@ type_name = "URL"
 # Classes that need to be imported
 imports = [ "java.net.URL" ]
 # Expression to convert the builtin type the custom type.  In this example, `{}` will be replaced with the int value.
-into_custom = "URL({})"
+lift = "URL({})"
 # Expression to convert the custom type to the builtin type.  In this example, `{}` will be replaced with the URL value.
-from_custom = "{}.toString()"
+lower = "{}.toString()"
 ```
 
 Here's how the configuration works in `uniffi.toml`.
@@ -164,16 +222,19 @@ Here's how the configuration works in `uniffi.toml`.
     custom type.  Defaults to the type name used in the UDL.  Note: The UDL
     type name will still be used in generated function signatures, however it
     will be defined as a typealias to this type.
-  * `into_custom`: Expression to convert the UDL type to the custom type.  `{}` will be replaced with the value of the UDL type.
-  * `from_custom`: Expression to convert the custom type to the UDL type.  `{}` will be replaced with the value of the custom type.
-  * `imports` (Optional) list of modules to import for your `into_custom`/`from_custom` functions.
+  * `lift`: Expression to convert the UDL type to the custom type.  `{}` will be replaced with the value of the UDL type.
+  * `lower`: Expression to convert the custom type to the UDL type.  `{}` will be replaced with the value of the custom type.
+  * `imports` (Optional) list of modules to import for your `lift`/`lower` functions.
 
-## Using Custom Types from other crates
+## Using custom types from other crates
 
-To use the `Handle` example above from another crate, these other crates just refer to the type
-as a regular `External` type - for example, another crate might use `udl` such as:
+To use custom types from other crates, use a typedef wrapped with the `[External]` attribute.
+For example, if another crate wanted to use the examples above:
 
 ```idl
 [External="crate_defining_handle_name"]
-typedef extern Handle;
+typedef i64 Handle;
+
+[External="crate_defining_log_record_name"]
+typedef dictionary LogRecord;
 ```