Should we change Valibot's API?

[fabian-hiller]

Hi folks, I am Fabian, the creator and maintainer of Valibot. Today I am writing this message to you to discuss the further development of Valibot's API. With currently more than 80,000 weekly downloads, the library is growing into a serious open source project used by thousands of JavaScript and TypeScript developers around the world.

In the last few days I received two API proposals from @Demivan and @xcfox. Both of them addressed similar pain points of Valibot's current API design. As our v1 release is getting closer and closer, it is important to me to involve the community in such a big decision.

Current pain points

Valibot's current mental model is divided into schemas, methods, validations, and transformations. Schemas validate data types like strings, numbers and objects. Methods are small utilities that help you use or modify a schema. Validations and transformations are used in the pipe argument of a schema. They can make changes to the input and check other details, such as the formatting of a string.

// With Valibot's current API
const EmailSchema = string([toTrimmed(), email(), endsWith('@example.com')]);

A drawback of this API design is that the current pipeline implementation is not modular. This increases the initial bundle size of simple schemas like string by more than 200 bytes. This is almost 30% of the total bundle size.

Another pain point is that the current pipeline implementation does not allow you to transform the data type. This forced me to add a transform method, resulting in two places where transformations can happen.

// With Valibot's current API
const NumberSchema = transform(string([toTrimmed(), decimal()]), (input) => {
  return parseInt(input);
});

Speaking of methods, it can quickly become confusing if you need to apply multiple methods to the same schema, as these functions must always be nested.

// With Valibot's current API
const LengthSchema = brand(
  transform(optional(string(), ''), (input) => input.length),
  'Length'
);

The last pain point that comes to mind is that the current API design gives you less control over the input and output type of a schema. When working with form libraries, it can be useful to have an input type of string | null for the initial values, but an output type of just string for a required field.

The pipe function

After several design iterations, @Demivan came up with the idea of a pipe function. Similar to the current pipe argument, it can be used for validations and transformations. The first argument is always a schema, followed by various actions or schemas.

// With the new `pipe` function
const LoginSchema = object({
  email: pipe(string(), minLength(1), email()),
  password: pipe(string(), minLength(8)),
});

// With Valibot's current API
const LoginSchema = object({
  email: string([minLength(1), email()]),
  password: string([minLength(8)]),
});

The big difference is that the pipe function also allows you to transform the data type. This would allow us to move methods like transform and brand into the pipeline. This prevents nesting of functions for these methods and simplifies the mental model.

With the pipe function, the mental model is reduced to schemas, methods, and actions. Actions are always used within the pipe function to further validate and transform the input and type of a schema.

Alternative names for pipe are flow (idea by @mtt-artis), schema (idea by @genki), compose (idea by @MohammedEsafi), and vali (idea by @Hugos68). Please share your thoughts.

The advantages

The pipe function makes Valibot even more modular, resulting in a smaller bundle size for very simple schemas without a pipeline. For very complex schemas it reduces function nesting when using transform and brand.

// With the new `pipe` function
const LengthSchema = pipe(
  optional(string(), ''),
  transform((input) => input.length),
  brand('Length')
);

// With Valibot's current API
const LengthSchema = brand(
  transform(optional(string(), ''), (input) => input.length),
  'Length'
);

It also gives you more control over the input and output type, and simplifies the mental model by eliminating the confusion between the transform method and the pipeline transformations of the current API.

// With the new `pipe` function
const NumberSchema = pipe(
  string(),
  toTrimmed(),
  decimal(),
  transform(parseInt)
);

// With Valibot's current API
const NumberSchema = transform(
  string([toTrimmed(), decimal()]),
  parseInt
);

Besides that, the pipe function would also allow us to easily add a metadata feature to Valibot. This could be interesting when working with databases to define SQL properties like PRIMARY KEY.

// With the new `pipe` function
const UserSchema = pipe(
  object({
    id: pipe(string(), uuid(), primaryKey()),
    name: pipe(string(), maxLength(32), unique()),
    bio: pipe(string(), description('Text ...')),
  }),
  table('users')
);

The disadvantages

The main disadvantage of the pipe function is that it requires a bit more code to write for medium sized schemas, and for more complex schemas you may end up nesting multiple pipelines.

// With the new `pipe` function
const NumberSchema = pipe(
  union([pipe(string(), decimal()), pipe(number(), integer())]),
  transform(Number)
);

// With Valibot's current API
const NumberSchema = transform(
  union([string([decimal()]), number([integer()])]),
  Number
);

Let's discuss

Your opinion matters to me. I encourage everyone to share their thoughts. Even quick feedback like "I like this ... and I don't like that ..." is welcome and will help to shape Valibot's future API design. Please discuss with me here on GitHub or share your thoughts on Twitter.

[juliusmarminge]

definetely in favor of something like this, the current api feels very limiting and hard to read once you go past the simple cases

[fabian-hiller]

Thank you for your feedback! What do you think about the new mental model? What do you think of the name pipe for these wrapper functions? Have you encountered other pain points not mentioned here?

[juliusmarminge]

i haven't dug too deep into valibot for the reason above. the pipe operator looks solid, very similar to builder pattern but with reduced bundle size.

[fredericoo]

really like it too. I have some mildly complex schemas that take me a while to understand and pipe would make it much simpler to read

[jchatard]

This mental model is so much easier to reason about. Using valibot becomes more flat that way. I like the he term pipe.

The nesting was very confusing to me.

That's a go! For me. :-)

[fabian-hiller]

Thank you for your feedback on this!

[xcfox]

I think a non-modular pipe is acceptable for best development experience and readability.
My idea is to think of each schema as a pipe, which at the same time reduces the cost of code migration:

// With the new `pipe` arguments
const LengthSchema = optional(string(), [
  transform((input) => input.length),
  brand('Length'),
])

// With Valibot's current API
const LengthSchema = brand(
  transform(optional(string(), ''), (input) => input.length),
  'Length'
)

// With the new `pipe` arguments
const NumberSchema = string(
  [toTrimmed(), decimal(), transform(parseInt)]
);

// Maybe we can add a new `from` method to ensure that NumberSchema is number().
const NumberSchema = number(
  [from(string(toTrimmed(), decimal()), parseInt)]
);

// With Valibot's current API
const NumberSchema = transform(
  string([toTrimmed(), decimal()]),
  parseInt
);

// With the new `pipe` arguments
const LoginSchema = object({
  email: string([message("Email"), minLength(1), email()]),
  password: string([minLength(8)]),
});

// With Valibot's current API
const LoginSchema = object({
  email: string("Wrong Email", [minLength(1), email()]),
  password: string([minLength(8)]),
});

[Demivan]

@xcfox Playground that you linked does not use arrays for schemas. Is the link correct?
When using arrays, there are issues with typing, as all array elements need to have the same type.

1. How would types work with transforms?
2. Is there any way to prevent incorrect usage like [number(), maxLength(32)]?

[fabian-hiller]

Thanks for the review and response @Demivan. @xcfox I appreciate your research and feedback! Even if I decide to go in a different direction with Valibot, I am grateful for your input. It helps me a lot to make the right decisions in the long run.

[xcfox]

@Demivan Here is an arrays version.

The reason I don't use array is that TypeScript always treats arrays loosely, requiring repeated use of the as const phrase to get the right array.

I haven't yet found a way to have TypeScript constrain the items in an array based on the previous item of the array. As you can see, the downside of this implementation is that elements in an array can't sense other elements within the same array.

1. Input types need to be declared manually in order for transforms to work.
2. I didn't find a way to constrain [number(), maxLength(32)], the elements in the array can't sense each other, but with number(maxLength(32)) it's possible.

[fabian-hiller]

I rewrote a minimal proof without function overloading at TS playgroud

This implementation is not completely type safe. I know it is just a draft, but is it even possible with your implementation to make the following code type safe? It is important that subsequent actions know the output of previous actions.

// The second transformation is invalid and should display a type error
const LengthSchema = string(transform((value) => value.length), transform((value) => value.length));

[xcfox]

@fabian-hiller Here I write a demo at TS playground.

TypeScript gives an error when the type does not match the output of the previous element.
While this error may not seem very elegant, it shows the possibilities of constraining array types.

[sillvva]

Would the pipe function replace all pipe arguments, including forward() on object schemas?

[fabian-hiller]

Yes, it will replace the pipe argument. I do not plan to support both to keep the mental model and API design consistent and simple. The forward method will still be required to forward issues to specific fields, but feel free to contact me if you have alternative ideas for the design and implementation of forward.

[sillvva]

That's good. I support the pipe function.

The disadvantages of nesting pipelines mentioned don't concern me, because I can break them up quite easily:

const strDecimal = pipe(string(), decimal());
const numInteger = pipe(number(), integer());
const numberSchema = pipe(union([strDecimal, numInteger]), transform(Number));

Which is similar to how I nest schemas

[MohammedEsafi]

Valibot might make the code a bit harder to read because of nested calls, but that's what helps keep the bundle size small. On the other hand, Zod.js has a cleaner design but comes with a bigger bundle size. So, it's a tradeoff to consider

If we can design an API that looks good and keeps the bundle size small, Valibot would be the winner. It's all about having a nice API design without sacrificing bundle size. let's think

[mtt-artis]

I prefer this new API.
However, the first parameter in a pipe function is generally the value to pass into a pipeline of functions. To me, this pipe behaves more like a flow.

https://gcanti.github.io/fp-ts/modules/function.ts.html#flow

[fabian-hiller]

Thanks for your feedback! I will take a look at it and think about the name. pipe seemed like a good fit to me because it is a neutral term that works for validations, transformations, and future features like metadata that we might add.

[Hugos68]

I really like the pipe function, it simplifies the API, string and min aren't different anymore, they are just validators on that attribute, it just makes sense, and if it adds benefits in terms of bundle that is win win for me

[fabian-hiller]

I agree! Thanks for your feedback!

[sandros94]

I like the new proposal. In particular the advantage of transforming data types. But my gut doesn't like those repeated pipe functions (except nested ones) and I'm starting to think if that there could be a way to catch and automatically apply them

[fabian-hiller]

Thanks for your feedback! Can you explain this in more detail? Can you provide a code example that shows your thoughts?

[sandros94]

Thanks for your feedback! Can you explain this in more detail? Can you provide a code example that shows your thoughts?

Sorry for the late reply, was offsite and busy.

I was just wandering if it was possible to infer the requirement of pipe within operations like object:

// With the new `pipe` function
const LoginSchema = object({
  email: pipe(string(), minLength(1), email()),
  password: pipe(string(), minLength(8)),
});

// inferring the `pipe` requirement via a list
const LoginSchema = object({
  email: [string(), minLength(1), email()],
  password: [string(), minLength(8)],
})

Exactly like @lukemorton then suggested later on.

P.S.: I also agree with the schema naming instead of pipe to make it more understandable to non-developer people that might end up in your project review session... (or more simply new developers that are just starting out, something I've indeed experienced)

[xcfox]

I think an additional schema or pipedObject could be implemented as an extension to object:

// Instead of this:
const UserSchema = pipe(
  object({
    id: pipe(string(), uuid(), primaryKey()),
    name: pipe(string(), maxLength(32), unique()),
    bio: pipe(string(), description('Text ...')),
    isStudent: boolean(),
  }),
  table('users')
)

// With `pipedObject`:
const UserSchema = pipedObject(
  {
    id: [string(), uuid(), primaryKey()],
    name: [string(), maxLength(32), unique()],
    bio: [string(), description('Text ...')],
    isStudent: boolean(),
  },
  table('users')
)

• This improves code readability and developer experience,
• It is tree-shakeable

The downside is: methods that modify the schema type like fallback, brand don't work on the type.
For strictly typed methods like brand, that I think it's a very rare case, the pipe is still needed:

const UserSchema = pipedObject({
  id: pipe(string(), brand('id'), uuid(), primaryKey()),
  name: [string(), maxLength(32), unique()],
  bio: [string(), description('Text ...')],
  isStudent: boolean(),
})

In most cases, developers only need to use pipedObject for the better development experience.
In very simple applications, the developer should choose object for small bundle size.
In more complex cases, use pipe for strict type safety.

[fabian-hiller]

Alternative names for pipe

Please share your thoughts. I will be happy to add your idea to the list as well.

• flow (idea by @mtt-artis)
• schema (idea by @genki)
• compose (idea by @MohammedEsafi)
• vali (idea by @Hugos68)
• of (idea by @xcfox)
• from (idea by @xcfox)
• val (idea by @xcfox)

You can vote here: https://strawpoll.com/e2nar3xWegB

[christophsturm]

what about field? I think its better than schema because its just one field and not a schema. (a schema consists of fields) pipe to me sounds like an implementation detail. what you do is declare a field. unless there many use cases where don't need a pipe and just declare string() or boolean()

[Demivan]

@christophsturm It is not just for fields. It will work with objects and any top level schemas.

[xcfox]

From the results of the poll, most people preferred pipe, which means that close to half of the people disliked pipe.

Just export defalut pipe in the index file, then everyone can easily use their favorite name.

import val, { object, string, minLength, email } from 'valibot'

const LoginSchema = object({
  email: val(string(), minLength(1), email()),
  password: val(string(), minLength(8)),
})

import pipe, { string, optional, transform, brand } from 'valibot'

const LengthSchema = pipe(
  optional(string(), ''),
  transform((input) => input.length),
  brand('Length')
)

[fabian-hiller]

I would not say they dislike it. They just liked other options better. Looking to the future, I think a consistent name is preferable. My vision is to provide simple but powerful primitives with clear ways to use them. I would find it confusing if an external blog post or YouTube video used a different name than the official documentation. Even if it is just a name, it can make the usage and mental model much harder to understand.

[brandonpittman]

It’s dying in the straw poll but “of” is a great idea. It is used in FP to create wrapper objects and reads well: “name: of(string(), minLength(8))”.

[kovalchukq]

I like new API. It's so much easier to read and understand.

[fabian-hiller]

Thank you for your feedback!

[juliusmarminge]

Would it be possible to include a .parse() method on this operator? would eliminate the need for typeschema in certain cases, wihtout needing to include it on every schema which was a bundle size concern before iirc?

[fabian-hiller]

No, I do not think this is an option for Valibot as a modular library. It would make the API less consistent. Also, the implementation you are probably looking for is exception based and needs to import ValiError as well. This would increase the bundle size even more unnecessarily if you only use safeParse where ValiError is not needed.

[samualtnorman]

will this have any impact on #176?

[Demivan]

It should be possible to implement this kind of type branding with new pipe.

[fabian-hiller]

It may be possible, but not with the same DX as with Zod. I expect that brand will need more than one generic. One for the input type and one for the brand name. Since TypeScript forces us to define non or every generics, you will be forced to define the input type as well. Depending on the complexity of the input type, it is probably easier to define no generic and pass a symbol as the first argument to brand.

[Sec-ant]

The choice of terminology such as flow for left-to-right (LTR) composition as a higher-order function, and pipe for LTR evaluation/application, appears to be more prevalent in existing libraries, as evidenced in the refs below. If we see a schema as a function, with the value to be validated as the function's input, the construction of a schema through our API aligns more closely with the concept of flow. This consideration makes flow a more suitable choice for our context.

Refs:

• The withdrawn TC39 proposal for function pipe and flow, it also proposed compose for RTL composition.
• In fp-ts, flow is used for function composition, and pipe is used for evaluation/application
• lodash also uses flow for this purpose

However, there are also notable exceptions. Both Ramda and RxJS utilize the term pipe to denote left-to-right function composition, without employing the term flow.

[fabian-hiller]

Thank you for your research!

[Sec-ant]

We have to also consider the implementation of types as well.

In the current design, the input types of each member in the pipe argument are always the same so we can use a single generic parameter to enforce them all.

/**
 * Pipe type.
 */
type Pipe<TInput> = (BaseValidation<TInput> | BaseTransformation<TInput>)[];

With the new API, as suggested in the post, the data type can be changed. I'm not sure if there's a simple solution to correctly implement the type of this new API.

Existing libraries in typescript like ts-functional-pipe and fp-ts just use function overloads extensively to support various number of arguments, but up to a limit.

[Sec-ant]

I believe you mistakenly mixed up these two terms. Because in other libraries pipe is for cascading apply and flow is for combination. See my above comment: #463 (comment).

And honestly I don't see the necessity to introduce both of these two functions. In analogy, schema itself is the function, use that schema to parse a value is like applying arguments to the function. When we define our schemas, we don't consume it, so all we need is the combination logic, not application.

[juliusmarminge]

Existing libraries in typescript like ts-functional-pipe and fp-ts just use function overloads extensively to support various number of arguments, but up to a limit.

We do this in trpc too. We have made attempts to do a variadic generic version but that really kills ts performance.

[Sec-ant]

We have made attempts to do a variadic generic version

I'm actually curious about how you guys do that :)

[fabian-hiller]

This could work (it is just a draft):

function pipe<
  TInput,
  T1 = TInput,
  T2 = T1,
  T3 = T2,
  T4 = T3,
  T5 = T4,
  T6 = T5,
  T7 = T6,
  T8 = T7,
  T9 = T8,
  T10 = T9,
  T11 = T10,
  TOutput = T11,
>(
  v1: Schema<TInput, T1>,
  v2?: SchemaOrAction<T1, T2>,
  v3?: SchemaOrAction<T2, T3>,
  v4?: SchemaOrAction<T3, T4>,
  v5?: SchemaOrAction<T4, T5>,
  v6?: SchemaOrAction<T5, T6>,
  v7?: SchemaOrAction<T6, T7>,
  v8?: SchemaOrAction<T7, T8>,
  v9?: SchemaOrAction<T7, T8>,
  v10?: SchemaOrAction<T7, T8>,
  v11?: SchemaOrAction<T7, T8>,
  v12?: SchemaOrAction<T8, TOutput>
): Schema<TInput, TOutput>;

[Sec-ant]

Wow, at first glance I doubted it but after a while I have to admit this looks promising, much clever and better than overloading. Good to learn this, thank you @fabian-hiller.

[lukemorton]

Thanks for your work on Valibot so far Fabian. The reason I am drawn to Valibot over Zod is its cleaner API. Zod adds a lot more boilerplate than perhaps is necessary. Composable functions like string(), url(), email() coerce() just read so much cleaner.

This proposal is pretty exciting for making the DSL of Valibot even more consistent and intuitive. I'm a huge fan of the functional programming style of being able to build pipelines of composable functions that can then be executed to validate inputs. As proposed this change will remove the repetition of the arguments currently passed to schema methods and instead provide a consistent, reliable and memorisable framework for validation.

My main thought reading the proposal and seeing the poll on X was that pipe() is a functional programming term and pretty meaningless to validation. I’ve always liked the Rails Active Record Validations for an example of a lib that focuses more on a clean and readable API. As a developer, I want to be able to read a validation schema and it make sense in plain english. The closer we get to plain english, and having even non-technical people read and understand it, the better.

Adding a whole bunch of pipe() methods is going to be pretty boilerplate-y!

I'm not completely happy with any of my suggestions below, but here they are anyway:

Suggestion 1

Would it be cleaner to use arrays to define pipelines in objects instead? This proposal is essentially renaming pipe to schema, which has already been suggest, plus using arrays as the values in object() definitions.

// Instead of this:
const LoginSchema = object({
  email: pipe(string(), minLength(1), email()),
  password: pipe(string(), minLength(8)),
});

// With arrays:
const LoginSchema = object({
  email: [string(), minLength(1), email()],
  password: [string(), minLength(8)],
});

// Instead of this:
const UserSchema = pipe(
  object({
    id: pipe(string(), uuid(), primaryKey()),
    name: pipe(string(), maxLength(32), unique()),
    bio: pipe(string(), description('Text ...')),
  }),
  table('users')
);

// With schema wrapper:
const UserSchema = schema(
  object({
    id: [string(), uuid(), primaryKey()],
    name: [string(), maxLength(32), unique()],
    bio: [string(), description('Text ...')],
  }),
  table('users')
);

// Instead of:
const NumberSchema = pipe(union([strDecimal, numInteger]), transform(Number));

// With schema wrapper:
const NumberSchema = schema(union([strDecimal, numInteger]), transform(Number));

Suggestion 2

Standardise around a schema function with takes a spread of arguments that are either objects or composable functions.

// Instead of this:
const LoginSchema = object({
  email: pipe(string(), minLength(1), email()),
  password: pipe(string(), minLength(8)),
});

// With object passed to schema function:
const LoginSchema = schema({
  email: [string(), minLength(1), email()],
  password: [string(), minLength(8)],
});

// Instead of this:
const UserSchema = pipe(
  object({
    id: pipe(string(), uuid(), primaryKey()),
    name: pipe(string(), maxLength(32), unique()),
    bio: pipe(string(), description('Text ...')),
  }),
  table('users')
);

// With schema wrapper:
const UserSchema = schema(
  {
    id: [string(), uuid(), primaryKey()],
    name: [string(), maxLength(32), unique()],
    bio: [string(), description('Text ...')],
  },
  table('users')
);

// Instead of:
const NumberSchema = pipe(union([strDecimal, numInteger]), transform(Number));

// With schema wrapper:
const NumberSchema = schema(union([strDecimal, numInteger]), transform(Number));

Even with pipe() this change will be great. I'm just thinking about how it might be even more cleaner which isn't so easy with a language like JS/TS.

[mtt-artis]

Hi 👋
I don't agree with "it improves the readability of the code".

With pipe, I just have to read from top to bottom to understand the schema.

With with, we have another keyword to remember and you have to jump it when reading.

It's personal preference I guess.

[xcfox]

Maybe valibot could provide different ESM export where pipe function is attached to the type schemas.

@Demivan You're absolutely right, we can export another API.
I use valibot on the backend, it doesn't make much sense to keep the packaging size small on the backend (In fact, in the node.js world, back-end applications are rarely bundled).

This allows for the best development experience and readability for the backend, such as when used with trpc.

We can create a new package @valibot/chain where we don't need to care about tree-shakeable:

// With `valibot`:
const LoginSchema = object({
  email: pipe(string(), minLength(1), email()),
  password: pipe(string(), minLength(8)),
})

// With `@valibot/chain` :
const LoginSchema = object({
  email: string().minLength(1).email(),
  password: string().minLength(8),
})

// With `valibot`:
const UserSchema = pipe(
  object({
    id: pipe(string(), uuid(), primaryKey()),
    name: pipe(string(), maxLength(32), unique()),
    bio: pipe(string(), description('Text ...')),
    isStudent: boolean(),
  }),
  table('users')
)

// With `@valibot/chain` :
const UserSchema = object({
  id: string().uuid().with(primaryKey()),
  name: string().maxLength(32).with(unique()),
  bio: string().description('Text ...'),
  isStudent: boolean(),
}).with(table('users'))

// With `valibot`:
const NumberSchema = pipe(union([strDecimal, numInteger]), transform(Number))

// With `@valibot/chain` :
const NumberSchema = union([strDecimal, numInteger]).transform(Number)

[fabian-hiller]

We could do that, but right now the core package is already a full time job for me and I will not have time to work on another library. Why did you choose Valibot over Zod if you prefer Zod's chaining method?

I can imagine starting valibot-chain as a community project. I am happy to advise the project, but not able to actively work on it. This library could basically just reuse the core package. That way you could keep the effort to a minimum, and only have to think about the external API design.

[Sec-ant]

Just an immature thought, we (or the community) can provide build-time api transformation (chain -> pipe) tool / plugin for users who like the chain style api, and still want to enjoy the valibot ecosystem and also keep the runtime footprint small.

[fabian-hiller]

@Sec-ant true! That's an even better option! In the long run, I plan to work on an official Valibot compiler to improve runtime performance as well as codemods to migrate from other libraries like Zod with a single CLI command. For now I think it makes sense to start this as a community project, but if it gets a lot of traction I can imagine integrating it into the official repo and releasing it under @valibot/....

[jsudelko]

I'm on board with pipe. It improves guessability since the API is more consistent. Good for man. Good for GPT.

Though I am curious how the current approach compares to pipe in terms of performance. Is it much faster, much slower, or does it have a negligible change in performance? See https://github.com/moltar/typescript-runtime-type-benchmarks. Consider benchmarking the two approaches while implementing it.

[tsingson]

pipe is better.

[fabian-hiller]

Thank you! Yes, I will investigate the performance. Currently I do not expect a significant difference.

[sacrosanctic]

This is a positive direction.

I consider your lib a FP style library. So adding a composition function is the logical extension of this. Every FP style library provides one.

---

What would be even better though is if your functions work with FP libraries that already provide utility functions like pipe for LTR composition and compose for RTL composition.

The benefit is that you only need to provide a basic pipe for the general audience, but those who wish to have more flexibility can bring in another lib for advanced composition. Or maybe they just have a preference for RTL composition.

---

Another suggestion is that brackets should be removed for functions without terms

// current proposal
const LoginSchema = object({
  email: pipe(string(), minLength(1), email()),
  password: pipe(string(), minLength(8)),
});

// my suggestion
const LoginSchema = object({
  email: pipe(string, minLength(1), email),
  password: pipe(string, minLength(8)),
});

Reference

Ramda pipe

import R from 'ramda'
// const f = (x,y) => R.add(R.negate(R.Math.pow(x,y)))
const f = R.pipe(Math.pow, R.negate, R.add(2));

f(3, 4); // -(3^4) + 2

REPL

FP-TS flow

import { flow } from 'fp-ts/lib/function'

// const f = (x) => toString(multiply2(add1(x)))
const f = flow(add1, multiply2, toString) // this is equivalent

[fabian-hiller]

Thank you for your feedback!

What would be even better though is if your functions work with FP libraries that already provide utility functions like pipe for LTR composition and compose for RTL composition.

In theory this sounds great, but in practice it will not work because Valibot also tries to make every schema accessible. string() and minLength() do not return a function. They return an object which can contain further properties like the entries of an object or the key and value schema of a map. Therefore, I think a pipe function that we maintain and controll is the way to go for Valibot.

Another suggestion is that brackets should be removed for functions without terms

I haven't implemented this yet because I want to keep the API as standardized and simple as possible. If we supported string and string(), there would be two ways to define the same schema. This makes using a library more complicated and it also increase the source code and bundle size.

[yudinmaxim]

I am not a very experienced developer, especially in matters of validation and listing architecture.
Subjectively, I will say that what I see in the comparisons says that pipe() reads faster. At the same time, the constant writing of pipe(..), including the nested pipe(...pipe(...pipe(...))), makes reading less pleasant - there is always an extra word that is not particularly useful for understanding the model.
I liked the model proposed above with writing through an array - it is shorter, but it is not so clear.
I understand that the classic pipe requires a larger bundle size and therefore is not considered?
string().minLength(1).email()

it may be worth considering alternative writing options:
() => - just the arrow function will say that we have a pipe call here. And if it doesn't exist, it's not pipe() (but I suspect it will be ugly).
Another option is to try using a non-full name: pipe() -> p() or schema() -> sch(). But everyone can make such a decision for themselves)

As a result: the new version looks clearer and easier to read. It looks a little less convenient, with a large scheme, for reading (I already said that the constant pipe() will make your eyes blink). But definitely this way looks right.

It is a pity that all the schemes will have to be redone after the introduction of this innovation, if I understood correctly.

[fabian-hiller]

Thanks for your feedback! Yes, all previous schemas will need to be migrated after the upgrade. However, I plan to provide a codemod that will do the migration completely automatically in >80% of cases. If there are any other statements or questions you would like me to respond to, please let me know.

[IlyaSemenov]

The main problem with the classic pipe is probably not the bundle size (which is often not really important) but that it's not extensible. You can have string().email() but you can't really add string().twitter() or nonEmptyString().twitter() should you wish to (unless using runtime patch hacks, which is unreliable).

[nakanoasaservice]

I think the pipe function is very good. If I were to make a minor request, I would prefer to avoid nesting functions by using the pipe function, rather than nesting calls, even when defining optionals. However, this approach might be at a disadvantage in terms of implementation size or performance.

// I prefer this
const OptionalStr = pipe(string(), optional('foo'))

// over this
const OptionalStr = optional(string(), 'foo')

[Demivan]

Theoretically it is possible to do if optional is placed before string in a pipe.

[fabian-hiller]

But how does optional know that it can also pass strings, and how does string know that it can also pass undefined?

[Demivan]

optional does not need to know that input could be a string. It only checks for undefined. If it encounters undefined it would mark input with something like undefinedAllowed: true, then type validations will skip if input is undefined.
I don't think it is a better than nesting, but theoretically possible.

[homersimpsons]

Maybe it would be easier / more correct to have boolean logic, for instance with an or someone could write:

const OptionalStr = or(undefined(), string())

by default, I see a pipe like an and where the property flows through transforms / validations. So we certainly needs an or version being branch or or or another name.

Note that the current union may be enough for this.

[fabian-hiller]

I think that’s exactly what union is doing.

[linkb15]

I like this new DX suggestions / discussions where everyone can pour out their thoughts over as the movement will affect the library moving on as well.

I like the pipe() here as well, it is easier to do transform and simpler in the mental model.

Need to give more complex schemas though to be able to tell the more differences.

like checkout schema for e-commerce for examples or forms when we need to do many complex stuffs with conditionals, combinations with some schemas, examples working with some other UI libraries, etc.

With cases like real world problems, it would be easier to understand in the experiences I think when it is being used with others stuffs in the ecosystem.

[dboune]

I'd be thrilled to see it possible to have precision control over the input type while still allowing further validation. I am again up at 1 a.m., fighting with form validation where I need input-only null, and stopped by to see if there have been any changes on this front before I pull my utils from an earlier project.

Overall, this looks like a positive direction. The usage seems straightforward.

[fabian-hiller]

Thanks for your feedback! Feel free to share sample code or describe your goal in more detail (what is the input, output and further validation). Then I can give you feedback if this API change makes it possible.

[dboune]

Hi Fabian,

It's precisely what you wrote about...

The last pain point that comes to mind is that the current API design gives you less control over the input and output type of a schema. When working with form libraries, it can be useful to have an input type of string | null for the initial values, but an output type of just string for a required field.

There are several tickets for it as well.

Without attempting to write code that isn't currently possible... replace number with whatever else as needed.

type Input = null | number;
type Output = number;

const schema = v.something; // some valid construct that produces the above types.

parse(schema, null); // Result: ValiError with whatever our messaging needs to be

[fabian-hiller]

With the pipe function you could write the following. The first schema of the pipe defines the input type, and the second defines the output type.

const Schema = pipe(nullable(number()), number());

[dboune]

That would be perfect! Developer UX and functional form validation all wrapped into one.

I'm thinking now on how this might affect multi-stage validations where there is potential for input -> validate -> mutate -> validate -> etc, and even crossing boundaries between server and client (say in a monorepo with shared code), and other similar patterns. Today it's common to keep multiple distinct definitions, and you might be moving toward something much more robust that could be able to track a lifecycle of some data through multiple stages.

I do this with typescript types alone so that changes in the backend filter out to the frontend, and cause type/lint errors when we have a divergence that would create an error. It's valuable.

[fabian-hiller]

Even though I am the author of Modular Forms, I am not that deep into multilevel validations with forms. Feel free to share a minimal example so I can understand it better. Then I can take it into account when working on Valibot and Modular Forms.

[IlyaSemenov]

Looking from the reusable schema developer perspective, I'm also +1 for extracting pipe out of schemas.

Currently, extending schemas means repetitive defaultArgs / mergePipes boilerplate, for example:

export function integerNumber(
  arg1?: ErrorMessage | Pipe<number>,
  arg2?: Pipe<number>,
) {
  const [error, pipe] = defaultArgs(arg1, arg2)
  return number(error, mergePipes([integer(error)], pipe))
}

From what I understand, in this case it will become as simple as:

const integerNumber = (error?: ErrorMessage) => pipe(number(error), integer(error))

which is a huge win.

[fabian-hiller]

Feel free to take a look at the draft PR for the pipe function I just created: #502

[LexSwed]

Just noticed it's not possible to provide a single error message for pipe validators. Sometimes I want to provide a generic string instead of passing the value multiple times. Maybe pipe can have an overload to allow passing an error message?

// from:
pipe(string('invalid string'), trim(), minLength('invalid string'), maxLength('invalid string'))
// to:
pipe([string(), trim(), minLength(), maxLength()], 'invalid string')

[fabian-hiller]

Thank you for your feedback! What is your use case? I would like to understand it better.

What you have in mind is already possible with our config method. Try it out in our playground.

const Schema = v.config(
  v.pipe(v.string(), v.maxLength(3)),
  { message: 'invalid string' }
);

You can also create your own message method:

import * as v from 'valibot';

function message<TSchema extends v.GenericSchema>(
  schema: TSchema,
  message: v.ErrorMessage<v.InferIssue<TSchema>>
): TSchema {
  return {
    ...schema,
    _run(dataset, config) {
      return schema._run(dataset, { ...config, message });
    },
  };
}

const Schema = message(v.pipe(v.string(), v.maxLength(3)), 'invalid string');

[LexSwed]

Thanks @fabian-hiller!

Yes, config works great for me, just missed it as an option 😄

[saturnonearth]

How would forward/custom work with this new API?

[fabian-hiller]

custom has been renamed to check. But both work the same way with the new API.

Here is an example: https://valibot.dev/guides/methods/#forward