We need your help to implement the pipe function 🙏

[fabian-hiller]

Based on #463 I started to rewrite Valibot and implement the pipe function. The results so far are promising. I was able to reduce the bundle size, improve type safety and fix some general issues. Unfortunately, I hit a TypeScript limitation or bug. My current implementation is completely typesafe. Even the issues are now fully typed. So I want to avoid compromising here at all costs.

Is there a TypeScript wizard in our community like @mattpocock or @ssalbdivad that can find a workaround? I have created a TypeScript playground with a minimal reproduction.

[ssalbdivad]

I took a quick look, and if I'm understanding the API correctly, the first issue can be resolved by ensuring the input type is correctly treated as contravariant so a broader type like string | any[] can be assigned to a narrower one like string. I made a couple adjustments to demonstrate that here:

https://tsplay.dev/WJ3lgw

However, you will run into a limitation inferring piped output with this approach. Without using HKTs, you will not be able to have a function like minLength that will return either a string or an array depending on the type of the input. You would have to separate it into two separate constraints, but you'd essentially run up against the same issue if you ever needed a narrowed instantiation of one of those types like number[].

The options for working around this are to define your actions as HKTs like those in HotScript or hkt-toolbelt so they can be applied at a type-level and return a result that depends on the input, or to use a different API. ArkType's next release uses chaining to handle this scenario:

Hopefully something like that could be made compatible with your tree-shaking strategy?

[fabian-hiller]

Thank you David for taking the time! Unfortunately, your solution does not solve the root of our problem. It only removes the TS error, but at the same time it changes the output type of Schema1 to string | any[] and that causes other problems.

[ssalbdivad]

Yes sorry my goal was not to solve the entire problem, but just to highlight what I understand to be an eventual limitation you'll encounter with this API.

Assuming you want to pipe some arbitrary list of transformations to some input and infer the narrowed output at each step, this cannot be achieved without some kind of HKT-based strategy.

I apologize if I misinterpreted the intent of the API.

I know you spoke with @Andarist about inferring inputs from context within the transform function. This is not a strategy I've ever used. To me, inference feels much more intuitive if the scopes are always handled bottom-up.

Have you considered removing the wrapper around the transform function altogether and instead interpreting a function passed directly to pipe as a transform? You could essentially "parse" out the same result.

[fabian-hiller]

Thank you for your feedback. I will think about it!

[brenelz]

Does this work the way you intend? Seems like the types flow pretty good but could just be accidental

https://tsplay.dev/NBdlxW

[fabian-hiller]

Thank you for your idea! The problem with this solution is that we lose type safety. Schema1 now contains a .pipe with a BaseAction instead of a MinLengthAction. For example, this prevents us from accessing Schema1.pipe[0].requirement in a type-safe way.

[brenelz]

ah i see. I figured I was probably missing something

[anuraghazra]

Does this work?

TSPlay

• Schema1 returns MinLength, Can safely access Schema1.pipe[0].requirement
• No TypeErrors

Let me know if I'm missing any edge case? 🤔

[fabian-hiller]

Like Ivan, I am blown away (and confused). 🤯😅
I will have a close look in the next few minutes to see if this workaround works for us.

[ssalbdivad]

@anuraghazra is a total boss 😎

[fabian-hiller]

The code is a little dirty, but it seems to work! 🥹 I am so grateful! 🙏 You probably saved all my work from the last few weeks. I was really afraid I would have to throw it all away. I will give another update this evening or at the weekend.

[anuraghazra]

I'm glad I was able to help. 😺

Wow. Why does this work?

To be honest even I'm not entirely sure.

My guess is that by specifying the union of BaseAction<InferOutput<TSchema>, InferOutput<TSchema>> type in the argument, we are making the pipeItem1's generic inference candidate much "better" thus TS is able to narrow down & return the concrete type.

Here's a good resource on TS inference algorithm: https://portal.gitnation.org/contents/lets-make-a-generic-inference-algorithm

But of course my understanding is limited in terms of how the TS compiler is internally handling this case, maybe @Andarist can explain better.

[Andarist]

TS is able to infer from into return type of minLength based on the contextual type of this position. That contextual type is TPipeItem1 if you don't use this union trick. So TS tries to instantiate this using the inferences that it was able to make so far. Nothing was inferred so far for TPipeItem1 so it's instantiated with a magical "silent never type" and this type is internal and non-inferrable. The latter doesn't quite matter all of that much because it doesn't hold any useful information for us anyway but I guess it's non-inferrable to avoid spoiling inference results with it.

So nothing interesting happens at this stage and, I guess that, later on TS just pulls constraints and stuff when it touches this minLength.

But when you are using this union trick the contextual type is TPipeItem1 | BaseAction<InferInput<TSchema>, InferOutput<TSchema>>. It's also instantiated with the inferences made so far and while TPipeItem1 is still instantiated with this magical silent never type it just gets eliminated from the union (T | never is just T). The other member of this union has better luck - it directly references type parameters that have some inference candidates already (TSchema). So this gets instantiated as BaseAction<string, string>.

Based on that TS infers through the return type of minLength (from BaseAction<string, string> to MinLengthAction<TInput, TRequirement>). I didn't follow this further but I think this is essentially what "unblocks" it here. It's sort of a timing issue and an implementation detail leak. It's certainly an interesting effect but I'd call it somewhat accidental. I don't think there is anything fundamental stopping it from working better in this situation without relying on this trick.

[gmaxlev]

I've heard that it's not possible to implement the pipe function in TypeScript "normally" like in other languages. That's why the pipe function in RxJS was written like that https://github.com/ReactiveX/rxjs/blob/master/packages/rxjs/src/internal/util/pipe.ts

[fabian-hiller]

I will take a look at it! Thank you!

[joshwashywash]

fp-ts is similar with their pipe function

https://github.com/gcanti/fp-ts/blob/a94d4f84d346da271e0450d7341b98e0a5739a0e/src/function.ts#L416

[fabian-hiller]

Feel free to take a look at the draft PR for the pipe function I just created: #502