undici and native Node.js fetch support

[valeriangalliat]

Since Node.js 17.5, we can pass --experimental-fetch to use the built-in undici fetch implementation.

Sadly even if fetch-cookie now theoretically supports WHATWG fetch standard (through response.headers.get('set-cookie') and splitCookieString), undici in most cases drop the set-cookie header, most likely because the spec says it's a forbidden header, and fetch-cookie cannot work with it.

As pointed out by @justingrant in whatwg/fetch#1384 (comment) (I'm following up here because this discussion is off topic for the linked issue), undici do expose the set-cookie header in some cases.

I confirmed on Node 17.5 and I am able to read the Set-Cookie header:

(await fetch('https://wikipedia.com/')).headers.get('set-cookie')
'WMF-Last-Access=26-Feb-2022;Path=/;HttpOnly;secure;Expires=Wed, 30 Mar 2022 12:00:00 GMT, WMF-Last-Access-Global=26-Feb-2022;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Wed, 30 Mar 2022 12:00:00 GMT, GeoIP=...; Path=/; secure; Domain=.wikipedia.org'

That's a good news but from my testing, the set-cookie header is not always returned. It seem to be related to responseTainting and the CORS implementation of undici but I couldn't figure precisely in what conditions undici will or will not drop the set-cookie header.

Typically in the earlier URL https://wikipedia.com/, it issues a redirect to https://www.wikipedia.org/ which is the actual response setting the cookie, as shown by:

curl 'https://www.wikipedia.org/' -H 'User-Agent:' -v 2>&1 | grep -i set-cookie
< set-cookie: WMF-Last-Access=26-Feb-2022;Path=/;HttpOnly;secure;Expires=Wed, 30 Mar 2022 12:00:00 GMT
< set-cookie: WMF-Last-Access-Global=26-Feb-2022;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Wed, 30 Mar 2022 12:00:00 GMT
< set-cookie: GeoIP=...; Path=/; secure; Domain=.wikipedia.org

But fetching that URL with undici returns null for the set-cookie header:

(await fetch('https://www.wikipedia.org/')).headers.get('set-cookie')
null

Also every test I've done on a http://localhost URL failed to return a set-cookie header with undici:

http.createServer((req, res) => res.setHeader('set-cookie', 'foo=bar').end()).listen(8080)
<ref *1> Server
(await fetch('http://localhost:8080/')).headers.get('set-cookie')
null
http.request('http://localhost:8080/', res => console.log(res.headers['set-cookie'])).end()
[ 'foo=bar' ]

And the same goes for every request I've done with redirect: 'manual', which is a requirement for fetch-cookie to do its job.

---

I'd love fetch-cookie to eventually work with the native Node.js fetch implementation but until we can reliably get access to the set-cookie response header this will not be possible.

[valeriangalliat]

Good news! Since nodejs/undici#1469 (nodejs/undici#1262) set-cookie headers are not removed anymore.

I just tested and fetch-cookie now works out of the box with Node 18's native fetch. 🎉