Skip to content

Conversation

FelixVaughan
Copy link
Contributor

This relates to...

#4444

Rationale

Preserve header shape and correct Host after DNS resolution to avoid routing issues/header corruption

Changes

Added addHostHeader to dns interceptor and parameterized tests for different header types.

Bug Fixes

Fixed DNS interceptor turning [string, string][]/iterables into numeric-key headers

Status

@@ -5,6 +5,43 @@ const DecoratorHandler = require('../handler/decorator-handler')
const { InvalidArgumentError, InformationalError } = require('../core/errors')
const maxInt = Math.pow(2, 31) - 1

function addHostHeader (headers, host) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, you don't need to parse the headers by yourself, this is done by undici automatically, we just need to standardise the way that these are forward to undici.

If Array, push host to the headers array, if an object, attach host as a new property

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've simplified the code!

A header case I'm slightly worried about though is non-array iterables.

request({ origin, path: '/', method: 'GET', headers: new Set([['f0', 'b0'], ['f1', 'b1']]) }).

These headers are valid under UndiciHeaders and a request like this (though rare) is possible; but the proposed array/object check would break this.

Another issue I've noticed is [string,string][] headers always raise an error due to the way header processing is handled in Request.js. Say headers = [['f0', 'b0']] or [['f0', 'b0'], ['f1', 'b1']]

if (Array.isArray(headers)) {
      if (headers.length % 2 !== 0) {
        throw new InvalidArgumentError('headers array must be even')
      }
      for (let i = 0; i < headers.length; i += 2) {
        processHeader(this, headers[i], headers[i + 1])
      }
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These headers are valid under UndiciHeaders and a request like this (though rare) is possible; but the proposed array/object check would break this.

Usually these are iterables, and we iterate over them, but not use utility functions; if that's the case, we can try to also check for Symbol.iterator to decide what to do with them.

Say headers = [['f0', 'b0']] or [['f0', 'b0'], ['f1', 'b1']]

That's somehow expected, we only expect them in the format [string, string], anything outside should throw

Copy link
Contributor Author

@FelixVaughan FelixVaughan Sep 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handling iterators might mean we can't avoid some normalization. However this can be streamlined with the normalizeHeaders function also used in cache.js

function normalizeHeaders (opts) {

diff --git a/lib/interceptor/dns.js b/lib/interceptor/dns.js
index 38287607..a878f39f 100644
--- a/lib/interceptor/dns.js
+++ b/lib/interceptor/dns.js
@@ -4,6 +4,15 @@ const { lookup } = require('node:dns')
 const DecoratorHandler = require('../handler/decorator-handler')
 const { InvalidArgumentError, InformationalError } = require('../core/errors')
 const maxInt = Math.pow(2, 31) - 1
+const { normalizeHeaders } = require('../util/cache.js')
+
+function addHostHeader (opts, host) {
+  if (Array.isArray(opts.headers)) {
+    return ['host', host, ...opts.headers]
+  }
+  const headers = normalizeHeaders(opts)
+  return { host, ...headers }
+}
 
 class DNSInstance {
   #maxTTL = 0
@@ -411,10 +420,7 @@ module.exports = interceptorOpts => {
           ...origDispatchOpts,
           servername: origin.hostname, // For SNI on TLS
           origin: newOrigin.origin,
-          headers: {
-            host: origin.host,
-            ...origDispatchOpts.headers
-          }
+          headers: addHostHeader(origDispatchOpts, origin.host)
         }
 
         dispatch(

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... that's a good catch; I'd suggest to move the normalize headers to core/utils and implement it within the DNS interceptor first, and we can check if we wanna do the same for the core request in a further issue

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... that's a good catch; I'd suggest to move the normalize headers to core/utils and implement it within the DNS interceptor first, and we can check if we wanna do the same for the core request in a further issue

This makes sense to me

function addHostHeader (headers = {}, host) {
if (Array.isArray(headers)) {
const header = ['host', host]
if (Array.isArray(headers[0])) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really support them? They should throw if request receives a multi-dimensional array.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They do throw for arrays, just not other iterables

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, let's do this: #4498 (comment)

@FelixVaughan FelixVaughan changed the title fix(dns): preserve header shape for [string,string][] and ensure Host fix(dns): preserve header shape and ensure Host Sep 4, 2025
}
return [...header, ...headers]
function addHostHeader (opts, host) {
if (Array.isArray(opts.headers)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The normalizeHeaders will normalize it to a plain JS object, feel free to append it once normalized.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for flat arrays for some reason. Refactored to accept types of [k1, v1, k2, v2, ...]

@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.56%. Comparing base (2a72563) to head (daf988c).
⚠️ Report is 671 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4498      +/-   ##
==========================================
- Coverage   94.17%   93.56%   -0.61%     
==========================================
  Files          90      103      +13     
  Lines       24559    32366    +7807     
==========================================
+ Hits        23128    30283    +7155     
- Misses       1431     2083     +652     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

lib/core/util.js Outdated
if (typeof src[Symbol.iterator] === 'function') {
const msg = 'opts.headers is not a valid header map'
for (const s of src) {
if (!Array.isArray(s)) throw new Error(msg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's an Undici error that is usable, let's make usage of it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants