[BUG] Scripting: onRequest return drops Content-Encoding header with custom value

[y-shih]

Environment

• Proxyman Version: 6.3.0
• macOS Version: 26.2 (25C56)
• Script Type: Scripting Tool

Problem

When onRequest returns the request object (even without any modification to headers), Proxyman drops the Content-Encoding header from the actual HTTP request sent to the server.

Our server uses a custom Content-Encoding value (not gzip/deflate/br) to indicate a proprietary body encoding. After enabling Scripting, this header disappears.

Steps to Reproduce

1. Create a Scripting rule matching any URL
2. Use the following minimal script:

function onRequest(context, url, request) {
    console.log("INPUT Content-Encoding:", request.headers["Content-Encoding"]);
    request.customPreviewerTabs = { "Test": JSON.stringify({ ok: true }) };
    console.log("OUTPUT Content-Encoding:", request.headers["Content-Encoding"]);
    return request;
}

3. Send a request that originally contains Content-Encoding: my_custom_value
4. Observe:
   • Console shows both INPUT and OUTPUT have Content-Encoding: my_custom_value
   • But the actual Request Headers in Proxyman UI (and sent to server) do NOT contain Content-Encoding

Expected Behavior

Returning request from onRequest should preserve all original headers, including Content-Encoding with custom values.

Actual Behavior

Content-Encoding header is dropped after onRequest returns, even though:

• It exists in request.headers when received
• It still exists in request.headers when returned
• The script does NOT modify request.headers

Evidence (Console Log)

[REQ] INPUT headers: Host, Content-Type, Connection, Accept, Content-Encoding, ...
[REQ] INPUT Content-Encoding: my_custom_value
[REQ] OUTPUT headers: Host, Content-Type, Connection, Accept, Content-Encoding, ...
[REQ] OUTPUT Content-Encoding: my_custom_value

But the actual HTTP request sent to server has no Content-Encoding header.

Impact

Server cannot detect the custom encoding and returns parse error.

Workaround Attempted

• Copying headers with spread operator {...request, headers: {...request.headers}} — same result
• Explicitly re-setting request.headers["Content-Encoding"] = originalValue — same result

None of these preserve the header in the actual outgoing request.

[NghiaTranUIT]

the problem is Proxyman removes the Accept-Encoding when constructing the Request after the scripting is executed. We do this way because we don't want to encode the body again.

Can you explain why you need to not remove the Content-Encoding? I though your server should detect it, if the Content-Encoding doesn't exist, your server should not decode the body

[y-shih]

Thanks for the quick reply.
In our case, we are not decoding nor modifying the request body at all. The script only adds customPreviewerTabs for UI display, and returns the original request object. request.headers["Content-Encoding"] is present both before and after our script, but Proxyman still drops Content-Encoding in the actual outgoing request.
So it seems this is not related to body decoding, but rather to a request re-build / header normalization path triggered by returning request from onRequest, which removes Content-Encoding when it has a non-standard value.
Could you confirm:

• Is Content-Encoding treated as a reserved/filtered header in Scripting?
• Is there any way to preserve a custom Content-Encoding value when returning a request?
• If not, would you consider this a bug in the scripting pipeline?

I can provide a minimal reproducible script and an endpoint that echoes received headers if helpful.

[sionnx]

the problem is Proxyman removes the Accept-Encoding when constructing the Request after the scripting is executed. We do this way because we don't want to encode the body again.

Can you explain why you need to not remove the Content-Encoding? I though your server should detect it, if the Content-Encoding doesn't exist, your server should not decode the body

you remove Content-Encoding on onResponse, right? Why?

When I capture traffic for some third‑party software, as soon as I enable “Run Script on onResponse”, the app’s requests start failing — even if I use the following no‑op script:

async function onResponse(context, url, request, response) {
  return response;
}

As soon as I turn off “Run Script on onResponse”, everything works again.

“I thought your server should detect it”

This is exactly my concern: is Proxyman designed only for debugging servers / clients you control yourself?

From my point of view, for a professional HTTP debugging/proxy tool, you should never silently change headers like Content-Encoding for arbitrary traffic. Apart from a few special cases like Content-Length (which obviously must be adjusted after modification), all other headers should be preserved exactly as they are. Silently dropping Content-Encoding breaks real‑world apps in ways that are extremely hard to diagnose.

[NghiaTranUIT]

I will add logic to perform the Body Compress depending on the Content-Encoding soon 👍

Sorry for this, it's because at the time we introduced the Scripting, it was hard to implement it, so we decided to drop the Content-Encoding. Most of the time it works, because modern networking libraries will auto do it (Not try decoding the body if there is no Content-Encoding header in the Request/Response)

[NghiaTranUIT]

@sion-codin @y-shih Here is the Beta build: https://download.proxyman.com/beta/Proxyman_6.3.0_Fix_scripting_remove_Content_Encoding.dmg

• Keep the Content Encoding, we don't remove it when using the Scripting Tool
• Perform the body compression if need
• If it's a custom encoding lib, Proxyman won't remove the header and the body

Please try and let me know your result 👍

[sionnx]

@NghiaTranUIT Thanks for your work — I can now confirm that the Content-Encoding header is preserved in the response headers.

However, when I enable “Run Script on onResponse”, the target app still fails to access the server, even with a no-op script.

By comparing the response headers before and after enabling the script, I found two additional changes:

1. The Transfer-Encoding header disappears.
2. A new Content-Length header is added.

My suggestions:

• Please keep the original Transfer-Encoding header as-is.
• If the original response does not contain a Content-Length header, Proxyman should not add one automatically.

My guess is that when scripting is enabled, Proxyman is effectively changing the response from Transfer-Encoding: chunked to an identity (non‑chunked) form, and this mismatch causes the client to fail when reading the response.

I’ve attached a screenshot of the header differences for reference.

[NghiaTranUIT]

the target app still fails to access the server, even with a no-op script.

I'm not sure why your networking library doesn't support it when the Transfer-Encoding is omitted. Do you write your own networking stack 🤔 Most modern network library handles this issue perfectly

I suppose I can't add the Transfer-Encoding back because it's a break-change 😿

[NghiaTranUIT]

@y-shih does the Beta build fix your problem 🤔

[sionnx]

Thanks for the quick reply!

Do you write your own networking stack 🤔 Most modern network library handles this issue perfectly

I’m not sure —This is a music software, not written by me. I’ll keep investigating what exactly is causing the problem. But in general, preserving the original ‎Transfer-Encoding is always the safest option.

I suppose I can't add the Transfer-Encoding back because it's a break-change

Why would that be a breaking change 🤔 From my perspective, restoring the original ‎Transfer-Encoding should not negatively affect existing users

[y-shih]

Update: Beta version partially fixes the issue, but triggers another bug

I tested Proxyman 6.4.0 Beta and can confirm that:

Content-Encoding header is now preserved - requests with custom Content-Encoding: tap_cryptor are correctly sent to our server, and we receive valid responses.

However, I encountered a new issue when using the custom Content-Encoding with binary/encrypted body:

New Bug: Clicking "Raw" tab causes Proxyman to hang/freeze

Steps to reproduce:

1. Intercept a request with custom Content-Encoding and encrypted binary body
2. Click the "Raw" tab in the Request panel
3. Proxyman becomes unresponsive for 20+ seconds (hang, not crash)

Environment:

• Proxyman Beta: 6.3.0 (60300)
• macOS: 26.2 (Build 25C56)
• Architecture: arm64

Hang Report Summary:

• Event: hang (21.76 seconds)
• The main thread is stuck in NSTextView layout code:
  • -[NSTextView setNeedsDisplayInRect:avoidAdditionalLayout:]
  • _NSFastFillAllLayoutHolesForGlyphRange
  • -[NSLayoutManager(NSPrivate) _recalculateUsageForTextContainerAtIndex:]

This appears to be caused by attempting to render large binary/encrypted data in a text view.

Hang report (key stack trace):

proxyman_request_original_tab_crash.log.zip