feat(httpclient): reuse httpclient #1409
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| package util | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "net" | ||
| "net/http" | ||
| "net/url" | ||
| "sync" | ||
| "time" | ||
|
|
||
| "one-api/common" | ||
|
|
||
| "golang.org/x/net/proxy" | ||
| ) | ||
|
|
||
| const ( | ||
| KeyOfMaxIdleConnsPerHost = "MAX_IDLE_CONNS_PER_HOST" | ||
| KeyOfMaxIdleConns = "MAX_IDLE_CONNS" | ||
| KeyOfIdleConnTimeout = "IDLE_CONN_TIMEOUT" | ||
| ) | ||
|
|
||
| var ( | ||
| innerHttpClients sync.Map | ||
|
|
||
| defaultMaxIdleConnsPerHost = 10 | ||
| defaultMaxIdleConns = 100 | ||
| defaultIdleConnTimeout = 60 | ||
| ) | ||
|
|
||
| func GetHttpClient(proxyURL string) (*http.Client, error) { | ||
| var err error | ||
| var parsedURL *url.URL | ||
| if proxyURL != "" { | ||
| parsedURL, err = url.Parse(proxyURL) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| } | ||
|
|
||
| if client, ok := innerHttpClients.Load(proxyURL); ok { | ||
| return client.(*http.Client), nil | ||
| } | ||
|
|
||
| var maxIdleConns = common.GetEnvOrDefault(KeyOfMaxIdleConns, defaultMaxIdleConns) | ||
| var maxIdleConnsPerHost = common.GetEnvOrDefault(KeyOfMaxIdleConnsPerHost, defaultMaxIdleConnsPerHost) | ||
| var idleConnTimeout = time.Duration(common.GetEnvOrDefault(KeyOfIdleConnTimeout, defaultIdleConnTimeout)) * time.Second | ||
|
|
||
| var transport *http.Transport | ||
| if proxyURL != "" { | ||
| switch parsedURL.Scheme { | ||
| case "http", "https": | ||
| transport = &http.Transport{ | ||
| Proxy: http.ProxyURL(parsedURL), | ||
| MaxIdleConns: maxIdleConns, | ||
| MaxIdleConnsPerHost: maxIdleConnsPerHost, | ||
| IdleConnTimeout: idleConnTimeout, | ||
| } | ||
|
|
||
| case "socks5", "socks5h": | ||
| // 获取认证信息 | ||
| var auth *proxy.Auth | ||
| if parsedURL.User != nil { | ||
| auth = &proxy.Auth{ | ||
| User: parsedURL.User.Username(), | ||
| Password: "", | ||
| } | ||
| if password, ok := parsedURL.User.Password(); ok { | ||
| auth.Password = password | ||
| } | ||
| } | ||
|
|
||
| // 创建 SOCKS5 代理拨号器 | ||
| // proxy.SOCKS5 使用 tcp 参数，所有 TCP 连接包括 DNS 查询都将通过代理进行。行为与 socks5h 相同 | ||
| dialer, err := proxy.SOCKS5("tcp", parsedURL.Host, auth, proxy.Direct) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| transport = &http.Transport{ | ||
| DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) { | ||
| return dialer.Dial(network, addr) | ||
| }, | ||
| MaxIdleConns: maxIdleConns, | ||
| MaxIdleConnsPerHost: maxIdleConnsPerHost, | ||
| IdleConnTimeout: idleConnTimeout, | ||
| } | ||
| default: | ||
| return nil, fmt.Errorf("unsupported proxy scheme: %s", parsedURL.Scheme) | ||
| } | ||
| } else { | ||
| transport = &http.Transport{ | ||
| MaxIdleConns: maxIdleConns, | ||
| MaxIdleConnsPerHost: maxIdleConnsPerHost, | ||
| IdleConnTimeout: idleConnTimeout, | ||
| } | ||
| } | ||
|
|
||
| var tmpClient *http.Client | ||
| if common.RelayTimeout > 0 { | ||
| tmpClient = &http.Client{ | ||
| Transport: transport, | ||
| Timeout: time.Duration(common.RelayTimeout) * time.Second, | ||
| } | ||
| } else { | ||
| tmpClient = &http.Client{ | ||
| Transport: transport, | ||
| } | ||
| } | ||
|
|
||
| innerHttpClients.Store(proxyURL, tmpClient) | ||
| return tmpClient, nil | ||
|
Comment on lines
+111
to
+112
There was a problem hiding this comment. 🛠️ Refactor suggestion Consider memory management for the client cache. The current implementation stores clients indefinitely in the sync.Map, which could lead to memory leaks if many different proxy URLs are used over time. Consider implementing a cleanup mechanism or LRU eviction. Potential solutions:
Do you want me to generate an implementation for any of these approaches? 🤖 Prompt for AI Agents |
||
| } | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
🧩 Analysis chain
Verify SOCKS5 proxy implementation behavior.
The comment on lines 74-75 states that proxy.SOCKS5 with "tcp" makes all connections including DNS queries go through the proxy (socks5h behavior), but this needs verification as the distinction between socks5 and socks5h schemes might not be properly handled.
🌐 Web query:
💡 Result:
The key difference between SOCKS5 and SOCKS5H proxy schemes centers on how DNS resolution is performed when making a proxied connection:
SOCKS5: The client (your local machine or application) performs DNS resolution. When you connect through a SOCKS5 proxy and provide a hostname, your client resolves that hostname to an IP address before sending the connection request to the proxy. The proxy then establishes a connection to the resolved IP. This method may expose your DNS queries locally, potentially leaking your intended targets[1][7].
SOCKS5H: The 'h' stands for hostname resolution by the proxy. When using SOCKS5H, your client sends the hostname (not the resolved IP address) to the proxy, which then performs the DNS lookup on your behalf and connects to the target host. This approach helps prevent DNS leaks and can be important for privacy and accessing geo-restricted content[1][7].
Here’s a table summarizing the difference:
How does
golang.org/x/net/proxyhandle each?golang.org/x/net/proxypackage only supports thesocks5://(SOCKS5) scheme and does not explicitly support thesocks5h://(SOCKS5H) scheme. When you provide a hostname, it is resolved locally before dialing through the proxy—consistent with standard SOCKS5 behavior[6].proxy.SOCKS5()function and pass a hostname, the Go DNS resolver is used, so DNS requests go out locally, not through the proxy[3][2].socks5h://so that hostname resolution can happen at the proxy. As of now, the package does not offer this functionality[6].Summary:
golang.org/x/net/proxydoes not currently support SOCKS5H: DNS will always be resolved locally by the client, regardless of the scheme[2][3][6][7].Citations:
Correct SOCKS5 proxy DNS behavior comment
The current comment implying that proxy.SOCKS5 over “tcp” will proxy DNS lookups (i.e. socks5h behavior) is inaccurate. The
golang.org/x/net/proxypackage only implements standard SOCKS5, which always resolves hostnames locally before dialing. It does not support remote (proxy-side) DNS resolution (socks5h).Please update the comment in
util/http_client.go(around lines 74–75) accordingly. For example:If you need DNS resolution by the proxy, you’ll have to implement or use a library that explicitly supports SOCKS5H.
📝 Committable suggestion
🤖 Prompt for AI Agents