[FEATURES/IDEAS/TESTING] Collection of ideas resulting from testing the dev branch

[ppenguin]

What's needed and why?

Preamble

The intent of this issue is to collect recent experiences with trying to use v1.6.x-dev and ideas for solutions to issues.

Background

a deployment under nomad where (for resource reasons) the BW instance was moved to a separate nomad node (on an arm64 VPS) and all upstream services run on another nomad node (amd64). There are services that need to be contacted both via http (normal WAF function for BW) and with TCP/UDP (via nginx stream functionality of BW).

Some observations

For now just a brain dump to not forget the topics:

1. USE_UDP should not be exclusive, we need also USE_TCP
2. we should allow multiple STREAM_PORTS (so both multi-site and multiple)
3. it is unintuitive to have to specify a REVERSE_PROXY_HOST if there is no http endpoint but only a stream. In fact, we should have LISTEN_STREAM_HOST, because typically we prefix the upstream for a proxy with http:// and for a stream it would be a bare IP or hostname. (The suitability of the term LISTEN in that context is debatable, because at least for the host we mean the upstream (forward)).

... to be updated ...

Related suspected bugs

BUG: my.server.com_LISTEN_STREAM_PORT is overwritten in config

I observed that the content of /etc/nginx/my.server.com/server-stream.conf had

listen 0.0.0.0:7777;

in it just after start (which was the value I configuered with LISTEN_STREAM_PORT, but a few seconds later it was reset to another port number?!!!

Ah, confirmed: this appears to be a bug in restoring config data from the cache or database, i.e. if there was an old value the new value doesn't overwrite it as expected.

BUG: http and stream forwarding doesn't work simultaneously

Related to observation (3) above: http doesn't work without http:// prefix in REVERSE_PROXY_HOST, but stream doesn't work with it. So we need either a dedicated STREAM_HOST or processing of the value, or process the proxy address. The former is much better, because it is necessary if we want to use the same vhost name for different upstreams for http and other backends.

Implementations ideas (optional)

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[TheophileDiot]

Hi, Thank you for opening this feature request ! 🎉
indeed the STREAM_HOST would be more intuitive.
Could you develop a bit more on your use case for multiple STREAM_PORTS ? Thank you
For the suspected bug, I'll have a look into it and let you know !

[ppenguin]

Could you develop a bit more on your use case for multiple STREAM_PORTS ?

In many cases where the infrastructure is not necessarily built on microservices, it could be pretty normal to expect that, say an user app or program, would want to connect to a non-http services.ourcorp.com:<port>, where <port> could be multiple ports, of which any should support specific configuration of any of the possibilities nginx offers (so at least tcp, udp or both). Since the preread selects based on vhost, it is an unnecessary limitation to not offer multiple ports configuration.

The alternative is highly unattractive, since it would require adding a new vhost to DNS, LetsEncrypt, the user app, etc, for each different non-http port that is exposed by the "functional unit" (e.g. a single container or task) behind BW.