using proxy server instead of normal HTTP server

[mreinsch]

I'd like to start a discussion with this pull request. It's still a bit rough, please bear with me.

This introduces a proxy server implementation (based on WEBrick) and configures the browser to use that proxy server. Thus the proxy server now receives any requests made by the browser. The proxy server internally uses the rack test framework to invoke the app to test. This allows us to easily test apps which use multiple domains.

Right now the proxy server forwards all requests to the web app (similar to the RackTest adapter), but it'd be relatively easy to extend this to actually forward some requests to remote servers or use net mocking frameworks to mock requests.

Besides the selenium proxy driver included in the pull request, I also created one for Poltergeist:

class Capybara::Poltergeist::ProxyDriver < Capybara::Poltergeist::Driver
  def initialize(app, opts={})
    phantomjs_options = opts.delete(:phantomjs_options) || []
    phantomjs_options.push(
      "--proxy=http://#{Capybara.server_host}:#{Capybara.server_port}",
      "--proxy-type=http",
      "--ignore-ssl-errors=true")
    super(app, opts.merge(phantomjs_options: phantomjs_options))
  end
  def needs_server?
    false
  end
  def needs_proxy?
    true
  end
end

Register those drivers like this:

Capybara.register_driver :poltergeist_proxy do |app|
  Capybara::Poltergeist::ProxyDriver.new(app)
end

Capybara.register_driver :selenium_proxy do |app|
  Capybara::Selenium::ProxyDriver.new(app)
end

One limitation, we need to configure the proxy driver's host and port to make sure they are available, for instance:

Capybara.configure do |config|
  config.javascript_driver = :selenium_proxy
  config.server_host = "127.0.0.1"
  config.server_port = 23798
end

I'd love to hear your comments on how we can get this into capybara, we need this mainly to test our multi-domain app. Thanks.

[abotalov]

Have you looked at https://github.com/jarib/browsermob-proxy-rb?

Also I haven't experienced any issues with testing multi-domain apps when I used Capybara + Selenium.

[jnicklas]

As an idea, I really like this. It would be very nice to have a simple way to mock out specific URLs via the proxy, it seems that this would be a fluent candidate for creating something like that, or even making it possible to use existing mocking frameworks like WebMock in the client. Interesting stuff, the idea of using a proxy is neat.

Implementation wise, I think this needs some work though, but I realize you just wanted to throw this out there. I'm not too happy about all the SSL stuff in there, and I also don't like that it comes with a preconfigured list of hosts it ignores. There's also a lot of copy-paste with Capybara::Server going on. I'm guessing you know all of that though, so let's work on making this into something we can merge.

I'd also liket to discuss the needs_proxy? method. Somehow this seems a bit strange to me. Some drivers will be capable of providing a proxy setting and some won't (terminus comes to mind). It seems more in line that some drivers are capable of using a proxy and some aren't, so maybe we should somehow have an option to enable the proxy, and a way for Capybara to instruct the driver to use a certain proxy server. What do you think?

[mreinsch]

Thanks for the feedback and sorry for the long delay!

I finally got some time to revise the pull request based on your feedback. The drivers now have a supports_proxy_protocol? method, which together with Capybara.use_proxy_protocol determines whether to use http or proxy. If proxy protocol is chosen, we'll tell the driver which host/port we're using, so it can start the browser with the correct settings.

So to use it, you only need the following

Capybara.configure do |config|
  config.use_proxy_protocol = true
  config.javascript_driver = :selenium
end

For poltergeist, the following patch works fine for me:

class Capybara::Poltergeist::Driver
  def supports_proxy_protocol?
    true
  end

  def setup_proxy_host(host, port)
    opts = (options[:phantomjs_options] ||= [])
    opts.delete_if {|o| o =~ /^--proxy/ }
    opts.push("--proxy=http://#{host}:#{port}", "--proxy-type=http")
    opts.push("--ignore-ssl-errors=true") unless opts.include?("--ignore-ssl-errors=true")
  end
end

I still need to make the list of ignored hosts configurable - idea there is to allow everyone to hook in their own code, so we can hopefully see stubs for some of the common sites people integrate with...

Anyway, please let me know if that is moving into a direction you'd be happy with. Thanks!

[mreinsch]

Regarding the SSL stuff, I'd prefer to not have that in there either, but it's required to test any SSL sites. The browsers use the CONNECT mechanism to tunnel the raw SSL stream through the proxy...

[twalpole]

selenium supports more than just Firefox - how will this work if the user is using selenium with chrome, ie, etc?

[mreinsch]

good question, I suppose all browsers will somehow support setting up a proxy. If they don't, we can have the supports_proxy_protocol return false for now.

[mreinsch]

I've completed the configuration, so it's now possible to plug in different rack apps for different hosts to mock out third party services - or to deliver 404 errors in case you just don't want to bother (like for google analytics).

[mateoqac]

Still a planning feature?