Parallel requests

Making parallel HTTP requests

Simple parallelism

When you initialize a new Faraday object, make sure to declare a proper adapter that supports parallel requests.

require 'typhoeus'

response1, response2 = nil

conn = Faraday.new(:url => "http://coolness.com") do |faraday|
  faraday.adapter Faraday::Adapter::Typhoeus
end

conn.in_parallel do
  resp1 = conn.get('/one')
  resp2 = conn.get('/two')

  # these will return nil here since the
  # requests haven't been completed
  resp1.body
  resp2.body
end

# at this point the response information you expected should be fully available to you.
resp1.body # resp1.status, etc
resp2.body

All of the response data will be nil (resp.body, resp.status, etc.) until all of the requests in parallel are completed. Which means you'll want to wait until you are outside the block to handle any of the responses.

You'll also want to be aware of failed requests. Since they happen in parallel, you should check each response and make sure they went through successfully.

responses = []

conn.in_parallel do
  responses << conn.get('/one')
  responses << conn.get('/one')
end

responses.each { |r| p r.status }

Advanced use

Using the manager directly.

manager = Typhoeus::Hydra.new(:max_concurrency => 10) # (200 is default)
#manager.disable_memoization

conn.in_parallel(manager) do ...

Explicitly setting the default parallel manager

conn = Faraday.new :parallel_manager => manager

More Information

• https://gist.github.com/2272636
• https://github.com/typhoeus/typhoeus