From c2eafbf2c6572f93cc513cbec35a46931e03a814 Mon Sep 17 00:00:00 2001 From: Florents Tselai Date: Sat, 2 Jan 2021 23:53:50 +0200 Subject: [PATCH] Change section --- README.md | 43 +++++++++++++++++++++---------------------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 89e20af..9ecc5b6 100644 --- a/README.md +++ b/README.md @@ -156,28 +156,6 @@ FROM 302 | http://www.google.ch/?gfe_rd=cr&ei=ACESWLy_KuvI8zeghL64Ag ``` -Using this extension as a background automated process without supervision (e.g as a trigger) may have unintended consequences for other servers. -It is considered a best practice to share contact information with your requests, -so that administrators can reach you in case your HTTP calls get out of control. - -Certain API policies (e.g. [Wikimedia User-Agent policy](https://meta.wikimedia.org/wiki/User-Agent_policy)) may even require sharing specific contact information -with each request. Others may disallow (via `robots.txt`) certain agents they don't recognize. - -For such cases you can set the `CURLOPT_USERAGENT` option - -```sql -SELECT http_set_curlopt('CURLOPT_USERAGENT', - 'Examplebot/2.1 (+http://www.example.com/bot.html) Contact abuse@example.com'); - -SELECT content::json ->> 'user-agent' FROM http_get('http://httpbin.org/user-agent'); -``` -``` - status | user_agent ---------+----------------------------------------------------------- - 200 | Examplebot/2.1 (+http://www.example.com/bot.html) Contact abuse@example.com -``` - - ## Concepts Every HTTP call is a made up of an `http_request` and an `http_response`. @@ -270,6 +248,27 @@ SELECT * FROM http_list_curlopt(); Will set the proxy port option for the lifetime of the database connection. You can reset all CURL options to their defaults using the `http_reset_curlopt()` function. +Using this extension as a background automated process without supervision (e.g as a trigger) may have unintended consequences for other servers. +It is considered a best practice to share contact information with your requests, +so that administrators can reach you in case your HTTP calls get out of control. + +Certain API policies (e.g. [Wikimedia User-Agent policy](https://meta.wikimedia.org/wiki/User-Agent_policy)) may even require sharing specific contact information +with each request. Others may disallow (via `robots.txt`) certain agents they don't recognize. + +For such cases you can set the `CURLOPT_USERAGENT` option + +```sql +SELECT http_set_curlopt('CURLOPT_USERAGENT', + 'Examplebot/2.1 (+http://www.example.com/bot.html) Contact abuse@example.com'); + +SELECT content::json ->> 'user-agent' FROM http_get('http://httpbin.org/user-agent'); +``` +``` + status | user_agent +--------+----------------------------------------------------------- + 200 | Examplebot/2.1 (+http://www.example.com/bot.html) Contact abuse@example.com +``` + ## Keep-Alive & Timeouts *The `http_reset_curlopt()` approach described above is recommended. The global variables below will be deprecated and removed over time.*