Simple Examples Options Header Options Special Options Errors Info Received headers Content Proxies Multi
libcurl is a free and easy-to-use client-side URL transfer
library, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS,
IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP,
SMTP, SMTPS, Telnet and TFTP. libcurl supports SSL certificates,
HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload,
proxies, cookies, user+password authentication (Basic, Digest,
NTLM, Negotiate, Kerberos), file transfer resume, http proxy
tunneling and more!
libcurl is highly portable, it builds and works identically on
numerous platforms, including Solaris, NetBSD, FreeBSD, OpenBSD,
Darwin, HPUX, IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows,
Amiga, OS/2, BeOs, Mac OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell
NetWare, DOS and more...
libcurl is free, thread-safe, IPv6 compatible, feature rich, well
supported, fast, thoroughly documented and is already used by many
known, big and successful companies and numerous applications.
use LibCurl::Easy;
# GET
print LibCurl::Easy.new(URL => 'http://example.com').perform.content;
# GET (download a file)
LibCurl::Easy.new(URL => 'http://example.com/somefile',
download => 'somefile').perform;
# HEAD
say LibCurl::Easy.new(:nobody, URL => 'http://example.com')
.perform.response-code;
# PUT (upload a file)
LibCurl::Easy.new(URL => 'http://example.com/somefile',
upload => 'somefile').perform;
# PUT (content from a string)
LibCurl::Easy.new(URL => 'http://example.com/somefile',
send => 'My Content').perform;
# DELETE
LibCurl::Easy.new(URL => 'http://example.com/file-to-delete',
customrequest => 'DELETE').perform;
# POST
LibCurl::Easy.new(URL => 'http://example.com/form.html',
postfields => 'name=foo&opt=value').perform;
If even those aren't easy enough, there is a tiny sub-class LibCurl::HTTP that adds aliases for the common HTTP methods:
use LibCurl::HTTP;
my $http = LibCurl::HTTP.new;
say $http.GET('http://example.com').perform.content;
say $http.GET('http://example.com', 'myfile').perform.response-code;
say $http.HEAD('http://example.com').perform.response-code;
$http.DELETE('http://example.com').perform;
$http.PUT('http://example.com', 'myfile').perform;
$http.POST('http://example.com/form.html', 'name=foo&opt=value').perform;
LibCurl::HTTP methods also enable failonerror by default, so any
HTTP response >= 400 will throw an error.
You can even import very simple subroutines ala WWW:
use LibCurl::HTTP :subs;
# Just GET content (will return Failure on failure):
say get 'https://httpbin.org/get?foo=42&bar=x';
# GET and decode received data as JSON:
say jget('https://httpbin.org/get?foo=42&bar=x')<args><foo>;
# POST content (query args are OK; pass form as named args)
say post 'https://httpbin.org/post?foo=42&bar=x', :some<form>, :42args;
# And if you need headers, pass them inside a positional Hash:
say post 'https://httpbin.org/post?foo=42&bar=x', %(:Some<Custom-Header>),
:some<form>, :42args;
# Same POST as above + decode response as JSON
say jpost('https://httpbin.org/post', :some<form-arg>)<form><some>;
my $curl = LibCurl::Easy.new(:verbose, :followlocation);
$curl.setopt(URL => 'http://example.com', download => './myfile.html');
$curl.perform;
say $curl.success;
say $curl.Content-Type;
say $curl.Content-Length;
say $curl.Date;
say $curl.response-code;
say $curl.statusline;
Of course the full power of libcurl is available, so you aren't limited to HTTP URLs, you can use FTP, SMTP, TFTP, SCP, SFTP, and many, many more.
Many of the libcurl
options are
available, mostly just skip the CURLOPT_, lowercase, and use '-'
instead of '_'. For example, instead of
CURLOPT_ACCEPT_ENCODING, use accept-encoding. When the
options are really boolean (set to 1 to enable and 0 to disable), you
can treat them like option booleans if you want, :option to
enable, and :!option to disable.
Just like libcurl, the primary option is URL, and can take many forms depending on the desired protocol.
Options can be set on a handle either on initial creation with
new(), or later with .setopt(). As a convenience, you can
also just treat them like normal object methods. These are all
equivalent:
my $curl = LibCurl::Easy.new(:verbose, :followlocation,
URL => 'http://example.com');
$curl.setopt(:verbose, followlocation => 1);
$curl.setopt(URL => 'http://example.com');
$curl.verbose(1);
$curl.followlocation(1);
$curl.URL('http://example.com');
postfields is actually
CURLOPT_COPYPOSTFIELDS so it will always copy the fields.
Some of the normal options have _LARGE versions. LibCurl always maps the option to the _LARGE option where they exist, so you don't have to worry about overflows.
These are the current options (If you want one not in this list, let me know):
CAinfo CApath URL accepttimeout-ms accept-encoding address-scope append autoreferer buffersize certinfo cookie cookiefile cookiejar cookielist customrequest default-protocol dirlistonly failonerror followlocation forbid-reuse fresh-connect ftp-skip-pasv-ip ftp-use-eprt ftp-use-epsv ftpport header http-version httpauth httpget httpproxytunnel ignore-content-length) infilesize localport localportrange low-speed-limit low-speed-time maxconnects maxfilesize maxredirs max-send-speed max-recv-speed netrc nobody noprogress nosignal password path-as-is post postfields postfieldsize protocols proxy proxyauth proxyport proxytype proxyuserpwd proxy-ssl-verifypeer proxy-ssl-verifyhost range redir-protocols referer request-target resume-from ssl-verifyhost ssl-verifypeer tcp-keepalive tcp-keepidle tcp-keepintvl timecondition timeout timeout-ms timevalue unix-socket-path unrestricted-auth use-ssl useragent username userpwd verbose wildcardmatch xoauth2_bearer
In addition to the normal libcurl special options that set headers (useragent, referer, cookie), there are some extra options for headers:
Content-MD5, Content-Type, Content-Length, Host, Accept,
Expect, Transfer-Encoding.
$curl.Host('somewhere.com'); # or $curl.setopt(Host => 'somewhere.com')
$curl.Content-MD5('...'); # or $curl.setopt(Content-MD5 => '...')
You can also add any other headers you like:
$curl.set-header(X-My-Header => 'something', X-something => 'foo');
You can clear a standard header by setting the header to '', or send a header without content by setting the content to ';'
$curl.set-header(Accept => ''); # Don't send normal Accept header
$curl.set-header(Something => ';'); # Send empty header
If you are reusing the handle, you can also clear the set headers:
$curl.clear-header();
This only clears the 'extra' headers, not useragent/referer/cookie.
In addition to the normal libcurl options, Raku LibCurl uses options for some special Raku functionality.
debugfunction replaces the libcurl
CURLOPT_DEBUGFUNCTION
callback, with one that looks like this:
sub debug(LibCurl::Easy $easy, CURL-INFO-TYPE $type, Buf $buf)
{...}
$curl.setopt(debugfunction => &debug);
xferinfo replaces the libcurl
CURLOPT_XFERINFOFUNCTION
(and
CURLOPT_PROGRESSFUNCTION)
with one that looks like this:
sub xferinfo(LibCurl::Easy $easy, $dltotal, $dlnow, $ultotal, $ulnow)
{...}
$curl.setopt(xferinfofunction => &xferinfo);
download specifies a filename to download into.
upload specifies a filename to upload from.
send specifies a Str or a Buf to send content from.
Finally there is a private option which replaces
CURLOPT_PRIVATE, and you can safely store any object in it.
In most circumstances, errors from libcurl functions will result in a thrown X::LibCurl exception. You can catch these with CATCH. You can see the string error, or cast to Int to see the libcurl error code.
For HTTP transfers, you can access the response code with
getinfo('response-code') or just .response-code. You can also
check that the response code is in the 2xx range with .success.
You might find the failonerror option useful to force an error if the HTTP code is equal to or larger than 400. That will cause an exception in those cases.
On an error, you may find extra human readable error
messages
with the .error method.
$curl.perform;
CATCH {
say "Caught!";
when X::LibCurl {
say "$_.Int() : $_";
say $curl.response-code;
say $curl.error;
}
}
After a transfer, you can retrieve internal information about the curl session with the .getinfo method.
You can explicitly request a single field, or a list of fields to get a hash, or just get all the fields as a hash. As in the options, there are also convenience methods for each info field.
say $curl.getinfo('effective-url');
say $curl.getinfo('response-code');
say $curl.getinfo(<effective-url response-code>); # Hash with those keys
say $curl.getinfo; # Hash of all info fields
say $curl.effective-url;
say $curl.response-code;
Fields currently defined are:
appconnect_time certinfo condition-unmet connect-time content-type cookielist effective-url ftp-entry-path header-size http-connectcode httpauth-avail lastsocket local-ip local-port namelookup-time num-connects os-errno pretransfer-time primary-ip primary-port proxyauth-avail redirect-url request-size response-code rtsp-client-cseq rtsp-cseq-recv rtsp-server-cseq rtsp-session-id size-download size-upload speed-download speed-upload ssl-engines total-time
You can retrieve the header fields in several ways as well.
say $curl.receiveheaders<Content-Length>; # Hash of all headers
say $curl.get-header('Content-Length');
say $curl.Content-Length;
If you did not specify the download option to download content
into a file, the content will be stashed in memory in a Buf object you
can access with the .buf() method.
If you understand that the content is decodable as a string, you can
call the .content($encoding = 'utf-8') method which will decode
the content into a Str, by default with the utf-8 encoding if
not specified.
say "Got content", $curl.content;
Forms now uses the libcurl MIME capability, but requires verison 7.56 or greater for forms.
There is a special POST option for multipart/formdata.
my $curl = LibCurl::Easy.new(URL => 'http://...');
# normal field
$curl.formadd(name => 'fieldname', contents => 'something');
# Use a file as content, but not as a file upload
$curl.formadd(name => 'fieldname', filecontent => 'afile.txt');
# upload a file from disk, give optional filename or contenttype
$curl.formadd(name => 'fieldname', file => 'afile.txt',
filename => 'alternate.name.txt',
contenttype => 'image/jpeg');
# Send a Blob of contents, but as a file with a filename
$curl.formadd(name => 'fieldname', filename => 'some.file.name.txt',
contents => "something".encode);
$curl.perform;
This will automatically cause LibCurl to POST the data.
libcurl has great proxy support, and you should be able to specify anything needed as options to LibCurl to use them. The easiest for most common cases is to just set the proxy option.
By default, libcurl will also respect the environment variables
http_proxy, ftp_proxy, all_proxy, etc. if any of those are
set. Setting the proxy string to "" (an empty string) will
explicitly disable the use of a proxy, even if there is an environment
variable set for it.
A proxy host string can also include protocol scheme (http://) and
embedded user + password.
LibCurl can be used to communicate with a unix socket by setting the
unix-socket-path option. You must still specify a host, but it is
ignored. For example, you could use the docker REST
API like this:
use LibCurl::Easy;
use JSON::Fast;
my $docker = LibCurl::Easy.new(unix-socket-path => '/var/run/docker.sock');
my $info = from-json($docker.URL("http://localhost/info").perform.content);
say $info<KernelVersion>;
say $info<OperatingSystem>;
There is an experimental stream capability which can bind a Channel to the data stream. Instead of retrieving the buffered data at once, each time data comes in it is sent through to the Channel. This is useful for long-lived connections that periodically send more data.
You can access it with Something like this:
my $curl = LibCurl::Easy.new(URL => ...);
my $stream = $curl.stream-out;
start react whenever $stream -> $in {
say "in: ", $in;
}
$curl.perform;
Suggestions for improving the interface for this capability welcome!
Raku LibCurl also supports the libcurl
multi interface.
You still have to construct LibCurl::Easy (or LibCurl::HTTP)
handles for each transfer, but instead of calling .perform, just add
the handle to a LibCurl::Multi.
use LibCurl::Easy;
use LibCurl::Multi;
my $curl1 = LibCurl::Easy.new(:verbose, :followlocation,
URL => 'http://example.com',
download => './myfile1.html');
my $curl2 = LibCurl::Easy.new(:verbose, :followlocation,
URL => 'http://example.com',
download => './myfile2.html');
my $multi = LibCurl:Multi.new;
$multi.add-handle($curl1);
$multi.add-handle($curl2);
$multi.perform;
say $curl1.statusline;
say $curl2.statusline;
You can also use an asynchronous callback to get a notification when
an individual transfer has completed. The callback takes place in the
same thread with all the transfers, so it should complete quickly (or
start a new thread for heavy lifting as needed). You can add
additional handles to the LibCurl::Multi at any time, even re-using
completed LibCurl::Easy handles (after setting URL, etc. as needed).
use LibCurl::Easy;
use LibCurl::Multi;
my $curl1 = LibCurl::Easy.new(:followlocation,
URL => 'http://example.com',
download => 'myfile1.html');
my $curl2 = LibCurl::Easy.new(:followlocation,
URL => 'http://example.com',
download => 'myfile2.html');
sub callback(LibCurl::Easy $easy, Exception $e)
{
die $e if $e;
say $easy.response-code;
say $easy.statusline;
}
my $multi = LibCurl::Multi.new(callback => &callback);
$multi.add-handle($curl1, $curl2);
$multi.perform;
LibCurl depends on libcurl, so
you must install that prior to installing this module. It may already
be installed on your system.
- For debian or ubuntu:
apt-get install libcurl4-openssl-dev - For alpine:
apk add libcurl - For CentOS:
yum install libcurl
There is another Raku interface to libcurl Net::Curl developed by Ahmad M. Zawawi. If you already use it and it works well for you, great, keep using it. Ahmad did a nice job developing it. I would encourage you to also take a look at this module. LibCurl provides a more 'rakuish' OO interface to libcurl than Net::Curl, and wraps things in a manner to make using it a little easier (IMHO).
Copyright © 2017 United States Government as represented by the Administrator of the National Aeronautics and Space Administration. No copyright is claimed in the United States under Title 17, U.S.Code. All Other Rights Reserved.
See NASA Open Source Agreement for more details.