Ruby bindings to RE2, a "fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python".
Current version: 2.15.0.rc1
Bundled RE2 version: libre2.11 (2024-07-02)
RE2('h.*o').full_match?("hello") #=> true
RE2('e').full_match?("hello") #=> false
RE2('h.*o').partial_match?("hello") #=> true
RE2('e').partial_match?("hello") #=> true
RE2('(\w+):(\d+)').full_match("ruby:1234")
#=> #<RE2::MatchData "ruby:1234" 1:"ruby" 2:"1234">While recent versions of Ruby have improved defences against regular expression denial of service (ReDoS) attacks, it is still possible for users to craft malicious patterns that take a long time to process by using syntactic features such as back-references, lookaheads and possessive quantifiers. RE2 aims to eliminate ReDoS by design:
Safety is RE2's raison d'être.
RE2 was designed and implemented with an explicit goal of being able to handle regular expressions from untrusted users without risk. One of its primary guarantees is that the match time is linear in the length of the input string. It was also written with production concerns in mind: the parser, the compiler and the execution engines limit their memory usage by working within a configurable budget – failing gracefully when exhausted – and they avoid stack overflow by eschewing recursion.
— Why RE2?
Install re2 as a dependency:
# In your Gemfile
gem "re2"
# Or without Bundler
gem install re2Include in your code:
require "re2"Full API documentation automatically generated from the latest version is available at https://mudge.name/re2/.
While re2 uses the same naming scheme as Ruby's built-in regular expression
library (with Regexp and
MatchData), its API is slightly
different:
Warning
RE2's regular expression syntax differs from PCRE and Ruby's built-in
Regexp library, see the
official syntax page for more
details.
The core class is RE2::Regexp which
takes a regular expression as a string and compiles it internally into an RE2
object. A global function RE2 is available to concisely compile a new
RE2::Regexp:
re = RE2('(\w+):(\d+)')
#=> #<RE2::Regexp /(\w+):(\d+)/>
re.ok? #=> true
re = RE2('abc)def')
re.ok? #=> false
re.error #=> "missing ): abc(def"Tip
Note the use of single quotes when passing the regular expression as
a string to RE2 so that the backslashes aren't interpreted as escapes.
When compiling a regular expression, an optional second argument can be used to change RE2's default options, e.g. stop logging syntax and execution errors to stderr with log_errors:
RE2('abc)def', log_errors: false)See the API documentation for RE2::Regexp#initialize for all the available options.
There are two main methods for matching: RE2::Regexp#full_match? requires the regular expression to match the entire input text, and RE2::Regexp#partial_match? looks for a match for a substring of the input text, returning a boolean to indicate whether a match was successful or not.
RE2('h.*o').full_match?("hello") #=> true
RE2('e').full_match?("hello") #=> false
RE2('h.*o').partial_match?("hello") #=> true
RE2('e').partial_match?("hello") #=> trueTip
Only extract the number of submatches you need as performance is improved with fewer submatches (with the best performance when avoiding submatch extraction altogether).
Both matching methods have a second form that can extract submatches as RE2::MatchData objects: RE2::Regexp#full_match and RE2::Regexp#partial_match.
m = RE2('(\w+):(\d+)').full_match("ruby:1234")
#=> #<RE2::MatchData "ruby:1234" 1:"ruby" 2:"1234">
m[0] #=> "ruby:1234"
m[1] #=> "ruby"
m[2] #=> "1234"
m = RE2('(\w+):(\d+)').full_match("r")
#=> nilRE2::MatchData supports retrieving submatches by numeric index or by name if present in the regular expression:
m = RE2('(?P<word>\w+):(?P<number>\d+)').full_match("ruby:1234")
#=> #<RE2::MatchData "ruby:1234" 1:"ruby" 2:"1234">
m["word"] #=> "ruby"
m["number"] #=> "1234"They can also be used with Ruby's pattern matching:
case RE2('(\w+):(\d+)').full_match("ruby:1234")
in [word, number]
puts "Word: #{word}, Number: #{number}"
else
puts "No match"
end
# Word: ruby, Number: 1234
case RE2('(?P<word>\w+):(?P<number>\d+)').full_match("ruby:1234")
in word:, number:
puts "Word: #{word}, Number: #{number}"
else
puts "No match"
end
# Word: ruby, Number: 1234By default, both full_match and partial_match will extract all submatches into the RE2::MatchData based on the number of capturing groups in the regular expression. This can be changed by passing an optional second argument when matching:
m = RE2('(\w+):(\d+)').full_match("ruby:1234", submatches: 1)
=> #<RE2::MatchData "ruby:1234" 1:"ruby">Warning
If the regular expression has no capturing groups or you pass submatches: 0, the matching method will behave like its full_match? or
partial_match? form and only return true or false rather than
RE2::MatchData.
If you want to repeatedly match regular expressions from the start of some input text, you can use RE2::Regexp#scan to return an Enumerable RE2::Scanner object which will lazily consume matches as you iterate over it:
scanner = RE2('(\w+)').scan(" one two three 4")
scanner.each do |match|
puts match.inspect
end
# ["one"]
# ["two"]
# ["three"]
# ["4"]RE2::Set represents a collection of
regular expressions that can be searched for simultaneously. Calling
RE2::Set#add with
a regular expression will return the integer index at which it is stored within
the set. After all patterns have been added, the set can be compiled using
RE2::Set#compile,
and then
RE2::Set#match
will return an array containing the indices of all the patterns that matched.
set = RE2::Set.new
set.add("abc") #=> 0
set.add("def") #=> 1
set.add("ghi") #=> 2
set.compile #=> true
set.match("abcdefghi") #=> [0, 1, 2]
set.match("ghidefabc") #=> [2, 1, 0]Warning
Note RE2 only supports UTF-8 and ISO-8859-1 encoding so strings will be
returned in UTF-8 by default or ISO-8859-1 if the :utf8 option for the
RE2::Regexp is set to false (any other encoding's behaviour is undefined).
For backward compatibility: re2 won't automatically convert string inputs to the right encoding so this is the responsibility of the caller, e.g.
# By default, RE2 will process patterns and text as UTF-8
RE2(non_utf8_pattern.encode("UTF-8")).match(non_utf8_text.encode("UTF-8"))
# If the :utf8 option is false, RE2 will process patterns and text as ISO-8859-1
RE2(non_latin1_pattern.encode("ISO-8859-1"), utf8: false).match(non_latin1_text.encode("ISO-8859-1"))This gem requires the following to run:
- Ruby 3.1 to 3.4.0-rc1
It supports the following RE2 ABI versions:
- libre2.0 (prior to release 2020-03-02) to libre2.11 (2023-07-01 to 2024-07-02)
Where possible, a pre-compiled native gem will be provided for the following platforms:
- Linux
- macOS
x86_64-darwinandarm64-darwin - Windows
x64-mingw-ucrt
SHA256 checksums are included in the release notes for each version and can be checked with sha256sum, e.g.
$ gem fetch re2 -v 2.14.0
Fetching re2-2.14.0-arm64-darwin.gem
Downloaded re2-2.14.0-arm64-darwin
$ sha256sum re2-2.14.0-arm64-darwin.gem
3c922d54a44ac88499f6391bc2f9740559381deaf7f4e49eef5634cf32efc2ce re2-2.14.0-arm64-darwin.gemGPG signatures are attached to each release (the assets ending in .sig) and can be verified if you import our signing key 0x39AC3530070E0F75 (or fetch it from a public keyserver, e.g. gpg --keyserver keyserver.ubuntu.com --recv-key 0x39AC3530070E0F75):
$ gpg --verify re2-2.14.0-arm64-darwin.gem.sig re2-2.14.0-arm64-darwin.gem
gpg: Signature made Fri 2 Aug 12:39:12 2024 BST
gpg: using RSA key 702609D9C790F45B577D7BEC39AC3530070E0F75
gpg: Good signature from "Paul Mucur <[email protected]>" [unknown]
gpg: aka "Paul Mucur <[email protected]>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7026 09D9 C790 F45B 577D 7BEC 39AC 3530 070E 0F75The fingerprint should be as shown above or you can independently verify it with the ones shown in the footer of https://mudge.name.
Warning
We strongly recommend using the native gems where possible to avoid the need for compiling the C++ extension and its dependencies which will take longer and be less reliable.
If you wish to compile the gem, you will need to explicitly install the ruby platform gem:
# In your Gemfile with Bundler 2.3.18+
gem "re2", force_ruby_platform: true
# With Bundler 2.1+
bundle config set force_ruby_platform true
# With older versions of Bundler
bundle config force_ruby_platform true
# Without Bundler
gem install re2 --platform=rubyYou will need a full compiler toolchain for compiling Ruby C extensions (see
Nokogiri's "The Compiler
Toolchain")
plus the toolchain required for compiling the vendored version of RE2 and its
dependency Abseil which includes CMake, a compiler
with C++14 support such as clang 3.4 or
gcc 5 and a recent version of
pkg-config. On
Windows, you'll also need pkgconf 2.1.0+ to avoid undefined reference
errors when attempting to
compile Abseil.
If you already have RE2 installed, you can instruct the gem not to use its own vendored version:
gem install re2 --platform=ruby -- --enable-system-libraries
# If RE2 is not installed in /usr/local, /usr, or /opt/homebrew:
gem install re2 --platform=ruby -- --enable-system-libraries --with-re2-dir=/path/to/re2/prefixAlternatively, you can set the RE2_USE_SYSTEM_LIBRARIES environment variable instead of passing --enable-system-libraries to the gem command.
- Thanks to Jason Woods who contributed the
original implementations of
RE2::MatchData#beginandRE2::MatchData#end. - Thanks to Stefano Rivera who first contributed C++11 support.
- Thanks to Stan Hu for reporting a bug with empty
patterns and
RE2::Regexp#scan, contributing support for libre2.11 (2023-07-01) and for vendoring RE2 and abseil and compiling native gems in 2.0. - Thanks to Sebastian Reitenbach for reporting
the deprecation and removal of the
utf8encoding option in RE2. - Thanks to Sergio Medina for reporting a bug when
using
RE2::Scanner#scanwith an invalid regular expression. - Thanks to Pritam Baral for contributing the
initial support for
RE2::Set. - Thanks to Mike Dalessio for reviewing the precompilation of native gems in 2.0.
- Thanks to Peter Zhu for ruby_memcheck and helping find the memory leaks fixed in 2.1.3.
- Thanks to Jean Boussier for contributing the
switch to Ruby's
TypedDataAPI and the resulting garbage collection improvements in 2.4.0. - Thanks to Manuel Jacob for reporting a bug when passing strings with null bytes.
All issues and suggestions should go to GitHub Issues.
This library is licensed under the BSD 3-Clause License, see LICENSE.txt.
Copyright © 2010, Paul Mucur.
The source code of RE2 is distributed in the ruby platform gem. This code is licensed under the BSD 3-Clause License, see LICENSE-DEPENDENCIES.txt.
The source code of Abseil is distributed in the ruby platform gem. This code is licensed under the Apache License 2.0, see LICENSE-DEPENDENCIES.txt.