Parser

Parser is a production-ready Ruby parser written in pure Ruby. It performs on par or better than Ripper, Melbourne, JRubyParser or ruby_parser.

Installation

Most recent version of Parser is 2.0; however, per release schedule, it stays in the beta status for a while. However, it handles much more input than stable 1.x branch, and for new work it is advisable to use the beta versions.

$ gem install parser --pre

Usage

Parse a chunk of code:

require 'parser/current'

p Parser::CurrentRuby.parse("2 + 2")
# (send
#   (int 2) :+
#   (int 2))

Access the AST's source map:

p Parser::CurrentRuby.parse("2 + 2").loc
# #<Parser::Source::Map::Send:0x007fe0ca8a69b8
#   @begin=nil,
#   @end=nil,
#   @expression=#<Source::Range (string) 0...5>,
#   @selector=#<Source::Range (string) 2...3>>

p Parser::CurrentRuby.parse("2 + 2").loc.selector.to_source
# "+"

Parse a chunk of code and display all diagnostics:

parser = Parser::CurrentRuby.new
parser.diagnostics.consumer = lambda do |diag|
  puts diag.render
end

buffer = Parser::Source::Buffer.new('(string)')
buffer.source = "foo *bar"

p parser.parse(buffer)
# (string):1:5: warning: `*' interpreted as argument prefix
# foo *bar
#     ^
# (send nil :foo
#   (splat
#     (send nil :bar)))

If you reuse the same parser object for multiple #parse runs, you need to #reset it.

You can also use the ruby-parse utility (it's bundled with the gem) to play with Parser:

$ ruby-parse -L -e "2+2"
(send
  (int 2) :+
  (int 2))
2+2
 ~ selector
~~~ expression
(int 2)
2+2
~ expression
(int 2)
2+2

$ ruby-parse -E -e "2+2"
2+2
^ tINTEGER 2                                    expr_end     [0 <= cond] [0 <= cmdarg]
2+2
 ^ tPLUS "+"                                    expr_beg     [0 <= cond] [0 <= cmdarg]
2+2
  ^ tINTEGER 2                                  expr_end     [0 <= cond] [0 <= cmdarg]
2+2
  ^ false "$eof"                                expr_end     [0 <= cond] [0 <= cmdarg]
(send
  (int 2) :+
  (int 2))

Features

• Precise source location reporting.
• Documented AST format which is convenient to work with.
• A simple interface and a powerful, tweakable one.
• Parses 1.8, 1.9, 2.0 and 2.1 (preliminary) syntax with backwards-compatible AST formats.
• Parsing error recovery.
• Improved clang-like diagnostic messages with location information.
• Written in pure Ruby, runs on MRI 1.8.7 or >=1.9.2, JRuby and Rubinius in 1.8 and 1.9 mode.
• Only two runtime dependencies: the gems ast and slop.
• Insane Ruby lexer rewritten from scratch in Ragel.
• 100% test coverage for Bison grammars (except error recovery).
• Readable, commented source code.

Documentation

Documentation for parser is available online on rdoc.info.

Known issues

Void value expressions

So-called "void value expressions" are not handled by Parser. For a description of what a void value expression is, see this gist and this Parser issue.

It is not clear which rules this piece of static analysis follows, or which problem does it solve. It is not implemented because there is no clear specification allowing us to verify the behavior.

Contributors

• Peter Zotov (whitequark)
• Magnus Holm (judofyr)

Acknowledgements

The lexer testsuite is derived from ruby_parser.

The Bison parser rules are derived from Ruby MRI parse.y.

Contributing

1. Make sure you have Ragel ~> 6.7 installed
2. Fork it
3. Create your feature branch (git checkout -b my-new-feature)
4. Commit your changes (git commit -am 'Add some feature')
5. Push to the branch (git push origin my-new-feature)
6. Create new Pull Request