tok

Go package to parse strings. Inpired by the cScanner implementation from clingo.

Scanner

Scanner is the main type to parse a string. It has basic functions to parse a text forward or backward. The basic functions exist for the following types:

• string

• fold

• rune

• anyrune

• between

• match

Basic functions return just a bool. Additional has the scanner read functions to read values. Read functions return the value and an error if it was not possible to read the value. The package has the following read functions:

• ReadBool

• ReadInt

• ReadUint

Reader

Readers can be used by the scanner to read from the scanner. tok has the following build-in Reader: Any, AnyFold, AnyRune, At, Between, BetweenAny, Body, Bool, Digit, Fold, Hex, Holey, Int, Janus, Lit, Many, Map, Match, Named, Not, Opt, Past, Rune, Seq, Set, SkipSeq, SkipWSSeq, Times, To, Uint, Wrap, WS, Zom

Mark Types

Marker

can be used to mark a scanner position

Token

can be used to mark a range of the scanner

Segment

can be used tag a Token with addition information

Grammar

A grammar is a Reader that has connected Rules. Check the grammar package with different grammars, like JSOM, MXT and Lua.

Graph

A graph allows to arrange the picked values hierarchically via Nodes. With the FlameStack function is it possible to generate a string that can be used to produce a FlameGraph:

$ flamegraph.pl flame.stack > graph.svg

Tracker

A Tracker can be coupled with a Scannar and track the movemend. Basket is a Tracker und can be used to Pick-Up Elements that where read by the Parser.

Log

A Log can be used to monitor and log the movemend in a Reader graph.