Antlr4 Java20 Grammar Query

[julianbass]

My use case is to count lines of Java code BUT only to include certain method types in the line counting. I intend to extend a base listener and implement the line counting function. But the listener methods are generated but never called when I parse some code. Separately, I've previously used an extended java base listener to calculate (for example) Cyclomatic complexity of my source code, but I am a grammar newbie.

I am using the Java20 grammar:
https://github.com/antlr/grammars-v4.git/java/java20

I've made a small change to the file Java20Lexer.g4

from this:
WS : [ \r\n\t\n\u000C]+ -> skip ;

to this:
`WS : [ \t\n\u000C]+ -> skip
;

NL : [\r\n]+
;`

I can see a new names are now included in the generated Java20Lexer.java code (I've edited this for brevity).

private static String[] makeRuleNames() { return new String[] { [...] "WS", "NL", "COMMENT", "LINE_COMMENT" }; }

private static String[] makeSymbolicNames() { return new String[] { [...], "WS", "NL", "COMMENT", "LINE_COMMENT" }; }

So, now I thought I could use this new Lexer rule in my grammar Java20Parser.g4, as follows

`// Paragraph 15.30
// ---------------

newLine
: NL
;`

As I expected, the base Listener Java20ParserBaseListener.java is generated with methods

/** * {@inheritDoc} * * <p>The default implementation does nothing.</p> */ @Override public void enterNewLine(Java20Parser.NewLineContext ctx) { } /** * {@inheritDoc} * * <p>The default implementation does nothing.</p> */ @Override public void exitNewLine(Java20Parser.NewLineContext ctx) { }

I can override these methods, but sadly they never get executed when I parse some code.

I've executed grun Java20 newLine CarTester.txt

My test file is simple and starts with

`public class FastCarTest {

@Test
public void test() {
	FastCar fastCar = new FastCar("BMW", "320M", 180);
	assertTrue(fastCar.getMake().equals("BMW"));
	assertTrue(fastCar.getModel().equals("320M"));
	assertTrue( fastCar.getSpeed() == 180);
}

}
...
`

The response I get from grun is line 1:0 mismatched input 'public' expecting NL

Any help to explain what I am doing wrong gratefully received.

[kaby76]

You'll have to leave the grammar as is when parsing your input because changing the lexing rule for whitespace will break everything in the parser. Lexing occurs first and it is not guided at all by the parser. If you made this change, you would have a huge task of modifying the parser grammar to optionally recognize a newline between everything else. There is a better way.

After parsing, what you will need is to first identify which methods you want to line count. You can do this with an Antlr Listener. You can then grab the text for the methods and just count newlines in the text.

A Trash script can perform essentially what you want. Trash automates the generation of a standardized driver application for a grammar. You can then use the other Trash tools and Unix wc to get a line count. You just need to figure out the test of the methods you are looking for. Below, I perform a wc on the block of text for all methodBody.

$ trparse in.txt | trxgrep ' //methodDeclaration/methodBody' | trtext | wc
CSharp 0 in.txt success 0.6792522
      6      15     198
05/10-11:15:17 ~
$ cat in.txt
public class FastCarTest {

@Test
public void test() {
        FastCar fastCar = new FastCar("BMW", "320M", 180);
        assertTrue(fastCar.getMake().equals("BMW"));
        assertTrue(fastCar.getModel().equals("320M"));
        assertTrue( fastCar.getSpeed() == 180);
}

}
05/10-11:16:47 ~

[julianbass]

Thanks so much for your help @kaby76. I was clearly going about this the wrong way. Your trash tools look excellent and very useful.

I'll investigate your suggested approach