public abstract class

DefaultJFlexLexer

extends Object
java.lang.Object
   ↳ com.pnfsoftware.jeb.util.encoding.jflex.DefaultJFlexLexer
Known Direct Subclasses

Class Overview

This is a default, and abstract implementation of a Lexer using JFLex with some utility methods that Lexers can implement. See flex examples at https://github.com/nordfalk/jsyntaxpane.

Summary

Fields
protected int offset
protected int tokenLength
protected long tokenStart
Public Constructors
DefaultJFlexLexer()
Public Methods
List<Token> parse(String s)
void parse(Reader in, List<Token> tokens)
abstract long yychar()
Return the char number from beginning of input stream.
abstract char yycharat(int pos)
Returns the character at position pos from the matched text.
abstract int yylength()
Returns the length of the matched text region.
abstract Token yylex()
This is called to return the next Token from the Input Reader
abstract void yyreset(Reader reader)
This will be called to reset the the lexer.
abstract String yytext()
Returns the text matched by the current regular expression.
Protected Methods
Token token(TokenType type, long start, int length)
Create and return a Token of given type from start with length offset is added to start
Token token(TokenType type)
Create and return a Token of given type.
Token token(TokenType type, int pairValue)
Create and return a Token of given type and pairValue.
Token token(TokenType type, int tStart, int tLength, int newStart, int newLength)
Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength params
[Expand]
Inherited Methods
From class java.lang.Object

Fields

protected int offset

protected int tokenLength

protected long tokenStart

Public Constructors

public DefaultJFlexLexer ()

Public Methods

public List<Token> parse (String s)

Throws
IOException

public void parse (Reader in, List<Token> tokens)

Throws
IOException

public abstract long yychar ()

Return the char number from beginning of input stream. This is NOT implemented by JFlex, so the code must be added to create this and return the private yychar field

public abstract char yycharat (int pos)

Returns the character at position pos from the matched text. It is equivalent to yytext().charAt(pos), but faster

Parameters
pos the position of the character to fetch. A value from 0 to yylength()-1.
Returns
  • the character at position pos

public abstract int yylength ()

Returns the length of the matched text region. This method is automatically implemented by JFlex lexers

public abstract Token yylex ()

This is called to return the next Token from the Input Reader

Returns
  • next token, or null if no more tokens.
Throws
IOException

public abstract void yyreset (Reader reader)

This will be called to reset the the lexer. This is created automatically by JFlex.

public abstract String yytext ()

Returns the text matched by the current regular expression. This method is automatically implemented by JFlex lexers

Protected Methods

protected Token token (TokenType type, long start, int length)

Create and return a Token of given type from start with length offset is added to start

protected Token token (TokenType type)

Create and return a Token of given type. start is obtained from yychar() and length from yylength() offset is added to start

protected Token token (TokenType type, int pairValue)

Create and return a Token of given type and pairValue. start is obtained from yychar() and length from yylength() offset is added to start

protected Token token (TokenType type, int tStart, int tLength, int newStart, int newLength)

Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength params