Class DefaultJFlexLexer
java.lang.Object
com.pnfsoftware.jeb.util.encoding.jflex.DefaultJFlexLexer
- Direct Known Subclasses:
JavaLexer
,PythonLexer
This is a default, and abstract implementation of a Lexer using JFLex with some utility methods
that Lexers can implement. See flex examples at https://github.com/nordfalk/jsyntaxpane.
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected int
protected int
protected long
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoid
protected Token
Create and return a Token of given type.protected Token
Create and return a Token of given type and pairValue.protected Token
Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength paramsprotected Token
Create and return a Token of given type from start with length offset is added to startabstract long
yychar()
Return the char number from beginning of input stream.abstract char
yycharat
(int pos) Returns the character at position pos from the matched text.abstract int
yylength()
Returns the length of the matched text region.abstract Token
yylex()
This is called to return the next Token from the Input Readerabstract void
This will be called to reset the the lexer.abstract String
yytext()
Returns the text matched by the current regular expression.
-
Field Details
-
tokenStart
protected long tokenStart -
tokenLength
protected int tokenLength -
offset
protected int offset
-
-
Constructor Details
-
DefaultJFlexLexer
public DefaultJFlexLexer()
-
-
Method Details
-
token
Helper method to create and return a new Token from of TokenType tokenStart and tokenLength will be modified to the newStart and newLength params -
token
Create and return a Token of given type from start with length offset is added to start -
token
Create and return a Token of given type. start is obtained fromyychar()
and length fromyylength()
offset is added to start -
token
Create and return a Token of given type and pairValue. start is obtained fromyychar()
and length fromyylength()
offset is added to start -
parse
- Throws:
IOException
-
parse
- Throws:
IOException
-
yyreset
This will be called to reset the the lexer. This is created automatically by JFlex. -
yylex
This is called to return the next Token from the Input Reader- Returns:
- next token, or null if no more tokens.
- Throws:
IOException
-
yycharat
public abstract char yycharat(int pos) Returns the character at position pos from the matched text. It is equivalent to yytext().charAt(pos), but faster- Parameters:
pos
- the position of the character to fetch. A value from 0 to yylength()-1.- Returns:
- the character at position pos
-
yylength
public abstract int yylength()Returns the length of the matched text region. This method is automatically implemented by JFlex lexers -
yytext
Returns the text matched by the current regular expression. This method is automatically implemented by JFlex lexers -
yychar
public abstract long yychar()Return the char number from beginning of input stream. This is NOT implemented by JFlex, so the code must be added to create this and return the private yychar field
-