class tokenizer

# class tokenizer | Abstract class to define a source of tokens for the parser. More...
Contains pure virtuals
Full name | `toSQLParse::tokenizer`
Definition | `#include <tosqlparse.h>`
Inherited by | editorTokenizer, stringTokenizer
List of all Methods
Annotated List
Files
Globals
Hierarchy
Index

#### Public Methods

- tokenizer (int offset=0,int line=0)
- tokenizer (toSyntaxAnalyzer &analyzer,int offset=0,int line=0)
- virtual ~tokenizer ()
- virtual QString getToken (bool forward=true,bool comment=false)
- virtual void setAnalyzer (toSyntaxAnalyzer &analyzer)
- virtual toSyntaxAnalyzer & analyzer ()
- virtual int line (void)
- virtual int offset (void)
- virtual void setOffset (int offset)
- virtual void setLine (int line)
- virtual QString remaining (bool eol)

#### Protected Members

- int Offset
- int Line

## Detailed Description

Abstract class to define a source of tokens for the parser.

int Offset | ### Offset

[protected]

int Line | ### Line

[protected]

tokenizer (int offset=0,int line=0) | ### tokenizer

Create a tokenizer. Optionally specify which line and offset to start at.

tokenizer (toSyntaxAnalyzer &analyzer,int offset=0,int line=0) | ### tokenizer

Create a tokenizer. Optionally specify which line and offset to start at.

~tokenizer () | ### ~tokenizer

[virtual]

QString getToken (bool forward=true,bool comment=false) | ### getToken

[pure virtual]

Get a token from the string.

Parameters:

forward | Go forward or backwards to get next token.
comment | Include comments as tokens.

void setAnalyzer (toSyntaxAnalyzer &analyzer) | ### setAnalyzer

[virtual]

Specify the syntax analyzer to use for the tokanizer

Parameters:

analyzer |

toSyntaxAnalyzer & analyzer () | ### analyzer

[virtual]

int line (void) | ### line

[virtual]

Get the current line of the tokenizer. A line is defined by a \n character

int offset (void) | ### offset

[virtual]

Current offset of the tokenizer (Should point to the character after the last token in the specified direction)

void setOffset (int offset) | ### setOffset

[virtual]

Set a new offset of tokenzer.

void setLine (int line) | ### setLine

[virtual]

Set new current line of tokenizer. This will not affect the current position of the tokenizer.

QString remaining (bool eol) | ### remaining

[pure virtual]

Get the data remaining after the current position.

Parameters:

eol | If true end of line, otherwise end of tokenizer.

Generated by: nneul on skyhawk on Wed Feb 23 19:49:58 2005, using kdoc 2.0a54.