Class: PHP_Parser_Tokenizer

Source Location: /PHP_ParserGenerator-0.1.7/Tokenizer.php

Class Overview

The tokenizer wrapper for parser - implements the 'standard?' yylex interface

Author(s):

Version:

$Id: Tokenizer.php 216607 2006-07-18 00:53:10Z cellog $

Variables

Methods

Inherited Variables

Inherited Methods

Class Details

[line 46]
The tokenizer wrapper for parser - implements the 'standard?' yylex interface

2 main methods:

Version: $Id: Tokenizer.php 216607 2006-07-18 00:53:10Z cellog $

[ Top ]

Class Variables

$debug = false

[line 55]

Debugging on/off

Access: public

Type:

boolean

[ Top ]

$lastComment =

[line 137]

The string of the last Comment Token

Access: public

Type:

string

[ Top ]

$lastCommentLine =

[line 128]

ID of the last Comment Token

Access: public

Type:

int

[ Top ]

$lastCommentToken =

[line 119]

ID of the last Comment Token

Access: public

Type:

int

[ Top ]

$line =

[line 76]

Current line.

Access: public

Type:

int

[ Top ]

$N = 0

[line 69]

Total Number of tokens.

Access: public

Type:

int

[ Top ]

$pos = -1

[line 83]

Current token position.

Access: public

Type:

int

[ Top ]

$token =

[line 91]

The current token (either a ord(';') or token numer - see php tokenizer.

Access: public

Type:

int

[ Top ]

$tokens =

[line 62]

Tokens - array of all the tokens.

Access: public

Type:

array

[ Top ]

$value =

[line 100]

The value associated with a token - eg. for T_STRING it's the string

Access: public

Type:

string

[ Top ]

$valueWithWhitespace =

[line 110]

The value associated with a token plus preceding whitespace, if any

This is only filled if whitespace attachment is turned on, for performance reasons

Access: public

Type:

string

[ Top ]

Method Detail

__construct (Constructor) [line 169]

none __construct(
string
$data, [
$options = array()])

Constructor

Load the tokenizer - with a string to tokenize. tidies up array, sets vars pos, line, N and tokens

Access: public

Parameters:

string	`$data`	—	PHP code to serialize
	`$options`	—

[ Top ]

advance [line 362]

boolean advance(
)

The main advance call required by the parser

return true if a token is available, false if no more are available. skips stuff that is not a valid token stores lastcomment, lastcommenttoken

Return: - true = have tokens
Access: public

[ Top ]

getLastComment [line 293]

array getLastComment(
)

get the last comment block (and reset it)

Return: ($commmentstring and $tokenPosition)
Access: public

[ Top ]

getValue [line 442]

void getValue(
)

[ Top ]

getWhitespace [line 232]

string getWhitespace(
)

Return the whitespace (if any) that preceded the current token

[ Top ]

globalSearch [line 271]

void globalSearch(
string
$var)

Compare global variable to search value, to see if we've

found a variable that must be documented

Parameters:

string $var — global variable found in source code

[ Top ]

haltLexing [line 222]

void haltLexing(
)

[ Top ]

handleMessage [line 240]

void handleMessage(
MsgServer_Msg
$msg)

Parameters:

MsgServer_Msg $msg —

[ Top ]

parseError [line 458]

string parseError(
)

return something useful, when a parse error occurs.

used to build error messages if the parser fails, and needs to know the line number..

Access: public

[ Top ]

stopTrackingWhitespace [line 261]

void stopTrackingWhitespace(
)

[ Top ]

trackWhitespace [line 256]

void trackWhitespace(
)

[ Top ]