Docs For Class Tokenizer

$config = null

[line 23]

The config data for the run.

Access: protected

Type:

\PHP_CodeSniffer\Config

[ Top ]

$eolChar = array()

[line 30]

The EOL char used in the content.

Access: protected

Type:

string

[ Top ]

$ignoredLines = array()

[line 51]

A list of lines being ignored due to error suppression comments.

Access: public

Type:

array

[ Top ]

$knownLengths = array()

[line 44]

Known lengths of tokens.

Var: int>
Access: public

Type:

array

[ Top ]

$tokens = array()

[line 37]

A token-based representation of the content.

Access: protected

Type:

array

[ Top ]

__construct (Constructor) [line 64]

void __construct(
string
$content, \PHP_CodeSniffer\Config
$config, [string
$eolChar = '\n'])

Initialise and run the tokenizer.

Throws: TokenizerException If the file appears to be minified.
Access: public

Overridden in child classes as:

CSS::__construct(): Initialise the tokenizer.

JS::__construct(): Initialise the tokenizer.

Parameters:

string	`$content`	—	The content to tokenize,
string	`$eolChar`	—	The EOL char used in the content.
\PHP_CodeSniffer\Config	`$config`	—	\| null $config The config data for the run.

[ Top ]

getTokens [line 116]

array getTokens(
)

Gets the array of tokens.

Access: public

[ Top ]

isMinifiedContent [line 95]

boolean isMinifiedContent(
string
$content, [string
$eolChar = '\n'])

Checks the content to see if it looks minified.

Access: protected

Parameters:

string	`$content`	—	The content to tokenize,
string	`$eolChar`	—	The EOL char used in the content.

[ Top ]

processAdditional [line 138]

void processAdditional(
)

Performs additional processing after main tokenizing.

Abstract:
Access: protected

Overridden in child classes as:

PHP::processAdditional(): Performs additional processing after main tokenizing.

CSS::processAdditional(): Performs additional processing after main tokenizing.

JS::processAdditional(): Performs additional processing after main tokenizing.

[ Top ]

replaceTabsInToken [line 269]

void replaceTabsInToken(

&$token, [string
$prefix = ' '], [string
$padding = ' '], array
$token)

Replaces tabs in original token content with spaces.

Each tab can represent between 1 and $config->tabWidth spaces, so this cannot be a straight string replace. The original content is placed into an orig_content index and the new token length is also set in the length index.

Access: public

Parameters:

array	`$token`	—	The token to replace tabs inside.
string	`$prefix`	—	The character to use to represent the start of a tab.
string	`$padding`	—	The character to use to represent the end of a tab.
	`&$token`	—

[ Top ]

tokenize [line 130]

array tokenize(
string
$string)

Creates an array of tokens when given some content.

Abstract:
Access: protected

Overridden in child classes as:

PHP::tokenize(): Creates an array of tokens when given some PHP code.

CSS::tokenize(): Creates an array of tokens when given some CSS code.

JS::tokenize(): Creates an array of tokens when given some JS code.

Parameters:

string $string — The string to tokenize.

[ Top ]

Class: Tokenizer

Class Overview

Author(s):

Copyright:

Variables

Methods

Child classes:

Inherited Variables

Inherited Methods

Class Details

Class Variables

$config = null

$eolChar = array()

$ignoredLines = array()

$knownLengths = array()

$tokens = array()

Method Detail

__construct (Constructor) [line 64]

Parameters:

getTokens [line 116]

isMinifiedContent [line 95]

Parameters:

processAdditional [line 138]

replaceTabsInToken [line 269]

Parameters:

tokenize [line 130]

Parameters: