Proposal for "PHP_LexerGenerator"

» Metadata » Status
  • Category: PHP
  • Proposer: Greg Beaver 
  • License: New BSD License
» Description
PHP_LexerGenerator generates a PHP 5-based token scanner (lexer).

The generated scanner is very efficient, as much as 2.5 times more efficient than other PHP scanner generators like csLex.

The file format is similar to re2c, which generates scanners in C. Learning to use this powerful lexer generator is easy, and the lexers work very well in conjunction with a parser to parse complex grammars.

This package needs to have error-checking of regular expressions prior to using them, and this will be added prior to release.

In addition, there are no regression tests, which will need fixing prior to a stable release.

Otherwise, it is ready for usage in the most complicated of lexers, as evidenced by a PGN file format lexer included as an example.
» Dependencies » Links
  • PHP 5.1.0
  • pcre extension
» Timeline » Changelog
  • First Draft: 2006-06-24
  • Proposal: 2006-06-24
  • Call for Votes: 2006-07-09
  • Greg Beaver
    [2006-06-30 16:38 UTC]

    - Changed license to new BSD license
    - updated download link to version 0.2.0
    - updated docs, the example lexer takes advantage of new regex features
    - Added regular expression parser, this adds tremendous flexibility
    * usage of internal sub-patterns is now allowed, and is passed to
    code handlers as variable $yy_subpatterns
    * usage of back-references is allowed for internal sub-patterns
    * usage of (?R) is allowed for recursive patterns and conditionals
    * usage of conditional patterns based on internal back-references
    or recursive back-reference (?(3)blah|foo) or (?(?R)blah|foo) is allowed
    * once-only patterns are allowed (?>blah)
    * usage of conditional patterns based on lookaheads are allowed
    (?(?=])x) (?(?=])x|y)
    * lookbehinds are impossible and are not allowed
    * setting PCRE internal options inside (?i) or (?i:blah) are not allowed
    - PHP_LexerGenerator_Exception extends PEAR_Exception

    I will call for votes in 7 days, please review now
  • Greg Beaver
    [2006-07-02 12:56 UTC]

    add "plex" command-line tool for generating lexers (similar to phplemon command for PHP_ParserGenerator)

    fix back-reference handling, including support for multipliers
    remove support for (?R) - it can't work unfortunately.

    update documentation and download link to version 0.2.1