Description: ------------ There are still some issues with processing Unicode, so although the parser code works, unit tests (i.e. LexerGeneratorTest::testLexerGeneratorUnicode()) fail.

can be solved by adding /u modified for all regexps matched by preg_match() function in generated file.

Adding an already-present /u to solve a file I/O problem is a fascinating solution. Feel free to send in a patch that demonstrates this. Alternatively you could actually run the unit test and look at the actual issue.

I am sorry. That comment is not related to your problem. I was unable to delete it. What I wanted to say is there should be a way to add /u to regexps in the generated file.

Okay, it's not well documented, but if you look at /tests/data/Unicode.plex, you'll see the new Unicode pragma: %unicode 1 This will add /u to the regexes. But the unit test fails thanks to a translation issue on file I/O. I also added %caseinsensitive 1 to make it easier to deal with case-less grammars.