Package home | Report new bug | New search | Development Roadmap Status: Open | Feedback | All | Closed Since Version 3.4.2

Bug #19804 JS Tokenizer wrong /**/ parsing
Submitted: 2013-01-29 08:21 UTC
From: dryabkov Assigned: squiz
Status: Closed Package: PHP_CodeSniffer (version 1.4.3)
PHP Version: Irrelevant OS:
Roadmaps: (Not assigned)    
Subscription  


 [2013-01-29 08:21 UTC] dryabkov (Denis Ryabkov)
Description: ------------ Tokenizer marks all code below /**/ as T_DOC_COMMENT

Comments

 [2013-01-29 08:28 UTC] squiz (Greg Sherwood)
-Assigned To: +Assigned To: squiz
It literally is just when using /**/ - even a single space will make it work. Sample code: alert('foo'); /**/ alert('foo');
 [2013-01-29 08:32 UTC] squiz (Greg Sherwood)
Here is the problem: Process char 14 => / (buffer: \n) => Added token T_WHITESPACE (\n) * buffer possibly contains token, looking ahead 8 chars * => Looking ahead 1 chars => /* * look ahead found more specific token (T_COMMENT), ignoring 14 * Process char 15 => * (buffer: /) * buffer possibly contains token, looking ahead 7 chars * => Looking ahead 1 chars => /** * look ahead found more specific token (T_DOC_COMMENT), ignoring 15 * * looking for end of comment * /* = T_COMMENT and /** = T_DOC_COMMENT Hence the confusion as it never finds the */ it is expecting.
 [2013-01-29 09:00 UTC] squiz (Greg Sherwood)
-Status: Assigned +Status: Closed
I couldnt think of a generic way of fixing this, so I had to put a specific fix in for it. Tokenizer now shows: Process char 14 => / (buffer: \n) => Added token T_WHITESPACE (\n) * buffer possibly contains token, looking ahead 8 chars * => Looking ahead 1 chars => /* * look ahead found more specific token (T_COMMENT), ignoring 14 * Process char 15 => * (buffer: /) * buffer possibly contains token, looking ahead 7 chars * => Looking ahead 1 chars => /** * look ahead ignored T_DOC_COMMENT, continuing * => Looking ahead 2 chars => /**/ Commit is here: https://github.com/squizlabs/PHP_CodeSniffer/commit/77a430b809ed1a66d46354c91 13011ccf433d5c9