Package home | Report new bug | New search | Development Roadmap Status: Open | Feedback | All | Closed Since Version 2.0.0a1

Bug #13921 js parsing fails for comments on last line of file
Submitted: 2008-05-16 19:03 UTC Modified: 2008-05-19 01:13 UTC
From: gggeek Assigned: squiz
Status: Closed Package: PHP_CodeSniffer (version 1.1.0RC1)
PHP Version: 5.2.6 OS: windows xp
Roadmaps: 1.1.0    

 [2008-05-16 19:03 UTC] gggeek (Gaetano Giunta)
Description: ------------ The current js parsing code does not work (a php warning is raised) for comments on the last line of the js file (or even if only followed by empty lines). This happens because js comments are terminated not at eol, but at the next found token instead. The fix involves imho changing the way comments are parsed: instead of ignoring (and parsing as separate tokens) all the chars that follow a comment opening token, the code should use the same approach used for strings , ie. continue greedy buffering of the source until the end of comment (either eol, eof or */) is found. This has the side advantage that there is no need to re-glue together comments later. Of course, unclode multiline comments at end of file could be tagged as error/warning Test script: --------------- parse a js file containing only this: // aa (make & either 1 or 2 lines long) Actual result: -------------- *** START TOKENIZING *** Process char 0 => / * buffer possibly contains token, looking ahead 8 chars * => Looking ahead 1 chars => // * look ahead found more specific token (T_COMMENT), ignoring 0 * Process char 1 => / * buffer possibly contains token, looking ahead 7 chars * => Looking ahead 1 chars => // => Looking ahead 2 chars => // a => Looking ahead 3 chars => // aa => Looking ahead 4 chars => // aa => Looking ahead 5 chars => // aa\n * look ahead found nothing * => Added token T_COMMENT (//) * looking for end of comment * Process char 2 => Process char 3 => a Process char 4 => a Process char 5 => Process char 6 => * found end of comment * Notice: Undefined offset: 3 in D:\htdocs\PHP_CodeSniffer\CodeSniffer\Tokenizers\JS.php on line 638 Call Stack: 0.0004 56568 1. {main}() D:\htdocs\PHP_CodeSniffer\scripts\phpcs:0 0.0201 1133784 2. PHP_CodeSniffer_CLI->process() D:\htdocs\PHP_CodeSniffer\scripts\phpcs:30 0.0231 1137792 3. PHP_CodeSniffer->process() D:\htdocs\PHP_CodeSniffer\CodeSniffer\CLI.php:334 0.1283 2619392 4. PHP_CodeSniffer->processFile() D:\htdocs\PHP_CodeSniffer\CodeSniffer.php:278 0.1359 2671040 5. PHP_CodeSniffer_File->start() D:\htdocs\PHP_CodeSniffer\CodeSniffer.php:600 0.1359 2672704 6. PHP_CodeSniffer_File->_parse() D:\htdocs\PHP_CodeSniffer\CodeSniffer\File.php:341 0.1393 2822792 7. PHP_CodeSniffer_File::tokenizeString() D:\htdocs\PHP_CodeSniffer\CodeSniffer\File.php:430 0.1393 2823304 8. PHP_CodeSniffer_Tokenizers_JS->tokenizeString() D:\htdocs\PHP_CodeSniffer\CodeSniffer\File.php:632 *** END TOKENIZING ***


 [2008-05-19 01:13 UTC] squiz (Greg Sherwood)
This bug has been fixed in CVS. If this was a documentation problem, the fix will appear on by the end of next Sunday (CET). If this was a problem with the website, the change should be live shortly. Otherwise, the fix will appear in the package's next release. Thank you for the report and for helping us make PEAR better. I've fixed this by just ensuring the stackPtr is valid before using it. I didn't rewrite the parsing code to change the rule that inline comments end on newlines because for the sniffs to work JS code needs to look like PHP code in the tokens array. This allows sniffs written for PHP to run on JS code with minimal (or no) changes. So I've made the JS tokeniser work in a similar way to the PHP one, even if the rules are not exactly the same for JS.