Package home | Report new bug | New search | Development Roadmap Status: Open | Feedback | All | Closed Since Version 1.6.7

Request #21077 Increase Crypt_GPG_Engine::CHUNK_SIZE
Submitted: 2016-06-07 08:52 UTC
From: alec Assigned: alec
Status: Closed Package: Crypt_GPG (version 1.4.1)
PHP Version: Irrelevant OS:
Roadmaps: 1.4.2    
Subscription  


 [2016-06-07 08:52 UTC] alec (Aleksander Machniak)
Description: ------------ Because of http://bugs.php.net/bug.php?id=35224 we set the chunk size to 8192. This bug however was fixed before PHP 5.2.1 which is our minimal requirement. I propose to make the chunk size bigger. From my tests with 8192*100 chunk and 1MB data, encryption performance is ~2 times better. So, 1MB sounds reasonable to me. However, we need some more investigation. The debug log shows me that smaller chunks are used still and the chunk size is different every time and the biggest I found is 82189. So, actually with chunk 8192*10 I got the same perf. improvement as with 8192*100.

Comments

 [2016-06-07 14:44 UTC] gauthierm (Michael Gauthier)
In my original testing (which was many years ago now) I found PHP internally split the work into 8192 byte chunks no matter what size was specified in the PHP code. This was debugged with strace at the time. I'd be delighted if that internal issue has been resolved and we can bump the chunk size up. My original tests were on 700 MB files and 8192 performed the best.
 [2016-06-08 14:40 UTC] alec (Aleksander Machniak)
It looks like the chunk size is only a suggestion for the stream operations. I.e. if I set it to 1MB it never used more than around 80kB (according to the debug log). Testing ~5MB file on gnupg-2.0.22 under linux 4.2.0 and ext4 (PHP7) I got following results: 4096 -> 20.0 sec. 8192 -> 4.0 sec. 16384 -> 2.9 sec. 65536 -> 2.6 sec. 819200 -> 2.8 sec. 8192000 -> 4.8 sec. However, if I add stream_set_chunk_size($pipe, self::CHUNK_SIZE); stream_set_read_buffer($pipe, self::CHUNK_SIZE); stream_set_write_buffer($pipe, self::CHUNK_SIZE); I got: 4096 -> 7.6 sec. 8192 -> 4.0 sec. 16384 -> 2.3 sec. 65536 -> 0.9 sec. 819200 -> 1.2 sec. 8192000 -> memory_limit (that's probably I'm testing this inside Roundcube Enigma plugin), but I expect worse result here. So, it looks like we can do 4x better than current code.
 [2016-06-13 17:31 UTC] alec (Aleksander Machniak)
I did more raw tests with bigger (60MB) file: 4096: 700 sec. 8192: 350 sec. 63536: 56 sec. 81920: 80 sec. And just for the record, encrypting this file using encryptFile() to skip passing the content over pipes takes 19 seconds on this machine.
 [2016-06-14 11:41 UTC] alec (Aleksander Machniak)
Just to be clear, when using encryptFile() we still use pipes. For completeness, doing the same using gnupg binary directly (no Crypt_GPG/PHP involved) takes only 4 seconds. Unfortunately gnupg has no --input argument, but it has --output which we should consider to use for encryptFile() and other commands that "redirect" the output to a file. That's however another story.
 [2016-06-14 13:20 UTC] alec (Aleksander Machniak)
-Status: Open +Status: Closed -Assigned To: +Assigned To: alec
This bug has been fixed in SVN. If this was a documentation problem, the fix will appear on pear.php.net by the end of next Sunday (CET). If this was a problem with the pear.php.net website, the change should be live shortly. Otherwise, the fix will appear in the package's next release. Thank you for the report and for helping us make PEAR better.
 [2016-06-17 17:07 UTC] alec (Aleksander Machniak)
-Roadmap Versions: +Roadmap Versions: 1.4.2