Couchbase

  • Log In Access more options
    • Online Help
    • GreenHopper Help
    • Agile Answers
    • Keyboard Shortcuts
    • About JIRA
    • JIRA Credits
    • What’s New
  • Dashboards Access more options (Alt+d)
  • Projects Access more options (Alt+p)
  • Issues Access more options (Alt+i)
  • Agile Access more options (Alt+g)
  • Couchbase.com Access more options
    • Why NoSQL?
    • Couchbase Server
    • Download
    • Learn
    • Careers
Couchbase PHP client library
  • Couchbase PHP client library
  • PCBC-111

Zlib compression broken

  • Agile Board
  • More Actions
  • Views
    • XML
    • Word
    • Printable

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Blocker Blocker
  • Resolution: Fixed
  • Affects Version/s: 1.0.5, 1.1.0-dp1, 1.1.0-dp2
  • Fix Version/s: None
  • Component/s: library
  • Security Level: Public
  • Labels:
    None

Description

Zlib compression is broken. This means large payloads are never compressed on the network.

class Compression extends CouchbaseTestCommon {
    function testCompression() {
        
        if (!extension_loaded('zlib')) {
            $this->markTestSkipped("Need Zlib extension");
        }
        
        $h = make_handle();
        $hnc = make_handle();
        
        $curcomp = couchbase_get_option($h, COUCHBASE_OPT_COMPRESSION);
        $this->assertEquals(COUCHBASE::COMPRESSION_NONE, $curcomp);
        
        couchbase_set_option($h, COUCHBASE_OPT_COMPRESSION,
                             COUCHBASE::COMPRESSION_ZLIB);
        $curcomp = couchbase_get_option($h, COUCHBASE_OPT_COMPRESSION);
        $this->assertEquals(COUCHBASE_COMPRESSION_ZLIB, $curcomp);
        
        couchbase_set_option($hnc, COUCHBASE_OPT_COMPRESSION,
                             COUCHBASE::COMPRESSION_NONE);
        
        $inikey = "couchbase.compression_threshold";
        $this->assertGreaterThan(0, ini_get($inikey));
        $this->assertTrue((bool)ini_alter($inikey,"10"));
        $this->assertEquals(10, ini_get($inikey));
        
        # Set a compressed value
        $v = str_repeat("long_value", 500);
        $ol = strlen($v);
        //$key = $this->mk_key();
        $key = "compressed_data";
        
        $rv = couchbase_set($h, $key, $v);
        $this->assertNotEmpty($rv);
        
        
        $comp = couchbase_get_option($h, COUCHBASE_OPT_COMPRESSION);
        print $comp . "\n";
        
        # Hack, append resets the flags, clearing the compression state.
        $rv = couchbase_append($hnc, $key, '_');
        $this->assertNotEmpty($rv);
        
        $ret = couchbase_get($hnc, $key);
        $this->assertNotNull($ret);
        $this->assertLessThan($ol, strlen($ret));
    }
}

Activity

Ascending order - Click to sort in descending order
  • All
  • Comments
  • Work Log
  • History
  • Activity
  • Gerrit Reviews
Hide
Permalink
Mark Nunberg added a comment - 06/Sep/12 6:47 PM
Apparently this results in (silent) memory corruption as well.. leading to unpredictable results. A similar test (pasted below) gives me this:


PHP Warning: PHP Startup: Unable to load dynamic library '/home/mnunberg/src/php-ext-couchbase-1.0.x/modules/pdo.so' - /home/mnunberg/src/php-ext-couchbase-1.0.x/modules/pdo.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHPUnit 3.6.12 by Sebastian Bergmann.

PHP Fatal error: Out of memory (allocated 3145728) (tried to allocate 4992063506150606153 bytes) in /home/mnunberg/src/php-couchbase-tests/unit/Compression.php on line 54
PHP Stack trace:
PHP 1. {main}() /usr/bin/phpunit:0
PHP 2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46
PHP 3. PHPUnit_TextUI_Command->run() /usr/share/php/PHPUnit/TextUI/Command.php:130
PHP 4. PHPUnit_TextUI_TestRunner->doRun() /usr/share/php/PHPUnit/TextUI/Command.php:192
PHP 5. PHPUnit_Framework_TestSuite->run() /usr/share/php/PHPUnit/TextUI/TestRunner.php:325
PHP 6. PHPUnit_Framework_TestSuite->runTest() /usr/share/php/PHPUnit/Framework/TestSuite.php:745
PHP 7. PHPUnit_Framework_TestCase->run() /usr/share/php/PHPUnit/Framework/TestSuite.php:772
PHP 8. PHPUnit_Framework_TestResult->run() /usr/share/php/PHPUnit/Framework/TestCase.php:751
PHP 9. PHPUnit_Framework_TestCase->runBare() /usr/share/php/PHPUnit/Framework/TestResult.php:649
PHP 10. PHPUnit_Framework_TestCase->runTest() /usr/share/php/PHPUnit/Framework/TestCase.php:804
PHP 11. ReflectionMethod->invokeArgs() /usr/share/php/PHPUnit/Framework/TestCase.php:942
PHP 12. Compression->testCompression() /usr/share/php/PHPUnit/Framework/TestCase.php:942
PHP 13. couchbase_get() /home/mnunberg/src/php-couchbase-tests/unit/Compression.php:54
couchbase_50493659660c9
Show
Mark Nunberg added a comment - 06/Sep/12 6:47 PM Apparently this results in (silent) memory corruption as well.. leading to unpredictable results. A similar test (pasted below) gives me this: PHP Warning: PHP Startup: Unable to load dynamic library '/home/mnunberg/src/php-ext-couchbase-1.0.x/modules/pdo.so' - /home/mnunberg/src/php-ext-couchbase-1.0.x/modules/pdo.so: cannot open shared object file: No such file or directory in Unknown on line 0 PHPUnit 3.6.12 by Sebastian Bergmann. PHP Fatal error: Out of memory (allocated 3145728) (tried to allocate 4992063506150606153 bytes) in /home/mnunberg/src/php-couchbase-tests/unit/Compression.php on line 54 PHP Stack trace: PHP 1. {main}() /usr/bin/phpunit:0 PHP 2. PHPUnit_TextUI_Command::main() /usr/bin/phpunit:46 PHP 3. PHPUnit_TextUI_Command->run() /usr/share/php/PHPUnit/TextUI/Command.php:130 PHP 4. PHPUnit_TextUI_TestRunner->doRun() /usr/share/php/PHPUnit/TextUI/Command.php:192 PHP 5. PHPUnit_Framework_TestSuite->run() /usr/share/php/PHPUnit/TextUI/TestRunner.php:325 PHP 6. PHPUnit_Framework_TestSuite->runTest() /usr/share/php/PHPUnit/Framework/TestSuite.php:745 PHP 7. PHPUnit_Framework_TestCase->run() /usr/share/php/PHPUnit/Framework/TestSuite.php:772 PHP 8. PHPUnit_Framework_TestResult->run() /usr/share/php/PHPUnit/Framework/TestCase.php:751 PHP 9. PHPUnit_Framework_TestCase->runBare() /usr/share/php/PHPUnit/Framework/TestResult.php:649 PHP 10. PHPUnit_Framework_TestCase->runTest() /usr/share/php/PHPUnit/Framework/TestCase.php:804 PHP 11. ReflectionMethod->invokeArgs() /usr/share/php/PHPUnit/Framework/TestCase.php:942 PHP 12. Compression->testCompression() /usr/share/php/PHPUnit/Framework/TestCase.php:942 PHP 13. couchbase_get() /home/mnunberg/src/php-couchbase-tests/unit/Compression.php:54 couchbase_50493659660c9
Hide
Permalink
Mark Nunberg added a comment - 06/Sep/12 10:28 PM
The problem reaches far deeper than simple zlib.

The compression headers themselves (the client uses a non-standard header, rather than gzip) is specified as a 'size_t', therefore this will break for any communication between cross-architecture clients.. i.e. i386 clients will not be able to properly encode lengths for x86_64 clients.

Additionally, 'simple' zlib compression won't work either because of said header.
Show
Mark Nunberg added a comment - 06/Sep/12 10:28 PM The problem reaches far deeper than simple zlib. The compression headers themselves (the client uses a non-standard header, rather than gzip) is specified as a 'size_t', therefore this will break for any communication between cross-architecture clients.. i.e. i386 clients will not be able to properly encode lengths for x86_64 clients. Additionally, 'simple' zlib compression won't work either because of said header.
Hide
Permalink
Mark Nunberg added a comment - 11/Sep/12 11:01 AM
http://review.couchbase.org/#/c/20604/
Show
Mark Nunberg added a comment - 11/Sep/12 11:01 AM http://review.couchbase.org/#/c/20604/

People

  • Assignee:
    Mark Nunberg
    Reporter:
    Mark Nunberg
Vote (0)
Watch (0)

Dates

  • Created:
    06/Sep/12 6:35 PM
    Updated:
    07/Nov/12 10:07 AM
    Resolved:
    07/Nov/12 10:07 AM

Agile

  • View on Board
  • Atlassian JIRA (v5.2.4#845-sha1:c9f4cc4)
  • Report a problem
  • Powered by a free Atlassian JIRA open source license for Couchbase. Try JIRA - bug tracking software for your team.