Couchbase

[CCBC-193] libcouchbase build is not checking install properly

Fri, 22 Mar 2013 19:56:36 -0500

I was trying to package the PHP client, which depends on libcouchase, and it is was refusing to build on some systems.

After looking into it a bit, there may be a couple of problems, but the big one is in the summary.

One problem is that it seems the dpkg -i doesn't actually get run. I added an echo of the label in there, and see that in the console output in Jenkins, but I don't see the lines under the label run. I'm stumped there.

[CCBC-208] libvbucket CRC32 hashing function returns wrong hashes

Thu, 9 May 2013 14:02:28 -0500

It seems that the CRC32 hashing function in libvbucket (hash_crc32()) returns an invalid hash.

For instance, hashing the key '1234567890' returns 9757 (0x261D), while it should return 639479525 (0x261DAEE5).

Every key I tried (on both win and linux) returns a 16bit hash.

The problem seems to be on line 85 in crc32.c.

Changing the return expression to 'return crc ^ ~0;' fixes the problem and computes the correct hash.

[CCBC-184] There are a lot of connections are established after restarting Couchbase server

Mon, 25 Feb 2013 01:08:00 -0600

1) Start Couchbase Server
2) Start to recieve some key from Couchbase server using a client based on libcouchbase library. We use about 250 threads for receiving key
3) Restart Couchbase server
4) After that a lot of connections are created (all of then in ESTABLISHED state). And soon quota for handlers is over.

[CCBC-181] enhance observe man page with a more complete example

Tue, 2 Apr 2013 01:36:45 -0500

The current observe man page doesn't really go into how to interpret results from the request. I needs to have that.

Ideally it should also show how one would do a regular poll to show checking for replication or persistence.

[CCBC-162] API reference uses old names, not current lcb_* names

Thu, 31 Jan 2013 03:11:36 -0600

The API reference for all functions uses libcouchbase_* names (libcouchbase_create, etc.) instead of the correct lcb_* names (lcb_create, etc.).

[CCBC-209] Make Windows builds work with jom

Fri, 10 May 2013 01:29:45 -0500

Jom supports parallel-make for windows' nmake (the equivalent to normal make's '-j' option). Right now compiling on windows is painfully slow.

Another alternative would be to make a solution file.

http://qt-project.org/wiki/jom

[CCBC-200] lcb_make_http_request doesn't respect timeouts

Fri, 5 Apr 2013 06:17:51 -0500

Run the attached testcase on Windows (after fixing the host and such). The lcb_wait after calling lcb_make_http_request will hang for a very long time (perhaps forever) if the server is no longer accessible.

I tested this by starting the client. When it prints "sleeping" I pull the ethernet cable on my machine. If the lcb_make_http_request is replaced by lcb_get then you'll get a timeout which is expected.

[CCBC-199] Problem with a newly created memcached bucket

Tue, 2 Apr 2013 06:28:35 -0500

See: http://www.couchbase.com/forums/thread/libcouchbase-problem-newly-created-memcached-bucket

[CCBC-195] config cache will crash client due to relocating buffers

Tue, 2 Apr 2013 03:30:20 -0500

See CCBC-190

During a cluster configuration change, the buffers are relocated while they are being processed. This causes crashes.

A solution would be to "request" a configuration refresh and have that take place once it is safe to move buffers around (i.e. once we are out of the buffer processing call chain).

[CCBC-194] Please add support for setsockopt() SO_KEEPALIVE & TCP_KEEPIDLE

Mon, 25 Mar 2013 12:25:05 -0500

Please add support for setsockopt() options SO_KEEPALIVE & TCP_KEEPIDLE through (suggestion)
lcb_connect() or some other such api so that I can control the connection's ability to not timeout
and have a firewall cause a disconnection from lack of inactivity...without such control, receive
NETWORK_ERROR from couchbase apis.

I cannot adjust the firewall, so I need to keep the connection alive.

[CCBC-192] Failure to handle host that has been removed from a cluster

Wed, 20 Mar 2013 21:38:41 -0500

We're encountering an issue where both the Ruby and Perl clients get() method will consistently, repeatedly just return an error. On the Perl client it is "Temporary error. try again later", and on Ruby it is "Bucket not found".

This occurs when we have several couchbase servers, and have connected using the node list option to supply all of them to the library.

We remove the first server in the node list via the Couchbase GUI, trigger a rebalance, and wait for it to complete.
Once the rebalance has completed, we start a script that simply attempts to connect and retrieve some keys we previously inserted. Instead of success, we find that it repeatedly gives the errors mentioned above.

note that if we remain connected to the cluster, then everything seems fine. The errors only start occurring when a client connects.

note also that the issue only occurs if the first server (or servers) in the list are down; if the first server listed is currently actively in the cluster, then we're OK.

It seems like the problem is that the now-removed server is still accepting connections on port 8091, and thus the client library thinks that this means it is a valid server. However because it's removed from the pool, it causes confusion and errors to the client.

Obviously, the correct/desired behaviour would be for the client library to behave as if the removed-server was not accepting connections, and to move onto the next server in the node list.

[CCBC-190] Conncache configuration not passing integration tests

Dependency — Mon, 1 Apr 2013 12:39:36 -0500

There are two primary issues which occur during server configuration changes (i.e. failover, rebalance):

1)
[SDKD(WARNING) 56.55 cbsdk.sdkd.remote remote.py:266] sdkd_lcb: src/event.c:211: parse_single: Assertion `(lcb_size_t)idx < c->instance->nservers' failed.

2)
The client may just hang.

[CCBC-189] Create a compat mode that cache the configuration objects

Tue, 26 Feb 2013 13:23:21 -0600

Create a compat mode that cache the configuration objects

[CCBC-187] Set TCP_NODELAY flag

Mon, 25 Feb 2013 01:14:48 -0600

We think it is good idea to set TCP_NODELAY flag to switch off the Nagle mode
(don't wait for sending a short packets)
Couchbase server, as we think, use this flag

[CCBC-186] Request to provide concrete examples in the documentation for OBSERVE functionality

Thu, 21 Feb 2013 15:17:58 -0600

http://support.couchbase.com/tickets/2608

Customer are interested in using OBSERVE functionality to allow them to check whether items have been replicated successfully, as they are worried about unreplicated items.

https://github.com/couchbase/libcouchbase/blob/master/man/man3couchbase/lcb_observe.3couchbase.txt
has basic info on the OBSERE functionality in libcouchbase, but we need to provide concrete examples on how to check multiple keys having been replicated successfully (they do a bulk write, so should be more efficient to wait for all of them, rather than wait for them individually)

Examples provided to the customer :

You will find the below example very useful for simple operations :

https://github.com/couchbase/php-ext-couchbase/blob/master/simple_observe.c

The below example would provide more insights when you have any performance or reliability issues :

https://github.com/couchbase/php-ext-couchbase/blob/master/observe.c

The basic idea here is to constantly "poll" the keys (sleeping within a given interval) until all of them are declared as "OK". The structures involved are the "observe_expectation" (persistence/replication requirements), the "pollprefs" (i.e. timeout, interval, etc.) and the "keystate" - this keeps tabs on how the actual status of each key matches up with the specified criteria. The lcb_observe callback is called for each "status report" for each key from each server; so it is necessary to check and see if the requirements have been met during each individual update.

[CCBC-183] Implement approach to specify behaviour for lcb_get_replica()

Fri, 15 Feb 2013 12:10:15 -0600

Implement approach to specify behaviour for lcb_get_replica()

[CCBC-177] Docs: Reference installation

Mon, 11 Feb 2013 05:36:17 -0600

Customers have lots of choices at their fingertips, and many of those choices will result in a non-functioning installation, or at least a lot of time spent figuring out what the right pieces are.

Can we please provide as much details as possible about a reference installation that we can be sure will work for a customer who is just getting started? This is outside of our support for any one operatin system, but more about telling the customer "if you install with these versions and packages, this code will work"

-OS
-other packages/modules required and/or tested with

[CCBC-174] Error handling documentation

Tue, 5 Feb 2013 02:39:18 -0600

Please create some documentation specifying possible error/failures to operations, what they "look" like in the logs/exceptions/stack traces and what our recommendation is on how to handle them.

i.e. tmp_oom, timeouts (connection/operation/java-internal/etc), "get miss" (it's technically a failure, let's make it overly obvious what it means), CAS failure, add() failure, replace() failure,

Some of this should be covered in the API reference, but this bug is specifically for a single page where this information is aggregated that a customer/user could read about how to handle errors.

[CCBC-170] After upgrading to 2.0.2 with CBC163, I now get timesouts again!!!

Wed, 6 Feb 2013 15:43:10 -0600

I was using 2.0.0 beta3 with the patch for TIMEOUTS manually applied....this was working fine but I had high CPU. This ran for weeks (if not a month) without any
timeouts at all...I was the happiest with lcb as I had been in a long time. However, after getting some complaints from my server admins and NOC, I upgraded to
2.0.2 with the fix for high CPU...

I rebuilt all modules and rolled to production. It's now been about 4 hours and I have received 2 timeouts as to where I had NONE for weeks without this change.

The load is now normal though but somehow something has broken because of this fix.

I looked at the change with CBC-163, but I don't see how this could have broken the code...but, it has - I promise. Can someone please take a look?

HELP!

[CCBC-169] Handle 30x HTTP response code when chosen node doesn't have active vbuckets

Dependency — Tue, 5 Mar 2013 16:08:38 -0600

Handle 30x HTTP response code when chosen node doesn't have active vbuckets

[CCBC-152] Provide lcb_check to (weakly) check for the existence of a key

Fri, 21 Dec 2012 13:27:18 -0600

This API will be a variant of OBSERVE and shall do a simple probe against the master node to see if the key exists in cache.

We need to have this in libcouchbase as normal observe is implemented by 'broadcasting' to all the nodes

[CCBC-145] Provide API for durability operations

Dependency — Fri, 21 Dec 2012 18:59:38 -0600

This feature will provide a callback-oriented API for durability (aka observe-poll) operations. Considering that the logic of implementing these operations has proved somewhat more complex than intended, it would be better to centralize this in libcouchbase.

The API will provide a "durability set" which is a group of keys to be monitored, a "per-key" callback for when each key reaches its persistence/durability requirements, a "set callback" for when all keys have been endured (or have encountered a condition in which no further retries can be made)

[CCBC-144] Standardize view option handling

Thu, 10 Jan 2013 15:47:26 -0600

Centralize view option conversion, validation, and construction so dependent code does not have to repeat the same boilerplate all the time. This makes it easier for client libraries to adapt to new view options, as well as provide common expected behavior with current options.

[CCBC-135] lcb_server_purge_implicit_responses: Assertion `nr == sizeof(req)' failed

Fri, 14 Dec 2012 13:04:43 -0600

Cannot reproduce, this is all the info I have

FAIL GetMulti - MgetOrdered [tests/phpt/GetMulti/MgetOrdered.phpt]

mnunberg@csure:/sources/lcb2.0-repo/php$ cat tests/phpt/GetMulti/*.out
PHP_COUCHBASE_OK
php: src/server.c:799: lcb_server_purge_implicit_responses: Assertion `nr == sizeof(req)' failed.

*Might* be related to ~~CCBC-120~~

To reiterate again, I cannot reproduce this - I just happened to notice it

[CCBC-128] Allow semi-static bootstrapping

Tue, 5 Mar 2013 16:08:38 -0600

I'm marking this as critical because many folks seem to frown upon the high delays (read http connection latency) when establishing a new couchbase client.

In this case, this would be a constructor option available for lcb_create which will accept an opaque 'map file'. This 'map file' will really just be a dump of a vbucket configuration. What makes this different from compat mode is that the clients will bootstrap when a network error or timeout occurs.

However in the normal model of spawn -> mc operation, where the cluster config is fairly static, there is no need for the latency induced by an extra http connection.

We'd need to implement file-locking semantics to write to this file (we'd maybe just ask for a path to read/write the config to/from, perhaps checking the timestamp, too).. as we'd have multiple clients trying to read/write to this. This is simple enough in Unix

This is in conjunction with a different discussion regarding how to best handle re-bootstrapping, where it was suggested that we really don't need the bootstrap connection in the first place.

There are inherent risks involved with this approach, for example, if one specifies the wrong nodes, then we only know about this during the re-bootstrap.

[CCBC-111] Make iops functions 'private', and discourage calls to run_event_loop/stop_event_loop

Mon, 10 Dec 2012 07:47:33 -0600

There's quite a bit of user code which calls io->run_event_loop and io->stop_event_loop.

These functions are largely private and are intended for use by libcouchbase itself. The fact that they are public API is only accidental, due to the fact that they are part of the iops structure.

These functions should never be called by user code:

1) They are a source of confusion (especially when mixed with lcb_wait)
2) They prevent any kind of wrapping/sanity logic done by lcb_wait (some already exists, and some can be implemented in future versions)

The following should be done

1) Remove the iops structure definition from being included by the main header file. Thus, #include <libcouchbase/couchbase.h> should not allow a dereference of the iops structure. Rather the iops structure should be defined in a new header file (iops.h) - so that users who want to explicitly manipulate and write event loop plugins can include it, but should otherwise not be present in 'user' code. types.h should include a predeclaration of the structure so that it may still be passed around as a simple pointer value.

2) Document the purpose of run_event_loop/stop_event_loop. Make it very clear that lcb_wait should be used in all synchronous use cases. If we have any examples using run_event_loop/stop_event_loop, remove them.

3) [ Optional? ] - In order to further discourage users from using these functions, throw a compiler warning/error for those functions unless LIBCOUCHBASE_INTERNAL is defined - tbd

[CCBC-110] Implement lcb_reconnect() which will reconnect all dead sockets

Tue, 20 Nov 2012 14:40:01 -0600

This functionality very important for long-living connection handles

[CCBC-108] Allow per-command retries for not-my-vbucket and other transient errors

Mon, 10 Dec 2012 07:47:32 -0600

We should avoid the possibility of 'infinite' errors and allow the user the ability of seeing the 'real' error which caused the command to fail..

http://review.couchbase.org/13177

[CCBC-127] libvbucket RPM is missing /usr/lib64/libvbucket.la file

Fri, 16 Nov 2012 17:51:51 -0600

The libvbucket1 rpm does not include the libvbucket.la file. The libcouchbase.la file references libvbucket.la as a dependency.

Linking via libtool against libcouchbase fails because of this. For example, compiling the php-ext-couchbase package fails like:

/bin/sh /root/code/php-ext-couchbase/libtool --mode=link cc -DPHP_ATOM_INC -I/root/code/php-ext-couchbase/include -I/root/code/php-ext-couchbase/main -I/root/code/php-ext-couchbase -I/usr/include/php -I/usr/include/php/main -I/usr/include/php/TSRM -I/usr/include/php/Zend -I/usr/include/php/ext -I/usr/include/php/ext/date/lib -I/usr/include/ -DHAVE_CONFIG_H -g -O2 -o couchbase.la -export-dynamic -avoid-version -prefer-pic -module -rpath /root/code/php-ext-couchbase/modules compress.lo couchbase.lo fastlz/fastlz.lo -lz -lcouchbase
grep: /usr/lib64/libvbucket.la: No such file or directory
/bin/sed: can't read /usr/lib64/libvbucket.la: No such file or directory
libtool: link: `/usr/lib64/libvbucket.la' is not a valid libtool archive
make: *** [couchbase.la] Error 1

I created a fake libvbucket.la by copying /usr/lib64/libcouchbase.la and editing it, and it allowed the compile to succeed.

[CCBC-116] Heavy errors during short network loss

Wed, 14 Nov 2012 11:55:10 -0600

During network loss to the nodes, there are heavy network errors - a combination of both timed-out operations as well as operations which fail with a network error. Might we want to provide an option to retry these operations?

[CCBC-100] Allow Incremental Row Callbacks

Sat, 8 Dec 2012 00:06:11 -0600

View query results may return large amounts of data. We should offer a row callback which returns a JSON string whose contents is a single row.

There are awesome JSON libraries to do this task, and it's something that I've found myself doing in the Perl client.. so why not move it into libcouchbase and let everyone be happy? :).

Basic idea:
Two callbacks are put into place. The first is a 'row callback' which receives rows, and the second is a complete callback, at which time any other non-row data is returned as a JSON object.

So for example

{ "total_rows" : 30,
  "rows" : [
   { "id" : "foo" },
   { "id" : bar" },
     ....

The row callback will first receive the string '{ "id" : "foo" }', then the second time around, '{"id":"bar"}'
The complete callback will return:
' { "total_rows" : 30 } '

[CCBC-83] Update the getting started to match the Beer sample DB

Thu, 18 Oct 2012 11:53:21 -0500

Once the sample DB has been integrated into the server, update the getting started (both on the web page and in the documentation) to use the sample database.

[CCBC-84] Updated screencast for /develop pages

Thu, 12 Jul 2012 00:56:01 -0500

Create an updated screencast to ship with the new 2.0 developer SDKs

[CCBC-49] API reference is complete

Mon, 10 Dec 2012 07:47:32 -0600

API reference is complete

[CCBC-205] Provide stricter semantics/behavior for configuration cache.

Wed, 1 May 2013 16:43:06 -0500

We may want to create a 'cache-only' mode in which creation of the client fails if the cache cannot be created (e.g. a bad path was passed, or the file is not writeable, etc.)

Currently, the cache is mostly a transparent extra, but there is no way for a user to determine whether the mechanism is at all working.

[CCBC-3] libcouchbase lacks a tutorial

Duplicate — Thu, 2 May 2013 16:33:11 -0500

libcouchbase lacks a tutorial document

[CCBC-204] configuration cache does not work for memcached buckets

Wed, 8 May 2013 06:31:57 -0500

Currently, the configuration cache relies on not_my_vbucket responses as an indicator that the configuration needs to be updated. In an environment with memcached buckets only though, there will not be such a response and thus we need another way to reliably update the configuration.

One thought would be to simply ensure the configuration cache is not more than NN seconds old. The process who has the configuration connection would simply need to touch it every NN-1 seconds if the configuration connection is still active.

If that process exits, then after NN seconds, processes will race to create a new configuration cache file.

The only downside I can see is that certain failure modes (stale TCP socket) would still need to be handled.

[CCBC-207] Failed connection attempts will hang until timeout interval is reached on win32

Wed, 8 May 2013 15:55:16 -0500

Failed connection errors are not detected in the select() call. This is because the select call does not add the socket to the exception fdset.

MSDN documentation states that failed connections are delivered through exceptfds, and that only successful connections are delivered as writability notifications. See http://msdn.microsoft.com/en-us/library/windows/desktop/ms740141(v=vs.85).aspx

[CCBC-198] Instance should close REST connection upon receiving valid config cache

Sat, 11 May 2013 13:28:28 -0500

If a valid config cache configuration is received, there is no point in keeping the REST socket open; (one of the reasons someone would even want to use a cached config is in order to reduce load from the server - so we should try to minimize connections in such cases).

Close the socket immediately (or eventually) if a valid configuration is obtained. Re-open as necessary.

[CCBC-137] libcouchbase test improvements

Sat, 11 May 2013 13:31:12 -0500

Generic improvements and fixes for libcouchbase test suite

[CCBC-139] add timeout tests for libcouchbase

Sat, 8 Dec 2012 00:11:17 -0600

add timeout tests for libcouchbase

[CCBC-4] libcouchbase lacks sample applications

Sat, 11 May 2013 13:31:34 -0500

Libcouchbase requires sample applications for completeness.

[CCBC-142] test libcouchbase observe functionality

Sat, 8 Dec 2012 00:11:17 -0600

test libcouchbase observe functionality

[CCBC-210] segfault from lcb_wait and libev

Thu, 16 May 2013 04:52:14 -0500

When running a test with a broken cluster, I happened to have PHP segfault after a long, long wait.

#0 0xffffffffffffffff in ?? ()
#1 0x00002aab69d1fab4 in ?? () from /usr/lib64/libcouchbase.so.2
#2 0x00002aab69d16421 in ?? () from /usr/lib64/libcouchbase.so.2
#3 0x00002aab6b271e73 in ev_invoke_pending () from /usr/lib64/libev.so.4
#4 0x00002aab6b276688 in ev_run () from /usr/lib64/libev.so.4
#5 0x00002aab6b06dd86 in ?? () from /usr/lib64/libcouchbase_libev.so
#6 0x00002aab69d1e698 in lcb_wait () from /usr/lib64/libcouchbase.so.2
#7 0x00002aab69af22b3 in php_couchbase_view_impl (ht=2,
    return_value=0x178de850, return_value_ptr=0x0, this_ptr=0x178dcf28,
    return_value_used=1, oo=1, uri_only=0)
    at /tmp/tmpwlG54H/couchbase-1.1.5/views.c:347
#8 0x00002aab69adfe86 in zim_couchbase_view (ht=395387424, return_value=0x1,
    return_value_ptr=0x11, this_ptr=0x0, return_value_used=16)
    at /tmp/tmpwlG54H/couchbase-1.1.5/apidecl.c:768
#9 0x00000000005e78e9 in ?? ()
#10 0x00000000005bd9ab in execute ()
#11 0x0000000000599f45 in zend_execute_scripts ()
#12 0x000000000054ad48 in php_execute_script ()
#13 0x0000000000622cde in ?? ()
#14 0x0000003d16a1d994 in __libc_start_main () from /lib64/libc.so.6
#15 0x0000000000421789 in _start ()

[CCBC-211] libcouchbase may touch a failed node after failover

Tue, 21 May 2013 10:24:35 -0500

Library version: v2.0.6 (compiled from the source code, on x64)
Server version: v2.0.1
Number of nodes in my test setup: 3 (2 x CentOS 6.2 x64, and 1 x CentOS 5.8 x86, if it's of any relevance)

Steps to reproduce:

1. Create a new cluster
2. Print the returned value by lcb_get_server_list function (there shoud be 3 servers printed)
3. Bring down network on one of servers
4. Try to put several values, until one of them is routed to the failed node. The library should return an error, which is expected.
5. Repeat 2
6. Manually fail over the failed node.
7. Print the returned value by lcb_get_server_list function (there should be 2 servers printed)
8. Try to put several values, until one of them is routed to the failed node. This behaviour is not expected.
9. Start rebalance.
10. While rebalancing takes place, a request might be routed to the failed node. It is not expected as well.
11. Once rebalance is completed, no requests are routed to the failed node. It is as expected.

Thanks in advance for your support.

[CCBC-212] IO Enhancements

Dependency — Tue, 21 May 2013 11:46:07 -0500

This super bug is an umbrella task for refactoring our IO system to be more streamlined, readable, and compatible with other I/O models.

[CCBC-213] Unify internal connect() routines

Dependency — Tue, 21 May 2013 11:45:13 -0500

This will unify the connection routines in terms of error handling and callback invocations.

[CCBC-214] Re-design internal operations in a queue/linked list structure

Dependency — Tue, 21 May 2013 11:46:07 -0500

Re-design internal operations in a queue/linked list structure

[CCBC-129] Add libevent support for win32

Wed, 21 Nov 2012 12:53:20 -0600

I created a NMakefile and made changes to the code to support libevent on windows.

Files affected:
iofactory.c
plugin-libevent.c

New file:
NMakefile.libevent

I uploaded a zipfile of the changes if you are interested in merging them.

[CCBC-101] allow google test suite to be preserved between invocations of make distclean and friends

Tue, 18 Sep 2012 11:35:28 -0500

additionally we should make it into a DSO and link the tests against that.. would make it somewhat quicker..

This is obviously not a major issue.. but would be nice :)

[CCBC-78] Improved Sample Code for "Simple Example"

Thu, 5 Jul 2012 16:25:15 -0500

Hi,

I received a variety of content corrections from a customer for the C SDK. I made all possible, however some require changes in the sample code the SDK Team provides. The existing sample code is here:

http://www.couchbase.com/docs/couchbase-sdk-c-1.0/simple_example.html

The customer requests:

- The tutorial example bails from callbacks with exit(), rather than
passing control back to main (after setting a global flag, for example,
or one passed in via the cookie, etc.).
- The tutorial should also show use of libcouchbase_remove() and what to
expect from libcouchbase_mget() afterwards (the latter is in the Python
docs, but not the C docs).
- The tutorial should use libcouchbase_destroy().
- Some mention of whether a disconnect is needed or not would be
appropriate.

Let me know if you are able to demonstrate use of these other methods in the sample code and send the code. I will add the new sample and document.

Also let me know if a C client needs to be destroyed and if so how.

[CCBC-161] libcouchbase_flush wrongly documented to "flush entire cluster"

Thu, 2 May 2013 16:32:40 -0500

The docs say that libcouchbase_flush() will "Flush the entire cluster" and "libcouchbase_flush may be used to remove all key/value pairs from the entire cluster."

It actually flushes a bucket, not the entire cluster.

[CCBC-206] Improve libcouchbase rpm instructions

Thu, 2 May 2013 16:35:16 -0500

There is currently no documentation about installing libcouchbase from the rpm or deb files.
The only documentation is for installation with yum or apt-get.

Since some packages are optional, more instructions would be nice.