Urgent Help - AWS EBS IO charges due to Couchbase DP4 activity

I suspect it could be the auto-compaction which may be disabled through the bucket settings.

Also, note that we do not recommend EC2 Micro instances, as there have been issues when the systems are starved of resources.

In any event, I would recommend moving to 2.0 Beta, which was posted this morning and checking the io-- possibly disabling auto-compaction. Beta is at couchbase.com/couchbase-server/beta

what are the implication of disabling auto compaction?

Is it something that you would recommend for production?

Can it be turned on later when we traffic starts to increase?

Finally - when turned back on, what are the cluster side effects during the transition to auto compaction (if at all)?

I tried disabling the auto compaction via the admin interface (web) - removed the checkboxes on database fragmentation and view fragmentation

did a stop , and start

nothing changed - same IO activity is observed.

Any other suggestion (other than upgrading - which might take me a few days)

The other IO activity it may be is related to the statistics it's collecting. There is no method of disabling that at the moment.

One workaround would be to shutdown the server, then move /opt/couchbase/var/lib/couchbase/mnesia contents to another device and put a symlink in it's place. Can you try that?

If possible, move it to the ephemeral local storage so it will still be persistent.

Since I'm running on a micro instance (free tier) I cannot use Instance Store (ephemeral local storage) since it is only available from a small instance upward.

If you have any other suggestion please let me know.

You can edit the file /opt/couchbase/etc/couchbase/static_config

And update loglevel for different component to error to reduce logging.
Restart couchbase-server then.

Eg:-
{loglevel_default, error}.
{loglevel_couchdb, error}.
{loglevel_ns_server, error}.
{loglevel_error_logger, error}.
{loglevel_user, error}.
{loglevel_menelaus, error}.
{loglevel_ns_doctor, error}.
{loglevel_stats, error}.
{loglevel_rebalance, error}.
{loglevel_cluster, error}.
{loglevel_views, error}.
{loglevel_mapreduce_errors, error}.

I changed the logging level per your instructions. stop, restart (two times) - I don't see any significant effect on disk IO. Same levels as before.

Those writes are stats updates.

We know it's going to cause problems for very small virtualised instances. And this will be fixed but definitely not before 2.0 is out.

In general we've found EC2 micro instances don't really work at all with couchbase as well as with it's predecessor membase. This is at least partly caused by extreme burstiness of CPU scheduling of micro instances that we've seen may cause timeouts to be hit quite easily.

For now avoid micro instances.

I'm not planning to run on micro instance when we launch. But for our current phase - it doesn't make any sense to pay the extra bucks. We prefer the charges as result of the excessive EBS IO.

I would suggest the following fixes:

1. easy access to stats level configuration through the admin interface
2. easy configuration of the log output path so that we can direct it to Instance Storage rather than EBS storage.

thanks!
CB