Big or Little Endian Byte Order to Create Base64 Keys ...

Gentlefolk,

In Couchbase, the keys are maintained in RAM. Hence, the size of the key is important. At least in my application it is. (I just saved 2.5 bytes per document by changing my keys and saved over 600 MB of RAM.) I want more savings and, hence, wish to compress my key space further.

My keys are derived from a 64 bit unsigned integer assigned by a foreign system. The next obvious step is to move from base10 key strings to base64 key strings.

I use Python v3.3. There is a method: int.to_bytes(length, byteorder, *, signed=False). "byteorder" could be either "big" or "little".

As the low order bits of this quite large number change very quickly, should I make my CB key little endian? Because of the way the bits are mapped to a string, the little endian mode will drive the greatest variation in keys between successive documents. But I lose easy sequential sorting on key value. (I don't currently use this property. It isn't clear that this sorting property is worth maintaining.)

Would little endian keys give me better key distribution across the memcache cluster? (I currently have decent key distribution across the cluster using the decimal form of the key.)

Anon,
Andrew

0 Answers

No answers yet