Unwanted single / double precision conversion during replication from CBL to CB server

Mobile Couchbase Lite
1

I’m reporting this issue on behalf of a user of the Couchbase Lite SDK for Dart I’m maintaining. Here is the original issue: Unwanted single / double precision conversion during replication from CBL to CB server · Issue #557 · cbl-dart/cbl-dart · GitHub

While investigating the problem, I found that the root cause is with the WriteFloat function in Fleece. It is used as part of JSON encoding and in turn when sending document data as JSON during replication. It appears that some floats are not correctly formatted as strings. 307.79998779296875F is converted in to "307.8".

Adding the following assertion to the WriteFloat test in NumericTests.cc results in an assertion failure:

expectDescription("307.79998779296875", 307.79998779296875F);

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FleeceTests is a Catch v2.13.8 host application.
Run with -? for options

-----------------------------------------------------------------------------------------------------------------------
WriteFloatDebug
-----------------------------------------------------------------------------------------------------------------------
/Users/terwesten/dev/fleece/Tests/NumericTests.cc:170
.......................................................................................................................

/Users/terwesten/dev/fleece/Tests/NumericTests.cc:171: FAILED:
  CHECK( floatStr(307.79998779296875F) == "307.79998779296875" )
with expansion:
  "307.8" == "307.79998779296875"

=======================================================================================================================
test cases: 1 | 1 failed
assertions: 2 | 1 passed | 1 failed

NB: NumericTests.cc is currently not included in the FleeceTests CMake target.

2 Likes
2

This is not something that is going to change. The underlying protocol represents a quantity in the smallest possible space. It does not keep the type information.

There are two possibilities here:

  • the type of a document field is known: if you put a double into the document, you explicitly retrieve a double from the document
  • add a type marker to the document:
"aTypedValue" : {
    "type": "double"
    "val": 307.79998779296875
}

… and, note that both doubles and floats are approximate values. They are subject to change due to rounding and representation. String comparison will not, in general, work. The best practice for comparing them is with within bounds: 307.79998779296875 = 307.79998779296869 ± 0.0000000000001

3

That’s not a valid test. “Float” refers to 32-bit IEE floating-point, which only has 24 bits of precision, about six decimal digits. It cannot represent the number 307.79998779296875 exactly; the closest representation is about 307.800. (It won’t be exactly that because 0.8 is an “infinite decimal” in binary that can’t be exactly represented with any precision.)

If you need more precision, you should be using the Double API functions. 64-bit IEE floats have about 15 decimal digits of precision.

There’s a lot of information online about the perils of expecting precision from floating point arithmetic. I don’t have any specific URLs handy, but I’m sure you can get some good info with a quick we search.

4

To clarify: Fleece does store the type information, which can be either Float (32-bit) or Double (64-bit). You choose which type of value to create.

When converting a number to decimal form Fleece is careful not to lose accuracy – it will only display a rounded form if it will parse back to the exact same binary representation.

In this case, parsing “307.79998779296875” and “307.8” to 32-bit float does result in the exact same value.

1 Like
5

@jens is being polite. That is a correction. The string comparison will not work for all the reasons that we both cited. Storing a double in a document, though, will get you a double back.

1 Like
6

@jens has recognized the issue and the fix will be in the upcoming release.

1 Like
7

@jianminzhao Is there a rough timeline for when the next release will become available? Thanks!

1 Like