let now = Date()
someMutableDocument.setDate(now, forKey: "now")
I save this document to a collection, then fetch it again and use the SDK to convert the value of “now” back into a native Swift Date. That seems to work if you don’t look closely:
Original Date: 2025-01-13 07:30:14 +0000
Fetched Date: 2025-01-13 07:30:14 +0000
But an equality test between the two dates fails because Couchbase has actually lost precision. Using .timeIntervalSince1970, here’s the two dates:
ISO-8601 places no limit on the number of decimal places of precision and simply says, “both parties need to agree.” Swift’s microsecond precision (6 decimal places) is actually less precision than what’s offered in Go, Rust, .Net, and other languages for which Couchbase offers SDKs.
If I store the timestamp as a Double instead of using the SDK’s date functions, I get back the exact value every time. Is this rounding due to JavaScript on the backend or SQL++ date function limitations?
When I use the official Couchbase SDKs to put native Swift objects into the database and then fetch them, I expect to get back the exact same values. Not ones that have been rounded. I realize we’re talking about microseconds, but technically Couchbase is causing data corruption with Dates. The SDKs ought to provide a warning on their date functions that precision is limited.
Background:
We’re evaluating Couchbase as a replacement for Realm, so I’m testing all the edge-cases to find sharp corners.
TL;DR: If you need to control how much precision your dates express, you should store them as doubles. (If you need the string format for SQL++ data functions, you could store the fractional seconds as a separate value and then combine them when converting back to a Date object.
I believe we followed the behavior of Server’s SQL++ API in encoding seconds to three decimal places.
Swift’s microsecond precision (6 decimal places)
Swift stores dates internally as 64-bit floats, so its precision is rather larger than microseconds. If the accuracy is microseconds, that’s likely due to the underlying kernel clock. (You can get nanosecond precision by using different clocks but they’re not based on a fixed epoch so they’re not useful for dates.)
When I use the official Couchbase SDKs to put native Swift objects into the database and then fetch them, I expect to get back the exact same values.
Couchbase is a JSON database, not a native-platform-objects database. Technically that means it doesn’t even recognize dates as a datatype, but they’re so common in databases that SQL++ defines functions for working with string-encoded dates. And for convenience our API includes methods for converting platform date/time objects to/from such strings.
Not ones that have been rounded.
They have to be rounded somehow! You expect microsecond precision. If we implemented that, users of some other platform whose dates have nanosecond precision would complain. And what about physicists at CERN who need picosecond or femtosecond precision for recording particle collsions?
Expecting lossless operations on floating-point numbers is a well-known fallacy. (I’ve noticed that recent versions of Clang offer a compiler warning when using == to compare two floats, because it’s so often a cause of bugs!) That said, CBL uses some tricky algorithms to avoid losing precision when encoding floating point numbers to ASCII – this is surprisingly hard to do. Coincidentally, we swiped that code from the Swift standard library
Mmm, it’s simpler than that. I see the native SDKs as a contract: “You give me a Date from language X and I’ll store it. When you ask for it back, I’ll give you the exact same value that you stored.”
The underlying storage engine is an implementation detail. Could be JSON or Postgres or binary or a wall of lava lamps, a la Cloudflare. Doesn’t matter. The SDK offers a contract that the value will round-trip without loss.
The precision should match the platform. So if language X offers femtosecond precision in its date type, the SDK for X should handle that.
Right now, the date() functions are misleading because they don’t have any warning about a loss of precision. The existing API functionality ought to be called setDateString() and carry that warning. The setDate() API ought to deliver the same precision as the platform’s native date type and do what is necessary to guarantee that.
It might seem trivial to argue over microseconds. But if I’m feeding the value into a hash to determine object equality, I’m going to get misleading results.
Use the “Leave additional Feedback” link on the SDK documentation page to open a Documentation Ticket to add a caveat to the setDate() method. (The API Reference doesn’t have a feedback link).
You give me a Datefrom languageX and I’ll store it. When you ask for it back, I’ll give you the exact same value that you stored.
Storing as a serialized object is the only universal solution that the database or SDK doesn’t need to know all the native formats. But it would not work when a date from language X is retrieved in a language Y that cannot represent that date; or the database itself needs to perform a function on that date other than equals/not-equals (i.e. birthDate < 1998, year(hireDate) ). So we have trade-offs.
That said, it seems that storing to microsecond precision would solve the Swift Date issue, and I don’t know why couchbase has chosen milliseconds instead of microseconds, since microseconds epoch will fit in 64 bits for 292,000 years, and milliseconds already takes 51 bits. Date Functions | Couchbase Docs
Sure. If a developer is accessing the database from multiple platforms with different conventions for Date handling, it’s now on them to ensure no precision is lost. But if a developer is only ever accessing the date via one Couchbase SDK (here, Swift), I would expect to get the exact same value out as the one I stored.
I assumed this probably had something to do with limitations in SQL++ date functions. It is an odd choice because essentially every language offers at least microsecond precision in Dates.
If you want to store a platform object that way, you should use the platform’s serialization mechanisms; that’s the only way to ensure it. Swift has the Codable interface which supports dates. You can take the resulting data blob and base64-encode it to store it in the database. Or, as I said, you can get the raw 64-bit float and store that.
Our Date API supports our own formats which are cross-platform and cross-language.
Sure. If a developer is accessing the database from multiple platforms with different conventions for Date handling, it’s now on them to ensure no precision is lost. But if a developer is only ever accessing the date via one Couchbase SDK (here, Swift), I would expect to get the exact same value out as the one I stored.
Sorry, Couchbase Lite is not a Swift-centric database. It’s cross-platform, language-neutral, and supports a number of languages. Couchbase Server supports a lot more. So no, we do not and cannot assume a developer is only ever accessing a date via one SDK. Our data format is JSON.
Keep in mind that a 64-bit float only has 52 bits of precision available to it; the other bits are reserved for exponent and sign. (A 64-bit integer of course has more space, but when working with JSON you have to assume the document can go through a parser that converts all numbers to float64, like JavaScript and many others.)
Ok. I mean, I can work around Couchbase here. But I still think it’s disingenuous to offer native Date types in your SDK’s API when the underlying storage engine does not accurately store those types in a non-destructive way. Especially because it’s an issue for every language you support, all of which offer greater date precision than 3 decimal places. This isn’t some niche, Swift-only issue.
You publish an SDK that says, “Give us a Swift Date; we can handle it.” But you really can’t. And there’s no warning in the SDK that lets me know the date API on MutableDocument is limited and I should be very careful if I’m expecting to get back an exactly-equal date.
Competitors don’t have this problem. Realm (also JSON-based document storage), for instance, yields the same date that I stored.
I guess what concerns me more than the issue itself is the attitude that this is totally fine and working as it should. The drawbacks and consequences are just dismissed out-of-hand. That’s okay this time, because I have a workaround. But if I marry Couchbase, what happens in the future where I hit a design snag that does NOT have a workaround?
I opened a ticket earlier on your behalf using the Leave Feedback link in the Swift SDK documentation - including a link to this thread. I don’t know if tickets opened via Leave Feedback are public or not - in case they are the ticket is
If another customer hits a snag that doesn’t have a work-around, we promise not to break your existing applications when fixing their snag.
Unfortunately, storing dates with six decimal precision would break existing things.
There are changes regarding nanosecond precision in dates coming in Couchbase Server 8
p.s. Realm did have a problem with date precision when they were only storing seconds.
@mreiche Oh, don’t worry, I opened many GitHub issues for Realm idiosyncrasies over the years. Nothing’s perfect. Plus, Couchbase hasn’t nuked its entire native ecosystem from orbit randomly, so the scoreboard is currently:
Couchbase: 1
Realm: 0
I will never have anything to do with MongoDB again. Ever.
IIRC, we didn’t initially support getting/setting Date objects in Couchbase Lite. They were added as a convenience, because ISO-8601 dates are a pretty common format in customer databases and because N1QL has functions to operate on them, so it would make sense to offer a mechanism to convert to and from that representation.
And yet, you are the only person whom I’ve ever heard complain about this in the 13 years I’ve been working on Couchbase Lite. ¯\_(ツ)_/¯
Q: Do you have a compelling need to store dates/times down to microsecond precision? Or is this more of a philosophical issue?
