Unexpected (mis) behaviour parsing JSON

Ruby SDK
1

First thing, I want to post this as a bug report, but you don’t have github Issues and your Jira is apparently only accessible to Couchbase employees.

We are having very odd issues, apparently with JSON being parsed between the Couchbase server and the data being returned from a Model. We have a fairly simple set of mapreduce views that appear to be unparseable by the YAJI library used by Couchbase Ruby Client.

Having traced the issue, it looks like at some point about here the JSON parser silently fails and returns a nil value instead of the correct Couchbase View object.

The JSON objects passed to the parser look like so:

["{\"total_rows\":147,\"rows\":[\r\n{\"id\":\"216a9481-2eee-4bf4-a3c8-75eaba1f4ea3\",\"key\":\"102e7db2-47c2-41ae-88ba-e8cb08de3fa2\",\"value\":{\"id\":\"216a9481-2eee-4bf4-a3c8-75eaba1f4ea3\",\"release\":{\"type\":\"release\",\"redirect_ids\":null,\"redirect_reason\":null,\"title\":\"Vous Etes Fous!\",\"labels\":[{\"label\":{\"id\":\"102e7db2-47c2-41ae-88ba-e8cb08de3fa2\"},\"catalog_number\":\"1990/2\"}],\"bar_code\":null,\"sdrm_code\":null,\"release_type\":null,\"c_line\":{\"year\":1990},\"distributor\":null,\"medias\":[{\"id\":\"16c5e03a-660f-49bd-8e71-44e842b6e5c7\",\"tracks\":[{\"id\":\"c92117bb-5274-47a6-80fb-6dbda016aa59\",\"position\":\"A\",\"recording\":{}},{\"id\":\"548308f8-85cc-4798-88d0-7097750da473\",\"position\":\"B\",\"recording\":{}}]}],\"release_events\":[{\"date\":\"1990-05-18\",\"area\":{\"name\":\"Belgium\",\"country_codes\":[\"BE\"]}}],\"artists\":[],\"external_id\":null},\"catalog_number\":\"1990/2\"}},\r\n{\"id\":\"8be52a2b-3636-43a9-8768-8468ca9562d7\",\"key\":\"102e7db2-47c2-41ae-88ba-e8cb08de3fa2\",\"value\":{\"id\":\"8be52a2b-3636-43a9-8768-8468ca9562d7\",\"release\":{\"type\":\"release\",\"redirect_ids\":null,\"redirect_reason\":null,\"title\":\"Vous Etes Fous!\",\"labels\":[{\"label\":{\"id\":\"102e7db2-47c2-41ae-88ba-e8cb08de3fa2\"},\"catalog_number\":\"1990/2\"}],\"bar_code\":null,\"sdrm_code\":null,\"release_type\":\"CD\",\"c_line\":{\"year\":1990},\"distributor\":null,\"medias\":[{\"id\":\"6631e2c2-b54e-41b8-b4b9-386eba48a810\",\"tracks\":[{\"id\":\"76e7a069-e498-46e6-ac6e-adeecfbb6333\",\"position\":\"1\",\"recording\":{}},{\"id\":\"a3814078-ef2b-465b-b2ef-d7bb57127cc5\",\"position\":\"2\",\"recording\":{}}]}],\"release_events\":[{\"date\":\"1990-05-18\",\"area\":{\"name\":\"Belgium\",\"country_codes\":[\"BE\"]}}],\"artists\":[],\"external_id\":null},\"catalog_number\":\"1990/2\"}}\r\n]\r\n}", "\n", nil]

We have multiple other models and views with almost identical structure that work as expected. This issue seems persistent across something in our data. In several tests with several JSON parsers we have not been able to identify a problem with the JSON.

Unless there is an easy, overlooked solution, we will likely fix this manually using an alternate parser and a monkey patch, so look forward to a pull request in the coming days.

2

Thanks for finding this issue. I will take a look at the data you have posted. If you will manage to write a standalone script to reproduce it, it would be great.

3

@grant, okay I think I’ve reproduced that. I will fix the issue soon

require 'rubygems'
require 'yaji'

parser = YAJI::Parser.new(:filter => '/rows/')

objects = []
parser.on_object { |obj| objects << obj }

data = ['{"rows":[{"value":{}}]}']
data.each { |datum| parser << datum }

abort('reproducible') if objects.size != 1
4

@grant, I’ve posted the fix here: https://github.com/avsej/yaji/commit/6b352d63ec0775ff2ef59de069ad7e5b7bb5d538

you can install it from git and check, tomorrow I will push it on rubygems. Basically the couchbase dependency restriction will pick up this fix version automatically (it will be 0.3.6) .

5

Awesome! Thanks. You just saved me a couple hours of patching and testing.