[MB-4511] JSON encoding during View Indexing over escapes double quotes in values Created: 06/Dec/11  Updated: 18/Jun/13  Resolved: 07/Dec/11

Status: Resolved
Project: Couchbase Server
Component/s: couchbase-bucket
Affects Version/s: 2.0-developer-preview-3
Fix Version/s: 2.0-developer-preview-3
Security Level: Public

Type: Bug Priority: Critical
Reporter: Benjamin Young Assignee: damien
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File beer_90_Minute_IPA.json     File beer.json     File json_encoding_issue    
Flagged:
Impediment

 Description   
I'm getting "invalid_json" warnings during indexing/view runs for JSON docs that are indeed valid both prior to being added to the bucket and after retrieval in the UI.

It seems to center around escaped double quotes in content.

Steps to reproduce:
1. add both of the attached .json docs to a new, empty bucket.
2. access any of the production views. The two that should return results with this single, content document in place are "brewery_beers" and "by_key_value"

After clicking "Show Results" for one of these views (or otherwise triggering view generation), Couchbase Server will generate an invalid_json error message (see the "json_encoding_issue" attached).

An example of the invalidated JSON is below (abbreviated "description" value from the IPA doc):
\"Esquire Magazine calls our 90 Minute .IPA., \"\"\\\"\"\"perhaps the best I.P.A. in America.\"\"\\\"\"\" An Imperial I.P.A. brewed to be savored from a snifter....\"

The invalid_json error is specifically:
"parse error: after key and value, inside map, I expect ',' or '}'\n"

Additionally, we should add a test for this, but that likely deserves it's own ticket.

 Comments   
Comment by Aleksey Kondratenko [ 07/Dec/11 ]
traced down to ejson and/or mccouch issue. By printing original value and value from validate here's what I get:


Value: <<"{\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \\\"perhaps the best I.P.A. in America.\\\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}">>
Json:<<"\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \"\"\\\"\"\"perhaps the best I.P.A. in America.\"\"\\\"\"\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}">>

So ejson:validate is messing with escaped quotes. Assigning to Filipe, who is familiar with ejson.
Comment by Filipe Manana [ 07/Dec/11 ]
Assigning to Aaron, as he wrote the validation feature for ejson recently. It's only enabled bia mccouch afaik.
Comment by Aleksey Kondratenko [ 07/Dec/11 ]
(n_0@127.0.0.1)11> io:format("~s~n", [ejson:decode(<<"\"a\\\"sd\"">>)]).
a"sd
ok
(n_0@127.0.0.1)12> io:format("~s~n", [element(2, ejson:validate(<<"\"a\\\"sd\"">>, <<"_">>))]).
"a""\"""sd"
ok
Comment by Filipe Manana [ 07/Dec/11 ]
A side note, ejson:validate functionality should definitely have unit tests. Unfortunately it's not the current case.
Features so self contained like this, should always have unit tests.
Comment by Sharon Barr (Inactive) [ 07/Dec/11 ]
Damien, can you fix that along your changes?
Comment by Sharon Barr (Inactive) [ 07/Dec/11 ]
http://review.couchbase.org/#change,11528
Comment by Aleksey Kondratenko [ 08/Dec/11 ]
http://review.couchbase.org/11552
Comment by Thuan Nguyen [ 22/Dec/11 ]
Integrated in github-couchdb-preview #221 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/221/])
    
Comment by Thuan Nguyen [ 30/Dec/11 ]
Integrated in github-couchdb-preview #226 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/226/])
    
Generated at Thu Nov 27 17:37:55 CST 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.