Couchbase Server

JSON encoding during View Indexing over escapes double quotes in values

Details

  • Flagged:
    Impediment

Description

I'm getting "invalid_json" warnings during indexing/view runs for JSON docs that are indeed valid both prior to being added to the bucket and after retrieval in the UI.

It seems to center around escaped double quotes in content.

Steps to reproduce:
1. add both of the attached .json docs to a new, empty bucket.
2. access any of the production views. The two that should return results with this single, content document in place are "brewery_beers" and "by_key_value"

After clicking "Show Results" for one of these views (or otherwise triggering view generation), Couchbase Server will generate an invalid_json error message (see the "json_encoding_issue" attached).

An example of the invalidated JSON is below (abbreviated "description" value from the IPA doc):
\"Esquire Magazine calls our 90 Minute .IPA., \"\"\\\"\"\"perhaps the best I.P.A. in America.\"\"\\\"\"\" An Imperial I.P.A. brewed to be savored from a snifter....\"

The invalid_json error is specifically:
"parse error: after key and value, inside map, I expect ',' or '}'\n"

Additionally, we should add a test for this, but that likely deserves it's own ticket.
  1. File
    beer_90_Minute_IPA.json
    06/Dec/11 12:30 PM
    0.5 kB
    Benjamin Young
  2. File
    beer.json
    06/Dec/11 12:30 PM
    1 kB
    Benjamin Young
  3. File
    json_encoding_issue
    06/Dec/11 12:30 PM
    12 kB
    Benjamin Young

Activity

Ascending order - Click to sort in descending order
Aleksey Kondratenko made changes -
Field Original Value New Value
Assignee Aleksey Kondratenko [ alkondratenko ]
Hide
Permalink
Aleksey Kondratenko added a comment -
traced down to ejson and/or mccouch issue. By printing original value and value from validate here's what I get:


Value: <<"{\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \\\"perhaps the best I.P.A. in America.\\\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}">>
Json:<<"\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \"\"\\\"\"\"perhaps the best I.P.A. in America.\"\"\\\"\"\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}">>

So ejson:validate is messing with escaped quotes. Assigning to Filipe, who is familiar with ejson.
Show
Aleksey Kondratenko added a comment - traced down to ejson and/or mccouch issue. By printing original value and value from validate here's what I get: Value: <<"{\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \\\"perhaps the best I.P.A. in America.\\\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}">> Json:<<"\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \"\"\\\"\"\"perhaps the best I.P.A. in America.\"\"\\\"\"\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}">> So ejson:validate is messing with escaped quotes. Assigning to Filipe, who is familiar with ejson.
Aleksey Kondratenko made changes -
Assignee Aleksey Kondratenko [ alkondratenko ] Filipe Manana [ Filipe Manana ]
Hide
Permalink
Filipe Manana added a comment -
Assigning to Aaron, as he wrote the validation feature for ejson recently. It's only enabled bia mccouch afaik.
Show
Filipe Manana added a comment - Assigning to Aaron, as he wrote the validation feature for ejson recently. It's only enabled bia mccouch afaik.
Hide
Permalink
Aleksey Kondratenko added a comment -
(n_0@127.0.0.1)11> io:format("~s~n", [ejson:decode(<<"\"a\\\"sd\"">>)]).
a"sd
ok
(n_0@127.0.0.1)12> io:format("~s~n", [element(2, ejson:validate(<<"\"a\\\"sd\"">>, <<"_">>))]).
"a""\"""sd"
ok
Show
Aleksey Kondratenko added a comment - ( n_0@127.0.0.1 )11> io:format("~s~n", [ejson:decode(<<"\"a\\\"sd\"">>)]). a"sd ok ( n_0@127.0.0.1 )12> io:format("~s~n", [element(2, ejson:validate(<<"\"a\\\"sd\"">>, <<"_">>))]). "a""\"""sd" ok
Aleksey Kondratenko made changes -
Fix Version/s 2.0-developer-preview-3 [ 10247 ]
Affects Version/s 2.0-developer-preview-3 [ 10247 ]
Priority Major [ 3 ] Critical [ 2 ]
Hide
Permalink
Filipe Manana added a comment -
A side note, ejson:validate functionality should definitely have unit tests. Unfortunately it's not the current case.
Features so self contained like this, should always have unit tests.
Show
Filipe Manana added a comment - A side note, ejson:validate functionality should definitely have unit tests. Unfortunately it's not the current case. Features so self contained like this, should always have unit tests.
Sharon Barr made changes -
Assignee Aaron Miller [ aaron ] Damien Katz [ damien ]
Hide
Permalink
Sharon Barr added a comment -
Damien, can you fix that along your changes?
Show
Sharon Barr added a comment - Damien, can you fix that along your changes?
Hide
Permalink
Sharon Barr added a comment -
http://review.couchbase.org/#change,11528
Show
Sharon Barr added a comment - http://review.couchbase.org/#change,11528
Sharon Barr made changes -
Status Open [ 1 ] Resolved [ 5 ]
Resolution Fixed [ 1 ]
Hide
Permalink
Aleksey Kondratenko added a comment -
http://review.couchbase.org/11552
Show
Aleksey Kondratenko added a comment - http://review.couchbase.org/11552
Hide
Permalink
Thuan Nguyen added a comment -
Integrated in github-couchdb-preview #221 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/221/])
    
Show
Thuan Nguyen added a comment - Integrated in github-couchdb-preview #221 (See [ http://qa.hq.northscale.net/job/github-couchdb-preview/221/ ])     
Hide
Permalink
Thuan Nguyen added a comment -
Integrated in github-couchdb-preview #226 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/226/])
    
Show
Thuan Nguyen added a comment - Integrated in github-couchdb-preview #226 (See [ http://qa.hq.northscale.net/job/github-couchdb-preview/226/ ])     
Peter Wansch made changes -
Component/s bucket-engine [ 10010 ]
Component/s mccouch [ 10071 ]

People

  • Assignee:
    Damien Katz
    Reporter:
    Benjamin Young
Vote (0)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: