base64 value display in GUI (Beta 2.0)
Hi,
Since beta 2.0, it looks like non-json values are stored in base64 in database.
All data is still decoded when I use memcache or couchbase libs so that's OK, however the GUI displays the base64-encoded value on the document page and also when previewing a random doc on the view page.
You can reproduce easily using the Couchbase GUI to create a new document and delete all sample content (including curly brackets) and only type some text between double-quotes and saving. It will update the content zone with its base64-encoded version.
You can verify this as well on the view page but you need a bucket with almost no document or you'll have trouble getting the random doc to be your test one.
This is using the Windows 2.0.0 build 1723 version.
I'm not sure at which level this should be fixed and am no professional programmer but I fixed this on my system by editing the following 2 files (sorry for windows paths):
In C:\Program Files\Couchbase\Server\lib\ns_server-2.0.0r_331_g08fb51b\priv\public\js\documents.js
On line 444, replace this line:
self.jsonCodeEditor.setValue(isError ? '' : JSON.stringify(obj.json, null, " "));
With this one:
self.jsonCodeEditor.setValue(isError ? '' : (obj.meta.type=="base64" ? '"'+decodeURIComponent(escape(window.atob( obj.json )))+'"' : JSON.stringify(obj.json, null, " ")));
In C:\Program Files\Couchbase\Server\lib\ns_server-2.0.0r_331_g08fb51b\priv\public\js\views.js
Insert the following line at line 1254:
if (doc.meta.type=="base64") { doc.json = decodeURIComponent(escape(window.atob( doc.json ))); }
I'm just using the window.atob() function so I think it does not work in IE. You guys have probably a better way to decode base64.
I just thought you'd like to know.
Sorry, I wasn't aware this feature would be removed. Probably didn't search enough...
May I ask the reason why this is removed ? Is it causing UI performance problems ?
I find this too bad as we are using Couchbase with Nginx to read/write non-jason values with the memc module and would have liked to see them in UI.
It would be easy for us to make it set json content instead but we haven't yet found an efficient way to parse json result on a GET (in Nginx)
We cannot show arbitrary blobs in UI. Post 2.0 we'll consider doing something. But right now document editing/browsing UI is very much json-oriented.
Hi Guys, this issue is not related to storing binary data. I upgraded my server to 2.0 beta today on windows 2008 server and my c# client which stores json objects in the data store is now showing the same encrypted string when i try to edit the document with the Admin UI!!!
I use the latest source code for the .net client from github and this is what the code and object class look like:
var user = new FbUser
{
//Id = key,
UserId = id,
IsDealer = isdealer,
IsRobot = isrobotic,
Owe = 0,
Owes = 0,
UserName = String.IsNullOrEmpty(userName) ? id.ToString() : Util.Truncate(userName, 50).Trim(),
Bookmark = false,
IsFan = false,
Note = strnote,
UserPin = pin,
Email = Util.Truncate(stremail, 150).Trim(),
IsAppUser = appuser,
Thumbnail = Util.Truncate(strthumbnail, 250),
Credits = isdealer ? Convert.ToDecimal(ConfigurationManager.AppSettings["NewDealerFreeChips"]) : Convert.ToDecimal(ConfigurationManager.AppSettings["NewUserFreeChips"]),
AccessToken = Util.Truncate(token, 256),
ActivationKey = Util.Truncate(activationKey, 250),
DeviceId = Util.Truncate(deviceId, 128),
DeviceModel = Util.Truncate(sysModel, 50),
DeviceType = Util.Truncate(sysVer, 50),
CreatedTime = DateTime.Now
};
Entities.Couchbase.couchbaseClient.StoreJson(StoreMode.Replace, user.type + "_" + id, user);
public abstract class ModelBase
{
[JsonProperty("id")]
public string Id { get; set; }
[JsonProperty("type")]
public abstract string type { get; }
[JsonProperty("rev")]
public abstract string rev { get; }
[JsonProperty("createdTime")]
public System.DateTime CreatedTime { get; set; }
}
public class FbUser : ModelBase
{
public override string type { get { return "fbUser"; } }
public override string rev { get { return JsonConvert.SerializeObject(DateTime.Now, new IsoDateTimeConverter()); } }
[JsonProperty("userId")]
public long UserId { get; set; }
[JsonProperty("loginDate")]
public System.DateTime LoginDate { get; set; }
Any ideas as to how to change the .net api so that your UI treats it as what it is! JSON....
Thanks in advance
ezrover, as pointed out above editing anything non-json in couchbase server's UI doesn't really work. We plan to disable showing and editing non-json documents in final 2.0.
Hi Alkondratenko, Could you please point out to me which part of my JSON object that I am storing with the latest version of .Net client in above code is NON Json?!!!
This is as Json as it gets... The other thing i noticed is the _id parameter is now storing as null whereas on the previous version it held the actual id of the row.
I need to know which part of my json object is flawed that causes the new Admin UI to display it as Base64 since it thinks it not json!
Do you c what i mean?
Thanks
Sorry misunderstood you. Your code sample is unreadable (likely forum software problem). You should point your question to .net sdk folks.
I c. Good idea. Thanks so much.
FWIW, the forum does allow one to post code and it'll even syntax highlight.
if a document contains only one numeric value generated by atomic increment function in API, then we won't be able to debug views and view documents from UI.
Right now the functionality of those increment operations is limited. It can't work with JSON documents, only with blob documents.
Remove the ability to display base64 in UI will affect the users who use increment/decrement operations.
Something I noticed on 2.0.0-1723-rel-community: if you go to the UI and create a document but add a _id, it will no longer display it as JSON when attempting to edit the document, it displays Base64, I wonder if this is a bug or intentional.
We had a number of bugs in this area. It's hard for me to say how recent 1723 is but in some more recent release it should be fixed.
Author: Aaron Miller
Date: Wed Aug 8 12:15:23 2012 -0700
MB-6143 Use JSON.org JSON validator
We don't need to actually parse JSON bodies, and the cJSON parser is not
strict enough, as it will accept JSON values followed by garbage, and
other problem cases.
Change-Id: I77969880309ea52f21150f930397400f0226fe98
Reviewed-on: http://review.couchbase.org/19374
Reviewed-by: Aliaksey Kandratsenka
Tested-by: Aaron Miller
and
Author: Aaron Miller
Date: Tue Oct 23 17:57:17 2012 -0700
MB-7006 Add tests for JSON validation
Change-Id: I7ece614adf3dc6c8be92d6168d3e80f4aae7418c
Reviewed-on: http://review.couchbase.org/21905
Tested-by: buildbot
Reviewed-by: Steve Yen
Tested-by: Steve Yen
commit f7fa4f21aaaf8d25d490c0291a320331c16431fc
Author: Aaron Miller
Date: Tue Oct 23 17:01:48 2012 -0700
MB-6961 Fix validation of numbers as JSON
Change-Id: Ibe14f516283ccdfd26795a0b9e9659d3b5cf8932
Reviewed-on: http://review.couchbase.org/21901
Reviewed-by: Chris Anderson
Reviewed-by: Chiyoung Seo
Tested-by: Chiyoung Seo
Reviewed-by: Damien Katz
Reviewed-by: Steve Yen
Now _any_ json value (including object literals, arrays, numbers and strings) should be recognized as json document by couchbase.
But note that there's still lots of issues in Couchbase document editing UI, unfortunately. Don't expect it to work well with that generalized json documents yet.
We don't plan to have support for showing non-json documents in final 2.0 UI. Note, all clients currently support and will continue supporting non-json values. It's going to be just UI limitation.
Base64 is just artifact of current implementation.