Couchbase Server

[RN 2.0.1] Map function failure in a view for a document causes it to be unavailable for all other views in the design document

Details

Description


As per Filipe's comments:

One thing to note is, when the map function of a view fails with a document X, map results for other views of the same design document won't be computed for that same document X.
Only way to avoid this is by having the views in different design documents. This is a limitation inherited from Apache CouchDB, maybe we can get rid of it post 2.0.

This can be little confusing for the users as views appear to be independent of each other.

See http://www.couchbase.com/issues/browse/CBSE-238 for an example case.

Activity

Ascending order - Click to sort in descending order
Farshid Ghods made changes -
Field Original Value New Value
Issue Type Improvement [ 4 ] Bug [ 1 ]
Fix Version/s 2.0 [ 10114 ]
Affects Version/s recent-builds-2.0 [ 10349 ]
Hide
Permalink
Farshid Ghods added a comment -
Dipti,

should we assign this to MC for documentation ?
Show
Farshid Ghods added a comment - Dipti, should we assign this to MC for documentation ?
Farshid Ghods made changes -
Assignee Farshid Ghods [ farshid ] Dipti Borkar [ dipti ]
Hide
Permalink
Dipti Borkar added a comment -
I consider this a bug. Filipe, can you provide feedback on how this can be fixed?

In this case, why did the view fail in the first case?
Show
Dipti Borkar added a comment - I consider this a bug. Filipe, can you provide feedback on how this can be fixed? In this case, why did the view fail in the first case?
Dipti Borkar made changes -
Assignee Dipti Borkar [ dipti ] Filipe Manana [ FilipeManana ]
Dipti Borkar made changes -
Priority Major [ 3 ] Blocker [ 1 ]
Hide
Permalink
Filipe Manana added a comment -
This is not a trivial change to do, and I think it's a bit late to do it.

Not sure if I understand your question about why it failed.
It's a map function run time error for a particular document, we log/report the error and move on. And, results for other views in the same design document for that particular document are not considered.
CouchDB has had this behaviour for years, I don't remember it being otherwise.
Show
Filipe Manana added a comment - This is not a trivial change to do, and I think it's a bit late to do it. Not sure if I understand your question about why it failed. It's a map function run time error for a particular document, we log/report the error and move on. And, results for other views in the same design document for that particular document are not considered. CouchDB has had this behaviour for years, I don't remember it being otherwise.
Hide
Permalink
Volker Mische added a comment - - edited
It's a bit like a compiler, if one source file fails, the program can't be build. The solution: fix your map functions.
Show
Volker Mische added a comment - - edited It's a bit like a compiler, if one source file fails, the program can't be build. The solution: fix your map functions.
Hide
Permalink
Dipti Borkar added a comment -
The solution cannot be fix your map functions. Its very non-intuitive behavior for a database and makes development tough.

Will need to fix in 2.0.1.
Show
Dipti Borkar added a comment - The solution cannot be fix your map functions. Its very non-intuitive behavior for a database and makes development tough. Will need to fix in 2.0.1.
Dipti Borkar made changes -
Labels 2.0-release-notes
Fix Version/s 2.0.1 [ 10399 ]
Fix Version/s 2.0 [ 10114 ]
Affects Version/s 2.0 [ 10114 ]
Hide
Permalink
Dipti Borkar added a comment -
where is the mapreduce_errors file located?
Show
Dipti Borkar added a comment - where is the mapreduce_errors file located?
Hide
Permalink
Filipe Manana added a comment -
For development builds at least, it's in the same directory as all other log files.
Show
Filipe Manana added a comment - For development builds at least, it's in the same directory as all other log files.
Hide
Permalink
Steve Yen added a comment -
bug-scrub: keeping in 2.0.1 for now, as two customers have reported running into this -- adoption issue.
Show
Steve Yen added a comment - bug-scrub: keeping in 2.0.1 for now, as two customers have reported running into this -- adoption issue.
Hide
Permalink
Volker Mische added a comment -
Moving to 2.0.2, it's out of scope for 2.0.1.

Filipe, feel free to move back if you don't agree.
Show
Volker Mische added a comment - Moving to 2.0.2, it's out of scope for 2.0.1. Filipe, feel free to move back if you don't agree.
Volker Mische made changes -
Fix Version/s 2.0.2 [ 10418 ]
Fix Version/s 2.0.1 [ 10399 ]
Dipti Borkar made changes -
Labels 2.0-release-notes 2.0-release-notes customer
Fix Version/s 2.0.1 [ 10399 ]
Fix Version/s 2.0.2 [ 10418 ]
Hide
Permalink
Dipti Borkar added a comment -
Need to move back as its a customer issue and a community issue hit by several folks. As discussed in the bug scrub on Friday, this is a blocker for 2.0.1.
If we cannot make it in this coming week, we can revisit.
Show
Dipti Borkar added a comment - Need to move back as its a customer issue and a community issue hit by several folks. As discussed in the bug scrub on Friday, this is a blocker for 2.0.1. If we cannot make it in this coming week, we can revisit.
Hide
Permalink
Dipti Borkar added a comment -
per bug-scrub: Filipe: I think you have made progress on this. Can you please update the bug re:fix?
Show
Dipti Borkar added a comment - per bug-scrub: Filipe: I think you have made progress on this. Can you please update the bug re:fix?
Hide
Permalink
Dipti Borkar added a comment -
We are trying to lock-down functional bugs by Friday.
Show
Dipti Borkar added a comment - We are trying to lock-down functional bugs by Friday.
Hide
Permalink
Filipe Manana added a comment -
http://review.couchbase.org/#/c/24205/
http://review.couchbase.org/#/c/24206/
Show
Filipe Manana added a comment - http://review.couchbase.org/#/c/24205/ http://review.couchbase.org/#/c/24206/
Hide
Permalink
Dipti Borkar added a comment -
Thanks :-)
Show
Dipti Borkar added a comment - Thanks :-)
Hide
Permalink
Filipe Manana added a comment -
merged
Show
Filipe Manana added a comment - merged
Filipe Manana made changes -
Status Open [ 1 ] Resolved [ 5 ]
Resolution Fixed [ 1 ]
Hide
Permalink
Thuan Nguyen added a comment -
Integrated in github-couchdb-preview #558 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/558/])
    MB-6895 Better map function error isolation (Revision cb32250806b28e9df6892f0ce652a9cb51949738)

     Result = SUCCESS
Filipe David Borba Manana :
Files :
* src/couchdb/couch_view_updater.erl
* src/mapreduce/win32/mapreduce.vcxproj.tpl.in
* src/mapreduce/mapreduce.cc
* src/couch_set_view/test/12-errors.t
* src/mapreduce/mapreduce.h
* src/mapreduce/Makefile.am
* src/mapreduce/test/01-map.t
* src/mapreduce/mapreduce_nif.cc
* src/mapreduce/nif_stl_allocator.h
* src/couch_set_view/src/couch_set_view_updater.erl
Show
Thuan Nguyen added a comment - Integrated in github-couchdb-preview #558 (See [ http://qa.hq.northscale.net/job/github-couchdb-preview/558/ ])      MB-6895 Better map function error isolation (Revision cb32250806b28e9df6892f0ce652a9cb51949738)      Result = SUCCESS Filipe David Borba Manana : Files : * src/couchdb/couch_view_updater.erl * src/mapreduce/win32/mapreduce.vcxproj.tpl.in * src/mapreduce/mapreduce.cc * src/couch_set_view/test/12-errors.t * src/mapreduce/mapreduce.h * src/mapreduce/Makefile.am * src/mapreduce/test/01-map.t * src/mapreduce/mapreduce_nif.cc * src/mapreduce/nif_stl_allocator.h * src/couch_set_view/src/couch_set_view_updater.erl
Hide
Permalink
Deepkaran Salooja added a comment -
For dev-subset queries, with one view having error in mapfuction is causing other views not to give results. Working fine for production/fullset queries.

1. Install 2.0.1-150-rel
2. Create default bucket, load 100000 items using mcsoda
3. Create 1 ddoc, 2 views with view v1 defined to generate run-time error:

curl -X PUT -H 'Content-Type: application/json' 'http://Administrator:password@10.3.3.95:8092/default/_design/dev_d1' \
   -d '{"views":{"v1":{"map":"function(doc, meta){\nemit(doc.foo.bar, null);\n}"}, "v2":{"map":"function(doc, meta){\nemit(meta.id, doc.name);\n}"}}}'

4. Query the views:

Development subset:

Expected error for v1 -
curl -X GET 'http://10.3.3.95:8092/default/_design/dev_d1/_view/v1?stale=false&limit=10'
{"total_rows":0,"rows":[
],
"errors":[
{"from":"local","reason":"{function_clause,\n [{couch_view_mapreduce,'-map/1-lc$^1/1-1-',\n [{error,<<\"TypeError: Cannot read property 'bar' of undefined\">>}]},\n {couch_view_mapreduce,'-map/1-lc$^0/1-0-',1},\n {couch_view_mapreduce,map,1},\n {couch_view_updater,'-do_maps/5-fun-1-',5},\n {couch_view_updater,do_maps,5},\n {couch_view_updater,'-update/3-fun-0-',3}]}"}
]
}

Unexpected error for v2 -
curl -X GET 'http://10.3.3.95:8092/default/_design/dev_d1/_view/v2?stale=false&limit=10&#39;
{"total_rows":0,"rows":[
],
"errors":[
{"from":"local","reason":"{function_clause,\n [{couch_view_mapreduce,'-map/1-lc$^1/1-1-',\n [{error,<<\"TypeError: Cannot read property 'bar' of undefined\">>}]},\n {couch_view_mapreduce,'-map/1-lc$^0/1-0-',1},\n {couch_view_mapreduce,map,1},\n {couch_view_updater,'-do_maps/5-fun-1-',5},\n {couch_view_updater,do_maps,5},\n {couch_view_updater,'-update/3-fun-0-',3}]}"}
]
}


Fullset:

curl -X GET 'http://10.3.3.95:8092/default/_design/dev_d1/_view/v1?stale=false&limit=10&full_set=true&#39;
{"total_rows":0,"rows":[
]
}

curl -X GET 'http://10.3.3.95:8092/default/_design/dev_d1/_view/v2?stale=false&limit=10&full_set=true&#39;

{"total_rows":100002,"rows":[
{"id":"70one-00003e3b9e533668","key":"70one-00003e3b9e533668","value":null},
{"id":"70one-000053b1e684c9e7","key":"70one-000053b1e684c9e7","value":null},
{"id":"70one-00029153d12ae1c9","key":"70one-00029153d12ae1c9","value":null},
{"id":"70one-0002ac0d783338cf","key":"70one-0002ac0d783338cf","value":null},
{"id":"70one-0004d0b59e19461f","key":"70one-0004d0b59e19461f","value":null},
{"id":"70one-00053f5e11d1fe4e","key":"70one-00053f5e11d1fe4e","value":null},
{"id":"70one-00056c20eb5a0295","key":"70one-00056c20eb5a0295","value":null},
{"id":"70one-0006246bee639c7a","key":"70one-0006246bee639c7a","value":null},
{"id":"70one-00063cb5da1826fe","key":"70one-00063cb5da1826fe","value":null},
{"id":"70one-0006aabe0ba47a35","key":"70one-0006aabe0ba47a35","value":null}
]
}
Show
Deepkaran Salooja added a comment - For dev-subset queries, with one view having error in mapfuction is causing other views not to give results. Working fine for production/fullset queries. 1. Install 2.0.1-150-rel 2. Create default bucket, load 100000 items using mcsoda 3. Create 1 ddoc, 2 views with view v1 defined to generate run-time error: curl -X PUT -H 'Content-Type: application/json' ' http://Administrator: password@10.3.3.95 :8092/default/_design/dev_d1' \    -d '{"views":{"v1":{"map":"function(doc, meta){\nemit(doc.foo.bar, null);\n}"}, "v2":{"map":"function(doc, meta){\nemit(meta.id, doc.name);\n}"}}}' 4. Query the views: Development subset: Expected error for v1 - curl -X GET ' http://10.3.3.95:8092/default/_design/dev_d1/_view/v1?stale=false&limit=10&#39; {"total_rows":0,"rows":[ ], "errors":[ {"from":"local","reason":"{function_clause,\n [{couch_view_mapreduce,'-map/1-lc$^1/1-1-',\n [{error,<<\"TypeError: Cannot read property 'bar' of undefined\">>}]},\n {couch_view_mapreduce,'-map/1-lc$^0/1-0-',1},\n {couch_view_mapreduce,map,1},\n {couch_view_updater,'-do_maps/5-fun-1-',5},\n {couch_view_updater,do_maps,5},\n {couch_view_updater,'-update/3-fun-0-',3}]}"} ] } Unexpected error for v2 - curl -X GET ' http://10.3.3.95:8092/default/_design/dev_d1/_view/v2?stale=false&limit=10&#39; {"total_rows":0,"rows":[ ], "errors":[ {"from":"local","reason":"{function_clause,\n [{couch_view_mapreduce,'-map/1-lc$^1/1-1-',\n [{error,<<\"TypeError: Cannot read property 'bar' of undefined\">>}]},\n {couch_view_mapreduce,'-map/1-lc$^0/1-0-',1},\n {couch_view_mapreduce,map,1},\n {couch_view_updater,'-do_maps/5-fun-1-',5},\n {couch_view_updater,do_maps,5},\n {couch_view_updater,'-update/3-fun-0-',3}]}"} ] } Fullset: curl -X GET ' http://10.3.3.95:8092/default/_design/dev_d1/_view/v1?stale=false&limit=10&full_set=true&#39; {"total_rows":0,"rows":[ ] } curl -X GET ' http://10.3.3.95:8092/default/_design/dev_d1/_view/v2?stale=false&limit=10&full_set=true&#39; {"total_rows":100002,"rows":[ {"id":"70one-00003e3b9e533668","key":"70one-00003e3b9e533668","value":null}, {"id":"70one-000053b1e684c9e7","key":"70one-000053b1e684c9e7","value":null}, {"id":"70one-00029153d12ae1c9","key":"70one-00029153d12ae1c9","value":null}, {"id":"70one-0002ac0d783338cf","key":"70one-0002ac0d783338cf","value":null}, {"id":"70one-0004d0b59e19461f","key":"70one-0004d0b59e19461f","value":null}, {"id":"70one-00053f5e11d1fe4e","key":"70one-00053f5e11d1fe4e","value":null}, {"id":"70one-00056c20eb5a0295","key":"70one-00056c20eb5a0295","value":null}, {"id":"70one-0006246bee639c7a","key":"70one-0006246bee639c7a","value":null}, {"id":"70one-00063cb5da1826fe","key":"70one-00063cb5da1826fe","value":null}, {"id":"70one-0006aabe0ba47a35","key":"70one-0006aabe0ba47a35","value":null} ] }
Deepkaran Salooja made changes -
Resolution Fixed [ 1 ]
Status Resolved [ 5 ] Reopened [ 4 ]
Hide
Permalink
Filipe Manana added a comment -
http://review.couchbase.org/#/c/24474/

Doh, this is what happens when there are 3 different view engines.
Also, for spatial (GeoCouch) views, the old behaviour is expected - it will change only when it migrates to the superstart tree structure as production views.

Try the next build with that change merged.
thanks
Show
Filipe Manana added a comment - http://review.couchbase.org/#/c/24474/ Doh, this is what happens when there are 3 different view engines. Also, for spatial (GeoCouch) views, the old behaviour is expected - it will change only when it migrates to the superstart tree structure as production views. Try the next build with that change merged. thanks
Filipe Manana made changes -
Status Reopened [ 4 ] Resolved [ 5 ]
Resolution Fixed [ 1 ]
Hide
Permalink
Thuan Nguyen added a comment -
Integrated in github-couchdb-preview #561 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/561/])
    MB-6895 Fix badmatch on error (Revision da000c225be6ad4821bda8212c27190ecdbe3284)

     Result = SUCCESS
Filipe David Borba Manana :
Files :
* src/couchdb/couch_view_mapreduce.erl
Show
Thuan Nguyen added a comment - Integrated in github-couchdb-preview #561 (See [ http://qa.hq.northscale.net/job/github-couchdb-preview/561/ ])      MB-6895 Fix badmatch on error (Revision da000c225be6ad4821bda8212c27190ecdbe3284)      Result = SUCCESS Filipe David Borba Manana : Files : * src/couchdb/couch_view_mapreduce.erl
Hide
Permalink
Thuan Nguyen added a comment -
Integrated in github-couchdb-preview #562 (See [http://qa.hq.northscale.net/job/github-couchdb-preview/562/])
    MB-6895 Correct error processing for dev views (Revision 356ef20516464c5bd0c22dec643bd27fba618d18)

     Result = SUCCESS
Filipe David Borba Manana :
Files :
* src/couchdb/couch_view_mapreduce.erl
Show
Thuan Nguyen added a comment - Integrated in github-couchdb-preview #562 (See [ http://qa.hq.northscale.net/job/github-couchdb-preview/562/ ])      MB-6895 Correct error processing for dev views (Revision 356ef20516464c5bd0c22dec643bd27fba618d18)      Result = SUCCESS Filipe David Borba Manana : Files : * src/couchdb/couch_view_mapreduce.erl
Hide
Permalink
Dipti Borkar added a comment -
assuming this was because dev views use stock couch. So Filipe, does your comment mean that you fixed it for 2.0.1?
Show
Dipti Borkar added a comment - assuming this was because dev views use stock couch. So Filipe, does your comment mean that you fixed it for 2.0.1?
Hide
Permalink
Deepkaran Salooja added a comment -
Verified with build 2.0.1-153-rel.
Show
Deepkaran Salooja added a comment - Verified with build 2.0.1-153-rel.
Deepkaran Salooja made changes -
Status Resolved [ 5 ] Closed [ 6 ]
Karen Zeller made changes -
Resolution Fixed [ 1 ]
Status Closed [ 6 ] Reopened [ 4 ]
Assignee Filipe Manana [ FilipeManana ] Karen Zeller [ kzeller ]
Karen Zeller made changes -
Planned End (re-schedule end date based on new assignee)
Karen Zeller made changes -
Summary Map function failure in a view for a document causes it to be unavailable for all other views in the design document [RN 2.0.1] Map function failure in a view for a document causes it to be unavailable for all other views in the design document
Component/s documentation [ 10012 ]
Component/s view-engine [ 10060 ]
Hide
Permalink
Karen Zeller added a comment -
Added to RN 2.0.1:

<rnentry type="fix">

<version ver="2.0.0l"/>

<class id="index"/>

<issue type="cb" ref="MB-6895"/>


<rntext>

<para>
In past releases of Couchbase Server 2.0, if you had a map function that failed to index, other functions in the same
design document would also fail. This has been fixed.
</para>


</rntext>

</rnentry>
Show
Karen Zeller added a comment - Added to RN 2.0.1: <rnentry type="fix"> <version ver="2.0.0l"/> <class id="index"/> <issue type="cb" ref=" MB-6895 "/> <rntext> <para> In past releases of Couchbase Server 2.0, if you had a map function that failed to index, other functions in the same design document would also fail. This has been fixed. </para> </rntext> </rnentry>
Karen Zeller made changes -
Status Reopened [ 4 ] Resolved [ 5 ]
Resolution Fixed [ 1 ]
Hide
Permalink
Karen Zeller added a comment -
Added to RN 2.0.1:

<rnentry type="fix">

<version ver="2.0.0l"/>

<class id="index"/>

<issue type="cb" ref="MB-6895"/>


<rntext>

<para>
In past releases of Couchbase Server 2.0, if you had a map function that failed to index, other functions in the same
design document would also fail. This has been fixed.
</para>


</rntext>

</rnentry>
Show
Karen Zeller added a comment - Added to RN 2.0.1: <rnentry type="fix"> <version ver="2.0.0l"/> <class id="index"/> <issue type="cb" ref=" MB-6895 "/> <rntext> <para> In past releases of Couchbase Server 2.0, if you had a map function that failed to index, other functions in the same design document would also fail. This has been fixed. </para> </rntext> </rnentry>
Karen Zeller made changes -
Status Resolved [ 5 ] Closed [ 6 ]

People

  • Assignee:
    Karen Zeller
    Reporter:
    Deepkaran Salooja
Vote (1)
Watch (6)

Dates

  • Created:
    Updated:
    Resolved: