Feature Request: Log REST queries made by the Python client

Python SDK
1

Gentlefolk,

Logs are good. Logs are very good for learning how complex systems, such as Couchbase, operate. I would like the Python SDK to offer me the option of having my REST API calls logged for my inspection.

For example, I am having a problem getting a view query to function properly. Yet, the view operates in the console. What am I doing wrong? (That is a rhetorical question.) The way I would find out is to make sure the Python client is issuing the identical query as the console. If so, I would expect identical results. Then my task is to tweak my use of the client until it emits the matching API call and gets the same result.

That would be very good.

Andrew

2

Hello,

I have created this enhancement request:
http://www.couchbase.com/issues/browse/PYCBC-150

Feel free to monitor this ticket.

Regards
Tug
@tgrall

3

The object returned by query() is an iterator. Printing it will only print a string representation of the iterator state (and ‘rows returned’ is the number of rows currently yielded by the iterator; not the amount of rows inside the index).

You can implicitly iterate over the returned object by doing something like:

rows = list(cb.query(…))

In which the list() constructor will treat the return value as an iterable (as it should be) and simply return the rows.

4

Additionally, you can use the Query.encoded property to see how your query will look like when sending it over the network.

5

As added information (bonus points for actually helping me solve the problem):

I have a test database where my Python client sits on the Twitter sample endpoint and catches tweets. This works great and I am very happy with the results.

Now I want to write a second script to trim the database. (The sample endpoint produces about 12 GB/day.)

I have a view that emits documents using numeric key of a tweet’s creation time, e.g. 1174932714. The following URL from the CB view editor web page works great:

http://127.0.0.1:8092/twitter/_design/statuses/_view/by_date?connection_timeout=60000&limit=10&skip=0

resulting in:

{"total_rows":5847613,"rows":[ {"id":"st:13110261","key":1174932714,"value":null}, {"id":"st:24656161","key":1176299350,"value":null}, {"id":"st:193253582","key":1186546652,"value":null}, {"id":"st:243086362","key":1188766579,"value":null}, {"id":"st:767116463","key":1204735210,"value":null}, {"id":"st:790555585","key":1208374803,"value":null}, {"id":"st:823173396","key":1212135874,"value":null}, {"id":"st:837724080","key":1213791679,"value":null}, {"id":"st:844266941","key":1214499099,"value":null}, {"id":"st:906070011","key":1220300170,"value":null} ] }

That is what I expected and desired.

The below script should do the same thing:

import sys, time, calendar

from couchbase import Couchbase
from couchbase.exceptions import *
from couchbase.views.params import *

cb = Couchbase.connect(bucket=‘twitter’)

def main(args=sys.argv[1:]):

q = Query

q.limit = 10
q.stale = STALE_UPDATE_BEFORE

results = cb.query("statuses", "by_date", use_devmode=False, query=q)

print(results)
print(results.errors)

main()

Instead it prints:

View<Design=statuses, View=by_date, Query=, Rows Fetched=0> []

No errors are generated but no rows are fetched either.

Why? No clue. The REST API works but the Python client (v1.0.0-beta on C v2.0.6) doesn’t. As this is a pretty simple query, I doubt this is a version issue.

Is the Python client generating the same request as the view console? Don’t know. A log would be nice. BTW, the connect document does not claim to have any errors in picking up the server. When the query is put into the script sitting on the endpoint, the same result ensues.

Anon,
Andrew

6

Also, you’re using q = Query. Here you’re assigning to the class object which is probably not what you want.

Instead you should create a new instance; e.g. q= Query()

7

Mark and Tag,

Thank you for helping. Python is a new language to me and such subtleties are sometimes still lost on me. Thank you for educating me.

After adding the parentheses (

q = Query()
), I get the following result: View

It is an obvious improvement but still no rows are fetched.

In review, I have a known working CB view. What is different than the examples I’ve seen are that it uses a number, the epoch time, and not a string as the key. I have a string variant of the view in my development design doc. It exhibits the same behavior.

I am at a loss on how to proceed. Your help heretofore is appreciated. Any further help you care to share will be ecstatically accepted.

Anon,
Andrew

8

Having now seen your reply to my earlier comment, I have pulled the data out of the iterator and am totally happy.

Thank you,
Andrew

9

Mark,

Ahh … now I see, Thank you. I am getting these results just fine and am back on track.

Anon,
Andrew