Couchbase Server

Allow TAP to broadcast item persistence

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Duplicate
  • Affects Version/s: 1.7.1
  • Fix Version/s: 2.0
  • Component/s: couchbase-bucket
  • Security Level: Public
  • Labels:
    None

Description

Dustin, would it be hard to add the capability to TAP to send a "persisted" message when an item is actually written? Then they could hook up an external consumer to only keep the keys/values that have hit the system but not yet been persisted in some off-node system?

It's certainly possible. We have some rigging for observation for the sync on persist code. I'm sure we could extend that to a general "broadcast" type observer that a tap stream could grab. An interesting twist on sync on persist.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Farshid Ghods added a comment -
assigning to Dustn for feedback ?
Show
Farshid Ghods added a comment - assigning to Dustn for feedback ?
Hide
Permalink
Mike Wiederhold added a comment -
I'm going to be adding a change that will allow us to do something like this.

1. For trying to synchronize the active and replica we will need a field in our tap streams that will say where an item is in one node (eg. memory or disk)

2. For observe I really want to stream stuff back through tap. The idea would be to say watch this key with a certain cas value and tell me when it's persisted on 2 nodes and replicated on 3. The client would have a tap stream and when this stuff happened we would get a tap message from the server.
Show
Mike Wiederhold added a comment - I'm going to be adding a change that will allow us to do something like this. 1. For trying to synchronize the active and replica we will need a field in our tap streams that will say where an item is in one node (eg. memory or disk) 2. For observe I really want to stream stuff back through tap. The idea would be to say watch this key with a certain cas value and tell me when it's persisted on 2 nodes and replicated on 3. The client would have a tap stream and when this stuff happened we would get a tap message from the server.
Hide
Permalink
Dustin Sallings added a comment -
I don't think it makes sense to have every client have a tap connection just to know when something happened.

There are other state transition events we'd like to push through tap already (e.g. evicted, fetched). Persisted isn't a bad idea. I think anything involving sending tiny, key-specific bits of data through tap is just another case of "we need unsolicited responses."
Show
Dustin Sallings added a comment - I don't think it makes sense to have every client have a tap connection just to know when something happened. There are other state transition events we'd like to push through tap already (e.g. evicted, fetched). Persisted isn't a bad idea. I think anything involving sending tiny, key-specific bits of data through tap is just another case of "we need unsolicited responses."
Hide
Permalink
Chiyoung Seo added a comment -
This will be supported by OBSERVE command in 2.0.
Show
Chiyoung Seo added a comment - This will be supported by OBSERVE command in 2.0.

People

  • Assignee:
    Mike Wiederhold
    Reporter:
    Perry Krug
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: