XDCR Mapping Not Working on CB 4.5.0 Server with ElasticSearch

I have a document on CouchBase which features nested types. When using XDCR to replicate to ElasticSearch 2.4.0, the mapping does not display correctly and only _index, _type, _id, _score, meta.rev.meta.flags, meta.experiation and meta.id show up (head plugin). My document is as follows:

{
“type”: “mydatatype”,
“idmaster”: “ABCDE0000000126”,
“cid”: “123456789”,
“dob”: “19980101”,
“phone”: " ",
“gender”: “M”,
“spouse_first”: " ",
“other_name_cnt”: 2,
“other_addr_cnt”: 1,
“names”: [
{
“idname”: 1234,
“number”: 0,
“firstname”: “JOHN”,
“middlename”: " ",
“surname”: “SMITH”,
“surname_second”: " ",
“generation”: " ",
“created”: “20061104”,
“fullname”: “JOHN SMITH”
},
{
“idname”: 1235,
“number”: 2,
“firstname”: “JOHN”,
“middlename”: " ",
“surname”: “SMITH”,
“surname_second”: " ",
“generation”: " ",
“created”: “20041102”,
“fullname”: “JOHN SMITH”
},
{
“idname”: 1236,
“number”: 3,
“firstname”: “JOHN”,
“middlename”: “R”,
“surname”: “SMITH”,
“surname_second”: " ",
“generation”: " ",
“created”: “20040410”,
“fullname”: “JOHN SMITH”
}
],
“addresses”: [
{
“idaddress”: 4567,
“number”: 0,
“street_num”: “1234”,
“street_predir”: " ",
“street_name”: “MAIN”,
“street_suffix”: “ST”,
“street_postdir”: " ",
“unit_type”: " ",
“unitid”: " ",
“city”: “ANYTOWN”,
“state”: “MA”,
“zip”: 12345,
“zip_plusfour”: “1234”,
“created”: “20050603”,
“updated”: “20050603”,
“street_addr”: "1234 MAIN ST "
},
{
“idaddress”: 4568,
“number”: 1,
“street_num”: “4567”,
“street_predir”: " ",
“street_name”: “FRONT”,
“street_suffix”: “ST”,
“street_postdir”: " ",
“unit_type”: " ",
“unitid”: " ",
“city”: “OTHERTOWN”,
“state”: “IA”,
“zip”: 23456,
“zip_plusfour”: “3456”,
“created”: “20040410”,
“updated”: “20040410”,
“street_addr”: "4567 FRONT ST "
}
]
}

My mapping document http://servername:9200/sandbox/_mapping is as follows:

“sandbox”: {
“mappings”: {
“couchbaseDocument”: {
"_source": {
“includes”: [
“meta."
]
},
“properties”: {
“doc”: {
“properties”: {
“addresses”: {
“properties”: {
“city”: {
“type”: “string”
},
“created”: {
“type”: “string”
},
“idaddress”: {
“type”: “long”
},
“number”: {
“type”: “long”
},
“state”: {
“type”: “string”
},
“street_addr”: {
“type”: “string”
},
“street_name”: {
“type”: “string”
},
“street_num”: {
“type”: “string”
},
“street_postdir”: {
“type”: “string”
},
“street_predir”: {
“type”: “string”
},
“street_suffix”: {
“type”: “string”
},
“unit_type”: {
“type”: “string”
},
“unitid”: {
“type”: “string”
},
“updated”: {
“type”: “string”
},
“zip”: {
“type”: “long”
},
“zip_plusfour”: {
“type”: “string”
}
}
},
“cid”: {
“type”: “string”
},
“click”: {
“type”: “string”
},
“dob”: {
“type”: “string”
},
“gender”: {
“type”: “string”
},
“idmaster”: {
“type”: “string”
},
“names”: {
“properties”: {
“created”: {
“type”: “string”
},
“firstname”: {
“type”: “string”
},
“fullname”: {
“type”: “string”
},
“generation”: {
“type”: “string”
},
“idname”: {
“type”: “long”
},
“middlename”: {
“type”: “string”
},
“number”: {
“type”: “long”
},
“surname”: {
“type”: “string”
},
“surname_second”: {
“type”: “string”
}
}
},
“other_addr_cnt”: {
“type”: “long”
},
“other_name_cnt”: {
“type”: “long”
},
“phone”: {
“type”: “string”
},
“spouse_first”: {
“type”: “string”
},
“type”: {
“type”: “string”
},
“with JSON”: {
“type”: “string”
}
}
},
“meta”: {
“include_in_all”: false,
“properties”: {
“expiration”: {
“type”: “long”,
“include_in_all”: false
},
“flags”: {
“type”: “long”,
“include_in_all”: false
},
“id”: {
“type”: “string”,
“include_in_all”: false
},
“rev”: {
“type”: “string”,
“include_in_all”: false
}
}
}
}
},
“couchbaseCheckpoint”: {
"_source": {
“includes”: [
"meta.

,
“doc."
]
},
“dynamic_templates”: [
{
“store_no_index”: {
“mapping”: {
“include_in_all”: false,
“index”: “no”,
“store”: “no”
},
“match”: "

}
}
],
“properties”: {
“doc”: {
“include_in_all”: false,
“properties”: {
“uuid”: {
“type”: “string”,
“index”: “no”,
“include_in_all”: false
}
}
},
“meta”: {
“type”: “object”,
“include_in_all”: false
}
}
},
default”: {
"_source": {
“includes”: [
“meta.*”
]
},
“properties”: {
“meta”: {
“type”: “object”,
“include_in_all”: false
}
}
}
}
}
}

I have nothing setup in my elasticsearch.yml file (other than password), what am I doing wrong?

Hi Todd - a quick glance at what you’re reporting is that XDCR replication is working but you have something unexpected in your index mapping that’s causing very little information to be indexed for each document. Can you try a very “non-selective” index mapping and see if that works for you?

Otherwise, if you think XDCR is at fault, you can check the goxdcr.log file for errors. I wouldn’t suspect that though unless you’re seeing that the replication stops before the total number of documents are transferred.
-Will

1 Like

Thanks Will, yes, definitely XDCR is working however I am not seeing the fields. Do you have a sample or reference document for "non-selective’ index mappings. Thanks in Advance.