Couchbase Mobile 2.0, introduces powerful Full Text Search (FTS) capabilities on your JSON Documents. This is part of the new Query interface based on N1QLa linguagem de consulta declarativa do Couchbase que estende o SQL para JSON. Se você estiver familiarizado com SQL, vai se sentir em casa com a semântica da nova API.
Full Text Search enables natural lanugage querying. This is the third in a series of posts that discusses the query interface in Couchbase Lite. This blog assumes you are familiar with the fundamentals, so if you haven’t done so already, be sure to review the postagem anterior primeiro. Se estiver interessado, os links para blogs que discutem outros recursos da interface do Query são fornecidos no final desta publicação.
You can download the latest pre-release version of Couchbase Mobile 2.0 from aqui.
Histórico
Se você estava usando as versões 1.x do Couchbase Mobile, provavelmente está familiarizado com Visualizações de mapas para criar índices e consultas. Na versão 2.0, você não precisa mais criar visualizações e funções de mapa! Em vez disso, uma interface simples permite a criação de índices e você pode usar uma interface do Query Builder para construir suas consultas. A nova interface de consulta é mais simples de usar e muito mais poderosa em comparação. Vamos descobrir alguns de seus recursos nesta postagem.
Projeto de amostra
While the examples discussed here use Swift for iOS, note that barring some minor differences, the same query interface is supported on the Android and Windows platforms as well.
Portanto, com alguns pequenos ajustes, você poderá reutilizar os exemplos de consulta desta postagem ao trabalhar com outras plataformas.
Siga as instruções abaixo se você estiver interessado em um projeto Swift de amostra
- Clone o iOS Swift Playground do GitHub
1$ git clone https://github.com/couchbaselabs/couchbase-lite-ios-api-playground - Siga as instruções de instalação no manual correspondente LEIAME para criar e executar o playground.
Modelo de dados de amostra
Usaremos o banco de dados Travel Sample localizado aqui. Você pode incorporar esse banco de dados pré-criado em seu aplicativo móvel e começar a usá-lo para suas consultas.
O conjunto de dados de amostra inclui vários tipos de documentos, conforme identificado pelo tipo no documento. Vamos nos concentrar em documentos de tipo "marco" . The JSON document model is shown below. For brevity, we have omitted some of the properties that are not relevant to this post from the model below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
{ "atividade": "see", "endereço": "84 rue Claude Monet", "alt": "Fondation Claude Monet", "cidade": "Giverny", "content" (conteúdo): "the house is quietly eccentric and highly interesting in an Orient-influenced style, and includes Monet's collection of [http://www.intermonet.com/japan/ Japanese prints]. There are no original Monet paintings on the site - the real drawcard, is the gardens around the house ...", "país": "França", "direções": nulo, "email": nulo, "geo": { "precisão": "ROOFTOP", "lat": 49.0753489, "longo": 1.5337884 }, "horas": "open April-October Mo-Su 9:30-18:00", "id": 10061, "imagem": nulo, "name" (nome): "Monet's House", "telefone": "+33 232512821", "price" (preço): "€9, $5 students, €4 4.00 disabled, under-7s free", "estado": "Haute-Normandie", "título": "Giverny", "tollfree": nulo, "tipo": "marco", "url": "http://www.fondation-monet.com/" } |
** Consulte o modelo acima para cada um dos exemplos de consulta abaixo. **
O identificador do banco de dados
Nas consultas abaixo, usaremos o Banco de dados API para abrir/criar o banco de dados CouchbaseLite.
|
1 2 |
var opções = DatabaseConfiguration() deixar db = tentar Banco de dados(nome: kDBName, configuração: opções) |
O básico
Full Text Search enables natural lanugage querying. In our post on the Fundamentos da consulta, we discussed the como e regex expressions for pattern matching operations. FTS supercedes that capability by enabling support for stemming, relevance based ranking e locale-specific natural language querying.
Full Text Searches are case insensitive and use the partida query expression. In order to perform FTS, you must create Full Text Index on appropriate properties. You can create index on one or more properties.
Stemming
Before we proceed with the examples, first a word on Stemming. Stemming is the process of reducing words to their root stem word. So for instância, “catty”, “catlike” and “cats” are reduced to the word “cat”. So searching for the term “cats” would give us results that match “cat”, “catlike” and so on.
Couchbase Lite currently supports Stemming in the following languages
* danish
* dutch
* english
* finnish
* french
* german
* hungarian
* italian
* norwegian
* portuguese
* romanian
* russian
* spanish
* swedish
* turkish
If no specific language is used, the tokenizer will still break the text into words at Unicode whitespace characters. So it should work, although less well, with any language that puts spaces between words.
Full Text Index
O nome that is associated with the index during creation is important. The query examples that we will see later will refer to the appropriate index via the name
Single Property Index
O exemplo a seguir cria um fullTextIndex no “content” property of a Documento. Stemming is enabled by default and the locale is assumed to be the locale of the device. While not shown below, you also have the option of specifying if “accents” have to be ignored or not via the ignoreAccents option. By default, accents are not ignored.
|
1 2 |
deixar ftsIndex = IndexBuilder.fullTextIndex(itens: FullTextIndexItem.propriedade("content" (conteúdo))) tentar db.createIndex(ftsIndex,withName: "ContentFTSIndex") |
Multiples Property Index
O exemplo a seguir cria um fullTextIndex em “content” e “name” properties of a Documento
|
1 2 |
deixar ftsIndex = IndexBuilder.fullTextIndex(itens: FullTextIndexItem.propriedade("content" (conteúdo)),FullTextIndexItem.propriedade("name" (nome))) tentar db.createIndex(ftsIndex,withName: "ContentAndNameFTSIndex") |
Index without stemming
O exemplo a seguir cria um fullTextIndex no “content” property of a Documento with stemming disabled. Stemming is enabled by default using the current device language settings. Setting language to nil will disable stemming.
|
1 2 |
deixar ftsIndex = IndexBuilder.fullTextIndex(itens: FullTextIndexItem.propriedade("content" (conteúdo))).language(nulo) tentar db.createIndex(ftsIndex,withName: "ContentFTSIndexNoStemming") |
FTS Search with Stemming
The query below fetches the id e conteúdo properties of "marco" tipo documents containing the term “Mechanical” no “content” property. We use the “ContentFTSIndex” that was created earlier.
Solicitação
|
1 2 3 4 5 6 7 8 9 |
deixar ftsExpression = FullTextExpression.índice("ContentFTSIndex") deixar pesquisa = Criador de consultas .selecionar(SelecionarResultado.expressão(Meta.id), SelecionarResultado.expressão(Expressão.propriedade("content" (conteúdo)))) .de(Fonte de dados.banco de dados(db)) .onde( Expressão.propriedade("tipo").igual a(Expressão.string ("marco")) .e( ftsExpression.partida("Mechanical"))) .limite(Expressão.int(limite)) |
Sample Response
The response to the above query will include documents that contain the terms “mechanical”, “mechanism”, “mechanisms”, “mechanic” and so on.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
[ { "id": "landmark_21703", "content" (conteúdo): "The Swiss luxury watch manufacturer, founded in 1851, is known for precise mechanics." }, { "id": "landmark_2592", "content" (conteúdo): "Here you can see the mechanisms that drive San Francisco's famed cable cars, as well as plenty of cable car memorabilia and information on the history of the cable cars." }, { "id": "landmark_26144", "content" (conteúdo): "This scenic section of the Golden Gate National Recreation Area is a favorite for hikers, bikers and beach-goers, with rugged coastal highlands and deep sand dunes. Hang gliding is quite popular here, with several shops for hang gliders in the area. Nearby is the remnants of Battery Davis, a WWII-era military defense mechanism." }, { "id": "landmark_33234", "content" (conteúdo): "Western-style steakhouse features a huge, ‘country’ bar and even a mechanical bull for those ready for a faux bull-riding adventure. The menu is vast and surprisingly inexpensive. And again, the saloon is a longtime trendy destination along the Sunset Strip for raucous good times." } ] |
FTS Search sem Stemming
The query below fetches the id e conteúdo properties of "marco" tipo documents containing the exact term “Mechanical” no “content” property. We use the “ContentFTSIndexNoStemming” that was created earlier which specified the option to disable stemming.
Solicitação
|
1 2 3 4 5 6 7 8 |
deixar ftsExpression = FullTextExpression.índice("ContentFTSIndexNoStemming") deixar pesquisa = Criador de consultas .selecionar(SelecionarResultado.expressão(Meta.id), SelecionarResultado.expressão(Expressão.propriedade("content" (conteúdo)))) .de(Fonte de dados.banco de dados(db)) .onde(Expressão.propriedade("tipo").igual a(Expressão.string ("marco")) .e( ftsExpression.partida("Mechanical"))) .limite(Expressão.int(limite)) |
Sample Response
The response to the above query will include documents that contain exactly the term “mechanical” in it. Note again that all searches are case insensitive.
|
1 2 3 4 5 6 7 |
[ { "id": "landmark_33234", "content" (conteúdo): "Western-style steakhouse features a huge, ‘country’ bar and even a mechanical bull for those ready for a faux bull-riding adventure. The menu is vast and surprisingly inexpensive. And again, the saloon is a longtime trendy destination along the Sunset Strip for raucous good times." } ] |
FTS Search on Multiple Properties
The query below fetches the id , nome e conteúdo properties of "marco" tipo documents containing the term “Mechanical” in either the “name” ou o “content” property. We use the “ContentAndNameFTSIndex” that was created earlier. This index enabled indexing on the “name” e “content” propriedades
Solicitação
|
1 2 3 4 5 6 7 8 9 |
deixar ftsExpression = FullTextExpression.índice("ContentAndNameFTSIndex") deixar pesquisa = Criador de consultas .selecionar(SelecionarResultado.expressão(Meta.id), SelecionarResultado.expressão(Expressão.propriedade("name" (nome))), SelecionarResultado.expressão(Expressão.propriedade("content" (conteúdo)))) .de(Fonte de dados.banco de dados(db)) .onde(Expressão.propriedade("tipo").igual a(Expressão.string ("marco")) .e( ftsExpression.partida("Mechanical"))) .limite(Expressão.int(limite)) |
Sample Response
The response to the above query will include documents that contain the term “mechanical” (or variants of it derived through stemming) in either the “name” or “content” property.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
[ { "id": "landmark_10062", "name" (nome): "Natural Mechanical Museum", "content" (conteúdo): "Founded by the Guillemard brothers: Jean-pierre, René and Gérard currently run restorations and exhibitions with the help of an enthusiasts team who devoted their time and known-how to the Patrimony preservation. The museum origin is a private collection of steam internal combustion engines; founded in 1955 by the Guillemard family a GIVERNY residents since generations." }, { "id": "landmark_21703", "name" (nome): "Patek Philippe Salons", "content" (conteúdo): "The Swiss luxury watch manufacturer, founded in 1851, is known for precise mechanics." }, { "id": "landmark_25929", "name" (nome): "Cable Car Museum", "content" (conteúdo): "Here you can see the mechanisms that drive San Francisco's famed cable cars, as well as plenty of cable car memorabilia and information on the history of the cable cars" }, { "id": "landmark_26144", "name" (nome): "Fort Funston", "content" (conteúdo): "This scenic section of the Golden Gate National Recreation Area is a favorite for hikers, bikers and beach-goers, with rugged coastal highlands and deep sand dunes. Hang gliding is quite popular here, with several shops for hang gliders in the area. Nearby is the remnants of Battery Davis, a WWII-era military defense mechanism" } ] |
FTS Search with Logical Expressions
In an earlier example, you saw that by disabling stemming, you can look for the exact search string. But what if you wanted to look for more than one search term ? The partida query expression accepts logical expressions including AND and OR.
The query below fetches the id e conteúdo properties of "marco" tipo documents containing the term “Mechanical” ou “Mechanism” no “content” property. We use the “ContentFTSIndexNoStemming” that was created earlier to disable stemming.
Solicitação
|
1 2 3 4 5 6 7 8 |
deixar ftsExpression = FullTextExpression.índice("ContentFTSIndexNoStemming") deixar pesquisa = Criador de consultas .selecionar(SelecionarResultado.expressão(Meta.id), SelecionarResultado.expressão(Expressão.propriedade("content" (conteúdo)))) .de(Fonte de dados.banco de dados(db)) .onde(Expressão.propriedade("tipo").igual a(Expressão.string ("marco")) .e( ftsExpression.partida("Mechanical OR Mechanism"))) .limite(Expressão.int(limite)) |
Sample Response
The response to the above query will include documents that contain the eactly the terms “mechanical” or “mechanism” in the “content” property.
|
1 2 3 4 5 6 7 8 9 10 11 |
[ { "id": "landmark_26144", "content" (conteúdo): "This scenic section of the Golden Gate National Recreation Area is a favorite for hikers, bikers and beach-goers, with rugged coastal highlands and deep sand dunes. Hang gliding is quite popular here, with several shops for hang gliders in the area. Nearby is the remnants of Battery Davis, a WWII-era military defense mechanism" }, { "id": "landmark_33234", "content" (conteúdo): "Western-style steakhouse features a huge, ‘country’ bar and even a mechanical bull for those ready for a faux bull-riding adventure. The menu is vast and surprisingly inexpensive. And again, the saloon is a longtime trendy destination along the Sunset Strip for raucous good times." } ] |
FTS Search with Wilcard Expression
You can use the “*” character in the search string to represent zero or more character matches.
The query below fetches the id e conteúdo properties of "marco" tipo documents containing the term “walt*” no “content” property. This will match all search terms that start with “walt” followed by zero or more characters. We use the “ContentFTSIndex” that was created earlier.
OBSERVAÇÃO: One could argue that the use of wildcard in the search term could be a naive way of implementing stemming. But then you may end up with derived forms that may not correspond to the terms derived through stemming. So it is preferrable to use stemming if that’s what you need.
Solicitação
|
1 2 3 4 5 6 7 8 |
deixar ftsExpression = FullTextExpression.índice("ContentFTSIndex") deixar pesquisa = Criador de consultas .selecionar(SelecionarResultado.expressão(Meta.id), SelecionarResultado.expressão(Expressão.propriedade("content" (conteúdo)))) .de(Fonte de dados.banco de dados(db)) .onde(Expressão.propriedade("tipo").igual a(Expressão.string ("marco")) .e( ftsExpression.partida("walt*"))) .limite(Expressão.int(limite)) |
Sample Response
The response to the above query will include documents that contain the terms “walt”, “Walter”, “Waltham”,“Walthamstow” and so on.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
[ { "id": "landmark_10134", "content" (conteúdo): "On the Ibrox tour, you get access to the home dressing room and hear a recorded message from Walter Smith and Ally McCoist before climbing the marble staircase, visit the illustrious trophy room, the blue room and the managers office. Tickets, except for matches against Celtic, are available online from the clubs website, ticket centre at the stadium and club outlets at JJB Sports Stores in Glasgow city centre." }, { "id": "landmark_16104", "content" (conteúdo): "Presents the history of Waltham Forest. The building was constructed to be a work house and has since been used as a police station and a private home. Its collection includes the Bremer car, built by engineer Frederick Bremer in 1892 it has a claim to being this first petrol-driven car made in Britain." }, { "id": "landmark_16105", "content" (conteúdo): "The ancient nucleus of present day Walthamstow centred around the 12 th century St.Marys Church " }, { "id": "landmark_16574", "content" (conteúdo): "Impressive hall architecture complete with tours most days.The Dorothy Chandler Pavilion is open to the public Christmas Eve day with almost round the clock performances by amateur cultural arts groups.The Walt Disney Hall has daily tours ,check website for schedules." }, { "id": "landmark_8631", "content ": "Museum about famous Scottish authors, focussing on Robert Burns, Sir Walter Scott and Robert Louis Stevenson " } ] |
FTS Search with Stop Words
Stop Words refer to common words in a language. In English, this would be terms like “the”, “is”, “and” , “which” and so on.
Example 1: Search String contains stop words
Couchbase Lite ignores stop words that appear in search string.
The query below fetches the id e conteúdo properties of "marco" tipo documents containing the term “on the history” no “content” property. We use the “ContentFTSIndex” that was created earlier.
Couchbase Lite ignores the stop words “on” and “the”, so you would fetch documents that only include the term “history” and derived forms of the stem word
Solicitação
|
1 2 3 4 5 6 7 |
deixar ftsExpression = FullTextExpression.índice("ContentFTSIndex") deixar pesquisa = Criador de consultas.selecionar(SelecionarResultado.expressão(Meta.id), SelecionarResultado.expressão(Expressão.propriedade("content" (conteúdo)))) .de(Fonte de dados.banco de dados(db)) .onde(Expressão.propriedade("tipo").igual a(Expressão.string ("marco")) .e( ftsExpression.partida("on the history"))) .limite(Expressão.int(limite)) |
Sample Response
The response to the above query will include documents that contain the terms “history” and derived forms of this word such as “historical”
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
[ { "id": "landmark_10019", "content" (conteúdo): "Museum on military engineering and the history of the British Empire. A quite extensive collection that takes about half a day to see. Of most interest to fans of British and military history or civil engineering." }, { "id": "landmark_10083", "content" (conteúdo): " Tours take about 45 min. In front the building, George Square, the citys notional centre, is populated by several statues of civic leaders and famous figures from history and is often used for outdoor events." }, { "id": "landmark_10093", "content" (conteúdo): "The auditorium has now garnered some world fame for being the place where the Susan Boyle audition - one of the most downloaded YouTube video clips in history - was filmed." }, { "id": "landmark_10101", "content" (conteúdo): "This museum has a large collection of artifacts and exhibits showcasing the history of the city. If you don't want to pay to enter the museum itself, you can just walk into the building (which contains three separate museums) and look at some historical photographs on the walls of the atrium." }, { "id": "landmark_10105", "content" (conteúdo): "The Peoples Palace is a great folk museum, telling the history of Glasgow and its people, from various perspectives, displaying details of Glasgow life (including one of Billy Connolly's banana boots). The Winter Gardens, adjacent, is a pleasant greenhouse with a reasonable cafe.)" } ] |
Example 2: Ignoring Stop Words while Searching
By default, Couchbase Lite ignores stop words within the search content.
The query below fetches the id e conteúdo properties of "marco" tipo documents containing the terms “blue fin yellow fin” no “content” property. We use the “ContentFTSIndex” that was created earlier.
Couchbase Lite ignores stop words during search, so you would fetch documents that include the terms “blue”, “fin” and “yellow” in that order, separated by any number of stop words.
Solicitação
|
1 2 3 4 5 6 7 |
deixar ftsExpression = FullTextExpression.índice("ContentFTSIndex") deixar pesquisa = Criador de consultas.selecionar(SelecionarResultado.expressão(Meta.id), SelecionarResultado.expressão(Expressão.propriedade("content" (conteúdo)))) .de(Fonte de dados.banco de dados(db)) .onde(Expressão.propriedade("tipo").igual a(Expressão.string ("marco")) .e( ftsExpression.partida("blue fin yellow fin"))) .limite(Expressão.int(limite))<código classe="swift"> |
Sample Response
The response to the above query will include documents that contain the terms “blue”, “fin” and “yellow” separated by any number of stop words such as “blue fin and yellow fin”
|
1 2 3 4 5 6 7 |
[ { "id": "landmark_18840", "content" (conteúdo): "This large aquarium specializes in exhibiting local sea life in typical local habitat displays, and has many spectacular exhibits. It is particularly known for its Kelp Forest exhibit, three stories high, filled with several varieties of giant kelp and a wide variety of marine animal species, and also for its million-gallon Open Sea exhibit with large blue fin and yellow fin tunas, mahi-mahis, sharks (including an occasional Great White Shark as a very temporary visitor, before being released back to the ocean), ocean sunfish (mola-molas) and sea turtles. The best exhibits include a large tank of silver sardines that swim around and around above one's head, and one of rescued sea otters deemed unreturnable to the wild and therefore kept at the aquarium. )" } ] |
FTS Search with Ranking
Você pode usar o FullTextFunction.rank to specify the rank order of the search results. This is useful to rate the matches in order of best match.
The query below fetches the id e conteúdo properties of "marco" tipo documents containing the term “attract” no “content” property. The documents are ordered in descending order according to rank which means that the document which the maximum number of matches is sorted higher than the rest.
Solicitação
|
1 2 3 4 5 6 7 8 |
deixar ftsExpression = FullTextExpression.índice("ContentFTSIndexNoStemming") deixar pesquisa = Criador de consultas.selecionar(SelecionarResultado.expressão(Meta.id), SelecionarResultado.expressão(Expressão.propriedade("content" (conteúdo)))) .de(Fonte de dados.banco de dados(db)) .onde(Expressão.propriedade("tipo").igual a(Expressão.string ("marco")) .e( ftsExpression.partida("attract"))) .orderBy(Pedidos.expressão(FullTextFunction.classificação("ContentFTSIndexNoStemming")).descendente()) .limite(Expressão.int(limite)) |
Sample Response
The response to the above query will include documents that include the term “attract” or derived versions of it. Documents with the maximum number of matches are sorted higher.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
[ { "id": "landmark_22056", "content" (conteúdo): "Top paid-for visitor attraction in Wales including a farm, indoor vintage funfair, zoo and indoor and outdoor adventure play. All-weather family attraction with 50% of attractions undercover." }, { "id": "landmark_16309", "content" (conteúdo): "The London Bridge Experience and London Tombs are two scare attractions for one price and have been voted the UK's Best Year Round Scare Attraction for three years running." }, { "id": "landmark_25216", "content" (conteúdo): "TA seaside amusement park located near the southern end of Mission Beach, Belmont Park is a landmark with a number of shops, restaurants, an arcade, and a bunch of rides. The big attraction is the Giant Dipper, a historic roller coaster that is one of the only two remaining oceanfront roller coasters still operating on the west coast. Among the other rides is a FlowRider (a simulated wave attraction which you can bodyboard on), an antique carousel, bumper cars, slides, pendulum rides, tilt-a-whirl, and a trampoline " }, { "id": "landmark_1059", "content" (conteúdo): "Aims to tell the history of flight throughout the 20th Century, and has a large collection of aircraft, including British Airways Concorde G-BOAA. Another rather good attraction (and well worth the look) is the De-Havilland Comet 4C, a derivative of the Worlds first jetliner." } ] |
Limitações
While the FTS capabily in Couchbase Lite 2.0 is extremely powerful and would suffice for use cases typical on an embedded database, there are a few limitations
- Match Expression can only be at the top-level or top-level AND expression. This means that the following expression is not allowed ftsExpression.match(“attract”).or(ftsExpression2.match(“museum”))
- Custom Language Tokenizers
The list of supported languages was specified earlier. At the time of writing this post, you cannot plug in a custom tokenizer in order to extend support to other languages - Fuzzy Search Support
We cannot specify a “fuzziness” factor on the query that may result in less relevant matches being considered - Facets
There is no support for faceted search
Bear in mind that Couchbase Lite is an embedded database. So one could argue that the FTS capabilities does not have to be as extensive as a server side database implementation. The support for these will be evaluated in future releases.
O que vem a seguir
This blog post looked at how you can leverage the Full Text Search (FTS) capabilities in the new Query API in Couchbase Mobile 2.0. This is a start. Expect to see more functionality in future releases. You can download the latest release from our downloads página.
Aqui estão algumas outras postagens relacionadas ao Couchbase Mobile Query que podem ser de interesse
- Isso postagem no blog discute os fundamentos
- Isso postagem no blog discute como consultar coleções de matrizes
- Isso postagem no blog discute como fazer consultas JOIN
Se tiver dúvidas ou comentários, deixe um comentário abaixo ou entre em contato comigo pelo Twitter @rajagp ou envie-me um e-mail priya.rajagopal@couchbase.com. O Fóruns do Couchbase são outro bom lugar para entrar em contato com perguntas.
