Previously, we showed how to use Couchbase RAG capabilities through a Python app that allows the user to ‘chat’ with their PDF ou with X. It’s simple to build, but can we build it simpler? I have been playing a lot with Couchbase Shell recently and it should allow me to do something similar.
Set up a scope and collection
I am assuming you are already familiar with Shell do Couchbase (cbsh), and have a configured cluster and model.
Create and select a scope and collection, and then create a primary index:
|
1 2 3 4 5 |
> scopes criar pdf > cb–ambiente escopo pdf > collections criar pdf > cb–ambiente coleção pdf > consulta “CREATE PRIMARY INDEX ON `default`:`cbsh`.`pdf`.`pdf`” |
Turn a PDF into chunked text
There are a variety of tools allowing you to convert a pdf to text. On most Linux distributions, you should find pdftotext.
|
1 |
> pdftotext ~/monopolyInstruction.pdf |
This will create a text version of the file with the same path, but with a .txt extension.
Com Nushell (cbsh is based on Nushell) it’s easy to split text thanks to the split command. The problem is finding the right delimiter you need to chunk out the file. Fortunately, it supports multiline string, so I copied and pasted text from the file between two paragraphs. However, you should be able to do something more sophisticated using regex. That’s the difference between blog material and production 😇.
|
1 2 3 |
> open ~/monopolyInstruction.txt |split fileira “ ::: ::: “|wrap text |
This will get you a table of text strings. To import it to Couchbase we wrap them in a text field, in a content JSON object, add a randomly generated UUID and upsert the result.
|
1 2 3 |
> open ~/monopolyInstruction.txt |split fileira “ ::: ::: “|wrap text |wrap conteúdo | each { inserir identidade { random uuid } } | doc atualização upsert |
The next step is to create embeddings, or vector representations of the text:
|
1 |
> consulta “SELECT meta().id as id, p.* from pdf as p” | wrap conteúdo| vector enrich–doc text | doc atualização upsert |
Then create the vector search index. Here it’s called pdf, index the field textVector, create 1536 dimensions vector and use l2_norm for similarity algorithm as it’s the default.
|
1 |
> vector criar–index pdf textVector 1536 |
I have imported the rules of Monopoly and I am asking how to get out of jail. In the original example, we had one answer with context and one without.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
> ask “how to get out of jail” Se você ou someone você know é em jail e needs para ser released, aqui estão some general etapas para take: Contact a lawyer: Se você ter legal representation ou know de a lawyer who can help com your case, reach fora para them para assistance em navigating o legal process. Obtain a bail bond: Em many cases, individuals can ser released de jail by posting bail. Este requires paying a definir quantidade de money para o court, which é typically returned once o individual attends todos required court dates. Attend court hearings: It‘s important to comply with all court requirements, including attending scheduled hearings and following any conditions of release set by the court. Seek support: Consider reaching out to family members, friends, or local organizations that may be able to provide assistance or guidance during this challenging time. Please keep in mind that the process of getting released from jail can vary depending on the specific circumstances of the case and the jurisdiction. It’s always best para consult com legal professionals para personalizado advice e assistance. |
And with context:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
> deixar question = “how to get out of jail”; vector enrich–text $question | vector pesquisar pdf textVector | selecionar identidade |doc obter| selecionar conteúdo.text | ask $question 👤 Laurent Doguin 🏠 Capela em ☁️ cbsh.gitlog.pdf > deixar question = “how to get out of jail”; vector enrich–text $question | vector pesquisar pdf textVector | selecionar identidade |doc obter| selecionar conteúdo.text | ask $question Embedding batch 1/1 Você can obter fora de jail by following these methods: **Roll Doubles:** Se você roll a double com o white dice ligado any de your next três turns, você can immediately move fora de Jail. Você then move o number de spaces shown by your doubles roll. **“Get Out of Jail Free” Card:** Se você ter a “Get Out of Jail Free” card, você can usar ele para obter fora de Jail without rolling doubles. Este card can ser obtained by purchasing ele de outro player ou drawing ele de o Chance ou Comunidade Chest cards. **Pay Fine:** Você can also choose para pay a fine de $50 before você roll o dice ligado either de your next two turns. Depois paying o fine, você estão free para move e continuar playing. Remember, se você do não roll doubles by your third turn ou usar a “Get Out of Jail Free” card, você deve pay o $50 fine para obter fora de Jail. |
Let’s simplify this by putting everything in a script. This is the content of myScript.nu:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
def initRAGPipeline [] { scopes criar pdf cb–ambiente escopo pdf collections criar pdf cb–ambiente coleção pdf consulta “CREATE PRIMARY INDEX ON `default`:`cbsh`.`pdf`.`pdf`” vector criar–index pdf textVector 1536 } def storeRAGDoc [] { wrap text |wrap conteúdo | each { inserir identidade { random uuid } } | doc atualização upsert consulta “SELECT meta().id as id, p.* from `pdf` as p” | wrap conteúdo| vector enrich–doc text | doc atualização upsert } def myAsk [$question: string] { deixar norag = ask $question deixar rag = vector enrich–text $question | vector pesquisar pdf textVector | selecionar identidade |doc obter| selecionar conteúdo.text | ask $question {“rag”:$rag, “norag”:$norag} } |
You can source the script file and then call those functions:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
> origem ./ragDemo/ragScript.nu > initRAGPipeline > open monopolyInstruction.txt |split fileira “ ::: ::: “| loja > myAsk “how to get out of jail” Embedding batch 1/1 ╭───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ rag │ Here estão o ways para obter fora de jail em o game de Monopoly: │ │ │ │ │ │ 1. **Roll Doubles:** A most common way para obter fora de jail é by rolling doubles ligado your turn. Se você roll │ │ │ doubles com o regular white dice ligado any de your next três turns after being enviado para jail, você can immediately move your token fora de jail e advance o corresponding │ │ │ number de spaces. Remember aquele você can only usar o white dice para this purpose. │ │ │ │ │ │ 2. **Using “Get Out of Jail Free” Card:** Se você ter a “Get Out of Jail Free” card, você can │ │ │ usar ele para obter fora de jail without rolling doubles. Simply present o card para o Banker para obter fora de jail para free. A card é then returned para o bottom de o deck. │ │ │ │ │ │ 3. │ │ │ **Purchase o Card:** Se outro player has a “Get Out of Jail Free” card e é willing para sell ele, você can purchase o card de them at a mutually agreed–upon preço. │ │ │ Este allows você para obter fora de jail even se você don‘t have the card yourself. │ │ │ │ │ │ 4. **Pay the Fine:** If you do not roll doubles within three turns or do not have a “Get Out of │ │ │ Jail Free” card, you must pay a fine of $50 to the Bank before you roll the dice on either of your next two turns. Once you pay the fine, you are immediately released from │ │ │ jail and can move your token as per the dice roll. │ │ │ │ │ │ These are the four main ways to get out of jail in Monopoly. │ │ norag │ If you or someone you know is currently in jail and looking to get released, here are some general steps to consider: │ │ │ │ │ │ 1. Contact a lawyer: A criminal defense attorney can │ │ │ provide guidance on legal options and help navigate the legal process for release. │ │ │ │ │ │ 2. Attend court hearings: It is important to attend all court hearings and follow any │ │ │ conditions set by the court to demonstrate cooperation with the legal system. │ │ │ │ │ │ 3. Consider bail: If bail is an option, you may be able to pay a set amount to be released from │ │ │ jail pending trial. If you cannot afford the bail amount, you may seek assistance from a bail bond agent. │ │ │ │ │ │ 4. Seek alternative options: Depending on the circumstances of your │ │ │ case, there may be alternative options for release such as pretrial services, diversion programs, or supervised release. │ │ │ │ │ │ 5. Follow legal advice: It is crucial to follow the │ │ │ advice of your legal counsel and comply with all legal requirements to increase the chances of a successful release. │ │ │ │ │ │ It’s important para note aquele o process de getting fora de │ │ │ jail can vary depending ligado o specific circumstances de o case e o laws em your jurisdiction. Para personalizado guidance, ele‘s recommended para speak com a lawyer ou │ │ │ legal professional specializing em criminal law. │ ╰───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ |
Here, you can see the same kind of result we achieved in the Python RAG demo, but this time using Couchbase Shell. It should be easier to manipulate, change or extend, because you don’t need to deploy an app or know Python. However, it will be less flexible than what you can achieve with Python and Langchain.
If this interests you, stay tuned–more AI and Couchbase Shell content is on the way!
- Saiba mais sobre Shell do Couchbase
- e Couchbase vector search capabilities
Deixe um comentário
Você precisa fazer o login para publicar um comentário.