Previously, we showed how to use Couchbase RAG capabilities through a Python app that allows the user to ‘chat’ with their PDF o with X. It’s simple to build, but can we build it simpler? I have been playing a lot with Couchbase Shell recently and it should allow me to do something similar.
Set up a scope and collection
I am assuming you are already familiar with Couchbase Shell (cbsh), and have a configured cluster and model.
Create and select a scope and collection, and then create a primary index:
|
1 2 3 4 5 |
El resultado de tu pregunta se muestra a continuación: alcances crear PDF El resultado de tu pregunta se muestra a continuación: cb–env alcance PDF El resultado de tu pregunta se muestra a continuación: colecciones crear PDF El resultado de tu pregunta se muestra a continuación: cb–env colección PDF El resultado de tu pregunta se muestra a continuación: consulta “CREATE PRIMARY INDEX ON `default`:`cbsh`.`pdf`.`pdf`” |
Turn a PDF into chunked text
There are a variety of tools allowing you to convert a pdf to text. On most Linux distributions, you should find pdftotext.
|
1 |
El resultado de tu pregunta se muestra a continuación: pdftotext ~/monopolyInstruction.PDF |
This will create a text version of the file with the same path, but with a .txt extension.
With Nushell (cbsh is based on Nushell) it’s easy to split text thanks to the split command. The problem is finding the right delimiter you need to chunk out the file. Fortunately, it supports multiline string, so I copied and pasted text from the file between two paragraphs. However, you should be able to do something more sophisticated using regex. That’s the difference between blog material and production 😇.
|
1 2 3 |
El resultado de tu pregunta se muestra a continuación: open ~/monopolyInstruction.txt |split fila “ ::: ::: “|wrap texto |
This will get you a table of text strings. To import it to Couchbase we wrap them in a text field, in a content JSON object, add a randomly generated UUID and upsert the result.
|
1 2 3 |
El resultado de tu pregunta se muestra a continuación: open ~/monopolyInstruction.txt |split fila “ ::: ::: “|wrap texto |wrap contenido | each { insert identificación { random uuid } } | doc. actualizar o insertar |
The next step is to create embeddings, or vector representations of the text:
|
1 |
El resultado de tu pregunta se muestra a continuación: consulta “SELECT meta().id as id, p.* from pdf as p” | wrap contenido| vector enrich–doc. texto | doc. actualizar o insertar |
Then create the vector search index. Here it’s called pdf, index the field textVector, create 1536 dimensions vector and use l2_norm for similarity algorithm as it’s the default.
|
1 |
El resultado de tu pregunta se muestra a continuación: vector crear–index PDF textVector 1536 |
I have imported the rules of Monopoly and I am asking how to get out of jail. In the original example, we had one answer with context and one without.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
El resultado de tu pregunta se muestra a continuación: ask “how to get out of jail” If you o someone you know is en jail y needs a be released, Aquí are some general steps a take: Contact a lawyer: If you have legal representation o know de a lawyer who lata help con your case, reach out a them para assistance en navigating el legal process. Obtain a bail bond: In many cases, individuals lata be released desde jail by posting bail. Este requires paying a establecer amount de money a el court, which is typically returned once el individual attends todo required court dates. Attend court hearings: It‘s important to comply with all court requirements, including attending scheduled hearings and following any conditions of release set by the court. Seek support: Consider reaching out to family members, friends, or local organizations that may be able to provide assistance or guidance during this challenging time. Please keep in mind that the process of getting released from jail can vary depending on the specific circumstances of the case and the jurisdiction. It’s always best a consult con legal professionals para personalizado advice y assistance. |
And with context:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
El resultado de tu pregunta se muestra a continuación: dejar question = “how to get out of jail”; vector enrich–texto $question | vector buscar PDF textVector | seleccionar identificación |doc. conseguir| seleccionar contenido.texto | ask $question 👤 Laurent Doguin 🏠 capella en ☁️ cbsh.gitlog.PDF El resultado de tu pregunta se muestra a continuación: dejar question = “how to get out of jail”; vector enrich–texto $question | vector buscar PDF textVector | seleccionar identificación |doc. conseguir| seleccionar contenido.texto | ask $question Embedding batch 1/1 You lata conseguir out de jail by following these methods: **Roll Doubles:** If you roll a double con el white dice en any de your next three turns, you lata immediately move out de Jail. You then move el number de spaces shown by your doubles roll. **“Get Out of Jail Free” Card:** If you have a “Get Out of Jail Free” card, you lata use it a conseguir out de Jail without rolling doubles. Este card lata be obtained by purchasing it desde another player o drawing it desde el Chance o Comunidad Chest cards. **Pay Fine:** You lata also choose a pay a fine de $50 before you roll el dice en either de your next two turns. After paying el fine, you are free a move y continue playing. Remember, si you do no roll doubles by your third turn o use a “Get Out of Jail Free” card, you must pay el $50 fine a conseguir out de Jail. |
Let’s simplify this by putting everything in a script. This is the content of myScript.nu:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
def initRAGPipeline [] { alcances crear PDF cb–env alcance PDF colecciones crear PDF cb–env colección PDF consulta “CREATE PRIMARY INDEX ON `default`:`cbsh`.`pdf`.`pdf`” vector crear–index PDF textVector 1536 } def storeRAGDoc [] { wrap texto |wrap contenido | each { insert identificación { random uuid } } | doc. actualizar o insertar consulta “SELECT meta().id as id, p.* from `pdf` as p” | wrap contenido| vector enrich–doc. texto | doc. actualizar o insertar } def myAsk [$question: cadena] { dejar norag = ask $question dejar rag = vector enrich–texto $question | vector buscar PDF textVector | seleccionar identificación |doc. conseguir| seleccionar contenido.texto | ask $question {“rag”:$rag, “norag”:$norag} } |
You can source the script file and then call those functions:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
El resultado de tu pregunta se muestra a continuación: fuente ./ragDemo/ragScript.nu El resultado de tu pregunta se muestra a continuación: initRAGPipeline El resultado de tu pregunta se muestra a continuación: open monopolyInstruction.txt |split fila “ ::: ::: “| store El resultado de tu pregunta se muestra a continuación: myAsk “how to get out of jail” Embedding batch 1/1 ╭───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ rag │ Here are el ways a conseguir out de jail en el game de Monopoly: │ │ │ │ │ │ 1. **Roll Doubles:** El most común way a conseguir out de jail is by rolling doubles en your turn. If you roll │ │ │ doubles con el regular white dice en any de your next three turns after being sent a jail, you lata immediately move your token out de jail y advance el corresponding │ │ │ number de spaces. Remember that you lata solo use el white dice para este purpose. │ │ │ │ │ │ 2. **Using “Get Out of Jail Free” Card:** If you have a “Get Out of Jail Free” card, you lata │ │ │ use it a conseguir out de jail without rolling doubles. Simply present el card a el Banker a conseguir out de jail para free. El card is then returned a el bottom de el deck. │ │ │ │ │ │ 3. │ │ │ **Purchase el Card:** If another player has a “Get Out of Jail Free” card y is willing a sell it, you lata purchase el card desde them en a mutually agreed–upon price. │ │ │ Este allows you a conseguir out de jail even si you don‘t have the card yourself. │ │ │ │ │ │ 4. **Pay the Fine:** If you do not roll doubles within three turns or do not have a “Get Out of │ │ │ Jail Free” card, you must pay a fine of $50 to the Bank before you roll the dice on either of your next two turns. Once you pay the fine, you are immediately released from │ │ │ jail and can move your token as per the dice roll. │ │ │ │ │ │ These are the four main ways to get out of jail in Monopoly. │ │ norag │ If you or someone you know is currently in jail and looking to get released, here are some general steps to consider: │ │ │ │ │ │ 1. Contact a lawyer: A criminal defense attorney can │ │ │ provide guidance on legal options and help navigate the legal process for release. │ │ │ │ │ │ 2. Attend court hearings: It is important to attend all court hearings and follow any │ │ │ conditions set by the court to demonstrate cooperation with the legal system. │ │ │ │ │ │ 3. Consider bail: If bail is an option, you may be able to pay a set amount to be released from │ │ │ jail pending trial. If you cannot afford the bail amount, you may seek assistance from a bail bond agent. │ │ │ │ │ │ 4. Seek alternative options: Depending on the circumstances of your │ │ │ case, there may be alternative options for release such as pretrial services, diversion programs, or supervised release. │ │ │ │ │ │ 5. Follow legal advice: It is crucial to follow the │ │ │ advice of your legal counsel and comply with all legal requirements to increase the chances of a successful release. │ │ │ │ │ │ It’s important a note that el process de getting out de │ │ │ jail lata vary depending en el specific circumstances de el case y el laws en your jurisdiction. Para personalizado guidance, it‘s recommended a speak con a lawyer o │ │ │ legal professional specializing en criminal law. │ ╰───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ |
Here, you can see the same kind of result we achieved in the Python RAG demo, but this time using Couchbase Shell. It should be easier to manipulate, change or extend, because you don’t need to deploy an app or know Python. However, it will be less flexible than what you can achieve with Python and Langchain.
If this interests you, stay tuned–more AI and Couchbase Shell content is on the way!
- Aprende más sobre Couchbase Shell
- y Búsqueda vectorial de Couchbase capacidades
Deja un comentario
Lo siento, debes estar conectado para publicar un comentario.