이전에는 사용자가 '채팅'을 할 수 있는 Python 앱을 통해 Couchbase RAG 기능을 사용하는 방법을 보여드렸습니다. PDF로 또는 와 X. 구축은 간단하지만 더 간단하게 구축할 수 있을까요? 최근에 Couchbase Shell을 많이 사용해봤는데 비슷한 작업을 할 수 있을 것 같습니다.
범위 및 컬렉션 설정
이미 다음 사항에 대해 잘 알고 계시리라 생각합니다. 카우치베이스 셸 (cbsh)를 실행하고 클러스터와 모델을 구성합니다.
범위와 컬렉션을 생성하고 선택한 다음 기본 인덱스를 만듭니다:
|
1 2 3 4 5 |
> scopes create pdf > cb-env scope pdf > collections create pdf > cb-env collection pdf > query "CREATE PRIMARY INDEX ON `default`:`cbsh`.`pdf`.`pdf`" |
PDF를 청크 텍스트로 변환
PDF를 텍스트로 변환할 수 있는 다양한 도구가 있습니다. 대부분의 Linux 배포판에서 다음을 찾을 수 있습니다. pdftotext.
|
1 |
> pdftotext ~/monopolyInstruction.pdf |
이렇게 하면 경로가 같지만 파일 이름에 .txt 확장자입니다.
함께 Nushell (cbsh는 Nushell 기반) 분할 명령 덕분에 텍스트를 쉽게 분할할 수 있습니다. 문제는 파일을 분할하는 데 필요한 올바른 구분 기호를 찾는 것입니다. 다행히 여러 줄 문자열을 지원하므로 파일에서 두 단락 사이의 텍스트를 복사하여 붙여넣었습니다. 하지만 정규식을 사용하면 좀 더 정교한 작업을 할 수 있습니다. 이것이 블로그 자료와 프로덕션의 차이점입니다 😇.
|
1 2 3 |
> open ~/monopolyInstruction.txt |split row " ::: ::: "|wrap text |
이렇게 하면 텍스트 문자열 테이블을 얻을 수 있습니다. 이를 Couchbase로 가져오려면 텍스트 필드, 콘텐츠 JSON 객체, 무작위로 생성된 UUID를 추가하고 결과를 업서트합니다.
|
1 2 3 |
> open ~/monopolyInstruction.txt |split row " ::: ::: "|wrap text |wrap content | each { insert id { random uuid } } | doc upsert |
다음 단계는 임베딩 또는 텍스트의 벡터 표현을 만드는 것입니다:
|
1 |
> query "SELECT meta().id as id, p.* from pdf as p" | wrap content| vector enrich-doc text | doc upsert |
그런 다음 벡터 검색 인덱스를 만듭니다. 여기서는 PDF라고 하며 필드를 색인화합니다. 텍스트 벡터를 사용하여 1536 차원 벡터를 생성하고 l2_norm 를 유사도 알고리즘의 기본값으로 설정합니다.
|
1 |
> vector create-index pdf textVector 1536 |
모노폴리의 규칙을 가져와서 감옥에서 빠져나오는 방법을 묻고 있습니다. 원래 예제에서는 컨텍스트가 있는 답변과 없는 답변이 있었습니다.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
> ask "how to get out of jail" If you or someone you know is in jail and needs to be released, here are some general steps to take: Contact a lawyer: If you have legal representation or know of a lawyer who can help with your case, reach out to them for assistance in navigating the legal process. Obtain a bail bond: In many cases, individuals can be released from jail by posting bail. This requires paying a set amount of money to the court, which is typically returned once the individual attends all required court dates. Attend court hearings: It's important to comply with all court requirements, including attending scheduled hearings and following any conditions of release set by the court. Seek support: Consider reaching out to family members, friends, or local organizations that may be able to provide assistance or guidance during this challenging time. Please keep in mind that the process of getting released from jail can vary depending on the specific circumstances of the case and the jurisdiction. It's always best to consult with legal professionals for personalized advice and assistance. |
그리고 컨텍스트와 함께:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
> let question = "how to get out of jail"; vector enrich-text $question | vector search pdf textVector | select id |doc get| select content.text | ask $question 👤 Laurent Doguin 🏠 capella in ☁️ cbsh.gitlog.pdf > let question = "how to get out of jail"; vector enrich-text $question | vector search pdf textVector | select id |doc get| select content.text | ask $question Embedding batch 1/1 You can get out of jail by following these methods: **Roll Doubles:** If you roll a double with the white dice on any of your next three turns, you can immediately move out of Jail. You then move the number of spaces shown by your doubles roll. **"Get Out of Jail Free" Card:** If you have a "Get Out of Jail Free" card, you can use it to get out of Jail without rolling doubles. This card can be obtained by purchasing it from another player or drawing it from the Chance or Community Chest cards. **Pay Fine:** You can also choose to pay a fine of $50 before you roll the dice on either of your next two turns. After paying the fine, you are free to move and continue playing. Remember, if you do not roll doubles by your third turn or use a "Get Out of Jail Free" card, you must pay the $50 fine to get out of Jail. |
모든 것을 스크립트에 넣어서 단순화해 보겠습니다. 다음은 다음과 같은 내용입니다. myScript.nu:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
def initRAGPipeline [] { scopes create pdf cb-env scope pdf collections create pdf cb-env collection pdf query "CREATE PRIMARY INDEX ON `default`:`cbsh`.`pdf`.`pdf`" vector create-index pdf textVector 1536 } def storeRAGDoc [] { wrap text |wrap content | each { insert id { random uuid } } | doc upsert query "SELECT meta().id as id, p.* from `pdf` as p" | wrap content| vector enrich-doc text | doc upsert } def myAsk [$question: string] { let norag = ask $question let rag = vector enrich-text $question | vector search pdf textVector | select id |doc get| select content.text | ask $question {"rag":$rag, "norag":$norag} } |
스크립트 파일을 소싱한 다음 해당 함수를 호출할 수 있습니다:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
> source ./ragDemo/ragScript.nu > initRAGPipeline > open monopolyInstruction.txt |split row " ::: ::: "| store > myAsk "how to get out of jail" Embedding batch 1/1 ╭───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ rag │ Here are the ways to get out of jail in the game of Monopoly: │ │ │ │ │ │ 1. **Roll Doubles:** The most common way to get out of jail is by rolling doubles on your turn. If you roll │ │ │ doubles with the regular white dice on any of your next three turns after being sent to jail, you can immediately move your token out of jail and advance the corresponding │ │ │ number of spaces. Remember that you can only use the white dice for this purpose. │ │ │ │ │ │ 2. **Using "Get Out of Jail Free" Card:** If you have a "Get Out of Jail Free" card, you can │ │ │ use it to get out of jail without rolling doubles. Simply present the card to the Banker to get out of jail for free. The card is then returned to the bottom of the deck. │ │ │ │ │ │ 3. │ │ │ **Purchase the Card:** If another player has a "Get Out of Jail Free" card and is willing to sell it, you can purchase the card from them at a mutually agreed-upon price. │ │ │ This allows you to get out of jail even if you don't have the card yourself. │ │ │ │ │ │ 4. **Pay the Fine:** If you do not roll doubles within three turns or do not have a "Get Out of │ │ │ Jail Free" card, you must pay a fine of $50 to the Bank before you roll the dice on either of your next two turns. Once you pay the fine, you are immediately released from │ │ │ jail and can move your token as per the dice roll. │ │ │ │ │ │ These are the four main ways to get out of jail in Monopoly. │ │ norag │ If you or someone you know is currently in jail and looking to get released, here are some general steps to consider: │ │ │ │ │ │ 1. Contact a lawyer: A criminal defense attorney can │ │ │ provide guidance on legal options and help navigate the legal process for release. │ │ │ │ │ │ 2. Attend court hearings: It is important to attend all court hearings and follow any │ │ │ conditions set by the court to demonstrate cooperation with the legal system. │ │ │ │ │ │ 3. Consider bail: If bail is an option, you may be able to pay a set amount to be released from │ │ │ jail pending trial. If you cannot afford the bail amount, you may seek assistance from a bail bond agent. │ │ │ │ │ │ 4. Seek alternative options: Depending on the circumstances of your │ │ │ case, there may be alternative options for release such as pretrial services, diversion programs, or supervised release. │ │ │ │ │ │ 5. Follow legal advice: It is crucial to follow the │ │ │ advice of your legal counsel and comply with all legal requirements to increase the chances of a successful release. │ │ │ │ │ │ It's important to note that the process of getting out of │ │ │ jail can vary depending on the specific circumstances of the case and the laws in your jurisdiction. For personalized guidance, it's recommended to speak with a lawyer or │ │ │ legal professional specializing in criminal law. │ ╰───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ |
여기에서는 Python RAG 데모에서 얻은 것과 동일한 종류의 결과를 볼 수 있지만 이번에는 Couchbase Shell을 사용합니다. 앱을 배포하거나 Python을 알 필요가 없으므로 조작, 변경 또는 확장하기가 더 쉬울 것입니다. 하지만 파이썬과 랭체인으로 달성할 수 있는 것보다는 유연성이 떨어질 것입니다.
관심이 있으시다면 계속 지켜봐 주세요. 더 많은 AI 및 Couchbase Shell 콘텐츠가 준비 중입니다!
-
- 자세히 알아보기 카우치베이스 셸
- 그리고 카우치베이스 벡터 검색 기능
