Continuous Integration and Continuous Deployment are now common software development practices. In the world of databases, this translates into needing on-demand, stateful, ephemeral environments.
Provisioning a stateless environment is not tied to any particular source of data. All that is needed is to run the code you want to test in your CI environment. This is the basis of most CI/CD tools and won’t be covered in this article.
The slightly harder part comes from the dependencies the application needs to be tested properly, which is often referred to as external services. Couchbase being one of them. There are different ways to get those, through Docker containers for instance, or hosted in your test infrastructure, or some external as a Service solution. It does not really matter as long as they are available while running your test. Good practices would be to use Environment Variables to refer to those instances.
Assuming these services are running, like a Couchbase Free Tier instance or a Docker container, the next step is to make sure that they are configured correctly, and seeded with the data needed for the test.
A while ago, I posted about using Couchbase Shell in GitHub actions. This will tell you the basics about using Couchbase Shell with GitHub Actions, but this can be applied to most CI/CD solutions as well. Today, I want to go further and show you some useful scripts to clone a cluster or elements of a cluster for your on demand environments.
Using Couchbase Shell to clone environments
When using Couchbase Shell, the first thing that comes to mind when wanting to do something is, is there a function for that? As of now we don’t have a function to clone something. Most of the available functions reflect our APIs capabilities and we have no cloning APIs today. But, we have the ability to write scripts, which means we can make our own!
The first thing that comes to mind when managing databases is often to recreate the structure and schemas. As Couchbase is Schemaless, this will only consist of the existing buckets, scopes, collections, and indexes in the source cluster. The first step is to export that structure so it can be reimported later. This function will list every bucket, then inner scopes and collections, and add them to an array. Then it will list all indexes and add them to the output JSON.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
# Exports all buckets, scopes, collections and indexes # for the given cluster def 내보내기–클러스터–struct [ 원천: 문자열 # The cluster to export ] { mut 내보내기 = [] 하다 buckets = buckets —clusters $원천 # List the buckets of the given cluster ~를 위해 양동이 안으로 $buckets { mut scope_structs = [] 하다 범위 = 범위 —clusters $원천 —양동이 $양동이.이름 ~를 위해 범위 안으로 $범위 { 하다 컬렉션 = (컬렉션 —clusters $원천 —양동이 $양동이.이름 —범위 $범위.범위 | reject –i 클러스터) $scope_structs ++= [{ 범위: $범위.범위, 컬렉션: $컬렉션 }] } # Merge the scopes with the bucket object and add it to the export array 하다 buc = ( $양동이 | merge {범위: $범위_structs } ) $내보내기 ++= [ $buc ] } 하다 indexes = 질의 indexes —definitions —비활성화–context —clusters $원천 하다 output = { buckets: $내보내기, indexes: $indexes } 반환 $output } |
This works because under the hood, Couchbase Shell is using Nushell, a new type of shell that is portable (meaning it works the same way on Linux, Windows, or OS X, which is great for CI/CD scripts having to support different OS), and that considers any structure data as a DataFrame, making the manipulation of JSON extremely easy.
To try it out, run cbsh, then source the file containing the function. For me it’s ci_scripts.nu. I have a cluster already configured in my cbsh config, called 로컬
|
1 2 3 4 |
로랑 도귄 at 로컬 안으로 여행–sample.inventory._default < 원천 CI–scripts.nu 로랑 도귄 at 로컬 안으로 여행–sample.inventory._default < 내보내기–클러스터–struct 로컬 | 저장 로컬–클러스터–내보내기.JSON |
Now if you open local-cluster-export.json, you will get the structure of your cluster:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 |
{ “buckets”: [ { “cluster”: “로컬”, “이름”: “트래블 샘플”, “type”: “카우치베이스”, “replicas”: 0, “min_durability_level”: “none”, “ram_quota”: 209715200, “flush_enabled”: 거짓, “cloud”: 거짓, “max_expiry”: 0, “scopes”: [ { “scope”: “재고”, “collections”: [ { “collection”: “공항”, “max_expiry”: “inherited” }, { “collection”: “항공사”, “max_expiry”: “inherited” }, { “collection”: “route”, “max_expiry”: “inherited” }, { “collection”: “landmark”, “max_expiry”: “inherited” }, { “collection”: “hotel”, “max_expiry”: “inherited” } ] }, { “scope”: “tenant_agent_00”, “collections”: [ { “collection”: “사용자”, “max_expiry”: “inherited” }, { “collection”: “bookings”, “max_expiry”: “inherited” } ] }, { “scope”: “tenant_agent_01”, “collections”: [ { “collection”: “사용자”, “max_expiry”: “inherited” }, { “collection”: “bookings”, “max_expiry”: “inherited” } ] }, { “scope”: “tenant_agent_02”, “collections”: [ { “collection”: “사용자”, “max_expiry”: “inherited” }, { “collection”: “bookings”, “max_expiry”: “inherited” } ] }, { “scope”: “tenant_agent_03”, “collections”: [ { “collection”: “사용자”, “max_expiry”: “inherited” }, { “collection”: “bookings”, “max_expiry”: “inherited” } ] }, { “scope”: “tenant_agent_04”, “collections”: [ { “collection”: “사용자”, “max_expiry”: “inherited” }, { “collection”: “bookings”, “max_expiry”: “inherited” } ] }, { “scope”: “_default”, “collections”: [ { “collection”: “_default”, “max_expiry”: “inherited” } ] }, { “scope”: “_system”, “collections”: [ { “collection”: “_query”, “max_expiry”: “” }, { “collection”: “_mobile”, “max_expiry”: “” } ] } ] } ], “indexes”: [ { “bucket”: “트래블 샘플”, “scope”: “_system”, “collection”: “_query”, “이름”: “#primary”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE PRIMARY INDEX `#primary` ON `travel-sample`.`_system`.`_query`”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_airportname”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_airportname` ON `travel-sample`(`airportname`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_city”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_city` ON `travel-sample`(`city`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_faa”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_faa` ON `travel-sample`(`faa`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_icao”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_icao` ON `travel-sample`(`icao`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “항공사”, “이름”: “def_inventory_airline_primary”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE PRIMARY INDEX `def_inventory_airline_primary` ON `travel-sample`.`inventory`.`airline` WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “공항”, “이름”: “def_inventory_airport_airportname”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_inventory_airport_airportname` ON `travel-sample`.`inventory`.`airport`(`airportname`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “공항”, “이름”: “def_inventory_airport_city”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_inventory_airport_city` ON `travel-sample`.`inventory`.`airport`(`city`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “공항”, “이름”: “def_inventory_airport_faa”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_inventory_airport_faa` ON `travel-sample`.`inventory`.`airport`(`faa`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “공항”, “이름”: “def_inventory_airport_primary”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE PRIMARY INDEX `def_inventory_airport_primary` ON `travel-sample`.`inventory`.`airport` WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “hotel”, “이름”: “def_inventory_hotel_city”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_inventory_hotel_city` ON `travel-sample`.`inventory`.`hotel`(`city`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “hotel”, “이름”: “def_inventory_hotel_primary”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE PRIMARY INDEX `def_inventory_hotel_primary` ON `travel-sample`.`inventory`.`hotel` WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “landmark”, “이름”: “def_inventory_landmark_city”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_inventory_landmark_city` ON `travel-sample`.`inventory`.`landmark`(`city`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “landmark”, “이름”: “def_inventory_landmark_primary”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE PRIMARY INDEX `def_inventory_landmark_primary` ON `travel-sample`.`inventory`.`landmark` WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “route”, “이름”: “def_inventory_route_primary”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE PRIMARY INDEX `def_inventory_route_primary` ON `travel-sample`.`inventory`.`route` WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “route”, “이름”: “def_inventory_route_route_src_dst_day”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_inventory_route_route_src_dst_day` ON `travel-sample`.`inventory`.`route`(`sourceairport`,`destinationairport`,(distinct (array (`v`.`day`) for `v` in `schedule` end))) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “route”, “이름”: “def_inventory_route_schedule_utc”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_inventory_route_schedule_utc` ON `travel-sample`.`inventory`.`route`(array (`s`.`utc`) for `s` in `schedule` end) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “재고”, “collection”: “route”, “이름”: “def_inventory_route_sourceairport”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_inventory_route_sourceairport` ON `travel-sample`.`inventory`.`route`(`sourceairport`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_name_type”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_name_type` ON `travel-sample`(`name`) WHERE (`_type` = “User“) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_primary”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE PRIMARY INDEX `def_primary` ON `travel-sample` WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_route_src_dst_day”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_route_src_dst_day` ON `travel-sample`(`sourceairport`,`destinationairport`,(distinct (array (`v`.`day`) for `v` in `schedule` end))) WHERE (`type` = “route“) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_schedule_utc”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_schedule_utc` ON `travel-sample`(array (`s`.`utc`) for `s` in `schedule` end) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_sourceairport”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_sourceairport` ON `travel-sample`(`sourceairport`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” }, { “bucket”: “트래블 샘플”, “scope”: “_default”, “collection”: “_default”, “이름”: “def_type”, “status”: “Ready”, “storage_mode”: “memory_optimized”, “replicas”: 0, “definition”: “CREATE INDEX `def_type` ON `travel-sample`(`type`) WITH { “defer_빌드“:true }”, “cluster”: “로컬” } ] } |
I have deleted that bucket for the purpose of this test, to reimport it later: buckets drop travel-sample.
The next logical step is to have a function that takes this file as input and recreate the complete structure in another cluster:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# Import all buckets, scopes and collections structure # in the given cluster def 가져오기–클러스터–struct [ destination: 문자열 # The cluster to import ] { 하다 structure = $안으로 # Assigning the piped structure to a variable 하다 buckets = $structure.buckets ~를 위해 양동이 안으로 $buckets { $양동이 | _create–양동이–definition $destination ~를 위해 범위 안으로 ($양동이.범위 | where not ( $그것.범위 | str starts–~와 함께 “_” ) ) { print $“Create scope ($destination)_($bucket.name)_($scope.scope)” 범위 만들다 —clusters $destination —양동이 $양동이.이름 $범위.범위 ~를 위해 col 안으로 $범위.컬렉션 { print $“Create collection ($destination)_($bucket.name)_($scope.scope)_($col.collection)” 컬렉션 만들다 —clusters $destination —양동이 $양동이.이름 —범위 $범위.범위 $col.컬렉션 } } } 하다 indexes = $structure.indexes $indexes | _create–indexes $destination # Nushell allows you to use other functions you created } def _create–indexes [ destination: 문자열 # the cluster where to create indexes ] { 하다 indexes = $안으로 ~를 위해 색인 안으로 $indexes { print $“Recreating index ($index.name) on cluster ($destination) with: “ print $색인.definition 질의 $색인.definition —비활성화–context —clusters $destination } } |
Now to run that function:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
로랑 도귄 at 로컬 안으로 여행–sample.inventory._default < 열다 로컬–클러스터–내보내기.JSON | 가져오기–클러스터–struct 카펠라 로랑 도귄 at 로컬 안으로 여행–sample.inventory._default < 열다 로컬–클러스터–내보내기.JSON | 가져오기–클러스터–struct 로컬 Create 양동이 local_travel–sample ~와 함께 200 할당량, 유형 카우치베이스, 0 replicas, none 내구성, 0 expiry Create 범위 local_travel–sample_inventory Create 컬렉션 local_travel–sample_inventory_airport Create 컬렉션 local_travel–sample_inventory_airline Create 컬렉션 local_travel–sample_inventory_route Create 컬렉션 local_travel–sample_inventory_landmark Create 컬렉션 local_travel–sample_inventory_hotel Create 범위 local_travel–sample_tenant_agent_00 Create 컬렉션 local_travel–sample_tenant_agent_00_users Create 컬렉션 local_travel–sample_tenant_agent_00_bookings Create 범위 local_travel–sample_tenant_agent_01 Create 컬렉션 local_travel–sample_tenant_agent_01_users Create 컬렉션 local_travel–sample_tenant_agent_01_bookings Create 범위 local_travel–sample_tenant_agent_02 Create 컬렉션 local_travel–sample_tenant_agent_02_users Create 컬렉션 local_travel–sample_tenant_agent_02_bookings Create 범위 local_travel–sample_tenant_agent_03 Create 컬렉션 local_travel–sample_tenant_agent_03_users Create 컬렉션 local_travel–sample_tenant_agent_03_bookings Create 범위 local_travel–sample_tenant_agent_04 Create 컬렉션 local_travel–sample_tenant_agent_04_users Create 컬렉션 local_travel–sample_tenant_agent_04_bookings Recreating 색인 #primary on cluster local with: 생성하다 PRIMARY INDEX `#primary` ON `travel-sample`.`_system`.`_query` Recreating 색인 def_airportname ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_airportname` ON `여행–sample`(`airportname`) 함께 { “defer_build”:참인 } Recreating 색인 def_city ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_city` ON `여행–sample`(`city`) 함께 { “defer_build”:참인 } Recreating 색인 def_faa ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_faa` ON `여행–sample`(`faa`) 함께 { “defer_build”:참인 } Recreating 색인 def_icao ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_icao` ON `여행–sample`(`icao`) 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_airline_primary ~에 클러스터 로컬 ~와 함께: 생성하다 PRIMARY INDEX `def_inventory_airline_primary` ON `여행–sample`.`inventory`.`항공사` 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_airport_airportname ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_inventory_airport_airportname` ON `여행–sample`.`inventory`.`airport`(`airportname`) 함께 { “defer_build”:참인 }Recreating 색인 def_inventory_airport_city ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_inventory_airport_city` ON `여행–sample`.`inventory`.`airport`(`city`) 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_airport_faa ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_inventory_airport_faa` ON `여행–sample`.`inventory`.`airport`(`faa`) 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_airport_primary ~에 클러스터 로컬 ~와 함께: 생성하다 PRIMARY INDEX `def_inventory_airport_primary` ON `여행–sample`.`inventory`.`airport` 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_hotel_city ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_inventory_hotel_city` ON `여행–sample`.`inventory`.`hotel`(`city`) 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_hotel_primary ~에 클러스터 로컬 ~와 함께: 생성하다 PRIMARY INDEX `def_inventory_hotel_primary` ON `여행–sample`.`inventory`.`hotel` 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_landmark_city ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_inventory_landmark_city` ON `여행–sample`.`inventory`.`landmark`(`city`) 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_landmark_primary ~에 클러스터 로컬 ~와 함께: 생성하다 PRIMARY INDEX `def_inventory_landmark_primary` ON `여행–sample`.`inventory`.`landmark` 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_route_primary ~에 클러스터 로컬 ~와 함께: 생성하다 PRIMARY INDEX `def_inventory_route_primary` ON `여행–sample`.`inventory`.`route` 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_route_route_src_dst_day ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_inventory_route_route_src_dst_day` ON `여행–sample`.`inventory`.`route`(`sourceairport`,`destinationairport`,(distinct (array (`v`.`day`) ~를 위해 `v` 안으로 `schedule` end))) 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_route_schedule_utc ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_inventory_route_schedule_utc` ON `여행–sample`.`inventory`.`route`(array (`s`.`utc`) ~를 위해 `s` 안으로 `schedule` end) 함께 { “defer_build”:참인 } Recreating 색인 def_inventory_route_sourceairport ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_inventory_route_sourceairport` ON `여행–sample`.`inventory`.`route`(`sourceairport`) 함께 { “defer_build”:참인 } Recreating 색인 def_name_type ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_name_type` ON `여행–sample`(`이름`) 어디 (`_type` = “User”) 함께 { “defer_build”:참인 } Recreating 색인 def_primary ~에 클러스터 로컬 ~와 함께: 생성하다 PRIMARY INDEX `def_primary` ON `여행–sample` 함께 { “defer_build”:참인 } Recreating 색인 def_route_src_dst_day ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_route_src_dst_day` ON `여행–sample`(`sourceairport`,`destinationairport`,(distinct (array (`v`.`day`) ~를 위해 `v` 안으로 `schedule` end))) 어디 (`유형` = “route”) 함께 { “defer_build”:참인 } Recreating 색인 def_schedule_utc ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_schedule_utc` ON `여행–sample`(array (`s`.`utc`) ~를 위해 `s` 안으로 `schedule` end) 함께 { “defer_build”:참인 } Recreating 색인 def_sourceairport ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_sourceairport` ON `여행–sample`(`sourceairport`) 함께 { “defer_build”:참인 } Recreating 색인 def_type ~에 클러스터 로컬 ~와 함께: 생성하다 INDEX `def_type` ON `여행–sample`(`유형`) 함께 { “defer_build”:참인 } |
And there you have it, functions that allow you to export and import the data structure from one cluster to another. While this is a good starting point, there are still questions about how to reimport data, or about granularity. Also, you may not want to export and import a complete cluster.
Filtering buckets to import is fairly easy as Nushell allows you to filter dataframes:
|
1 2 |
로랑 도귄 at 로컬 안으로 여행–sample.inventory._default < 열다 로컬–클러스터–내보내기.JSON | { buckets: ( $안으로.buckets | where 이름 == ‘travel-sample’), indexes :( $안으로.indexes | where 양동이 == ‘travel-sample’) } |
This will recreate a JSON object containing only a bucket named 여행 샘플 and indexes for this bucket.
From there you should be all set to manage basic cluster structure. What about the data? There are different ways you can import data with cbsh, as it covers most key/value operations as well as any INSERT/UPSERT queries. And then we have the doc import command. Its usage is fairly straightforward, all you need is a list of rows with an identified id field. This can be anything that can be turned into a dataframe for Nushell (XML, CSV, TSV, Parquet, and more). And of course, it can be a JSON file from a Couchbase SQL++ query. This is an example that will save a query result to a file and import that file back to a collection:
|
1 2 3 4 5 6 7 8 |
# Save file content to filename 하다 filename = $“temp_($src_bucket)_($src_scope)_($src_collection).json” 하다 질의 = “SELECT meta().id as meta_id, meta().expiration as expiration, c.* FROM `” + $src_bucket + “`.” + $src_scope + “.” + $src_collection + ” c” 질의 —비활성화–context —clusters $p.소스 $질의 | 저장 –f $filename # Import the file content and print the results print $“Import collection content from ($src)_($src_bucket)_($src_scope)_($src_collection) to ($dest)_($dest_bucket)_($dest_scope)_($dest_collection)” print ( 의사 가져오기 —양동이 $p.dest_bucket —범위 $p.dest_scope —컬렉션 $p.dest_collection —clusters $p.dest —아이디–column meta_아이디 $filename ) |
That’s one particular example but the whole point of using scripting language is to make them your own. You will find a more complete example in this GitHub Gist. It has support for environment variables for source and destination and you can decide to either clone all buckets of a cluster, a specific bucket, scope, or collection.
Don’t hesitate to drop us a comment here or on 디스코드, we are always looking for suggestions to improve the global Couchbase experience.
댓글 남기기
댓글을 달기 위해서는 로그인해야합니다.