Aaron Benton is an experienced architect who specializes in creative solutions to develop innovative mobile applications. He has over 10 years experience in full stack development, including ColdFusion, SQL, NoSQL, JavaScript, HTML, and CSS. Aaron is currently an Applications Architect for Shop.com in Greensboro, North Carolina and is a Couchbase Community Champion.
FakeIt Series 4 of 5: Working with Existing Data
So far in our FakeIt series we’ve seen how we can Generate Fake Data, Share Data and Dependencies, and use Definitions for smaller models. Today we are going to look at the last major feature of FakeIt, which is working with existing data through inputs.
Rarely as developers do we get the advantage of working on greenfield applications, our domains are more often than not a comprised of different legacy databases and applications. As we are modeling and building new applications, we need to reference and use this existing data. FakeIt allows you to provide existing data to your models through JSON, CSV or CSON files. This data is exposed as an inputs variable in each of a models *run and *build functions.
Users Model
We will start with our users.yaml model that we updated to in our most recent post to use Address 그리고 Phone definitions.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
이름: Users 유형: object 열쇠: _id 데이터: min: 1000 max: 2000 properties: _id: 유형: 문자열 설명: 그 document 아이디 지어진 ~에 의해 그 prefix “user_” 그리고 그 users 아이디 데이터: post_build: “`user_${this.user_id}`” 문서 유형: 유형: 문자열 설명: 그 document 유형 데이터: 가치: “user” user_id: 유형: integer 설명: An auto–incrementing 번호 데이터: 빌드: document_색인 first_name: 유형: 문자열 설명: 그 users 첫 번째 이름 데이터: 빌드: faker.이름.firstName() last_name: 유형: 문자열 설명: 그 users last 이름 데이터: 빌드: faker.이름.lastName() 사용자 이름: 유형: 문자열 설명: 그 사용자 이름 데이터: 빌드: faker.internet.userName() 비밀번호: 유형: 문자열 설명: 그 users 비밀번호 데이터: 빌드: faker.internet.비밀번호() email_address: 유형: 문자열 설명: 그 users 이메일 address 데이터: 빌드: faker.internet.이메일() created_on: 유형: integer 설명: An epoch time 의 when 그 사용자 was created 데이터: 빌드: 새로운 날짜(faker.날짜.past()).시간 가져오기() addresses: 유형: object 설명: An object containing 그 집 그리고 work addresses ~를 위해 그 사용자 properties: 집: 설명: 그 users 집 address schema: $ref: ‘#/definitions/Address’ work: 설명: 그 users work address schema: $ref: ‘#/definitions/Address’ main_phone: 설명: 그 users 메인 phone 번호 schema: $ref: ‘#/definitions/Phone’ 데이터: post_build: | 삭제 이것.main_phone.유형 반환 이것.메인_phone additional_phones: 유형: array 설명: 그 users additional phone numbers items: $ref: ‘#/definitions/Phone’ 데이터: min: 1 max: 4 definitions: Phone: 유형: object properties: 유형: 유형: 문자열 설명: 그 phone 유형 데이터: 빌드: faker.무작위의.arrayElement([ ‘Home’, ‘Work’, ‘Mobile’, ‘Other’ ]) phone_number: 유형: 문자열 설명: 그 phone 번호 데이터: 빌드: faker.phone.phoneNumber().교체하다(/[^0–9]+/g, ”) 확장 프로그램: 유형: 문자열 설명: 그 phone 확장 프로그램 데이터: 빌드: chance.bool({ likelihood: 30 }) ? chance.integer({ min: 1000, max: 9999 }) : null Address: 유형: object properties: address_1: 유형: 문자열 설명: 그 address 1 데이터: 빌드: `${faker.address.streetAddress()} ${faker.address.streetSuffix()}` address_2: 유형: 문자열 설명: 그 address 2 데이터: 빌드: chance.bool({ likelihood: 35 }) ? faker.address.secondaryAddress() : null locality: 유형: 문자열 설명: 그 city / locality 데이터: 빌드: faker.address.city() 지역: 유형: 문자열 설명: 그 지역 / 상태 / province 데이터: 빌드: faker.address.stateAbbr() postal_code: 유형: 문자열 설명: 그 zip code / postal code 데이터: 빌드: faker.address.zipCode() country: 유형: 문자열 설명: 그 country code 데이터: 빌드: faker.address.countryCode() |
Currently, our Address definition is generating a random country. What if our ecommerce site only supports a small subset of the 195 countries? Let’s say we support six countries to start with: US, CA, MX, UK, ES, DE. We could update the definitions country property to grab a random array element:
(For brevity the other properties have been left off of the model definition)
|
1 2 3 4 5 6 |
... country: 유형: 문자열 설명: 그 country code 데이터: 빌드: faker.무작위의.arrayElement([‘US’, ‘CA’, ‘MX’, ‘UK’, ‘ES’, ‘DE’]); |
While this would work, what if we have other models that rely on this same country info, we would have to duplicate this logic. We can achieve this same thing by creating a countries.json file, and adding an inputs property to the data property that can be an absolute or relative path to our input. When are model is generated, our countries.json file will be exposed to each of the models build functions via the inputs argument as inputs.countries
(For brevity the other properties have been left off of the model definition)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
이름: Users 유형: object 열쇠: _id 데이터: min: 1000 max: 2000 inputs: ./countries.JSON properties: ... definitions: ... country: 유형: 문자열 설명: 그 country code 데이터: 빌드: faker.무작위의.arrayElement(inputs.countries); countries.JSON [ “US”, “CA”, “MX”, “UK”, “ES”, “DE” ] |
By changing one existing line and adding another line in model we have provided existing data to our Users model. We can still generate a random country, based on the countries our application supports. Lets test our changes by using the following command:
|
1 |
fakeit 콘솔 —count 1 models/users.yaml |
Products Model
Our ecommerce application is using a separate system for categorization, we need to expose that data to our randomly generated products so that we are using valid category information. We will start with the products.yaml that we defined in the FakeIt Series 2 of 5: Shared Data and Dependencies post.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
products.yaml 이름: 제품 유형: object 열쇠: _id 데이터: min: 4000 max: 5000 properties: _id: 유형: 문자열 설명: 그 document 아이디 데이터: post_build: `product_${이것.product_id}` 문서 유형: 유형: 문자열 설명: 그 document 유형 데이터: 가치: product product_id: 유형: 문자열 설명: Unique 식별자 representing a specific product 데이터: 빌드: faker.무작위의.UUID() 가격: 유형: 더블 설명: 그 product 가격 데이터: 빌드: chance.floating({ min: 0, max: 150, fixed: 2 }) sale_price: 유형: 더블 설명: 그 product 가격 데이터: post_build: | 하다 sale_price = 0; 만약 (chance.bool({ likelihood: 30 })) { sale_price = chance.floating({ min: 0, max: 이것.price * chance.floating({ min: 0, max: 0.99, fixed: 2 }), fixed: 2 }); } 반환 sale_price; display_name: 유형: 문자열 설명: Display 이름 의 product. 데이터: 빌드: faker.commerce.productName() short_description: 유형: 문자열 설명: Description 의 product. 데이터: 빌드: faker.lorem.paragraphs(1) long_description: 유형: 문자열 설명: Description 의 product. 데이터: 빌드: faker.lorem.paragraphs(5) keywords: 유형: array 설명: An array 의 keywords items: 유형: 문자열 데이터: min: 0 max: 10 빌드: faker.무작위의.word() availability: 유형: 문자열 설명: 그 availability 상태 의 그 product 데이터: 빌드: | 하다 availability = ‘In-Stock’; 만약 (chance.bool({ likelihood: 40 })) { availability = faker.무작위의.arrayElement([ ‘Preorder’, ‘Out of Stock’, ‘Discontinued’ ]); } 반환 availability; availability_date: 유형: integer 설명: An epoch time 의 when 그 product 이다 available 데이터: 빌드: faker.날짜.recent() post_build: 새로운 날짜(이것.availability_date).시간 가져오기() product_slug: 유형: 문자열 설명: 그 URL friendly 버전 의 그 product 이름 데이터: post_build: faker.helpers.slugify(이것.display_name).toLowerCase() category: 유형: 문자열 설명: Category ~를 위해 그 Product 데이터: 빌드: faker.commerce.department() category_slug: 유형: 문자열 설명: 그 URL friendly 버전 의 그 category 이름 데이터: post_build: faker.helpers.slugify(이것.category).toLowerCase() image: 유형: 문자열 설명: Image URL representing 그 product. 데이터: 빌드: faker.image.image() alternate_images: 유형: array 설명: An array 의 alternate images ~를 위해 그 product items: 유형: 문자열 데이터: min: 0 max: 4 빌드: faker.image.image() |
Our existing categories data has been provided in CSV format.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
categories.csv “category_id”,“category_name”,“category_slug” 23,“Electronics”,“electronics” 1032,“Office Supplies”,“office-supplies” 983,“Clothing & Apparel”,“clothing-and-apparel” 483,“Movies, Music & Books”,“movies-music-and-books” 3023,“Sports & Fitness”,“sports-and-fitness” 4935,“Automotive”,“automotive” 923,“Tools”,“tools” 5782,“Home Furniture”,“home-furniture” 9783,“Health & Beauty”,“health-and-beauty” 2537,“Toys”,“toys” 10,“Video Games”,“video-games” 736,“Pet Supplies”,“pet-supplies” |
Now we need to update our products.yaml model to use this existing data.
(For brevity the other properties have been left off of the model definition)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
이름: 제품 유형: object 열쇠: _id 데이터: min: 4000 max: 5000 inputs: – ./categories.csv pre_build: globals.current_category = faker.무작위의.arrayElement(inputs.categories); properties: ... category_id: 유형: integer 설명: 그 Category 아이디 ~를 위해 그 Product 데이터: 빌드: globals.current_category.category_아이디 category: 유형: 문자열 설명: Category ~를 위해 그 Product 데이터: 빌드: globals.current_category.category_이름 category_slug: 유형: 문자열 설명: 그 URL friendly 버전 의 그 category 이름 데이터: post_build: globals.current_category.category_slug ... |
There are a few things to notice about how we’ve updated our products.yaml model.
- inputs: is defined as an array not a string. While we are only using a single input, you can provide as many input files to your model as necessary.
- A pre_build function is defined at the root of the model. This is because we cannot grab a random array element for each of our three category properties as the values would not match. Each time an individual document is generated for our model, this pre_build function will run first.
- Each of our category properties build functions reference the global variable set by the pre_build function on our model.
We can test our changes by using the following command:
|
1 |
fakeit 콘솔 —count 1 models/products.yaml |
결론
Being able to work with existing data is an extremely powerful feature of FakeIt. It can be used to maintain the integrity of randomly generated documents to work with existing system, and can even be used to transform existing data and import it into Couchbase Server.
Up Next
Previous
- FakeIt Series 1 of 5: Generating Fake Data
- FakeIt Series 2 of 5: Shared Data and Dependencies
- FakeIt Series 3 of 5: Lean Models through Definitions
This post is part of the Couchbase Community Writing Program
댓글 남기기
댓글을 달기 위해서는 로그인해야합니다.