{"id":3187,"date":"2017-04-06T10:36:30","date_gmt":"2017-04-06T17:36:30","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=3187"},"modified":"2023-06-19T03:02:33","modified_gmt":"2023-06-19T10:02:33","slug":"fakeit-series-working-existing-data","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/fakeit-series-working-existing-data\/","title":{"rendered":"FakeIt Series 4 of 5: Working with Existing Data"},"content":{"rendered":"

Aaron Benton is an experienced architect who specializes in creative solutions to develop innovative mobile applications. He has over 10 years experience in full stack development, including ColdFusion, SQL, NoSQL, JavaScript, HTML, and CSS. Aaron is currently an Applications Architect for Shop.com in Greensboro, North Carolina and is a Couchbase Community Champion<\/a>.<\/p>\n

\"Aaron<\/p>\n

FakeIt Series 4 of 5: Working with Existing Data<\/strong><\/h3>\n

So far in our FakeIt<\/a><\/u>\u00a0series we’ve seen how we can Generate Fake Data<\/a><\/u>, Share Data and Dependencies<\/a><\/u>, and use Definitions\u00a0for smaller models<\/a>. Today we are going to look at the last major feature of FakeIt, which is working with existing data through inputs.<\/p>\n

Rarely as developers do we get the advantage of working on greenfield applications, our domains are more often than not a comprised of different legacy databases and applications. As we are modeling and building new applications, we need to reference and use this existing data. \u00a0FakeIt<\/a><\/u>\u00a0allows you to provide existing data to your models through JSON, CSV or CSON files. This data is exposed as an inputs\u00a0variable in each of a models *run\u00a0and *build\u00a0functions.<\/p>\n

Users Model<\/strong><\/h3>\n

We will start with our users.yaml\u00a0model that we updated to in our most recent post<\/a><\/u>\u00a0to use Address <\/strong>and Phone<\/strong>\u00a0definitions.<\/p>\n

name:\u00a0Users\r\ntype:\u00a0object\r\nkey:\u00a0_id\r\ndata:\r\n\u00a0 min:\u00a01000\r\n\u00a0 max:\u00a02000\r\nproperties:\r\n\u00a0 _id:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The document id built by the prefix \"user_\"\u00a0and the users id\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 post_build:\u00a0\"`user_${this.user_id}`\"\r\n\u00a0 doc_type:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The document type\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 value:\u00a0\"user\"\r\n\u00a0 user_id:\r\n\u00a0 \u00a0 type:\u00a0integer\r\n\u00a0 \u00a0 description:\u00a0An auto-incrementing number\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0document_index\r\n\u00a0 first_name:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The users first name\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.name.firstName()\r\n\u00a0 last_name:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The users last name\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.name.lastName()\r\n\u00a0 username:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The username\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.internet.userName()\r\n\u00a0 password:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The users password\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.internet.password()\r\n\u00a0 email_address:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The users email address\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.internet.email()\r\n\u00a0 created_on:\r\n\u00a0 \u00a0 type:\u00a0integer\r\n\u00a0 \u00a0 description:\u00a0An epoch time of when the user was created\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0new Date(faker.date.past()).getTime()\r\n\u00a0 addresses:\r\n\u00a0 \u00a0 type:\u00a0object\r\n\u00a0 \u00a0 description:\u00a0An object containing the home and work addresses for the user\r\n\u00a0 \u00a0 properties:\r\n\u00a0 \u00a0 \u00a0 home:\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The users home address\r\n\u00a0 \u00a0 \u00a0 \u00a0 schema:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0$ref: '#\/definitions\/Address'\r\n\u00a0 \u00a0 \u00a0 work:\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The users work address\r\n\u00a0 \u00a0 \u00a0 \u00a0 schema:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0$ref: '#\/definitions\/Address'\r\n\u00a0 main_phone:\r\n\u00a0 \u00a0 description:\u00a0The users main phone number\r\n\u00a0 \u00a0 schema:\r\n\u00a0 \u00a0 \u00a0$ref: '#\/definitions\/Phone'\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 post_build:\u00a0|\r\n\u00a0 \u00a0 \u00a0 \u00a0delete this.main_phone.type\r\n\u00a0 \u00a0 \u00a0 \u00a0return this.main_phone\r\n\u00a0 additional_phones:\r\n\u00a0 \u00a0 type:\u00a0array\r\n\u00a0 \u00a0 description:\u00a0The users additional phone numbers\r\n\u00a0 \u00a0 items:\r\n\u00a0 \u00a0 \u00a0$ref: '#\/definitions\/Phone'\r\n\u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 min:\u00a01\r\n\u00a0 \u00a0 \u00a0 \u00a0 max:\u00a04\r\ndefinitions:\r\n\u00a0 Phone:\r\n\u00a0 \u00a0 type:\u00a0object\r\n\u00a0 \u00a0 properties:\r\n\u00a0 \u00a0 \u00a0 type:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The phone type\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.random.arrayElement([ 'Home', 'Work', 'Mobile', 'Other'\u00a0])\r\n\u00a0 \u00a0 \u00a0 phone_number:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The phone number\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.phone.phoneNumber().replace(\/[^0-9]+\/g, '')\r\n\u00a0 \u00a0 \u00a0 extension:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The phone extension\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0chance.bool({ likelihood: 30\u00a0}) ? chance.integer({ min: 1000, max: 9999\u00a0}) : null\r\n\u00a0 Address:\r\n\u00a0 \u00a0 type:\u00a0object\r\n\u00a0 \u00a0 properties:\r\n\u00a0 \u00a0 \u00a0 address_1:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The address 1\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0`${faker.address.streetAddress()} ${faker.address.streetSuffix()}`\r\n\u00a0 \u00a0 \u00a0 address_2:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The address 2\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0chance.bool({ likelihood: 35\u00a0}) ? faker.address.secondaryAddress() : null\r\n\u00a0 \u00a0 \u00a0 locality:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The city \/ locality\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.address.city()\r\n\u00a0 \u00a0 \u00a0 region:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The region \/ state \/ province\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.address.stateAbbr()\r\n\u00a0 \u00a0 \u00a0 postal_code:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The zip code \/ postal code\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.address.zipCode()\r\n\u00a0 \u00a0 \u00a0 country:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The country code\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.address.countryCode()<\/pre>\n
 <\/p>\n
Currently, our Address<\/strong>\u00a0definition is generating a random country. What if our ecommerce site only supports a small subset of the 195 countries? Let’s say we support six countries to start with: US, CA, MX, UK, ES, DE. We could update the definitions country\u00a0property to grab a random array element:<\/p>\n
(For brevity the other properties have been left off of the model definition)<\/em><\/p>\n
...\r\n\u00a0 \u00a0 \u00a0 country:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The country code\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.random.arrayElement(['US', 'CA', 'MX', 'UK', 'ES', 'DE']);<\/pre>\n
While this would work, what if we have other models that rely on this same country info, we would have to duplicate this logic. We can achieve this same thing by creating a countries.json\u00a0file, and adding an inputs\u00a0property to the data\u00a0property that can be an absolute or relative path to our input. When are model is generated, our countries.json\u00a0file will be exposed to each of the models build functions via the inputs\u00a0argument as inputs.countries<\/p>\n
(For brevity the other properties have been left off of the model definition)<\/em><\/p>\n
name:\u00a0Users\r\ntype:\u00a0object\r\nkey:\u00a0_id\r\ndata:\r\n\u00a0 min:\u00a01000\r\n\u00a0 max:\u00a02000\r\n\u00a0 inputs:\u00a0.\/countries.json\r\nproperties:\r\n...\r\ndefinitions:\r\n...\r\n\u00a0 \u00a0 \u00a0 country:\r\n\u00a0 \u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 \u00a0 description:\u00a0The country code\r\n\u00a0 \u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.random.arrayElement(inputs.countries);\r\n\r\ncountries.json\r\n\r\n[\r\n\u00a0\"US\",\r\n\u00a0\"CA\",\r\n\u00a0\"MX\",\r\n\u00a0\"UK\",\r\n\u00a0\"ES\",\r\n\u00a0\"DE\"\r\n]<\/pre>\n
By changing one existing line and adding another line in model we have provided existing data to our Users model. We can still generate a random country, based on the countries our application supports. Lets test our changes by using the following command:<\/p>\n
fakeit console --count 1 models\/users.yaml\r\n\r\n<\/pre>\n
\"blog4<\/p>\n
Products Model<\/strong><\/h3>\n
Our ecommerce application is using a separate system for categorization, we need to expose that data to our randomly generated products so that we are using valid category information. We will start with the products.yaml\u00a0that we defined in the FakeIt Series 2 of 5: Shared Data and Dependencies<\/a><\/u>\u00a0post.<\/p>\n
products.yaml\r\n\r\nname:\u00a0Products\r\ntype:\u00a0object\r\nkey:\u00a0_id\r\ndata:\r\n\u00a0 min:\u00a04000\r\n\u00a0 max:\u00a05000\r\nproperties:\r\n\u00a0 _id:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The document id\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 post_build:\u00a0`product_${this.product_id}`\r\n\u00a0 doc_type:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The document type\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 value:\u00a0product\r\n\u00a0 product_id:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0Unique identifier representing a specific product\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.random.uuid()\r\n\u00a0 price:\r\n\u00a0 \u00a0 type:\u00a0double\r\n\u00a0 \u00a0 description:\u00a0The product price\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0chance.floating({ min: 0, max: 150, fixed: 2\u00a0})\r\n\u00a0 sale_price:\r\n\u00a0 \u00a0 type:\u00a0double\r\n\u00a0 \u00a0 description:\u00a0The product price\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 post_build:\u00a0|\r\n\u00a0 \u00a0 \u00a0 \u00a0let sale_price = 0;\r\n\u00a0 \u00a0 \u00a0 \u00a0if (chance.bool({ likelihood: 30 })) {\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0sale_price = chance.floating({ min: 0, max: this.price * chance.floating({ min: 0, max: 0.99, fixed: 2 }), fixed: 2 });\r\n\u00a0 \u00a0 \u00a0 \u00a0}\r\n\u00a0 \u00a0 \u00a0 \u00a0return sale_price;\r\n\u00a0 display_name:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0Display name of product.\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.commerce.productName()\r\n\u00a0 short_description:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0Description of product.\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.lorem.paragraphs(1)\r\n\u00a0 long_description:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0Description of product.\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.lorem.paragraphs(5)\r\n\u00a0 keywords:\r\n\u00a0 \u00a0 type:\u00a0array\r\n\u00a0 \u00a0 description:\u00a0An array of keywords\r\n\u00a0 \u00a0 items:\r\n\u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 min:\u00a00\r\n\u00a0 \u00a0 \u00a0 \u00a0 max:\u00a010\r\n\u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.random.word()\r\n\u00a0 availability:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The availability status of the product\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0|\r\n\u00a0 \u00a0 \u00a0 \u00a0let availability = 'In-Stock';\r\n\u00a0 \u00a0 \u00a0 \u00a0if (chance.bool({ likelihood: 40 })) {\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0availability = faker.random.arrayElement([ 'Preorder', 'Out of Stock', 'Discontinued' ]);\r\n\u00a0 \u00a0 \u00a0 \u00a0}\r\n\u00a0 \u00a0 \u00a0 \u00a0return availability;\r\n\u00a0 availability_date:\r\n\u00a0 \u00a0 type:\u00a0integer\r\n\u00a0 \u00a0 description:\u00a0An epoch time of when the product is available\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.date.recent()\r\n\u00a0 \u00a0 \u00a0 post_build:\u00a0new Date(this.availability_date).getTime()\r\n\u00a0 product_slug:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The URL friendly version of the product name\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 post_build:\u00a0faker.helpers.slugify(this.display_name).toLowerCase()\r\n\u00a0 category:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0Category for the Product\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.commerce.department()\r\n\u00a0 category_slug:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The URL friendly version of the category name\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 post_build:\u00a0faker.helpers.slugify(this.category).toLowerCase()\r\n\u00a0 image:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0Image URL representing the product.\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0faker.image.image()\r\n\u00a0 alternate_images:\r\n\u00a0 \u00a0 type:\u00a0array\r\n\u00a0 \u00a0 description:\u00a0An array of alternate images for the product\r\n\u00a0 \u00a0 items:\r\n\u00a0 \u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 \u00a0 min:\u00a00\r\n\u00a0 \u00a0 \u00a0 \u00a0 max:\u00a04\r\n\u00a0 \u00a0 \u00a0 \u00a0 build:\u00a0faker.image.image()<\/pre>\n
Our existing categories data has been provided in CSV format.<\/p>\n
categories.csv\r\n\r\n\"category_id\",\"category_name\",\"category_slug\"\r\n23,\"Electronics\",\"electronics\"\r\n1032,\"Office Supplies\",\"office-supplies\"\r\n983,\"Clothing & Apparel\",\"clothing-and-apparel\"\r\n483,\"Movies, Music & Books\",\"movies-music-and-books\"\r\n3023,\"Sports & Fitness\",\"sports-and-fitness\"\r\n4935,\"Automotive\",\"automotive\"\r\n923,\"Tools\",\"tools\"\r\n5782,\"Home Furniture\",\"home-furniture\"\r\n9783,\"Health & Beauty\",\"health-and-beauty\"\r\n2537,\"Toys\",\"toys\"\r\n10,\"Video Games\",\"video-games\"\r\n736,\"Pet Supplies\",\"pet-supplies\"<\/pre>\n
Now we need to update our products.yaml\u00a0model to use this existing data.<\/p>\n
(For brevity the other properties have been left off of the model definition)<\/em><\/p>\n
name:\u00a0Products\r\ntype:\u00a0object\r\nkey:\u00a0_id\r\ndata:\r\n\u00a0 min:\u00a04000\r\n\u00a0 max:\u00a05000\r\n\u00a0 inputs:\r\n\u00a0 \u00a0 -\u00a0.\/categories.csv\r\n\u00a0 pre_build:\u00a0globals.current_category = faker.random.arrayElement(inputs.categories);\r\nproperties:\r\n...\r\n\u00a0 category_id:\r\n\u00a0 \u00a0 type:\u00a0integer\r\n\u00a0 \u00a0 description:\u00a0The Category ID for the Product\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0globals.current_category.category_id\r\n\u00a0 category:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0Category for the Product\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 build:\u00a0globals.current_category.category_name\r\n\u00a0 category_slug:\r\n\u00a0 \u00a0 type:\u00a0string\r\n\u00a0 \u00a0 description:\u00a0The URL friendly version of the category name\r\n\u00a0 \u00a0 data:\r\n\u00a0 \u00a0 \u00a0 post_build:\u00a0globals.current_category.category_slug\r\n...<\/pre>\n
There are a few things to notice about how we’ve updated our products.yaml\u00a0model.<\/p>\n