FakeIt Serie 1 de 5: Generación de datos falsos

Aaron Benton es un arquitecto experimentado especializado en soluciones creativas para desarrollar aplicaciones móviles innovadoras. Tiene más de 10 años de experiencia en desarrollo full stack, incluyendo ColdFusion, SQL, NoSQL, JavaScript, HTML y CSS. Aaron es actualmente Arquitecto de Aplicaciones para Shop.com en Greensboro, Carolina del Norte y es un Campeón de la comunidad Couchbase.

FakeIt Serie 1 de 5: Generación de datos falsos

There are countless blog posts on modelado de datos, key and document patterns. All of these posts give a great introduction into how to structure and model your documents in Couchbase, but none of them tell you what to do next. In this blog series we are going to answer the question, what do you after you’ve defined your data model?

Modelo de usuario

For this series we will be working with a greenfield e-commerce application. As with most e-commerce applications, our application is going to have users so this is where we will begin.

We have defined a basic user model to start with.

We’ve done the hardest part, which is defining our model, but now what?

• How do we represent this model?
• How do we document this model?
• Does this model rely on data from other models?
• How can data be generated from this model?
• How can we generate fake / test data?

Luckily for us there is a NodeJS project called FakeIt that can answer all of these questions for us. FakeIt is a command-line utility that generates fake data in json, yaml, yml, cson, or csv formats based on models which are defined in yaml. Data can be generated using any combination of FakerJS, ChanceJS, or Custom Functions. The generated data can be output in the following formats and destinations:

• json
• yaml
• cson
• csv
• Zip Archive of json, yaml, cson or csv files
• Servidor Couchbase
• Couchbase Sync Gateway Server

We can define a FakeIt model in YAML to represent our JSON model. This provides us a documented and data-typed model that we can communicate how our model should be structure and what the properties are for.

You’re probably saying to yourself, "great, I’ve defined my model in YAML but what good does this do me?" One of the biggest issues developers face when beginning development is having data to work with. Often times an exorbitant amount of time is spent manually creating documents, writing throw away code to populate a bucket. Additionally you may have a full or partial data dump of your database that has to be imported.

These are time consuming, tedious and in the case of a data dump do not provide any insight or documentation into the available models. We can add a few simple properties to our FakeIt model describing how our model should be generated, and through a single file we can create an endless amount of fake randomized documents.

We have added a data property to each of our models properties describing how that value should be generated. FakeIt supports 5 different ways to generate a value:

• pre_build: function to initialize the value
• construir: function that builds a value
• fake: A FakerJS template string i.e. {{internet.userName}}
• value: A static value to use
• post_build: a function that runs after every property in the model has been set

These build functions are a JavaScript function body. Each of these functions is passed the following variables that can be used at the time of its execution:

• documents – An object containing a key for each model whose value is an array of each document that has been generated
• globals – An object containing any global variables that may have been set by any of the run or build functions
• inputs – An object containing a key for each input file used whose value is the deserialized version of the files data
• faker – A reference to FakerJS
• chance – A reference to ChanceJS
• document_index – This is a number that represents the currently generated document’s position in the run order
• require – This is the node require function, it allows you to require your own packages. For better performance require and seth them in the pre_run function.

For example, if we look at the username properties build function it would look like this:

Now that we have defined how our model should be generated, we can start to generate some fake data with it.

With our users model saved to a file models/users.yaml, we can output data directly to the console using the command

Using this same model we can generate 100 JSON files and save them into a directory named output/ using the command

Additionally, we can create a zip archive of 1,000 JSON files using the command:

fakeit directory –count 1000 –verbose output/users.zip models/users.yaml

1	fakeit directorio -cuente 1000 -verbose salida/usuarios.zip modelos/usuarios.yaml

We can even generate a single CSV file of our model using the following command:

Whether you are using JSON files, Zip Archives or CSV files all of these can be imported into Couchbase Server by using the CLI tools cbdocloader (for *.json and *.zip files) or cbimport (for *.json and *.csv files)

While generating static files is beneficial, there is still the extra step of having to import them into Couchbase Server through the available CLI tools. FakeIt also supports Couchbase Server and Sync Gateway as output destinations. We can generate 10,000 JSON documents from our users.yaml model, and output them to a bucket named ecommerce on a Couchbase Server running locally using the command:

Conclusión

We’ve seen how we can represent a user’s JSON model using YAML to document and describe how a properties value should be generated. That single users.yaml file can be output to the console, JSON files, Zip archive of JSON files, CSV files, and even directly into Couchbase. FakeIt is a fantastic tool to speed up your development and generate larger development datasets. You can save your FakeIt models as part of your codebase for easy repeatable datasets by any developer.

FakeIt is a tool to ease development and testing of your Couchbase deployment. While it can generate large amounts of data, it is not a true load testing tool. There are CLI tools available for load testing and sizing such as cbc-pillowfight y cbworkloadgen

A continuación

• FakeIt Serie 2 de 5: Datos compartidos y dependencias
• FakeIt Series 3 de 5: Modelos Lean a través de definiciones
• FakeIt Series 4 de 5: Trabajar con datos existentes
• FakeIt Series 5 de 5: Desarrollo móvil rápido con Sync-Gateway

Este post forma parte del Programa de Escritura de la Comunidad Couchbase