{"id":16042,"date":"2024-07-24T13:54:13","date_gmt":"2024-07-24T20:54:13","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=16042"},"modified":"2024-07-30T11:05:08","modified_gmt":"2024-07-30T18:05:08","slug":"privacy-requirements-without-the-headache","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/","title":{"rendered":"Privacy Requirements Without the Headache"},"content":{"rendered":"<p>Any modern web application that interacts with users in different localities must contend with varying privacy requirements and user data consent guidelines. These can be overwhelming for any engineering team to handle, regardless whether the team is just you or fifty people.<\/p>\n<p>How do you simplify the process to remove the hassle and decrease the likelihood of non-compliance?<\/p>\n<p>In this guide, we\u2019ll walk through an implementation leveraging AWS services working in tandem with Couchbase to sanitize data before it ever reaches a data store. This simplified implementation can be utilized in any context and can be expanded to fit the use case you are building.<\/p>\n<p><i>tl;dr Interested in just seeing the code along with a README containing deployment instructions? Find all you need on <\/i><a href=\"https:\/\/github.com\/hummusonrails\/anonymize-data-aws-couchbase-example\"><i>GitHub<\/i><\/a><i>.<\/i><\/p>\n<h2>Overview<\/h2>\n<h3>Example Scenario<\/h3>\n<p>Imagine a small eCommerce platform that sells unique, handmade crafts. This site operates on a one-off purchase model, meaning users are not required to create accounts or log in. Shoppers simply browse the catalog, select items they wish to purchase, and proceed to checkout. During the checkout process, they provide essential information such as full name, mailing address, billing address, payment information, phone number, and email address. This Personally Identifiable Information (PII) is necessary to complete the purchase and ensure the item is shipped to the correct address.<\/p>\n<p>Once the order is placed, the data travels through various microservices. The order processing service validates the details and checks inventory, the payment service securely handles the billing information to process transactions, and the shipping service uses the provided mailing address to arrange delivery.<\/p>\n<p>The site is designed to give users a seamless and easy shopping experience. There are no accounts. No need to go through a lengthy registration process. Simply choose the item to buy, make the payment along with shipping information and it is done. However, while the customer has had a frictionless shopping experience, their personal data cannot have the same frictionless experience in your application. To maintain privacy and comply with regulations, this PII must be sanitized before being stored in the long-term purchase history database.<\/p>\n<p>While this scenario may seem unique, it is actually a situation that may occur in our applications all the time. Every application that collects any aspect of a user\u2019s information must ensure rigorous consent and be in compliance with the laws and regulations of every locality it operates in across the globe. This can be an enormous challenge, and one that carries with it severe ramifications if not done properly.<\/p>\n<h3>Solution Workflow<\/h3>\n<p>In order to effectively handle privacy requirements for live data, we will need to leverage the real-time data capabilities of AWS and Couchbase to transform the data as soon as it is created. It is legally challenging to store private data even for a moment in a database according to GDPR regulations. As such, not only is it good data practice to add the data only after it has been sanitized, it may even be a legal requirement.<\/p>\n<p>The solution we are crafting will leverage AWS Simple Queue Service (SQS), Elastic Container Registry (ECR), and a Lambda function working in partnership with Couchbase Capella, the fully managed Database as a Service (DBaaS). This workflow can be implemented with any message sender to the queue service, which will initiate the data sanitizing process. In other words, it is plug and play capable.<\/p>\n<p>The following workflow visually demonstrates how the solution operates.<\/p>\n<p><a href=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2024\/07\/image1-3.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-16043\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2024\/07\/image1-3-1024x383.png\" alt=\"\" width=\"900\" height=\"337\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image1-3-1024x383.png 1024w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image1-3-300x112.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image1-3-768x287.png 768w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image1-3-1536x574.png 1536w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image1-3-1320x493.png 1320w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image1-3.png 1999w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\" \/><\/a><\/p>\n<p>First, any message sender sends data to the queue service. This triggers the Lambda function with the message. The Lambda function processes and transforms the data removing all PII from the message. Finally, the sanitized data is then sent to Couchbase to be safely added to the data store.<\/p>\n<p>Now that we understand the workflow, let\u2019s build it!<\/p>\n<h2>Implementation<\/h2>\n<h3>Configuring Couchbase Capella<\/h3>\n<p>It is free to sign up and try Couchbase Capella, and if you have not done so yet, you can do so by navigating to <a href=\"https:\/\/cloud.couchbase.com\/\">cloud.couchbase.com<\/a> and creating an account using your GitHub or Google credentials, or by making a new account with an email address and password combination.<\/p>\n<p><a href=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2024\/07\/image4-1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-16044 alignleft\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2024\/07\/image4-1-406x1024.png\" alt=\"\" width=\"147\" height=\"371\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image4-1-406x1024.png 406w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image4-1-119x300.png 119w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image4-1-300x756.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image4-1.png 562w\" sizes=\"auto, (max-width: 147px) 100vw, 147px\" \/><\/a>Once you have done so, from within your Capella dashboard, you will create your first database. For the purposes of this walkthrough, let\u2019s name it <i>anonymize_data_example<\/i>.<\/p>\n<p>The summary of your new database will be presented on the left-hand side of the dashboard. Capella is multi-cloud and can work with AWS, Google Cloud or Azure. For this example, you will deploy to AWS.<\/p>\n<p>After you have created your database, you need to create a <i>bucket<\/i>. A <i>bucket<\/i> in Couchbase is the container where the data is stored. Each item of data, known as a <i>document<\/i>, is kept in JSON making its syntax familiar to most developers. You can name your bucket whatever you want. However, for the purposes of this walkthrough, let\u2019s name this bucket <i>anonymized_data_example<\/i>.<\/p>\n<p>Now that you have created both your database and your bucket, you are ready to create your database access credentials and to fetch your connection URL that you will be using in your Lambda function.<\/p>\n<p>Navigate to the <i>Connect<\/i> section in the Capella dashboard and take note of the <i>Connection String<\/i>.<\/p>\n<p><a href=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2024\/07\/image2-2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-16045\" src=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2024\/07\/image2-2-1024x583.png\" alt=\"\" width=\"900\" height=\"512\" srcset=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image2-2-1024x583.png 1024w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image2-2-300x171.png 300w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image2-2-768x438.png 768w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image2-2-1536x875.png 1536w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image2-2-1320x752.png 1320w, https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image2-2.png 1999w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\" \/><\/a><\/p>\n<p>Then, click on the <i>Database Access<\/i> link under section two. In that section, you will create credentials \u2013 a username and password \u2013 that your Lambda function will use to authenticate with the database. You can scope the credentials to the specific bucket you created or give it permission for all buckets and databases in your account. You need to make sure it has both read and write access, regardless.<\/p>\n<p>At this point, you are now ready to create the Lambda function that will be invoked each time a message is received by the queue service.<\/p>\n<h3>Creating the Lambda Function<\/h3>\n<p>Every application will have different PII that it may collect, and depending on the localities the application is operating in, it may also have different requirements of what it must remove before saving the data. In this example, we will remove the IP address and the surname of every message before sending it on to Couchbase.<\/p>\n<p>The function to sanitize the data is itself fairly straightforward:<\/p>\n<pre class=\"nums:false lang:js decode:true \">function anonymizeData(data) {\r\n\u00a0\u00a0\u00a0\u00a0data.user.last_name = '';\r\n\u00a0\u00a0\u00a0\u00a0delete data.user.ip_address;\r\n\u00a0\u00a0\u00a0\u00a0return data;\r\n}<\/pre>\n<p>This function, <code>anonymizeData<\/code>, will be invoked in the <code>handler<\/code> function of the code. If you are not familiar with a <code>handler<\/code> function, it is used by AWS Lambda to process incoming messages from the SQS queue, transform the data, and store it in Couchbase. Your main executable code must begin with a <code>exports.handler<\/code>.<\/p>\n<p>The <code>handler<\/code> function fetches the Couchbase credentials and connection string from the environment variables set inside the configuration of the Lambda function. Detailed instructions on setting environment variables for a Lambda function can be found in the <a href=\"https:\/\/docs.aws.amazon.com\/lambda\/latest\/dg\/configuration-envvars.html\">AWS documentation<\/a>. Then, the function parses the message it received from SQS and calls the <code>anonymizeData<\/code> function on it. Lastly, it <i>upserts<\/i> the data into the Couchbase bucket.<\/p>\n<pre class=\"nums:false lang:js decode:true\">exports.handler = async (event) =&gt; {\r\n\u00a0\u00a0\u00a0\u00a0console.log('Starting Lambda function');\u00a0\r\n\r\n\u00a0\u00a0\u00a0\u00a0if (!event.Records || !Array.isArray(event.Records)) {\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0throw new TypeError('event.Records is not iterable');\r\n\u00a0\u00a0\u00a0\u00a0}\r\n\r\n\u00a0\u00a0\u00a0\u00a0const connectionString = process.env.COUCHBASE_CONNECTION_STRING;\r\n\u00a0\u00a0\u00a0\u00a0const username = process.env.COUCHBASE_USERNAME;\r\n\u00a0\u00a0\u00a0\u00a0const password = process.env.COUCHBASE_PASSWORD;\r\n\u00a0\u00a0\u00a0\u00a0const bucketName = process.env.COUCHBASE_BUCKET_NAME;\r\n\r\n\u00a0\u00a0\u00a0\u00a0try {\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0const cluster = await couchbase.connect(connectionString, {\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0username: username,\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0password: password\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0});\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0console.log('Connected to Couchbase cluster');\r\n\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0const bucket = cluster.bucket(bucketName);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0const collection = bucket.defaultCollection();\r\n\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0for (const record of event.Records) {\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0console.log('Processing record:', record);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0const payload = JSON.parse(record.body);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0const transformedData = anonymizeData(payload);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0console.log('Transformed data:', transformedData);\r\n\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0await collection.upsert(transformedData.record_id, transformedData);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0console.log('Data upserted:', transformedData.record_id);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0\u00a0\u00a0} catch (error) {\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0console.error('Error during processing:', error);\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0throw error;\r\n\u00a0\u00a0\u00a0\u00a0}\r\n};<\/pre>\n<p>To set up the Lambda function using the AWS CLI, you need to create a new Lambda function and specify the necessary IAM role and handler. Here\u2019s how you can do it:<\/p>\n<pre class=\"nums:false wrap:true lang:sh decode:true\">aws lambda create-function --function-name AnonymizeDataExampleFunction \\\r\n  --package-type Image \\\r\n  --code ImageUri=&lt;your-account-id&gt;.dkr.ecr.&lt;your-region&gt;.amazonaws.com\/anonymize_data_example_image:latest \\\r\n  --role arn:aws:iam::&lt;your-account-id&gt;:role\/&lt;your-lambda-execution-role&gt;\r\n\r\n<\/pre>\n<p>Make sure to replace <em><code>&lt;your-account-id&gt;<\/code><\/em>, <code>&lt;your-region&gt;<\/code>, and <code>&lt;your-lambda-execution-role&gt;<\/code> with your actual AWS account ID, region, and the ARN of the IAM role you created for the Lambda function. This command sets up the Lambda function to use the Docker image you will publish to AWS ECR.<\/p>\n<p>The Lambda function is now ready to be packaged in a Docker image and published to the AWS ECR service.<\/p>\n<h3>Deploying the Docker Image to ECR<\/h3>\n<p>To deploy the Docker image to AWS ECR, we start by building the Docker image for the linux\/amd64 platform. This ensures compatibility with the x86_64 architecture of our Lambda function. We use a <i>Dockerfile<\/i> that installs the necessary tools and dependencies, sets the working directory, copies the required files, installs the dependencies, and copies the Lambda function code. The <i>Dockerfile<\/i> also specifies the command to run the Lambda function.<\/p>\n<p>Here is the <i>Dockerfile<\/i>:<\/p>\n<pre class=\"nums:false lang:default decode:true \">FROM public.ecr.aws\/lambda\/nodejs:18\r\n\r\n# Install necessary tools and dependencies\r\nRUN yum -y install gcc-c++ tar gzip findutils\r\n\r\n# Set the working directory\r\nWORKDIR \/var\/task\r\n\r\n# Copy package.json and package-lock.json\r\nCOPY package*.json .\/\r\n\r\n# Install dependencies\r\nRUN npm install\r\n\r\n# Copy the function code\r\nCOPY index.js .\/\r\n\r\n# Command to run the Lambda function\r\nCMD [ \"index.handler\" ]\r\n<\/pre>\n<p>Once the Dockerfile is ready, we build the Docker image with the specified platform:<\/p>\n<pre class=\"nums:false lang:default decode:true\">docker build --platform linux\/amd64 -t anonymize_data_example_image .\r\n<\/pre>\n<p>After building the image, we tag it appropriately for the AWS ECR repository:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true\">docker tag anonymize_data_example_image:latest &lt;your-account-id&gt;.dkr.ecr.&lt;your-region&gt;.amazonaws.com\/anonymize_data_example_image:latest<\/pre>\n<p>To push the Docker image to ECR, we first log in to the ECR registry using the AWS CLI to obtain the login credentials:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true \">aws ecr get-login-password --region &lt;your-region&gt; | docker login --username AWS --password-stdin &lt;your-account-id&gt;.dkr.ecr.&lt;your-region&gt;.amazonaws.com<\/pre>\n<p>With the credentials in place, we push the Docker image to the ECR repository:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true\">docker push &lt;your-account-id&gt;.dkr.ecr.&lt;your-region&gt;.amazonaws.com\/anonymize_data_example_image:latest<\/pre>\n<p>Finally, we update the Lambda function named <code>AnonymizeDataExampleFunction<\/code> to use the new Docker image from ECR. This involves specifying the image URI and ensuring that the Lambda function is configured to run the code from the newly pushed Docker image:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true \">aws lambda update-function-code --function-name AnonymizeDataExampleFunction --image-uri &lt;your-account-id&gt;.dkr.ecr.&lt;your-region&gt;.amazonaws.com\/anonymize_data_example_image:latest<\/pre>\n<p>By following these steps, the Docker image is successfully deployed to AWS ECR and the Lambda function is updated to use the new image, allowing the application to sanitize PII before storing it in Couchbase. This setup ensures that data processing complies with privacy requirements while leveraging the power of AWS and Couchbase.<\/p>\n<h3>Setting up the SQS Service<\/h3>\n<p>To facilitate the flow of data between the message sender and the Lambda function, we need to set up an Amazon SQS queue. The SQS queue will act as a buffer that receives and stores messages until they are processed by the Lambda function. Here\u2019s how you can set up an SQS queue and make it a trigger for the Lambda function using the AWS CLI.<\/p>\n<p>First, create a new SQS queue. This queue will receive messages containing user data from the message sender, which is the application.<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true \">aws sqs create-queue --queue-name AnonymizeDataExampleQueue<\/pre>\n<p>After creating the queue, you will receive a URL for the queue. This URL is necessary for sending messages to the queue and configuring it as a trigger for the Lambda function.<\/p>\n<p>Next, we need to get the ARN of the SQS queue, which is required to set up permissions and triggers. Use the following command to retrieve the ARN:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true \">aws sqs get-queue-attributes --queue-url https:\/\/sqs.&lt;your-region&gt;.amazonaws.com\/&lt;your-account-id&gt;\/AnonymizeDataExampleQueue --attribute-names QueueArn<\/pre>\n<p>Replace <code>&lt;your-region&gt;<\/code> and <code>&lt;your-account-id&gt;<\/code> with your actual AWS region and account ID.<\/p>\n<p>The output will include the <i>QueueArn<\/i>, which looks something like this:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true\">{\r\n\u00a0\u00a0\u00a0\u00a0\"Attributes\": {\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"QueueArn\": \"arn:aws:sqs:&lt;your-region&gt;:&lt;your-account-id&gt;:AnonymizeDataExampleQueue\"\r\n\u00a0\u00a0\u00a0\u00a0}\r\n}<\/pre>\n<p>Now, we need to grant the Lambda function permission to read messages from the SQS queue. Create a policy file named <i>lambda-sqs-policy.json<\/i> with the following content:<\/p>\n<pre class=\"nums:false lang:default decode:true \">{\r\n\u00a0\u00a0\"Version\": \"2012-10-17\",\r\n\u00a0\u00a0\"Statement\": [\r\n\u00a0\u00a0\u00a0\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Effect\": \"Allow\",\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Action\": \"sqs:ReceiveMessage\",\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Resource\": \"arn:aws:sqs:&lt;your-region&gt;:&lt;your-account-id&gt;:AnonymizeDataExampleQueue\"\r\n\u00a0\u00a0\u00a0\u00a0},\r\n\u00a0\u00a0\u00a0\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Effect\": \"Allow\",\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Action\": \"sqs:DeleteMessage\",\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Resource\": \"arn:aws:sqs:&lt;your-region&gt;:&lt;your-account-id&gt;:AnonymizeDataExampleQueue\"\r\n\u00a0\u00a0\u00a0\u00a0},\r\n\u00a0\u00a0\u00a0\u00a0{\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Effect\": \"Allow\",\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Action\": \"sqs:GetQueueAttributes\",\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"Resource\": \"arn:aws:sqs:&lt;your-region&gt;:&lt;your-account-id&gt;:AnonymizeDataExampleQueue\"\r\n\u00a0\u00a0\u00a0\u00a0}\r\n\u00a0\u00a0]\r\n}<\/pre>\n<p>Apply this policy to the Lambda execution role:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true \">aws iam put-role-policy --role-name &lt;your-lambda-execution-role&gt; --policy-name LambdaSQSPolicy --policy-document file:\/\/lambda-sqs-policy.json<\/pre>\n<p>Next, add the SQS queue as a trigger for the Lambda function:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true \">aws lambda create-event-source-mapping --function-name AnonymizeDataExampleFunction --batch-size 10 --event-source-arn arn:aws:sqs:&lt;your-region&gt;:&lt;your-account-id&gt;:AnonymizeDataExampleQueue<\/pre>\n<p>This command sets up the Lambda function to be triggered by messages arriving in the SQS queue.<\/p>\n<p>Now, your Lambda function is set to receive every message sent to the queue service. The workflow is complete! This setup ensures efficient handling of user data while maintaining compliance with privacy requirements.<\/p>\n<h3>Test the Implementation Locally<\/h3>\n<p>To ensure that everything works as expected, you can test the workflow by sending a message through to the queue directly yourself. From the command line execute the following:<\/p>\n<pre class=\"nums:false wrap:true lang:default decode:true \">aws sqs send-message --queue-url https:\/\/sqs.&lt;your-region&gt;.amazonaws.com\/&lt;your-account-id&gt;\/AnonymizeDataExampleQueue --message-body \"{\\\"record_id\\\": \\\"purchase_002\\\", \\\"item\\\": \\\"item_1\\\", \\\"user\\\": {\\\"first_name\\\": \\\"John\\\", \\\"last_name\\\": \\\"Doe\\\", \\\"ip_address\\\": \\\"192.168.1.1\\\"}, \\\"timestamp\\\": \\\"2024-07-01T12:34:56Z\\\"}\"<\/pre>\n<p>The message will trigger the Lambda function, which in turn, will add the data to your Couchbase bucket after sanitizing it. You can view the sample data you sent in the message by either logging into the Capella dashboard or in your <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=Couchbase.vscode-couchbase\">VSCode<\/a> or <a href=\"https:\/\/plugins.jetbrains.com\/plugin\/22131-couchbase\">Jetbrains<\/a> IDE directly by using their respective Couchbase extension.<\/p>\n<h2>Wrapping Up<\/h2>\n<p>The work to build, deploy and maintain any modern application today is an enormous task. As much as it is possible, parts of that task that are not directly related to the core business of the application should be simplified and abstracted away to reduce the cognitive load of the engineers, the SREs, and everyone else involved. Privacy regulations is one major item that must both be strictly adhered to and should also not cause any major headaches or increased workload.<\/p>\n<p>Building an automated plug and play workflow that can easily sit between any application that interacts with Personally Identifiable Information and the database that data is stored in will significantly ease the cognitive load, reduce the burden and streamline the development process. With leveraging AWS and Couchbase together, you can make that possible for your real-time data sanitization needs thereby ensuring both user integrity and engineering productivity.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Any modern web application that interacts with users in different localities must contend with varying privacy requirements and user data consent guidelines. These can be overwhelming for any engineering team to handle, regardless whether the team is just you or [&hellip;]<\/p>\n","protected":false},"author":85356,"featured_media":16046,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1815,2225,1816,1813],"tags":[9445,2135,9997],"ppma_author":[9985],"class_list":["post-16042","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-best-practices-and-tutorials","category-cloud","category-couchbase-server","category-security","tag-data-privacy","tag-gdpr","tag-pii"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.8 (Yoast SEO v25.8) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Privacy Requirements Without the Headache - The Couchbase Blog<\/title>\n<meta name=\"description\" content=\"Every application that collects any aspect of a user\u2019s information must ensure rigorous consent and be in compliance with the laws and regulations of every locality it operates in across the globe.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Privacy Requirements Without the Headache\" \/>\n<meta property=\"og:description\" content=\"Every application that collects any aspect of a user\u2019s information must ensure rigorous consent and be in compliance with the laws and regulations of every locality it operates in across the globe.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/\" \/>\n<meta property=\"og:site_name\" content=\"The Couchbase Blog\" \/>\n<meta property=\"article:published_time\" content=\"2024-07-24T20:54:13+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-07-30T18:05:08+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2024\/07\/image3-2.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"640\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Ben Greenberg, Senior Developer Evangelist\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ben Greenberg, Senior Developer Evangelist\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/\"},\"author\":{\"name\":\"Ben Greenberg, Senior Developer Evangelist\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/48efa1524aec97312d92f65a270c255d\"},\"headline\":\"Privacy Requirements Without the Headache\",\"datePublished\":\"2024-07-24T20:54:13+00:00\",\"dateModified\":\"2024-07-30T18:05:08+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/\"},\"wordCount\":1989,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image3-2.png\",\"keywords\":[\"data privacy\",\"gdpr\",\"pii\"],\"articleSection\":[\"Best Practices and Tutorials\",\"Couchbase Capella\",\"Couchbase Server\",\"Security\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/\",\"name\":\"Privacy Requirements Without the Headache - The Couchbase Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image3-2.png\",\"datePublished\":\"2024-07-24T20:54:13+00:00\",\"dateModified\":\"2024-07-30T18:05:08+00:00\",\"description\":\"Every application that collects any aspect of a user\u2019s information must ensure rigorous consent and be in compliance with the laws and regulations of every locality it operates in across the globe.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#primaryimage\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image3-2.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image3-2.png\",\"width\":1200,\"height\":640},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.couchbase.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Privacy Requirements Without the Headache\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#website\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"name\":\"The Couchbase Blog\",\"description\":\"Couchbase, the NoSQL Database\",\"publisher\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#organization\",\"name\":\"The Couchbase Blog\",\"url\":\"https:\/\/www.couchbase.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png\",\"width\":218,\"height\":34,\"caption\":\"The Couchbase Blog\"},\"image\":{\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/48efa1524aec97312d92f65a270c255d\",\"name\":\"Ben Greenberg, Senior Developer Evangelist\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/c9bda12524045d12a5878a2ef3fbe0de\",\"url\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/06\/T024FJS4M-U075H3NTJUR-b4c321d902e2-512.jpeg\",\"contentUrl\":\"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/06\/T024FJS4M-U075H3NTJUR-b4c321d902e2-512.jpeg\",\"caption\":\"Ben Greenberg, Senior Developer Evangelist\"},\"url\":\"https:\/\/www.couchbase.com\/blog\/author\/bengreenberg\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Privacy Requirements Without the Headache - The Couchbase Blog","description":"Every application that collects any aspect of a user\u2019s information must ensure rigorous consent and be in compliance with the laws and regulations of every locality it operates in across the globe.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/","og_locale":"en_US","og_type":"article","og_title":"Privacy Requirements Without the Headache","og_description":"Every application that collects any aspect of a user\u2019s information must ensure rigorous consent and be in compliance with the laws and regulations of every locality it operates in across the globe.","og_url":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/","og_site_name":"The Couchbase Blog","article_published_time":"2024-07-24T20:54:13+00:00","article_modified_time":"2024-07-30T18:05:08+00:00","og_image":[{"width":1200,"height":640,"url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2024\/07\/image3-2.png","type":"image\/png"}],"author":"Ben Greenberg, Senior Developer Evangelist","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Ben Greenberg, Senior Developer Evangelist","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#article","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/"},"author":{"name":"Ben Greenberg, Senior Developer Evangelist","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/48efa1524aec97312d92f65a270c255d"},"headline":"Privacy Requirements Without the Headache","datePublished":"2024-07-24T20:54:13+00:00","dateModified":"2024-07-30T18:05:08+00:00","mainEntityOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/"},"wordCount":1989,"commentCount":0,"publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image3-2.png","keywords":["data privacy","gdpr","pii"],"articleSection":["Best Practices and Tutorials","Couchbase Capella","Couchbase Server","Security"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/","url":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/","name":"Privacy Requirements Without the Headache - The Couchbase Blog","isPartOf":{"@id":"https:\/\/www.couchbase.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#primaryimage"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#primaryimage"},"thumbnailUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image3-2.png","datePublished":"2024-07-24T20:54:13+00:00","dateModified":"2024-07-30T18:05:08+00:00","description":"Every application that collects any aspect of a user\u2019s information must ensure rigorous consent and be in compliance with the laws and regulations of every locality it operates in across the globe.","breadcrumb":{"@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#primaryimage","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image3-2.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/07\/image3-2.png","width":1200,"height":640},{"@type":"BreadcrumbList","@id":"https:\/\/www.couchbase.com\/blog\/privacy-requirements-without-the-headache\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.couchbase.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Privacy Requirements Without the Headache"}]},{"@type":"WebSite","@id":"https:\/\/www.couchbase.com\/blog\/#website","url":"https:\/\/www.couchbase.com\/blog\/","name":"The Couchbase Blog","description":"Couchbase, the NoSQL Database","publisher":{"@id":"https:\/\/www.couchbase.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.couchbase.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.couchbase.com\/blog\/#organization","name":"The Couchbase Blog","url":"https:\/\/www.couchbase.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/2023\/04\/admin-logo.png","width":218,"height":34,"caption":"The Couchbase Blog"},"image":{"@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/48efa1524aec97312d92f65a270c255d","name":"Ben Greenberg, Senior Developer Evangelist","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.couchbase.com\/blog\/#\/schema\/person\/image\/c9bda12524045d12a5878a2ef3fbe0de","url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/06\/T024FJS4M-U075H3NTJUR-b4c321d902e2-512.jpeg","contentUrl":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/06\/T024FJS4M-U075H3NTJUR-b4c321d902e2-512.jpeg","caption":"Ben Greenberg, Senior Developer Evangelist"},"url":"https:\/\/www.couchbase.com\/blog\/author\/bengreenberg\/"}]}},"authors":[{"term_id":9985,"user_id":85356,"is_guest":0,"slug":"bengreenberg","display_name":"Ben Greenberg, Senior Developer Evangelist","avatar_url":{"url":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/06\/T024FJS4M-U075H3NTJUR-b4c321d902e2-512.jpeg","url2x":"https:\/\/www.couchbase.com\/blog\/wp-content\/uploads\/sites\/1\/2024\/06\/T024FJS4M-U075H3NTJUR-b4c321d902e2-512.jpeg"},"author_category":"","last_name":"Greenberg, Senior Developer Evangelist","first_name":"Ben","job_title":"Senior Developer Evangelist","user_url":"","description":""}],"_links":{"self":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/16042","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/users\/85356"}],"replies":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/comments?post=16042"}],"version-history":[{"count":0,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/posts\/16042\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media\/16046"}],"wp:attachment":[{"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/media?parent=16042"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/categories?post=16042"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/tags?post=16042"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.couchbase.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=16042"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}