Over the past couple of months it has been very difficult not to overhear conversations about GDPR and its implications on how we build and maintain software. Some were particularly memorable.
"No, I can't fire you right now, not when we're a month away from GDPR!"
- Random person overheard yelling on the phone in San Francisco
At Rollbar, we've been working hard to be ready. This is a story about how we prepared for GDPR.
What is GDPR?
GDPR{:target="_blank"} is a new, EU-wide regulation that replaces the 1995 EU Data Protection directive. It strengthens the privacy and control EU residents have over their personal data, and is broad in reach because any organizations that process data on EU residents must be compliant, even if they aren't based in the EU. Failure to comply may result in severe penalties of up to 4% of an offending organization's global revenue.
GDPR challenges
To us, GDPR changes everything. We've gone from a world where data is always an asset, to one where data is both an asset and a liability. This fundamentally changes how we think about collecting, keeping, and deleting data, and where the value for our customers is in each of those things.
We were faced with a few challenges in this project, including:
Hard deadlines
We practice agile software development. What that means to us is while we have a rough idea of when we'd like to release a feature, the exact release date depends on the progress we make, and the progress is primarily measured on a week-to-week basis.
GDPR is coming into effect on May 25th, 2018. Since both we and our customers need to be ready before then, we gave ourselves hard deadlines on this project. While this wasn't our first time doing so, it certainly was not our standard approach for most projects.
Scope and complexity
We typically build features to satisfy requirements that are a mix of product vision and customer requests. With GDPR however, much of the requirements are driven by what the law states. Of course conveniently the law isn't very specific about a lot of things. This means we had to first consult our legal counsel to determine how the law applies to our specific situation.
For example, one question we had was whether Rollbar is considered both a data controller and data processor. We know we are clearly a data processor, because we handle error data on behalf of our customers. But to a SaaS vendor like Segment - whose product we use - are we a data controller in their eyes, and if so, what does that mean?
To make matters more complex, even before we started this project we've had customers requesting features similar to - but not exactly the same as - what is needed for GDPR compliance. There were also operational requirements to contend with, such as making sure the performance of our databases is up to snuff when doing a lot of writes that would now include deletes.
Sorting out these overlapping and sometimes conflicting requirements was a big challenge on a scale new to us.
Important little details
If you're not familiar with Rollbar, a big part of our product value is the contextual metadata about the errors we collect. For every error occurrence, we include data such as number of affected end-users, IP addresses, browsers, OSes, and more. Our customers like Twilio{:target="_blank"} and Instacart{:target="_blank"} use them to figure out the impact of an error and triage accordingly.
Say you are a Rollbar customer who receives a request from an EU resident to erase all his or her personal data. We want to make sure that once you delete his or her data from all systems including Rollbar's, the quality of insights you get from Rollbar will not be degraded.
To make sure of that, we had to make decisions on many, many important little details we didn't necessarily have to think about before.
One such example is the number of unique person and IP address counter. We had to decide:
Should we decrement the count of persons and IPs if the data of one of those persons has been deleted? When should we not?
For instance, a request to delete data of one person could mean decrementing the count of one person and one IP address, or one person and 2 IP addresses, because that person may be encountering this error at home and at work. And what if the same IP address was used by 3 or more persons - do we decrement the count then?
GDPR changes everything not just because it turns data into both an asset and a liability, but also because it breaks the assumption that a historical ledger doesn't change. That presents a challenge especially for a product like ours.
"I can tell you, this is not as simple as deleting a row in a table"
Strategies we used and lessons learned
On the bright side, we took this project as an opportunity to address the needs of multiple stakeholders in one shot. Instead of just doing the bare minimum to check the boxes for GDPR compliance, we ended up building useful features and paying down tech debt at the same time.
We also learned a few valuable lessons and used several strategies that we can apply to future projects - here are some of them:
Solve problems in stages
When we built the feature to delete personal data of a specific person to support the GDPR right to erasure, we originally started with a way to do that synchronously. Only later we evolved it to become a job that is queued and processed asynchronously. By tackling a hard problem in stages, we were able to consistently make meaningful progress.
Look for the win-win
One of the things we did was to change the default option of the data scrubbing filters in our SDKs to exclude personal data. Customers now have to opt in to collect person names and email addresses. The SDKs now also come with an option to anonymize IP addresses. With these changes we've made it easier for customers to reduce their GDPR exposure upfront. This is a win-win solution because it also means we could potentially avoid some operational costs of doing deletes.
Trust your team
Our engineering team has grown quite a bit recently, and GDPR was by far the largest project with the most stakeholders and the most complexity, that we've undertaken since we've been this size. Fortunately, there was a high level of trust throughout the team: not just between management and engineering, but also among the engineers themselves. In the end, everyone came together, figured out what needed to be delivered, and executed confidently.
{: .highlightbox}
If you want to learn more about the features and policies we have in place to support GDPR compliance{:target="_blank"}, including Data Processing Agreement, person deletion API, customizable data retention, encryption at rest, and more, visit Rollbar.com/GDPR{:target="_blank"}.
Hope you enjoyed our GDPR story. If you aren't using a tool like Rollbar yet, you're practically outsourcing bug discovery to users. There's a better way to detect production issues. We built Rollbar to make your life easier. Give it a try - it's free!
