Greenhouse Shifts to Continuous Delivery Thanks To Rollbar

Challenge

Greenhouse Software, based in New York City, is the fastest-growing enterprise talent acquisition company. Thousands of the smartest and most successful companies like Cisco Meraki, Time Inc., and Airbnb use Greenhouse’s intelligent guidance to design and automate all aspects of hiring throughout their organizations, helping them compete for and win top talent. The company was founded in 2012 to help organizations of all sizes implement hiring best practices and scale recruiting more easily. Greenhouse has since expanded to offer an enterprise talent acquisition suite of solutions that does everything from applicant tracking to interview scheduling. Recently Greenhouse launched a new analytics product to surface key data on finding, assessing and keeping top talent.

To support thousands of customers and a system that receives 10,000 requests per minute, reliability at Greenhouse is paramount. “A lot of people depend on this software for their day-to-day work,” says Josh Bazemore, QA and Operations Lead at Greenhouse. “If you walk into an interview and you can’t ask the questions to your candidate, it would put you in a pretty tight spot.”

Josh oversees a team of QA engineers who are responsible for owning the release process and making sure the site is constantly online and working smoothly. With each new release – which happens a couple times a day as part of their continuous delivery workflow – they make sure there are no show-stopper bugs that prevent users from doing their job.

“When you move fast and release constantly, you need to be sure you’re not introducing any issues to deal with later,” Josh says. “It becomes very expensive to fix a bug after the release and slow down the entire delivery of the release process.”

Solution

To catch these bugs and continuously monitor the releases for any issues, Greenhouse relies heavily on Rollbar, and in particular, the Compliant SaaS product to protect sensitive customer data.

Greenhouse began using Rollbar back in 2012, when its engineering team was just two people and they were releasing every other week. Mike O’Neil was one of those first engineers at Greenhouse. Even then, he says, they realized how critical an automated tool like Rollbar was for catching the issues they could not.

“Software engineers make mistakes and code isn’t perfect. Plus, we integrate with third-party systems so anything can go wrong in the stack,” says Mike. “Having Rollbar there gave us this immediate ability to troubleshoot things.”

In the early days, Rollbar would catch an issue and email the team with a report. Now it’s a much more continuous process of Rollbar running alongside every release and alerting the teams via their release channel in Slack. Today more than 60 people within the Development, QA and Support teams use Rollbar daily to track and triage errors. Rollbar is integrated in a variety of their development environments and tracks the different Greenhouse products with every release so issues are surfaced early and often.

“If Rollbar is not open all the time, we are looking at it for a good 50% of the day,” Josh says. “I can’t imagine we’d be able to handle the day-to-day management of bugs in a release without a tool like Rollbar.”

Results

Making the shift from releasing bi-weekly to deploying multiple times a day has allowed Greenhouse to scale quickly and exceed customer expectations. “Every time a customer sees a bug, they lose a little bit of trust in us. With Rollbar, we can notice these and fix them quickly. It’s a lot easier for us to to build on that trust,” Josh says.

For the support team responsible for handling many of these issues with customers, Rollbar has been a true time-saver, according to Amanda Segal, a support engineer at Greenhouse. Rather than dig through complicated log files to reproduce a bug, the support engineers and customer success team can open up Rollbar and find it instantly. It’s also made it much easier to prioritize fixes, with Rollbar capturing whether an issue affects just one customer or many. For the end user, this translates into much quicker bug fixes.

“If you have a service like Rollbar in place, it empowers your support team to be able to sort through issues and turn things around quickly,” says Amanda. “Your customers are going to get the benefit of quick support, which is good for everybody.”

Rollbar gives Greenhouse a level of confidence in their releases that they don’t believe would be possible without it. By getting immediate – and automatic – feedback as to whether an issue will impact the end-user, the team can focus on building product and moving fast.

“It would be a challenge to release code as often as we do, with as many new features and changes as we’re pushing, if we didn’t have Rollbar,” Mike says. “I could do my job without it, but it would be a lot harder.”