At Rollbar we love workflow automation. With our new Notifications API, you can automate setting up of custom notification rules for all your Rollbar projects. As more of our customers switch to microservices, we wanted to build a programmatic way to set up these rules for multiple projects or services in just a few seconds, without having to go to the UI.
With this API, you’ll be able to -
- Set-up multiple Rollbar projects with the notification settings you want in a few minutes
- Get notified about new errors occurring in your production environment through Slack or PagerDuty
- Instantly see new errors happening in your environments right in your Slack channels
In this article we will go through automating the Slack and PagerDuty integration setup and defining some custom rules for notifications.
Prerequisites for Slack notifications
First, make sure you have a Slack channel ready for the notifications. You can either integrate with an already existing channel or create a new one. Let’s assume you create a Slack channel called #alerts.
Second, you need to obtain a write access token for your Rollbar project from the project general settings: Rollbar dashboard → Settings → Project Access Tokens.
Third, obtain your Service Account ID. You can find it here in your Rollbar's user settings page for Slack: Rollbar dashboard → Click your username in the top right corner → Account Settings → Connected Account → Slack → look for Notifications API section at the bottom.
Configuring Slack notifications
Now it’s time to actually link the project with the Slack channel. As you can see in the GIF above you can do it with the PUT /api/1/notifications/slack endpoint.
Assuming you created Slack channel #alerts and your Service Account ID is 12345, you can do it with the following API calls:
curl \
--location \
--request PUT 'https://api.rollbar.com/api/1/notifications/slack' \
--header 'X-Rollbar-Access-Token: <WRITE_PROJECT_ACCESS_TOKEN>' \
--header 'Content-Type: application/json' \
--data-raw '{
"enabled": true,
"service_account_id": <SERVICE ACCOUNT ID>,
"channel": "#alerts"
}'Setting up custom rules for your Slack notifications
Now the integration configuration is prepared to setup its trigger rules. We have prepared a vast range of different triggers and filters which really allow you to set up your Slack notifications in whatever way works for your team. You can find the complete reference in our Notifications API documentation, but for now here are a couple of examples.
When new errors start happening in production
This is probably the most straightforward scenario. You just want to be notified when a new type of error starts happening on our production environment. This is important as it is most likely your live users being affected by an issue. Let’s look at the following API call:
curl \
--location \
--request PUT 'https://api.rollbar.com/api/1/notifications/slack/rules' \
--header 'X-ROLLBAR-ACCESS-TOKEN: <WRITE_PROJECT_ACCESS_TOKEN>' \
--header 'Content-Type: application/json' \
--data-raw '[
{
"trigger": "new_item",
"filters": [
{ "type": "level", "operation": "gte", "value": "error" },
{ "type": "environment", "operation": "eq", "value": "production" }
]
}
]'new_item is fired when an error or message is seen for the first time. We then apply a level filter to only get notified of items with level greater or equal to error. At last, we apply the environment filter to only get this notification for new items in environment production.
Here is our brand new notification:
Get notified about occurrences skyrocketing in production
Now, let’s assume you’d like to notify your team about new trends of errors happening on your production environment. For this you can use the occurrence_rate trigger and the environment filter. In the following API call, we will set up notifications to be sent to #alerts when the same error is raised on production environment more than 20 times in a minute. You can achieve this by sending the following API call:
curl \
--location \
--request PUT 'https://api.rollbar.com/api/1/notifications/slack/rules' \
--header 'X-ROLLBAR-ACCESS-TOKEN: <WRITE_PROJECT_ACCESS_TOKEN>' \
--header 'Content-Type: application/json' \
--data-raw '[
{
"trigger": "occurrence_rate",
"filters": [
{ "type": "rate", "period": 60, "count": 20 },
{ "type": "level", "operation": "gte", "value": "error" },
{ "type": "environment", "operation": "eq", "value": "production" }
]
}
]'occurrence_rate trigger is fired when X occurrences are seen in Y minutes.
It comes together with a special filter rate that takes two parameters: count - the number of occurrences (X) and period - the timeframe (Y).
Next, we apply level filter to only get notified about items of level error and above.
And at last, we apply the environment filter to only get notified about items in the production environment.
Other triggers
Rollbar now offers a wide range of other triggers and filters. You can explore all of them in our Notifications API documentation. Just to bring up some of them:
exp_repeat_item- fired on 10 to n exponent occurrence (10th, 100th, 1,000th, 10,000th, … occurrence)reopened_item- an error/message is marked Active by a user.new_version- a new code version is detectedresolved_item- an error/message is marked Resolvedreactivated_item- an error/message occurs again after being marked Resolveddeploy- a new deploy is reported
Prerequisites for PagerDuty notifications
Having the Slack integration is awesome. Now you get Rollbar notifications right there in your Slack workspace. But how about PagerDuty? With Rollbar’s new Notifications API you can integrate that as well!
First, you need to make sure you have Rollbar integrated with your service. If you added the Rollbar integration when you were first setting up your account for the first service, you don’t have to do it again. But if you did not or want to integrate Rollbar for a different service, go to -
- Configuration → Services → select service you want to set up Rollbar notifications for → Integrations → + New Integration
- Provide
Rollbarfor the Integration Name,Rollbarfor the Integration Type,- check Use our API directly
- and pick Events API v2 in the Integration Type select box
- Click Add Integration
curl \
--location \
--request PUT 'https://api.rollbar.com/api/1/notifications/pagerduty' \
--header 'X-Rollbar-Access-Token: <WRITE_PROJECT_ACCESS_TOKEN>' \
--header 'Content-Type: application/json' \
--data-raw '{
"enabled": true,
"service_key": "<PAGER DUTY SERVICE INTEGRATION KEY> "
}'You can find the full reference of this endpoint in our PagerDuty Notifications API documentation.
Setting up custom rules for your PagerDuty notifications
Just like with Slack, let’s set up the trigger rules now. Here we will show again the same example notification for new Rollbar items:
curl \
--location \
--request PUT 'https://api.rollbar.com/api/1/notifications/pagerduty/rules' \
--header 'X-Rollbar-Access-Token: <WRITE_PROJECT_ACCESS_TOKEN>' \
--header 'Content-Type: application/json' \
--data-raw '[
{
"trigger": "new_item",
"filters": [
{ "type": "level", "operation": "gte", "value": "error" },
{ "type": "environment", "operation": "eq", "value": "production" }
]
}
]'Here is how a new incident will now be reported in PagerDuty when Rollbar detects a new error in your application:
The full reference of this endpoint can be found in our PagerDuty rules Notifications API documentation
Don't see a tool you use? Let us know!
With Rollbar’s new Notifications API you can fully integrate Rollbar into your project automation workflow. Slack and PagerDuty were the first two services in our priority list to implement. However, we are already working on expanding that list with email notifications being first in the line.
Whatever your communication workflow is, we are working hard at Rollbar to ensure you can integrate that with our product. If you would like to add notifications automation for a service we do not support yet, we would love to hear from you. Feel free to reach out to [email protected] and tell us about your request.