Escalation path branches
You might have people who are on-call just during business hours, or where you only want to wake people up when the priority is the highest for an alert. You can now set up rules around priority and/or working hours to ensure you page the right people at the right time without causing additional noise.
Below instructions help you create multiple escalation paths with rules, but you can still create simple escalation paths with the feature as in this help article.
How does it work?
You can start creating a new escalation path from scratch or edit your current paths. With this feature, you can
Set up working hours for all levels in the escalation path
Configure priorities in incident.io and connect them to your alert source
Configure notifications for your devices for high and low-urgency
Create branches based on priority and/or working hours
Choose which levels should send high or low-urgency notifications
Choose to escalate either within a time window or based on your working hours
1️⃣ Set up working hours for all levels in the escalation path
Setting up working hours will create the working hours for all levels in the escalation path created. You can choose the days, times, and the timezone. We don't currently support different working hours per level.
2️⃣ Configure priorities and connect them to your alert source
To use priorities in the escalation path branches, you can create them in Alert Attribute settings. Priorities are now a first-class citizen in incident.io, which means that you wouldn't need to create those as a custom field in the catalog just to connect with your alert source.
You can create priorities locally just in incident.io or bringing your priorities from your Alert source. We'll show both cases below
Create your priorities in incident.io
Head to Alert Attribute settings
Create Priorities, organise them with drag and drop and choose a default value
After creating the priorities, head to your Alert sources to set up the default priority per source.
Bring your priorities from your alert source i.e. Datadog
Go to your Alert sources and choose an alert source
Go to Configure tab and scroll down to Priorities
Choose to use a variable and Add new Expression
Choose query
Choose the payload and Parse
Write to field
$.metadata.priority(Example, can be a different payload variable name)Choose Alert priority from the drop-down
Choose the default if the query doesn't return the Alert priority
Save the setting by clicking 'Add'
Save the alert source
You will also be able to see your priorities in Catalog if created or brought through your Alert source.
3️⃣ Configure notifications for your devices for high and low-urgency
If you want to enable your workflows to use different notifications depending on priority or working hour branches, you can head to your Preferences and choose the notifications you'd like to receive in both cases.
4️⃣ Create branches based on priority and/or working hours
Now that you have all configurations created, you can start your path with or without conditions. Conditions we offer at the moment are working hours and priority.
You can choose to start from either
If you want to use both priority and working hours as rules, you should create three branches, like below. You can choose to start your escalations either with working hours or priorities. The example below will do the next
1st branch will send Alerts within working hours with high urgency notifications to the person on call from the Payments team and wait 10 minutes until escalating to the next level
Second branch
If P1, will send Alerts outside of working hours with high urgency notifications to the person on call from the Payments team and wait 5 minutes before escalating to the next level
If other than P1, will send Alerts outside working hours with low urgency notifications to the person on call from the Payments team and wait until working hours begin for that person to acknowledge, before escalating to the next level
5️⃣ Choose which levels should send high or low-urgency notifications
Earlier in this article, we showed how every person can set up their preferences on high and low-urgency alerts. To set them up in the Escalation path, just choose either High or Low in every level to choose how you'd want to notify the users.
6️⃣ Choose to escalate either within a time window or based on your working hours
If you have used Working hours as a condition in your branch, you'll receive a few new options in escalation delays: Working hours begin and working hours end.
This means that people who are on call will receive the notifications at that moment, but we'll wait until that time before we escalate to the next level
Escalate to Slack
Choose when an alert should be escalated to a Slack channel of your choice where anyone can acknowledge and start resolving the alert.
1️⃣ Head to Escalation paths
2️⃣ Click 'Level' to choose a channel
3️⃣ Search for the Slack channel
4️⃣ Save your escalation path
Example use cases
You can send all escalations to a shared Slack channel to let a group of people know that something is wrong, but still escalate to an actual responder at the same time by selecting "Don't wait" as a time to acknowledge option.
You can set up a branch to deviate low urgency escalations to a Slack channel.
Good to know
If you have an escalation path that first pages a slack channel, then the person being on call in the next level will be shown as 'Level 1' in your On-call status
If you have an escalation path that first pages someone else, then a slack channel, then you, you'll be shown as 'Level 2' in your On-call status
Escalate to everyone
Page everyone at the same time in a schedule or the people of your choice. When choosing this, everyone in the same level will be paged at the same time.
1️⃣ Head to Escalation paths
2️⃣ Go to a level
3️⃣ Choose a schedule or users
4️⃣ Choose 'Everyone'
Good to know
By choosing 'Everyone' it means that we will page everyone in that level. If a person from this level acks, we will not escalate to the next level - but we will notify to all people in the level until their notification preferences end. Read more about notification preferences here.
'All at once' setting will only page those people in a schedule who are on call
Read more about Escalation paths
If you have any feedback or feature requests on the feature, please send us a message to [email protected] - we'd love to hear from you!