Integrations — On-call and alerting
NerveStax and Splunk On-Call
What we do
What NerveStax does with Splunk On-Call
Route your Airflow, dbt and webhook failures to NerveStax first. An agent investigates each one with read-only tools, resolves the noise with its reasoning written down, and escalates the rest to Splunk On-Call with the likely cause and the evidence attached. Your rotations, escalation policies and paging rules stay where they are.
-
Triage before the page
Alerts land in NerveStax first. The agent reads the failed task's logs, the lineage around the asset and what changed recently, then resolves or escalates.
-
Close noise with a written reason
Retries that already recovered, transient infrastructure failures and fan-out from a single upstream break end as resolved, with the whole investigation recorded.
-
Escalate with cause and evidence
A real incident is routed to Splunk On-Call carrying the likely cause, what is downstream of it, and the checks behind that reading.
-
Any paging system with an API
Not on one of the named tools? Escalation can be delivered to any paging or alerting system that exposes an API, and the incident is also recorded in NerveStax.
-
Unsure pages anyway
A crash, a timeout, a missing or inconclusive verdict all escalate. Nothing is dropped silently, and an uncertain agent escalates rather than inventing a cause.
Background
About Splunk On-Call
Splunk On-Call, formerly VictorOps, routes alerts to the engineer on call and tracks the response. Data platform failures are usually only one of the alert sources it handles.
How it connects
What connecting it looks like
-
01
Create a REST endpoint
Add a REST integration for the routing key that should take data platform incidents, and copy the endpoint URL.
-
02
Add it in NerveStax
Set it as the escalation target for the workspace. It is stored encrypted and no screen ever shows it back.
-
03
Point your alerts at NerveStax
Use the per-project webhook URL for Airflow failure callbacks, dbt run and test results, or anything else that can POST JSON.
-
04
Read the transcripts
Check what was resolved and what was paged, and why. That tells you what else is worth routing through triage.
Scope
What this does not cover
- Escalation is one-way: NerveStax pages Splunk On-Call. It does not sync incident state back or write into incident timelines.
- Triage is read-only. Code fixes still arrive as a tested pull request a person approves.
- Any paging system with an API can receive escalations; the named tools are the ones we have built and use.
Questions
Splunk On-Call and NerveStax
Q01Is this the same as VictorOps?+
Yes. Splunk On-Call is the current name for VictorOps, and the integration is the same one. If your team still refers to routing keys and REST endpoints in VictorOps terms, nothing about the setup changes.
Q02Do you replace our paging tool?+
No. It keeps the rotations, escalation policies and notification rules; we page it. The difference is what it receives: an alert that has already been investigated, with the likely cause and the evidence attached, instead of a job name and a retry count.
Q03We use something else for paging. Can it still work?+
Yes, if it has an API that accepts an incoming incident. PagerDuty, Opsgenie and Splunk On-Call are built in; anything else with an API can be the target. Every escalation is also recorded in NerveStax with its diagnosis, so no team is left without the detail.
Q04What if the triage agent is wrong?+
Every verdict carries its transcript: each log it read, each check it ran. A wrong call is visible in seconds rather than found in a retro. And an uncertain agent does not guess, it escalates as inconclusive, so being wrong tends to mean paging you unnecessarily.
Q05Can it act on production during triage?+
No. Triage runs with read-only tools and no repository or write access while a raw alert payload is in its context, and payloads are fenced as untrusted input. When the fix is a code change, it arrives as a tested pull request a person approves.