Closed-Loop Rollback
After a Full-Auto action removes access, GraphnAI watches the hosts that depended on that access. If failed sign-ins or failed services appear there, it undoes the change or escalates it for review.
Requires role: Admin (the Closed-Loop Rollback section of a receipt and the Autonomy Activity tab); Operator or Admin (escalation and undo notifications) Related: Remediation Reversibility and Undo, Full-Auto Execution, Semi-Auto Execution, Select Fire™
Overview
Removing access can break something that quietly depended on it, such as a service or a scheduled task. Closed-Loop Rollback catches that for the actions Full-Auto runs.
Once the directory confirms a Full-Auto action, GraphnAI starts a rollback watch. The watch covers the hosts that used the account's access before the change. If those hosts start logging failed sign-ins for the account, or failed services, the watch trips. A hygiene action is then undone automatically. A containment action is escalated to a person instead.
The watch starts on its own and needs no setup. It's stored in the database, so it survives a server restart, and it extends itself when a sync overlaps it.
Semi-Auto actions don't get a watch. You authorized each one yourself, and you can undo it from the History tab. See Remediation Reversibility and Undo.
Prerequisites
- Full-Auto is armed. See Full-Auto Execution.
- At least one sync has completed for the domain.
- The account has Windows sign-in telemetry. Full-Auto won't act on an account without it, so every watched account has it.
- Admin role to see a watch's detail. Operators see the action's row and status, but not the hosts being watched.
A change to a protected object that someone confirmed by typing its name never gets a watch. Its only undo is the confirmed one on the History tab. See Confirming an Undo on a Protected Object.
For a hybrid identity (an on-premises account with an Entra ID twin), the watch adds the hosts the twin used. The signal comes from Windows events only. Failed cloud sign-ins in Entra ID aren't part of it yet.
How the Watch Works
When it starts
The watch starts when the directory confirms a Full-Auto action. The action's status stays Verified, and its receipt timeline shows "Rollback watch started". If the server restarts during the watch, GraphnAI starts it again from the saved record.
Which hosts it watches
The watch covers the hosts that used the account legitimately before the change. It finds them two ways and combines the results:
- Observed use: hosts where the account signed in successfully in the 30 days before the change (Windows event 4624).
- Access simulation: computers that lose access when the permissions are removed, taken from the same simulation Select Fire runs.
What trips it
Each sweep reads the raw Windows events for the watched hosts. It doesn't wait for detection alerts, which can lag during a sync. For each host, it compares failed sign-ins for the account (event 4625) after the change with that host's own history. The history covers the 30 days before the change, per host and per account.
A host trips the watch when any of these is true:
- Little or no history: the host averaged less than one failed sign-in a day for the account, and 3 or more new failures appear after the change. Most hosts fall here.
- A history of failures: failures after the change exceed the host's Average by more than 3.5 times its Spread.
- Failures that point to the change: failed sign-ins whose reason code points to the removed access, plus failed services on the host (Windows service events 7000, 7001, 7034 and 7038), add up to 3 or more.
Only watched hosts count. Failed sign-ins for the account from any other computer are ignored. An attacker who floods sign-in attempts against the account from their own machines can't trip the watch.
The watch window
The window is 90 minutes. That covers the known delays for the changes GraphnAI makes. For example, an Entra ID session can stay valid for up to an hour, and AdminSDHolder can take up to 60 minutes to reset protected permissions. Failures may not appear straight away.
If a sync runs during the window, the window is extended by the overlap, so a sync never hides a failure. The receipt shows the extension under the countdown, for example "Extended 12 minutes during a sync". GraphnAI checks every watch every 2 minutes.
Intent-Based Policy: When It Reverts and When It Escalates
Each action records its intent when it runs. The intent decides what a tripped watch does.
- Hygiene: the action removed access that shouldn't exist, such as an extra group membership. When the watch trips, GraphnAI undoes the action from its saved copy of the original state, the same way Undo does. The status reads Running while the undo runs, then Reverted once the directory confirms it. Full-Auto then won't repeat the same change on the same account for a cooldown period (24 hours by default). Its reason reads "Rolled back recently".
- Containment: the action stopped an active threat, such as disabling a compromised account. GraphnAI never undoes containment automatically, because that would let the threat back in. The status moves to Escalated for review, and Operators and Admins get a notification.
An action whose intent isn't recorded is treated as containment.
Failures after a containment action are often expected. A tool that used the compromised account stops working, and that's the point. The escalation lets a person decide whether the failing service was legitimate.
Escalation and the Circuit Breaker
When escalation occurs
A tripped watch escalates in these cases:
- Containment: the action was a containment, so it's never undone automatically.
- Undo failed: GraphnAI tried to undo a hygiene action, and the Identity Bridge couldn't apply it.
- Undo not confirmed: the undo ran, but the directory didn't confirm the original state.
- No safe automatic undo: the action has no undo that can run without a person, for example because it's Not reversible.
A short-lived problem, such as a bridge that's briefly offline, is retried on the next sweeps. After 10 tries, about 20 minutes, the action is escalated.
An escalated action moves to Escalated for review, and Operators and Admins get a notification. Nothing retries after that.
The circuit breaker
The breaker counts undone actions and failed escalation deliveries. If 5 or more happen within 10 minutes, something is wrong at scale: the signal is misfiring, or the policy doesn't fit your environment. The breaker then pauses Full-Auto, so its actions are held for a person to authorize in Semi-Auto. Admins get a "Full-Auto paused" notification.
While the breaker is open, an action held by it shows the reason "Paused after rollbacks". The pause lifts by itself once fewer than 5 of those events fall inside the last 10 minutes. Review what was undone before then. To keep Full-Auto from running while you investigate, turn Shadow mode back on in Admin > AI Provider > Autonomy.
Finding It in the UI
Remediation page (History tab)
Select Detection & Response > Remediation in the sidebar. The History tab opens. A Full-Auto action shows one of these statuses during and after its watch:
| Status | What it means |
|---|---|
| Verified | The directory confirmed the change. While the watch runs, Undo is disabled with the tooltip "Undo is available after the rollback watch ends." |
| Reverted | The watch tripped and GraphnAI undid the action. The directory confirmed the original state. |
| Escalated for review | The watch tripped and the action wasn't undone. A person needs to review it. |
Select a row to open its receipt. Its Timeline shows when the watch started, when it ends or ended, and when failures were seen on a watched host.
The Autonomy Activity tab (Admin only)
The Autonomy Activity tab on the same page shows Armed watches, Awaiting review and Full-Auto actions as counts. Below them are the running watches with their countdowns, the queue of escalated actions and tripped watches, and recent Full-Auto activity. Select an entry to open its receipt.
The Closed-Loop Rollback section (Admin only)
A Full-Auto action's receipt includes a Closed-Loop Rollback section.
Watch While the watch runs, this row reads "About 45 minutes left of 90 minutes", with a meter of the window used. A tripped watch reads "Failures seen" and when. A finished watch reads "Ended with no failed sign-ins or failed services on the watched hosts", or "Ended. No hosts were watched." when it had nothing to watch.
If a watched host fails "Undo the change automatically" for a hygiene action. "Alert a person, never undo" for a containment action.
Hosts watched The section lists each watched host, with a count in its title such as "Hosts watched (4)". For each host:
- Its name.
- Its history for the account: "Usually about 1.2 a day", or "No failed sign-ins before the change".
- After a trip, the failed sign-ins since the change, for example "5 failed sign-ins since the change".
A host whose failures are above its usual level carries an Above usual badge. The watch can also trip on failed services or on the reason codes of the failures, which the badge doesn't show.
Escalation An escalated action's receipt says why:
| Title | What it means |
|---|---|
| Containment held, as designed | Failures appeared, and the action was containment. Nothing was undone, as intended. |
| No safe automatic undo | Failures appeared, and this action can't be undone automatically. |
| Undo failed | GraphnAI tried to undo the action and the undo didn't complete. |
| Undo not confirmed | The undo ran, but the directory didn't confirm it. |
Notifications
| Notification | Who gets it | When |
|---|---|---|
| Closed-Loop Rollback undid a change | Operators and Admins | A watch tripped and the undo was confirmed. |
| Remediation escalated | Operators and Admins | A containment held, or an undo failed or wasn't confirmed. The message names the account and the change. |
| Full-Auto paused | Admins | The circuit breaker tripped. |
Step-by-Step: Investigating a Rollback Trip
Follow these steps after an escalation or undo notification, or when you see a Reverted or Escalated for review row.
- Select Detection & Response > Remediation in the sidebar.
- In the History tab, filter the status to Escalated for review or Reverted.
- Select the row to open its receipt.
- In Hosts watched, find the hosts marked Above usual.
- Compare each host's failures since the change with its usual level.
- Decide whether the failures came from something that needed the access.
A host with no failures before the change and several after it had something running on the removed access.
If the service needed the access, the undo was right. Change the access policy, or move the service off this account, before you run the remediation again.
If the failures were unrelated noise, the watch tripped on a false positive. After you investigate, run the action again from Select Fire™.
For an escalated containment action:
- Check that the failing service was using the compromised account, so its failure is expected.
- If a legitimate service failed, restore its access with your directory tools, and investigate the dependency.
Troubleshooting
A rollback watch has run for a long time
The window is 90 minutes, and a sync that overlaps it extends it. Check whether a sync is running on the Bridges page. Once the sync finishes, the watch either ends cleanly or trips.
The receipt has no Closed-Loop Rollback section
The section needs the Admin role. If you're an Admin, the action may not have had a watch:
- It ran in Semi-Auto. Only Full-Auto actions get a watch.
- It was a change to a protected object that someone confirmed by name.
- It didn't reach the Verified state.
If the watch couldn't be loaded, the section reads "Couldn't load the rollback watch. Reopen this entry to try again."
The watch has no hosts
The receipt reads "No hosts were found to watch." The account had no successful sign-ins in the 30 days before the change, and the simulation found no computers that lose access. The watch still runs and ends when the window closes, reading "Ended. No hosts were watched."
An escalation arrived and the receipt says "Undo failed"
GraphnAI tried to undo the action, and the Identity Bridge couldn't apply it. Read the Escalation section of the receipt for the cause. An escalated action can't be undone from the History tab, so reverse the change in the directory with your own tools. If the bridge was offline, reconnect it too.
The "Full-Auto paused" notification appeared
The circuit breaker counted 5 or more undone actions or failed escalation deliveries within 10 minutes. Review the recent Reverted rows in the History tab to find what caused them. Test the escalation channel too, because failed deliveries also count. Full-Auto resumes by itself when the burst ages out of the 10-minute window, so turn Shadow mode back on in Admin > AI Provider > Autonomy if you need more time.