What’s new in PulseAI 2.9
This release is about the model record. A registry entry has always said what a model is, who is accountable for it and how it is rated. It now also holds what happened when the model was reviewed, what is still open, and whether the numbers your monitoring produces are inside the limits you set.
A review history on every model
Each model has a Reviews tab. Admins and managers record a review with its date, what it covered, an outcome of approved, approved with conditions, or not approved, and a written summary. The person recording it is recorded as the reviewer, and a review cannot be entered in someone else’s name.
A review cannot be edited or deleted once it is recorded. If something was wrong, you record a new one, so the history shows what was concluded at the time. When the reviewer is also the model’s owner, the review is kept and marked as not independent.
Findings with an owner, a due date, and a history
A review can raise findings, and a finding can also be added on its own. Each has a severity, and can have a named owner and a due date. An open finding is shown as overdue from the day after its due date.
A finding is closed in one of two ways, and both need a note: resolved, saying what was done, or accepted, saying why the gap is being carried. A closed finding can be reopened, and every change is kept with the person and the time.
The next review date follows the risk tier
Recording a review sets the model’s next review date from its risk tier: 12 months for a high-risk model, 24 for medium, and 36 for low or experimental. An admin can change those intervals for your organization. A reviewer can bring the next review forward, for example after approving with conditions, but cannot push it past the interval.
For organizations in full governance mode, a model whose next review date passes with no later review gets a finding saying so. It closes when a later review is recorded.
Thresholds for the metrics your monitoring reports
Your monitoring already measures things like drift and accuracy. An admin can now declare those metrics on the model, each with a warning threshold and a breach threshold, and say whether higher or lower is worse. The Monitoring tab then shows each metric as within threshold, warning, or in breach, and marks it stale when nothing has arrived for a period you set.
Your monitoring sends the values in with a key created for that one model. A key can only send values for its model and cannot read anything, it expires, and it can be limited to particular networks. PulseAI does not compute these values or keep their long-term history: it shows the latest reading and a short list of recent ones, and your monitoring system stays the system of record.
A breach opens a finding, and only a person closes it
When a metric reaches its breach threshold, PulseAI opens a finding on the model with the value, the threshold in force and the time. One finding covers a continuing breach, so a metric that stays out of range does not fill the list.
If the metric recovers, its state returns to within threshold and the finding stays open until someone resolves or accepts it with a note. A reading from a monitoring job can raise a concern. It cannot say the concern has been dealt with.