Auditing access changes around quality review

Research findings

## Question and scope A useful scope is narrow enough to reproduce and specific enough to show whether excess access is being detected. State how duplicate requests are identified and whether they are merged, linked, or counted separately. Define the start event, the stop event, the pause rule, and the treatment of a missing client intake before collecting data. If the study is exploratory, label it exploratory and do not present the result as a stable baseline. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract. The scope is one defined population of quality review cases observed from first request through verified close in the access management console. If one request produces several team roster updates, define whether the unit is the request or each update before sampling. Define age from the first recorded request, not from the first time someone remembered to open the case. Separate the operational question from the compliance question; this method does not certify either outcome. At Outsourced Company, the quality review review should end with a dated note and a named owner for the next check.

## Population and unit A population that changes size during the day should be reported with the start, end, and net change, not only the closing total. Define whether a request that generates several team roster updates is one case or several before counting begins. Use plain language in the definition and add one example and one non-example for the reviewer. Separate cases opened before the window from cases opened and closed inside it, and report both counts. If the evidence is sensitive, record the approved reference and access path rather than copying the content. Include cases opened before the window but closed inside it only when the protocol says they belong to the population. Record the queue at the start and end of each observation day so additions and withdrawals are visible. The stop event should be the verified destination update, not the last message in the thread. If the queue was archived during the window, report the loss and exclude the affected period rather than assuming the cases were resolved. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.

## Definitions and measures Name the owner who can declare a case complete, and record that role in the data dictionary. A blank field, a not-applicable field, and a confirmed negative field should have different codes. Define whether a case that waits for a third party is open, paused, or excluded, and apply that rule to every owner. If two reviewers disagree, preserve both classifications until the definition is clarified. Record this decision in the study log with the case key, the review date, and the person who approved the classification. A case that has no destination change should be defined as cancelled or rejected, not counted as complete. Keep the original text and the coded value together so a reviewer can check the classification. Before collection, test the definitions on a missing quality review, a reopened case, and a handoff. A definition that depends on the reviewer’s memory is not reproducible enough for this study. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.

## Data dictionary and evidence The data dictionary should name the allowed values for status, exception reason, owner, and close evidence. Record the extraction time separately from the event time so a late entry is visible. Keep the original evidence reference next to the coded value so the reviewer can verify the classification. Define who may change a coded value and how the change is logged. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it. Each case needs the source, the observation time, the field supported, the owner, and the reviewer. Name the required fields for each article type and mark conditional fields in the dictionary. Record the data extraction date and the version of the task queue used for the population count. Record the source location, not just the file name, so a reviewer can reopen the exact evidence. At Outsourced Company, the reviewer should be able to explain this step to team lead from the case record alone. Test the dictionary on five cases before the full sample; ambiguous fields should be fixed before collection, not after. Use a controlled list for exception reasons; do not let free text become the analysis category. Store sensitive evidence in the approved repository and keep only the reference in the analysis file. Keep the original request, the approval, and the destination value linked to the same key. Use the same note in the case file and the study extract; do not create a second version in a personal document.

Discussion

## Sampling and collection Stop expanding the sample only when the review question is answered, not when the result starts to look favorable. Collect the case from the task queue, the timekeeping exception, and the approval record, and stop at the verified close. Include reopened, unresolved, and exception cases in the sample frame. Record the date of the sample and the version of the queue export. When a case needs an exception, record the reason, the owner, and the date before continuing the review. Use a random or systematic start rule for the first case, then take consecutive cases within each stratum. Collect the exception reason and the action taken before the close so the analysis can separate detection from resolution. If a stratum has fewer cases than planned, report the smaller count instead of replacing it with a convenience case. A case that is difficult to access is still part of the frame; report the access failure rather than replacing it silently. Keep the measurement note short enough to read with the case and specific enough to support the reported task cycle time. Sample across the full observation window, including the days when the queue was busiest and the days when coverage changed. Collect the case from the timekeeping system, the client intake, and the approval record, and stop at the verified close. Keep the sample selection reproducible by recording the export date, filter, and random seed or start rule. If a selected case is out of scope, replace it only with the next case in the same stratum and record the substitution. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.

Implications and limits

## Analysis and reporting For quality review, report the age distribution and the share of cases waiting on a named decision rather than a single average. If two measures disagree, report both and explain what each one counts. If the result is unstable, say which cases create the variation and what a larger sample would test. Report the numerator, denominator, observation window, and exclusion count for each measure. Link the observation to the exact field in the team roster and to the client report that supports it. A difference between two periods is a signal to inspect, not proof that a process change caused the change. The analysis should end with the decision the evidence can support and the decision it cannot support. Show the unresolved cases by owner, age, and dependency instead of reporting a single completion percentage. Report the number of cases that drive a small-sample result and the direction of the effect. Do not let a summary written later replace the evidence that was available at the first decision.

## Limitations The study cannot determine whether the professional decision was correct, only whether the record and handoff are reconstructable. Small samples are unstable; report the range and the cases that create the variation. Report the limits of the timekeeping system data, including fields that were unavailable or changed during the window. Record the access problems that prevented a case from being reviewed. If the case moved between owners, include both owner periods in the review note and name the handoff time. The record may be incomplete, delayed, or maintained outside the team, so the method cannot infer missing work from a clean summary. Changes in staffing, access, or the access management console during the window can create a difference that is not a process effect. Report the limits of the access management console data, including fields that were unavailable or changed during the window. State what a follow-up study would measure and which population it would include. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.

## Review and next action A quality review should end with an owner, a date, and one testable change, not a list of intentions. If the action changes offboarding gap, record the approval from operations manager before implementation. Set the next sample date before closing the review, so the action does not drift. Link the action to the case evidence that prompted it, not only to the meeting notes. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract. A quality review should end with an owner, a date, and one testable change, not a list of intentions. Repeat the protocol with the same definitions and window, then compare the same measures. Record the owner, the due date, and the exact change that will be tested at the next review. Record the owner, due date, and exact change that will be tested at the next review. At Outsourced Company, the quality review review should end with a dated note and a named owner for the next check.