Testing whether timekeeping monitoring catches silent failures

Research findings

## Question and scope The scope is one defined population of timekeeping cases observed from first request through verified close in the access management console. A count without a denominator is not a measurement, especially when the queue changes size during the day. Define age from the first recorded request, not from the first time someone remembered to open the case. Separate the operational question from the compliance question; this method does not certify either outcome. If the evidence is sensitive, record the approved reference and access path rather than copying the content. The scope is one defined population of timekeeping cases observed from first request through verified close in the reporting dashboard. If the population is too small to sample, say so and report the complete set rather than implying a larger study. Distinguish an administrative pause from a failure: a pause waits for a named decision, while a failure leaves the next action unowned. State whether the study covers one team, one client, or all cases in the defined service. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.

## Population and unit Record how duplicate requests were matched, merged, or kept separate, because that choice changes the denominator. The population should include unresolved cases, not only closures, or the study will overstate completion. The stop event should be the verified destination update, not the last message in the thread. Record the extraction date and the queue filter so another reviewer can reproduce the population count. Record this decision in the study log with the case key, the review date, and the person who approved the classification. Test the unit definition on a case that crosses owners, a case that is still open, and a case that was reopened. A case that moves between owners stays one case unless the process defines a new requirement at the handoff. A correction should be classified by the original case and by the correction event, with both dates retained. Report a case that moved between owners as one case with two owner periods unless the process creates a new requirement. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.

## Definitions and measures If the definition of completion changes, keep the old and new rules side by side for the transition period. A blank field, a not-applicable field, and a confirmed negative field should have different codes. A duplicate request is a separate intake event even when it points to the same underlying need; record the link. Define whether the measure counts requests, cases, or destination updates, and use that unit consistently. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it. If the definition of completion changes, keep the old and new rules side by side for the transition period. A blank field, a not-applicable field, and a confirmed negative field should have different codes. A correction is a change to a closed record; a clarification that does not change the record is a note, not a correction. A definition that depends on the reviewer’s memory is not reproducible enough for this study. At Outsourced Company, the reviewer should be able to explain this step to operations manager from the case record alone.

## Data dictionary and evidence If the source is a document, record the page or field; if the source is a system, record the record identifier and extraction time. Test the dictionary on a case with a missing client report and a case with a corrected value. Record the source identifier, the field name, the observation time, and the reviewer for every extracted value. Archive the dictionary with the study so a later reviewer can reproduce the categories. Use the same note in the case file and the study extract; do not create a second version in a personal document. Record the version or date of the client intake and the team roster value so a later reviewer can see what was known at the first action. Give every case a study key that is not the client name, member number, or candidate email. Keep the source reference and the coded value in the same row, and do not overwrite the original with a later correction. Use a separate field for reviewer notes so the coded values stay comparable. When a case needs an exception, record the reason, the owner, and the date before continuing the review. The data dictionary should name the allowed values for status, exception reason, owner, and close evidence. Give every case a study key that is not the client name, member number, or candidate email. Record the data extraction date and the version of the timekeeping system used for the population count. Use a separate field for reviewer notes so the coded values stay comparable. Keep the measurement note short enough to read with the case and specific enough to support the reported task cycle time.

Discussion

## Sampling and collection A convenience sample of recently closed cases will overrepresent easy work and underrepresent the tail that needs attention. Record the time of each observation so a later reviewer can distinguish a late entry from a changed decision. Keep the sample selection reproducible by recording the export date, filter, and random seed or start rule. Keep a table of planned, eligible, selected, and reviewed counts for each stratum. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved. Stop expanding the sample only when the review question is answered, not when the result starts to look favorable. Record the time of each observation so a later reviewer can distinguish a late entry from a changed decision. If a stratum has fewer cases than planned, report the smaller count instead of replacing it with a convenience case. Include at least one case from each owner in scope when the frame allows it. Link the observation to the exact field in the timekeeping system and to the team roster that supports it. If the population is small, report every case and the reason the sample is exhaustive rather than presenting a false precision. Record the time of each observation so a later reviewer can distinguish a late entry from a changed decision. If a stratum has fewer cases than planned, report the smaller count instead of replacing it with a convenience case. Use a random start and a fixed interval when the frame is ordered, and record both. Do not let a summary written later replace the evidence that was available at the first decision.

Implications and limits

## Analysis and reporting For timekeeping, report the age distribution and the share of cases waiting on a named decision rather than a single average. Use case-level review to explain an outlier before treating it as a process signal. Write the finding in plain language and separate what was observed from what the team believes caused it. When a measure is based on a small sample, state the case count and avoid ranking owners by the result. If the case moved between owners, include both owner periods in the review note and name the handoff time. A difference between two periods is a signal to inspect, not proof that a process change caused the change. Compare periods only after checking the case mix, the definitions, and the queue rules. Use a simple before-and-after comparison only when the definitions and observation window are the same. When a rate changes, check whether the case mix or the evidence standard changed at the same time. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.

## Limitations The cited public sources frame recordkeeping, privacy, security, or control practice; they do not provide a local result. Small samples are unstable; report the range and the cases that create the variation. State that the result describes the observed sample and not the entire industry or every team in the service. Report the limits of the access management console data, including fields that were unavailable or changed. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract. The method also cannot observe work that never entered the queue, which may be the most important missing population. The cited sources describe recordkeeping, privacy, security, or control practice; they do not supply a local benchmark. Record the response or access problems that prevented a case from being reviewed. State what a follow-up study would measure and which population it would include. At Outsourced Company, the timekeeping review should end with a dated note and a named owner for the next check.

## Review and next action Repeat the protocol after the change with the same definitions, window, and exclusions. If the action changes unowned task, record the approval from team lead before implementation. Close the review only when the owner has accepted the result and the next check is scheduled. If the action needs approval, record the approver and the date before the change is applied. If the evidence is sensitive, record the approved reference and access path rather than copying the content. If the action affects timekeeping error, route it to offshore worker and record the approval before implementation. Name the owner, the review date, and the decision the study is expected to inform. Record the owner, the due date, and the exact change that will be tested at the next review. Report a failed action with the same detail as a successful one so the team can learn from it. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.