Can’t Wait?

AQMS results are not, under any circumstances, used to measure the individual performance of Appeals employees. AQMS is designed and intended as a useful tool for executive and top level managers to use in determining strengths and weaknesses of the quality of the Appeals process.

Any feedback or other work products generated from AQMS are not used as a substitute for case reviews, case file reviews, on the job visits and workload reviews.

Closed cases are selected for AQMS review by a random sampling process that is statistically valid to the Area/Campus level for an annual period. The case sampling system is designed to ensure a representative number of cases are selected from each Appeals Area.

The criteria used in selecting the representative case samples are case source and type.

In addition to randomly selected cases, Closed Case Reviews may be conducted on certain types of cases at the request of Appeals Executive Management.

AQMSSELECT is the indicator automatically set when the case is selected for AQMS review based on programmed criteria.

When a case is selected for AQMS review, ACDS generates a partially completed form for the selected case which is completed by the APS function closing the case and attached to the case file. Upon closing the case, APS ensures documents identified on the form are included in the case file. APS then forwards the selected closed case to the appropriate AQMS reviewer/office as identified on the form.

The AQMS rating system provides an objective, consistent, and equitable scoring system for selected cases, and emphasizes Standards identified as key factors for ensuring an acceptable quality Appeals process.

The AQMS rating system consists of eight key Standards for Non-Collection Appeals cases and seven key Standards for Collection Appeals cases.

Quality Standards provide objective criteria upon which case quality is evaluated. Each one provides concise statements of Appeals' expectations of a quality work product.

Each Quality Standard contains one or more Attributes that define the prerequisite of a quality case and reflect Appeals' goals.

Some Attributes have Reason Codes. Reason Codes are specific examples of actions or tasks that are contrary to Appeals goals.

Each Attribute is worded as a question. When rating aspects of cases relevant to an Attribute, the AQMS Reviewer assigns an answer of "Met" or "Not Met" to each applicable Attribute. For an Attribute with Reason Codes, the rating of the Attribute is determined based on whether any of the Reason Codes for the Attribute are selected.

Reason Codes provide detailed information used for analytical purposes. The Reason Codes are used as a management tool to identify the cause(s) of a missed Attribute.

A narrative comment is required for some Reason Codes and Attributes. A narrative comment is a brief summary of the Reviewer’s observations about a given Attribute or Reason Code. They are used to explain the reasons Attributes are rated Not Met and to further explain why a Reason Code was selected. They are also used to provide favorable comments.

The rating assigned to an Attribute by the AQMS Reviewer is a component of the overall score of the related Standard.

The AQMS Review Check Sheet is a form outlining the Quality Standards, Attributes, and Reason Codes. AQMS Reviewers use the check sheet to systematically score each case.

The check sheet outlines the criteria for rating each Standard and includes a comment section for each Standard, Attribute, and Reason Code.

When completing the check sheet, Reviewers take into account the significance of both positive and negative aspects of the case file and their respective impact on the Standards, Attributes, and/or Reason Codes.

For many of the Attributes or Reason Codes, Reviewers are required to use the comment section to discuss NOT MET ratings on any Attribute or Reason Codes, and they are asked to share favorable comments regarding Standards and Attributes. Reviewers must ensure their comments are meaningful, clear and easily understood.

To provide statistically valid data on case quality for Appeals, it is important to have consistency among the members of the Review Staff in evaluating cases and to have a procedure in place to evaluate consistency.

Consistency checks are performed by having Reviewers independently evaluate the same case and comparing the consistency of the ratings for each Attribute and the selection of Reason Codes.

Group meetings are held by the AQMS Review Staff to discuss specific aspects of the consistency reviews and methods to ensure consistency in the review of cases.

Each year AQMS issues a report containing the statistical findings for Non-Collection case reviews and for Collection case reviews. Recommendations for each Appeals Organizational Function are included in the report.

The AQMS Reviewer’s Guides should be used when interpreting or analyzing the AQMS reports. The AQMS Reviewer’s Guides, check sheets, and the AQMS Reports are posted on the AQMS web site.

This report lists cases selected for AQMS review according to parameters selected by the user. The reports are created for the prior work day, prior work week, prior work month, or a specific period identified by entering a beginning and ending report date.

This is an optional report available for use as determined on a local basis.