3 Important Updates to Evaluation Reports and the CCC Milestone Review

Evaluation Reports

Beta No More

We are excited to announce that the new ‘beta’ Evaluation Reports will finally function as the primary reports within the software. The wait for a new and more valuable solution has been long, but once this next step is taken in the coming weeks, will have been well worth it. Old Evaluation Reports will continue to be phased out of the software. In a few months, they will no longer exist in New Innovations.

This next step in the transition will also allow all end users to see the new Evaluation Reports. If you believe this will cause them any confusion, please let them know ahead of time. An update notice will also be posted within the New Innovations Message Box.

Thank you to all who provided feedback as we made the journey through this transition!


Comment Box Grade Scale Mapping

We are pleased to announce that programs utilizing Comment Box Grade Scales on their evaluation forms are now able to map these comments to specific Subcompetencies. This update allows you to gather better faculty feedback and to provide stronger educational rationale for the milestone scores assigned to your residents.

Has Not Achieved Level 1

For those of you whose milestones include a “has not achieved level 1” option, we are pleased to announce that this designation will now be reflected on your evaluation forms. The update doesn’t require that you take any additional steps. All existing evaluations that include a Subcompetency grid will automatically include the “has not achieved level 1” option and thereby match your specialty’s published milestone format.

In addition to changing the way the Subcompetencies display, you now have the option to factor the “has not achieved level 1” responses as a zero value when calculating the average response.

Want to know more about these CCC Updates? Read More Below

As you probably know, very few specialties have published their milestones in a five-tier, Dreyfus and Dreyfus progression that includes multiple defining characteristics per level. Variations range from using a 0-4 scale, adding Critical Deficiencies, skipping Dreyfus and Dreyfus altogether and targeting “Ready for unsupervised practice,” to including a single descriptive narrative per level.

One additional variation employed by numerous specialties was to augment the five-level model and include a sixth level used to indicate when a resident hadn’t even reached competence in the first level; while there are various phrases used to encapsulate this concept of deficiency, we will use “Has not yet achieved Level 1” to represent them all. Historically New Innovation’s evaluations didn’t explicitly handle this scenario and, rather, grouped the “Has not achieved Level 1” concept under a generic “N/A” option. With our upcoming release, we are now explicitly handling the “Has not yet achieved Level 1” functionality. (NOTE: these updates only apply to specialties whose published milestones include “Has not achieved Level 1” as a column on the Subcompetency grid. It does not include specialties whose published milestones incorporate this concept in a checkbox below the Subcompetency grid)

While the format change will be the first update that you will notice, the “Has not yet achieved level 1” concept can also be incorporated into the numeric scoring system and averaging calculations in the CCC Milestone Reviews as well as in evaluation reports.

Upon release, any score of “Has not achieved Level 1” will be visualized within Evaluation Reports as ‘Level 0.’ Additionally, the Evaluation Report average will also include a 0 in its calculation whenever it encounters a “Has not achieved Level 1” response. The CCC Milestones Reviews includes similar functionality, but the functionality is controlled by a toggle option which excludes the “Has not achieved Level 1” values by default. This ‘toggle switch’ will also impact the Evaluation Reports in the near future. That addition will be announced once we near that time.

A quick note regarding responses that have been logged prior to our update: because assessors only had the option to give their subject a score from the Subcomptency grid or an “N/A,” we cannot implicitly assume that an “N/A” response given implicitly means that the resident has not yet achieved level one. Accordingly, the “N/A” responses submitted prior to this update will not count as a zero nor will they factor into the averaging calculations.

“Has Not Achieved Level 1″ pertains to the following specialties:

ACGME Specialties

1100 Anesthesiology – General 1102 Anesthesiology – Pain Management 1200 Dermatology – General 1250 Emergency Medicine – General 1350 Family Medicine – General 1800 Ophthalmology – General 1925 Anatomic Pathology – General 1950 Anatomic & Clinical Pathology – General 1975 Pathology, Clinical – General 2050 Physical Medicine & Rehabilitation – General 2200 Psychiatry – General 2400 Diagnostic Radiology – General 2650 Urology – General AOA Subspecialties

3102 Anesthesiology – Acute & Chronic Pain Management 3350 Dermatology – General 3351 Dermatology – Dermatopathology 3400 Diagnostic Radiology – General 3450 Emergency Medicine – General 3600 Family Medicine – General 3630 Family Medicine & Osteopathic Manipulative Treatment – General 4700 Ophthalmology – General 5050 Anatomic Pathology – General 5500 Psychiatry – General 5501 Psychiatry – Addiction Psychiatry 5900 Physical & Rehabilitation Medicine – General 6250 Urological Surgery – General 6650 Pain Medicine – General