Today’s post wraps up the series we’ve been writing with our friends at SurfWatch Labs on applying the Preparedness Cycle to Cyber Risk Management, with a final summary podcast and post to follow next week where I’ll talk to Jeff Peters about some common issues, best practices, and more. The complete blog and series can be accessed from the SurfWatch Labs blog (read parts one, two, three, four, and five a, five b, and five c here). Excerpts from this last post on Evaluating and Improving follow.
For those desiring a “deep dive” into exercise evaluation and improvement planning, review the guidance in the 2013 Homeland Security Exercise and Evaluation Program (HSEEP). That will provide details on the process of developing and conducting evaluation and improvement planning and documentation, addressing ideas such as Exercise Evaluation Guides (EEGs), data collection, after action reporting, and developing an improvement plan and corrective action program. Below, I’d like to share a few ideas for additional consideration.
Do What Works
The HSEEP guidance above provides specific approaches that work. Using well-established 
standards like Core Capabilities and EEGs provide common terms and references, and help promote consistency in evaluations and documentation. All good! However, not every exercise is resourced (nor really requires) the complete HSEEP approach. HSEEP is guidance and should be treated as exactly that. If you want to irritate an exercise pro, tell them you want an “HSEEP-compliant exercise” and watch their eyes roll into the deepest parts of their skull … What is critical is that you plan for evaluations hand-in-hand with training and exercises and that you have a deliberate approach. Your organization may have some specific ways you like to capture and report information or you may need to be mindful of certain sensitivities…
What is most important is that you know what you have available, deliberately plan as part of the training and exercise development process, and ensure evaluation does occur and is documented.
Get Buy-In
… Getting buy-in early and from the right people can save planners (particularly junior personnel) a lot of grief and greatly help support an effective and value-added evaluation. We want to gain buy-in into our approach to the evaluation, as well as to the activities supporting the evaluation and improvement planning… know who you’re going to be putting some focus on and get ahead of potential tensions and flare-ups — but engage them privately before doing so publicly… In both developing the evaluation process and in conducting the evaluation and after action activities, building support and getting others to invest in what you’re doing can grease the process and make it a lot more successful.
Seek Continuous Improvement
One of my favorite books is the classic Animal Farm and like Boxer, the hardworking but rather dim horse in that story, my typical approach to things is to put my head down, block out the noise and tell myself, “I will work harder.” After many years of ugly running and punishing my Achilles, I started cycling about a year ago. Applying my usual approach, I try to muscle through every challenge, which has some utility. But, when I take the time to look at my stats, assess parts of the ride and how I tackled them, compare with previous workouts, and otherwise assess and evaluate my performance, I’m able to better understand how I did and how I can improve. My goal is to keep getting better.
Develop a multi-year plan, establish goals and milestones, plan but be flexible, and seek to continuously improve the readiness and resilience of your organization through effective evaluation, corrections, and improvement planning.
In Animal Farm, Boxer’s valiant efforts end in the care of the “Horse Slaughterer and Glue Boiler,” 
and I’d prefer a smarter, more positive outcome. By properly planning and preparing for my ride evaluation, taking the appropriate amount of time to review, assess, and evaluate my performance, I am able to work towards continuous improvement and hopefully reaching the desired level of physical fitness. Hopefully… The same approach should be applied towards exercises and preparedness broadly…
To read the complete post, continue to SurfWatch Labs: “Preparedness & Cyber Risk Reduction Part Six: Evaluate & Improve”
This series is being written by Andy Jabbour, Gate 15’s Co-Founder and Managing Director. Andy leads Gate 15’s risk management and critical infrastructure operations with focus on Information Sharing, Threat Analysis, Operational Support & Preparedness Activities (Planning, Training & Exercise). Andy has years of experience working with partners across the critical infrastructure and homeland security enterprise to support national security and client business needs.
