In late 2002, the U.S. Government enacted a new law that was designed to hold each federal agency accountable to develop, document, and implement an agency-wide information security program, including for its contractors. The Federal Information Security Management Act (FISMA), was one of the first information security laws to require agencies to perform continuous assessments and develop procedures for detecting, reporting, and responding to security incidents.

With limited technological resources available for monitoring and assessing performance over time, however, agencies struggled to adhere to the law’s goals and intent. Ironically, although FISMA’s goal was to improve oversight of security performance, early implementation resulted in annual reviews of document based practices and policies. Large amounts of money were spent bringing in external audit firms to perform these assessments, producing more paper-based reports that, although useful for examining a wide set of criteria, failed to verify the effectiveness of security controls, focusing instead on their existence.

John Streufert, a leading advocate of performance monitoring at the State Department and later at DHS, estimated that by 2009, more than $440 million dollars per year was being spent on these paper-based assessments, with findings and recommendations becoming out of date before they could be implemented. Clearly, this risk assessment methodology was not yielding the outcomes the authors had in mind and in time, agencies began to look for solutions that could actually monitor their networks and provide real-time results.

Thanks to efforts by Streufert and others, it wasn’t long before “continuous monitoring” solutions existed. But, just as with all breakthrough technologies, early attempts at continuous monitoring were limited by high costs, difficult implementations and a lack of staffing resources. As continuous monitoring solutions made it into IT security budgets, organizations and agencies were challenged to make optimal use of tools that required tuning and constant maintenance to show value. False positives and missed signals led many IT teams to feel like they were drinking from a fire hose of data and the value of continuous monitoring in many cases was lost.

However, solutions today offer a number of benefits including easy operationalization, lower costs and reduced resource requirements.

Many options, such as outside-in performance rating solutions, require no hardware or software installation and have been shown to produce immediate results. These tools continuously analyze vast amounts of external data on security behaviors and generate daily ratings for the network being monitored, with alerts and detailed analytics available to identify and remediate security issues.

The ratings are objective measures of security performance, with higher ratings equaling a stronger security posture.

Used in conjunction with other assessment methods, organizations can use ratings to get a more comprehensive view of security posture, especially as they provide ongoing visibility over time instead of being based on a point in time result. The fidelity of “outside-in” assessments is very good when compared to the results of manual questionnaires and assessments because outside-in solutions eliminate some of the bias and confusion that may be seen in personnel responses. Additionally, outside-in performance monitoring can be used to quickly and easily verify effectiveness of controls, not just the existence of policies and procedures that may or may not be properly implemented.

These changes have made continuous performance monitoring and security ratings more appealing to organizations across the commercial and government space.  Organizations have learned that real-time, continuous performance monitoring can allow them to immediately identify and respond to issues and possibly avoid truly catastrophic events, as research has shown a strong correlation between performance ratings and significant breach events. Furthermore, as it becomes easier to monitor internal networks, organizations are beginning to realize the security benefits that can be gained through monitoring vendors and other third parties that are part of the business ecosystem.

Being able to monitor and address third party risk puts us squarely in the realm of next generation continuous monitoring, something many regulators are pushing to see addressed in current risk management strategies.