SMotW #10: Unsecured access points

Security Metric of the Week #10: Number of unsecured access points

As worded, this candidate metric potentially involves simply counting how many access points are unsecured.  In practice, we would have to define both "access points" and "unsecured" to avoid significant variations (errors) in the numbers depending on who was doing the counting.

Depending on how broadly or narrowly it is interpreted, "access points" might mean any of the following, if not something completely different:

• WiFi Access Points, specifically; 
• Legitimate/authorized points of access into/out of the corporate network e.g. routers, modems, gateways, WiFi Access Points, Bluetooth connections etc.;
• Both legitimate/authorized and illegitimate/unauthorized points of access into/out of the corporate network - assuming we can find and identify them as such;
• Designated security/access control points between network segments or networks e.g. firewalls and authentication/access control gateways;
• Physical points of access to/from the organization's buildings or sites - again both legitimate/authorized and illegitimate/unauthorized (e.g. unlocked or vulnerable windows, service ducts, sewers), assuming we can identify these too;
• Points of contact and communications between the organization's systems, processes and people and the outside world e.g. telephones, social media, email, face-to-face meetings, post ... 

Similarly, absolutely any access point might be deemed "unsecured" (more likely, "insecure") by a professionally-paranoid risk-averse security person who can envisage particular scenarios or modes of attack/compromise that would defeat whatever  controls are in place, or who knows through experience that controls sometimes fail in service.  Conversely, a non-security-professional might claim that every single access point is "secured" since he/she personally can't easily bypass/defeat it.  This kind of discrepancy could be resolved by some sort of rational decision process according to an assessment of the risks and the strength of the controls.  However, if the metric is used by management specifically to attempt to drive through security improvements at the access points, the people making the improvements tend to be the very same people who are assessing risks and controls, hence the metric would lose its objectivity and teeth.  Defining security standards for access points might help address the issue, and in fact that might be a useful spin-off benefit of using such a metric.

P	R	A	G	M	A	T	I	C	Score
95	80	90	70	85	77	45	75	55	75%

The PRAGMATIC score for this metric worked out at a very respectable 75% in the imaginary context of Acme Enterprises Inc.  It scored very well for Predictiveness (given that access control is a core part of security, so weaknesses in access control undermine most other controls) and Actionability (it is pretty obvious what needs to be done to improve the measurements: secure those vulnerable access points!).  The lowest-scoring element was Cost at 55% since defining security standards, locating potential access points and assessing them against the standards would undoubtedly be a labor-intensive process.

In the course of discussing the scoring, we considered possible variants of the metric itself and variations in the measurement process.  For instance, there might be advantages in reporting the proportion of access points that are unsecured: without more information about the total number of access points, recipients can't tell whether, say, 87 is a good number for the simple count version of this metric, whereas 87% is more Meaningful.  That straightforward change to the metric has a minor impact on the Cost since someone would have to count and/or estimate the total number of access points, and periodically revisit the calculation as things change.  We suspect Acme's management would like it too.

Furthermore, for some purposes, it would be worthwhile knowing just how insecure are the unsecured access points, implying a rating scheme, perhaps something as crude as a read/amber/green rating for the security of each access point identified, maybe with a clear (uncolored) rating for those that have yet to be assessed.  Assessments that involve penetration testing, IT audits or professional security reviews might well generate the additional information anyway in order to prioritize the follow-up activities needed to secure the unsecured.  In short, although the metric's Cost would increase, so would its value, hence it might still rate 55% (the PRAGMATIC parameter we call Cost for short is in reality Cost-effectiveness).

The previous two paragraphs demonstrate how the PRAGMATIC approach is more than simply a static rating or ranking scheme for metrics: it facilitates and encourages creative discussion and improvement of metrics that are under consideration, focusing most attention on whichever aspects hold back the overall PRAGMATIC score.  Given the specific situation of this candidate metric, it would be feasible, for instance, to trade-off Accuracy and precision to improve both the Cost and Timeliness scores by settling for rough but ideally informed and reasonable estimates of the proportions of secured versus unsecured access points instead of actual counts.  That might be a perfectly acceptable compromise for Acme's management.  The PRAGMATIC method provides the framework within which to frame this kind of sensible discussion.