table of content

Part 1 Building the foundation

1 Introducing cybersecurity metrics

1.1 Understanding metrics

1.2 Significance of metrics in cybersecurity

1.3 Traditional vs. innovative metrics

1.4 The changing landscape of cybersecurity

1.5 The importance of frameworks in cybersecurity

1.5.1 HITRUST framework

1.5.2 Center for Internet Security CIS Top 18 framework

1.5.3 NIST Cybersecurity Framework v2.0

1.6 AI and predictive cybersecurity metrics

1.7 Defending against AI

2 Cybersecurity analytics toolkit

2.1 Tool selection

2.1.1 Key tool selection factors

2.2 Dashboard development

2.2.1 Choosing the right metrics

2.2.2 Knowledge points

2.2.3 Dashboard example

2.3 Statistical analysis

2.3.1 Trend analysis

2.3.2 Correlation analysis

2.3.3 Probability distribution

2.4 Integrated security analytics environment

2.4.1 Integrated security analytics environment exercise

2.5 Continuous improvement and iteration

2.5.1 Iteration techniques

2.5.2 Continuous improvement cycle

3 Implementing a security metrics program

3.1 Introducing metrics program design

3.1.1 Key components of a security metrics program

3.2 METRICS methodology

3.2.1 Measure

3.2.2 Evaluate

3.2.3 Threshold

3.2.4 Report

3.2.5 Improve

3.2.6 Communicate

3.2.7 Sustain

3.3 Using the METRICS methodology example

3.3.1 Step 1: Measure

3.3.2 Step 2: Evaluate

3.3.3 Step 3: Threshold setup

3.3.4 Step 4: Report

3.3.5 Step 5: Improve

3.3.6 Step 6: Communicate

3.3.7 Step 7: Sustain

3.3.8 Exercise summary

3.4 Building a metrics dashboard

3.4.1 Different dashboards for different stakeholders

3.4.2 Security dashboard example

3.5 Tools and technologies for metrics collection

3.5.1 Open source tools for metrics collection

3.5.2 Commercial tools for metrics collection

3.5.3 Automation and integration

3.5.4 Scripting for metrics collection

3.6 Common pitfalls in metrics programs

3.6.1 Pitfall 1: Overreliance on quantitative metrics

3.6.2 Pitfall 2: Misalignment with business goals

3.6.3 Pitfall 3: Stagnation and complacency

4 Integrating metrics into business strategy

4.1 Business alignment

4.1.1 Business-aligned security metrics example

4.1.2 Mapping metrics to business performance

4.1.3 Supporting innovation with metrics

4.2 Security metrics alignment with business strategy exercise

4.2.1 Step 1: Understanding MediHealth’s business objectives

4.2.2 Step 2: Selecting key metrics

4.2.3 Step 3: Mapping metrics to business goals

4.2.4 Step 4: Measuring metrics

4.2.5 Step 5: Communicating the Metrics

4.3 Security metric reporting

4.3.1 Presenting metrics in executive reports

4.3.2 Demonstrating return on investment

4.3.3 Communication strategies

4.3.4 Metrics communication exercise

Part 2 The metrics that matter

5 Establishing the foundation

5.1 Governance

5.2 Organizational context

5.2.1 Understanding the differences between mission, vision, and values

5.2.2 Strategic objectives

5.2.3 Organizational metrics

5.2.4 Organizational metrics exercise

5.3 Risk management strategy

5.3.1 Risk mitigation

5.3.2 Risk management metrics

5.3.3 Risk management metrics exercise

5.4 Roles, responsibilities, and authorities

5.4.1 Roles, responsibilities, and authorities metrics

5.4.2 Roles and responsibilities metrics exercise

5.5 Policy, processes, and procedures

5.5.1 Policy, processes, and procedures metrics

5.5.2 Policy, processes, and procedures metrics exercise

5.6 Oversight

5.6.1 Governance structure

5.6.2 Simulated governance structure

5.6.3 Governance exercise

5.6.4 Oversight metrics

5.6.5 Oversight metrics exercise

5.7 Cybersecurity supply chain risk management

5.7.1 Supply chain risk management practices

5.7.2 Cybersecurity supply chain risk management metrics

5.7.3 Supply chain risk management metrics exercise

5.8 Governance metrics

5.9 Answer to exercise 5.6.3

6 Foundations of cyber risk

6.1 Identify

6.2 Asset management

6.2.1 Asset management metrics

6.2.2 Asset management metrics exercise

6.3 Risk assessments

6.3.1 NIST 800-30: Guide for conducting risk assessments

6.3.2 Risk assessment metrics

6.3.3 Risk assessment metrics exercise

6.4 Continuous improvement

6.4.1 Continuous improvement metrics

6.5 Identity metrics exercise

7 Protecting your assets

7.1 Identity management, authentication, and access control

7.1.1 Authentication

7.1.2 Access control

7.1.3 Identity and credential management

7.1.4 Context-based identity proofing and credential binding

7.1.5 User, service, and hardware authentication

7.1.6 Identity assertion, protection, conveyance, and verification

7.1.7 Access permissions management

7.1.8 Physical access controls

7.1.9 Types of security controls

7.1.10 Identity management, authentication, and access control metrics

7.1.11 Identity and access management dashboard exercise

7.2 Awareness and training

7.2.1 Awareness and training metrics

7.2.2 Awareness and training metrics exercise

7.3 Data security

7.3.1 Data at rest

7.3.2 Data in transit

7.3.3 Data in use

7.3.4 Data backup and recovery

7.3.5 Data security metrics

7.3.6 Data security metrics exercise

7.4 Configuration management

7.4.1 Software maintenance

7.4.2 Hardware maintenance

7.4.3 Log maintenance

7.4.4 Unauthorized software prevention

7.4.5 Secure software development

7.4.6 Platform metrics

7.4.7 Platform security metrics exercise

7.5 SDLC testing exercise

7.6 Technology infrastructure resilience

7.6.1 Networks

7.6.2 Environmental threats

7.6.3 Resilience mechanisms

7.6.4 Resource capacity

7.7 Protection metrics

7.8 Answer to exercise from section 7.5

8 Continuous threat detection

8.1 Implementing continuous threat monitoring systems

8.1.1 Defining strategy

8.1.2 Establishing architecture, implementing data collection, and analysis

8.1.3 Responding to findings

8.2 Open source alternative to continuous threat detection

8.3 Continuous monitoring metrics

8.3.1 Continuous monitoring metrics exercise

8.4 Understanding ATLAS

8.4.1 ATLAS methodology

8.4.2 Review and update

8.4.3 ATLAS benefits

8.5 Determining valid threat detections

8.5.1 False rejection rate

8.5.2 False acceptance rate

8.5.3 Equal error rate

8.5.4 FRR, FAR, and EER metrics

8.5.5 FRR, FAR, and ERR exercise

8.6 Adverse event analysis

8.6.1 Adverse event analysis metrics

8.6.2 Adverse event analysis metrics exercise

8.7 Alternatives to ATLAS and Wazuh

9 Incident management and recovery

9.1 Incident management

9.2 Planning and preparation

9.3 Testing an IRP

9.3.1 Tabletop exercise example

9.3.2 Tabletop exercise metrics

9.3.3 Table for tabletop exercise evaluation sample

9.4 Detection and documentation

9.4.1 Three-tier severity model

9.4.2 Five-tier severity model

9.4.3 Changing severity over time

9.4.4 Practical example of incident documentation

9.4.5 Chain of custody

9.4.6 Incident management metrics

9.4.7 Incident management metrics exercise

9.5 Incident triage and analysis

9.5.1 Case study: Financial institution data breach

9.5.2 Incident metrics

9.5.3 Incident metrics exercise

9.5.4 Incident metrics table

9.6 Notification and communication

9.7 Containing validated incidents

9.8 Evidence gathering and forensic analysis

9.9 Eradication of incidents

9.9.1 Incident response metrics

9.10 Recovering to operational status

9.11 Post-incident and lessons learned

9.11.1 Incident reporting and communication metrics

9.11.2 Incident reporting and communication metrics exercise

Part 3 Beyond the basics: Advanced analytics, machine learning, and AI

10 Advanced cybersecurity metrics

10.1 Risk exposure and predictive analysis

10.2 Risk exposure and predictive metrics

10.2.1 Risk exposure index

10.2.2 Predictive threat index

10.2.3 Risk exposure and predictive metrics exercise

10.2.4 Using open source tools to calculate REI and PTI

10.2.5 How cybersecurity teams work together with open source solutions

10.3 Advanced threat detection

10.3.1 Anomaly detection

10.3.2 Time to predict

10.3.3 Dynamic risk scoring

10.3.4 Advanced threat detection metric exercise

10.4 Effectiveness of AI in cybersecurity

10.4.1 False-positive suppression rate

10.4.2 AI-based decision accuracy rate

10.4.3 Effectiveness of AI in cybersecurity exercise

10.5 Cloud and network threat management

10.5.1 Cloud threat detection

10.5.2 Cloud threat detection accuracy

10.5.3 Attack surface index

10.5.4 Cloud and network threat management metric exercise

10.6 Incident management

10.6.1 Threat hunting

10.6.2 Incident impact estimation

10.6.3 Incident management metric exercise

10.7 Continuous improvement

10.7.1 Risk mitigation

10.7.2 Time to predict

10.7.3 Continuous improvement metric exercise

11 Advanced statistical analysis

11.1 Continuous improvement with statistical metrics

11.1.1 Key concepts for continuous improvement

11.1.2 Free resources for statistical analysis

11.2 Using statistical metrics for continuous improvement

11.3 Implementing a feedback loop

11.4 Finding hidden relationships in data

11.5 Using moving averages to track trends

11.5.1 Moving averages example

11.5.2 Detailed description of the plots

11.6 Grouping similar events for better insights

11.7 Forecasting cybersecurity trends

11.7.1 ARIMA in action

11.8 Bayesian inference

11.8.1 Interpreting the results

11.9 Statistical models for vulnerability management

11.9.1 Interpreting the results

12 Advanced machine learning analysis

12.1 Code requirements

12.1.1 What is AI

12.2 DBSCAN for threat detection

12.2.1 DBSCAN example

12.3 Random forest and SVM for threat detection

12.3.1 Random forest and SVM exercise

12.3.2 Isolation forest and autoencoders exercise

12.4 Natural language processing

12.4.1 NLP exercise

12.5 Deep learning

12.5.1 Deep learning exercise

12.6 Reinforcement learning

12.6.1 Reinforcement learning example in Python code

13 Generative AI in cybersecurity metrics

13.1 Understanding generative AI

13.2 Open source generative AI alternatives

13.2.1 LM Studio

13.2.2 Ollama

13.2.3 LM Studio vs. Ollama

13.2.4 Why we chose LM Studio for this chapter

13.3 A note on LLMs

13.4 Prompt engineering

13.4.1 Why prompt engineering matters

13.5 Prompt engineering in cybersecurity

13.5.1 Basics of prompt engineering

13.5.2 Open source vs. cloud API integration

13.6 Generating and analyzing cybersecurity data

13.6.1 Generating synthetic cybersecurity data

13.6.2 Analyzing the output

13.7 Enhancing reporting and visualization

13.7.1 Failed login reasons reporting

13.8 Automating incident response and trend analysis

13.8.1 Setting the context

13.9 Exploring generative AI

13.9.1 Learning resources

13.9.2 Inspiration to experiment

13.9.3 Your next steps

13.10 Limitations and ethical considerations

Overview

7 Protecting your assets

This chapter presents a layered, data-driven approach to protecting organizational assets, uniting policy, people, and technology to preserve confidentiality, integrity, and availability. It centers on identity and access management as the gatekeeper: establishing lifecycle-driven identities, strong authentication (including MFA, biometrics, and context-aware proofing), and granular authorization through models like RBAC and ABAC under the principle of least privilege. SSO and federation extend trusted access across domains, while service and hardware authentication further harden inter-service communications and device trust. Technical safeguards are complemented by physical controls, and all protections are organized as preventive, detective, and corrective measures, with metrics embedded throughout to guide risk decisions and balance false positives and negatives.

The chapter then traces data security across its lifecycle—at rest, in transit, and in use—emphasizing modern encryption practices (symmetric/asymmetric), current TLS versions, and robust hashing (e.g., SHA‑256 and above) while retiring obsolete methods. It highlights backup and recovery as critical defenses against ransomware and extortion, requiring encrypted, tested backups tied to clear RPO and RTO objectives. Equally vital is the human layer: role-relevant cybersecurity training and frequent awareness touchpoints that build a security-minded culture. Access permission governance, regular audits, and aligned policies keep privileges right-sized as roles and business needs evolve.

Operational resilience is built through disciplined configuration management, timely patching, secure hardware lifecycle practices, comprehensive logging, and a secure SDLC that integrates threat modeling, secure coding, SAST/DAST, penetration testing, code signing, hardening, and continuous monitoring/SOAR. Network resilience combines layered defenses, segmentation, and encryption with environmental protections, redundancy, failover, load balancing, and capacity planning—often leveraging elastic cloud resources—while accounting for hybrid cloud and third‑party risks through testing and tabletop exercises. Throughout, the chapter stresses metrics and continuous improvement: tracking IAM efficiency, training outcomes, encryption and backup coverage, baseline drift, and remediation speed to sustain a measurable, adaptive security posture.

An example of a Single Sign-On (SSO) process.

Federated identities require a user to be authenticated at their home organization, after which separate security domains use SAML to verify that they can trust and accept user identities from one another.

Security controls can be categorized by their type as well as their function.

The left shows Acme Corp’s identity and access management metrics before IAM improvements. On the right are the identity and access management metrics after IAM improvements.

The left side shows the review of training and awareness metrics before implementing a comprehensive training program. The right side shows the improvements after implementing the training program.

On the left shows the Secure Tech data security dashboard before implementing data protection practices. On the right is the data security dashboard after implementing data protection practices.

An overview of how security is integrated into each phase of the SDLC, highlighting critical security tasks associated with planning, development, testing, deployment, and maintenance stages.

The left side shows the SecureNet Inc. platform security dashboard before implementing the new security monitoring strategy. After implementing the new platform security monitoring strategy, the platform security dashboard is on the right.

Exercise: Match the testing method with the SDLC phase.

Answers to exercise 7.4.

Summary

Identity and access metrics measure how well users are managed and authenticated.
Authentication success and failure rates highlight security and user access reliability.
Training metrics track the scope and effectiveness of cybersecurity education programs.
Data security is evaluated by tracking encryption use and incident response success.
Metrics for platform security assess the maintenance and response to software vulnerabilities.
Resilience in infrastructure is measured by incident responses and system availability.

FAQ

What are Identity Management (IdM) and Identity Lifecycle Management (ILM), and why do they matter?

IdM is the process and technology that verifies identities, authenticates users, and authorizes access to systems and data. ILM governs each identity from creation to modification to offboarding so the right people have the right access at the right time—and lose it when they shouldn’t have it. Centralized IdM enforces consistent policies, scales with org changes, and provides a single source of truth for audits and compliance.

What authentication methods are available, and how should we choose among them?

Common options include: - Passwords: Familiar but vulnerable to reuse, phishing, and brute force; often need reinforcement. - Tokens/MFA/2FA: Codes from hardware fobs or apps; context-aware MFA adapts to risk (device, location). - Biometrics: Strong identity binding (fingerprint/face) but requires careful privacy handling. - Passwordless: Aligns with Zero Trust but slower to adopt in many orgs. Choice depends on asset risk, usability needs, and operational resources. Also authenticate services (tokens/certificates for APIs) and hardware (device allowlists, TPM-backed checks) to extend trust beyond users.

How do DAC, MAC, RBAC, and ABAC differ, and when should each be used?

- DAC (Discretionary): Owners grant permissions; flexible for collaboration but prone to human error. - MAC (Mandatory): Centralized labels and policies; highly secure for government/military contexts. - RBAC (Role-Based): Permissions tied to job roles; scalable, simpler administration for clear org structures. - ABAC (Attribute-Based): Policies evaluate attributes (user, resource, environment); most granular and dynamic. Pick based on your security requirements, governance model, and need for flexibility and scale.

What is the principle of least privilege, and how do we enforce it?

Least privilege grants only the minimum access needed to perform a task. Enforce it by: - Defining roles and standard access paths (RBAC) and adding context where needed (ABAC). - Reviewing access regularly and logging changes. - Aligning access to documented job attributes and removing access promptly at offboarding.

What is context-based identity proofing and credential binding?

It continuously evaluates signals (time, location, device, behavior) to validate that the current user context matches expectations. Credential binding ties access to known devices or environments so low-risk contexts reduce friction while high-risk contexts trigger additional verification (for example, extra MFA if logging in from a new country).

How do Single Sign-On (SSO) and federated identity work, and when should we use each?

SSO lets a user authenticate once, then reuse a secure token to access multiple services without reentering credentials. Federated identity extends trust across domains (partners) using standards like SAML, so an identity verified in one organization can access resources in another. Use SSO to simplify access within your estate; use federation to enable trusted access across separate organizations or security domains.

Why are physical access controls still critical to cybersecurity?

Physical breaches can lead to theft, vandalism, malware introduction (for example, USB drops), or rogue devices in data centers. Use layered defenses: locks and badges, biometric entry, visitor logs and escorts, cameras, tamper-evident logging of door access, and periodic audits and testing aligned to risk.

How should we protect data at rest, in transit, and in use?

- Data at rest: Encrypt storage, control access, run integrity checks. Use strong crypto and retire weak ciphers. - Data in transit: Use TLS (v1.2+), VPNs where appropriate, and endpoint authentication to prevent eavesdropping/tampering. - Data in use: Apply application-level controls and encryption where feasible, balancing performance and security. Hashing best practices: Use SHA-256/384/512 or SHA-3. Avoid MD5/SHA-1 due to collision weaknesses.

How do backups, RTO, and RPO help counter ransomware and data extortion?

Backups (encrypted in transit and at rest) enable recovery without paying ransom. Test restores regularly. Set: - RTO (Recovery Time Objective): Maximum acceptable time to restore service. - RPO (Recovery Point Objective): Maximum acceptable data loss measured in time between backups. Pair solid backup/recovery with strong preventive controls and monitoring to reduce extortion risk.

Which metrics should we track to know if our protections are working?

- IAM/auth/access: Provision/de-provision time; auth success/failure rates; identity audit issues; unauthorized physical access incidents; mean time to respond to auth anomalies. - Training/awareness: Completion rates, test scores, pre/post improvements, retraining counts, repeat failures, and incident reduction after training. - Data security: Ratio of encrypted to non-encrypted systems, deprecated crypto in use, incident counts, backup/test success rates, RPO/RTO compliance. - Platform/configuration: Deviations from baseline, patch cadence, unauthorized software installs, mean time to remediate misconfigs/update systems, uptime, and resource usage. Incorporate drift detection for real-time baseline enforcement.

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

pdf, ePub, online

$54.99 $32.99

you save $22.00 (40%)

include audio $24.99 $14.99

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

$54.99 $32.99

you save $22.00 (40%)

include audio $24.99 $14.99

eBook

pdf, ePub, online

$54.99 $32.99

you save $22.00 (40%)

include audio $24.99 $14.99

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more