What is security assessment and monitoring?

Answer

Security assessment and monitoring are continuous processes for evaluating control effectiveness, identifying vulnerabilities, detecting security events, and verifying compliance with requirements.

CPCSC requires organizations to not only implement security controls but also assess whether they work as intended and monitor for security issues. Understanding security assessment and monitoring requirements helps executives establish ongoing security validation programs that provide assurance controls remain effective as threats and systems evolve.

Why Assessment and Monitoring Matter

Implementing security controls is necessary but insufficient—organizations must verify controls actually work and continue working over time. Control failures can occur through misconfiguration, incomplete implementation, software bugs, incompatibility with other systems, or degradation over time.

Threats evolve as adversaries develop new attack techniques, vulnerabilities are discovered in software, and organizational systems and risks change. Compliance drift happens as changes accumulate that inadvertently weaken security—new systems are added without applying security standards, configurations change through normal operations, and personnel turnover leads to security knowledge loss.

Unknown vulnerabilities exist in all complex systems and software—assessment discovers them before adversaries exploit them. Security incidents must be detected promptly to limit damage through rapid response and containment.

For CPCSC Level 2 certification, external assessors evaluate not just whether security plans document appropriate controls, but whether controls are implemented and effective—this requires ongoing internal assessment and monitoring that provides evidence of control effectiveness.

ITSP.10.171 Security Assessment and Monitoring Requirements

The Security Assessment and Monitoring family includes multiple requirements that organizations must address:

• Develop and implement plans to assess security control effectiveness, conduct assessments at defined frequency, after significant changes to systems, or when security incidents occur
• Develop and implement plans to monitor security control effectiveness continuously or at defined frequency
• Provide assessment and monitoring results to organizational personnel or roles who can take appropriate action based on findings
• Employ independent assessors or assessment teams to conduct security control assessments—independence prevents conflict of interest where implementers assess their own work
• Develop and implement plans of action and milestones (POA&M) to document planned remedial actions to correct deficiencies identified during assessments and reduce or eliminate vulnerabilities
• Monitor remedial actions until deficiencies are corrected or risks are accepted

These requirements establish expectation of continuous security validation, not one-time assessment—security assessment is ongoing program, not project.

Types of Security Assessments

Organizations employ multiple assessment types to comprehensively evaluate security:

• Security control assessments systematically evaluate whether each ITSP.10.171 control is implemented and effective using assessment procedures from NIST SP 800-171A or similar frameworks
• Vulnerability assessments use automated scanning tools to identify known vulnerabilities in systems, applications, databases, and network devices based on current vulnerability databases
• Penetration testing simulates adversary attacks to identify exploitable vulnerabilities and evaluate defensive capabilities—testers actively attempt to breach security controls
• Configuration assessments verify systems are configured according to security baselines and industry hardening guides
• Code reviews analyze application source code or binaries for security vulnerabilities
• Architecture reviews evaluate security design of systems and networks to identify structural weaknesses
• Physical security assessments evaluate facility security, access controls, surveillance, and physical protections

Each assessment type provides different perspectives—comprehensive security validation uses multiple types rather than relying on single approach. Organizations should conduct assessments proportional to risk, with systems handling specified information receiving more frequent and thorough assessment than general-purpose systems.

Security Monitoring Approaches

Continuous monitoring provides ongoing visibility into security status between formal assessments. Organizations can implement several monitoring approaches:

• Security Information and Event Management (SIEM) systems aggregate and analyze logs from throughout the environment, correlate events to detect suspicious patterns, generate alerts for security incidents, and provide dashboards showing security status
• Security Operations Center (SOC) staffed by security analysts monitors SIEM and other security tools, investigates alerts, responds to incidents, and conducts threat hunting to proactively search for threats
• Automated compliance monitoring uses tools to continuously scan configurations, access permissions, and security settings to verify compliance with baselines and policies—deviations trigger alerts for investigation
• Network security monitoring analyzes network traffic for indicators of compromise, command-and-control communications, data exfiltration, or lateral movement by adversaries
• Endpoint detection and response (EDR) monitors endpoint systems for malicious process execution, suspicious file changes, or adversary techniques
• Vulnerability management continuously scans for new vulnerabilities and tracks remediation of identified issues
• File integrity monitoring detects unauthorized changes to critical system files or configurations

Monitoring provides real-time or near-real-time detection versus assessment's periodic evaluation—both are necessary for comprehensive security visibility.

Independent Assessment Requirements

ITSP.10.171 requires employing independent assessors to conduct security control assessments. Independence means assessors are not directly involved in implementing or operating the controls they assess—having developers assess their own code or administrators assess systems they manage creates conflict of interest and reduces objectivity.

Independence can be achieved through several approaches:

• Separate internal teams where organization has dedicated assessment team separate from implementation teams
• External assessors using third-party consulting firms or assessment organizations
• Cross-team assessment where team members assess other teams' work rather than their own

For CPCSC Level 2 certification, external assessment by accredited certification bodies is required every three years—these assessors are independent by definition. Between formal Level 2 assessments, internal assessment by independent teams or external consultants satisfies ongoing assessment requirements.

Independence improves assessment objectivity, increases credibility of results, identifies issues that implementers might miss due to familiarity blindness, and provides external perspective on security maturity. Organizations should clearly define who has assessment responsibilities and ensure appropriate separation from implementation and operations.

Vulnerability Management Process

Structured vulnerability management identifies and remediates security weaknesses systematically. The process includes several key phases:

Discovery uses vulnerability scanners to identify known vulnerabilities in systems, manual code review to find application vulnerabilities, penetration testing to discover exploitable issues, and threat intelligence about new vulnerabilities affecting technologies used.

Prioritization rates vulnerabilities by severity based on several factors:

• CVSS scores rating technical severity
• Exploitability assessing whether exploits exist and attacks are observed
• Asset criticality considering sensitivity of information on affected systems
• Exposure determining whether vulnerabilities are accessible to attackers

Remediation plans address vulnerabilities through multiple approaches:

• Patching to apply security updates from vendors
• Configuration changes to implement secure settings or workarounds
• Architectural changes to isolate vulnerable systems
• Compensating controls to mitigate risk when patching isn't immediately feasible
• Acceptance of risk for low-severity vulnerabilities in low-criticality systems after management approval

Verification confirms remediation was effective by rescanning or retesting. Tracking monitors status of all vulnerabilities from discovery through remediation using vulnerability management platforms or ticketing systems.

Metrics measure vulnerability management effectiveness through mean time to remediate vulnerabilities by severity, percentage of systems current on security patches, number of critical vulnerabilities open, and scan coverage. Organizations should define maximum remediation timeframes for vulnerabilities by severity—for example, critical vulnerabilities affecting specified information systems might require remediation within 15 days while low-severity issues allow 90 days.

Plans of Action and Milestones (POA&M)

POA&M documents formally track security deficiencies and remediation plans. Each POA&M entry includes several key components:

• Deficiency description including what control is deficient, what the gap is, what risk it creates, and how it was discovered
• Risk rating categorizes severity and urgency
• Remediation plan specifies what corrective action will be taken, who is responsible, what resources are required, and what dependencies exist
• Milestones establish timeline with target dates for remediation phases and final completion
• Status tracking monitors progress including current status, delays or impediments, and resource needs
• Risk mitigation documents interim compensating controls applied while remediation is in progress to reduce risk
• Acceptance workflow routes POA&M through appropriate approvals—senior management must accept risks for items requiring extended remediation periods
• Verification records how remediation effectiveness will be verified upon completion

Organizations maintain comprehensive POA&M tracking systems showing all deficiencies, remediation progress, and risk status. During CPCSC Level 2 assessments, assessors review POA&M to evaluate whether organizations identify deficiencies, take them seriously, and remediate them systematically rather than allowing deficiencies to accumulate indefinitely.

Well-maintained POA&M demonstrates mature security program even when deficiencies exist—no organization has perfect security, but good organizations track and address issues systematically.

Security Metrics and Reporting

Effective assessment and monitoring programs measure security and report to management. Organizations should track several types of metrics:

Technical metrics include:

• Number and severity of vulnerabilities discovered and remediated
• Percentage of systems current on patches
• Security incident counts and trends
• Mean time to detect and respond to incidents
• Compliance rates with security baselines
• Security control effectiveness scores

Operational metrics cover assessment completion rates, remediation timelines, POA&M aging, and resource allocation.

Risk metrics quantify residual risk levels, risk trends over time, and risk concentration by system or business unit.

Compliance metrics track compliance with ITSP.10.171 requirements, contractual obligations, and regulatory requirements.

Maturity metrics assess security program maturity against frameworks like CMMC or NIST Cybersecurity Framework.

Reporting provides several levels of visibility:

• Security dashboard for leadership showing high-level security status
• Detailed reports to security teams covering technical findings and recommendations
• Executive briefings highlighting critical issues, trends, and resource needs

Metrics should be actionable—measuring things that leadership can use to make decisions rather than interesting but irrelevant statistics. Regular reporting (monthly or quarterly) keeps security visible to management and enables data-driven resource allocation and prioritization.

Continuous Monitoring vs. Periodic Assessment

Organizations should balance continuous monitoring with periodic assessment. Each approach has distinct advantages and limitations.

Continuous monitoring provides real-time visibility, early detection of incidents and configuration drift, automated efficiency, and ongoing compliance evidence. However, it may focus on known issues rather than discovering new vulnerabilities, produce alert fatigue if not properly tuned, miss issues that automated tools don't detect, and create false confidence if monitoring coverage has gaps.

Periodic assessment through comprehensive manual evaluation discovers issues monitoring misses, provides independent validation, assesses controls holistically rather than technically, and satisfies compliance requirements for formal assessments. However, assessments are point-in-time snapshots that can miss changes between assessments, require significant resources and time, and may not detect active attacks occurring between assessments.

Best practices combine both approaches—continuous monitoring for ongoing visibility and rapid incident detection, with periodic comprehensive assessments (annually or after significant changes) to validate control effectiveness holistically. Together, they provide defense-in-depth visibility that neither alone can achieve.

Third-Party Assessment Services

Many organizations engage external assessment services to supplement internal capabilities. Several types of services are available:

• Managed Security Service Providers (MSSPs) offer continuous security monitoring and SOC services, providing 24/7 coverage without requiring internal staffing
• Vulnerability assessment services conduct regular automated and manual vulnerability testing
• Penetration testing firms provide independent security testing simulating adversary attacks
• Security audit firms assess compliance with standards and provide independent validation

Benefits include accessing specialized expertise, providing independent perspectives, augmenting limited internal resources, satisfying requirements for independent assessment, and leveraging economies of scale for expensive tools and skilled personnel.

Challenges include cost of external services, potential loss of security knowledge to external providers rather than building internal capability, dependency on external providers, and security risks from granting third parties access to systems.

Organizations should evaluate whether to build internal assessment capabilities, use external providers, or hybrid approaches based on their size, budgets, internal expertise, and assessment requirements. For CPCSC Level 2, external accredited assessment is mandatory every three years—organizations should prepare through internal assessments that identify and remediate issues before formal external assessment.

Assessment and Monitoring Challenges

Organizations face practical challenges implementing effective programs:

• Tool complexity and cost as comprehensive assessment and monitoring requires multiple expensive tools requiring specialized expertise
• Alert volume and false positives from monitoring systems can overwhelm security teams, causing real threats to be missed in noise—tuning is essential but time-consuming
• Skills shortage as effective security assessment and monitoring requires skilled personnel, but cybersecurity talent shortage makes recruiting and retaining analysts difficult
• Coverage gaps occur when some systems, applications, or security domains lack adequate assessment or monitoring
• Integration challenges arise when assessment and monitoring tools don't integrate well with each other or with IT systems
• Resource constraints particularly for smaller organizations make comprehensive programs difficult to achieve

Organizations should develop programs appropriate to their size and resources, focusing initially on highest-risk systems and gradually expanding coverage. Managed services, automation, and phased implementation help organizations with limited resources achieve effective assessment and monitoring.

Learn More

Additional resources on security assessment and monitoring:

• Protecting Specified Information (ITSP.10.171)
• Canadian Program for Cyber Security Certification