PCI DSS reporting details to ensure when contracting quarterly CDE tests

This is the second blog in the series focused on PCI DSS, written by an LevelBlue consultant. See the first blog relating to IAM and PCI DSS here.

There are several issues implied in the PCI DSS Standard and its associated Report on Compliance which are rarely addressed in practice. This occurs frequently on penetration and vulnerability test reports that I’ve had to assess.

First off is a methodology which matches the written policies and procedures of the entity seeking the assessment. I frequently see the methodology dictated by the provider, not by the client. As a client you should be asking (possibly different providers) at minimum for:

• Internal and external network vulnerability testing
• Internal and external penetration testing for both application and network layers
• Segmentation testing
• API penetration testing
• Web application vulnerability testing.

Each of these types of tests then needs to be applied to all appropriate in-scope elements of the cardholder data environment (CDE). Generally, you will provide either a list of URLs or a list of IP addresses to the tester. PCI requires that all publicly reachable assets associated with payment pages be submitted for testing. In as much as dynamic IP assignment is very common, especially in Cloud environments, ensure that you are providing a consistent set of addressing information across quarterly testing orders.

Make sure that the Approved Scanning Vendor (ASV) scans are attested scans, both by you and the ASV, and that the scan report shows enough detail to know what was scanned and the results. The first two summary pages are rarely enough for the assessor to work with since they may give a quantity of assets scanned and a quantity found, but no specific information on what was scanned.  

You will need to specify to the testing provider that each of the reports must include

• The tester’s credentials and training record showing appropriate training within the prior 12 months
• If it’s an internal resource performing the tests, explain in the report how they are independent of the organization managing the equipment being tested. (Admins report to CIO, testers report to CTO, for instance, although that could mean testers and developers were in the same organization and not necessarily independent).
• The date of the previous test completion (to prove “at least quarterly” (or annual) execution).
• The dates of the current test execution.
• Dates of remediation testing and exactly what it covered, along with a summary of the new results (just rewriting the old results is very difficult for the Qualified Security Assessor (QSA) to recognize at assessment time).
• All URLS and IP addresses covered, and explain any accommodations made for dynamic DNS assignments such as in the cloud platforms, any removals, or additions to the inventory from the previous test (deprecated platforms, in-maintenance and therefore undiscovered, cluster additions, etc.). Any assets that were under maintenance during the scheduled test must have a test performed on them as soon as they come back online, or they could languish without testing for substantial periods.
• Explain any resources, for which results are included in the report, but are not in fact part of the scope of the CDE and therefore may not need the remediations that an in-scope device does need (e.g., printers on CDE-adjacent networks).
• Explanations of why any issues found, and deemed failures, by the testing are not in fact germane to the overall security posture. (This may be internally generated, rather than part of the test report).
• Suspected and confirmed security issues that arose during the previous year are listed by the tester in the report with a description as to how the testing confirmed that those issues remain adequately remediated. At a minimum, anything addressed by the Critical Response Team should be included here.
• Any additional methodology to confirm the PCI requirements (especially for segmentation, and how the testing covered all segmentation methods in use).

In future PCI DSS 4.0 assessments, the testers must also prove that their test tools were up to date and capable of mimicking all current and emerging attacks. This does not mean another 100 pages of plugin revisions that a QSA cannot practically compare to anything. A new paradigm for test and system-under-test component revision level validation will have to be developed within the testing industry.

Credentialed internal vulnerability scans are also required by PCI DSS 4.0 requirement 11.3.1.2. This requires creation of the role(s) and privilege(s) to be assigned to the test userID, including a sufficient level of privilege to provide meaningful testing without giving the test super-user capabilities, per requirement 7. Management authorization to enable the accounts created for testing, and management validation of the role and of the credentials every six months.. Requirement 8 controls also apply to the credentials created for testing. These include, but are not limited to, 12-character minimum passwords, unique passwords, monitoring of the activity of the associated userID(s), and disabling the account(s) when not in use.