Vulnerability Testing Scope

Audited by Netcraft is an automated vulnerability scanning service which probes Internet-connected networks for security vulnerabilities and configuration errors. The service detects all responding hosts within an Internet address range, and then performs a methodical examination of each available service by performing tests for common misconfigurations and security weaknesses. Key features of the service include:

  • Over 7500 classes of tests (including many more sub-tests); 5 to 10 new tests and advisories added every week.
  • Automated host detection — Netcraft does not need to be informed of every network change.
  • Firewall mapping — shows services unintentionally exposed by maintenance or configuration errors.
  • Safe example exploits are embedded into the reports, where possible, for easy ‘click to test’ self-verification of fixes.
  • Management differential reporting highlights security changes between reports.
  • Descriptive severity grading and categorisation of exploit risk.
  • Online reports linked to clear and concise fix information – web database of fixes and sources of other information.
  • Non-disruptive — Denial-of-service exploits are reported but not executed, and test load is controlled.

Test Scope

The vulnerability scanning service covers all machines in the given internet address ranges from which responses were detected. For each machine detected, the services and characteristics of the machine are analysed.

TCP/IP characteristics ICMP responses and other TCP/IP characteristics of the machine are examined. These are used to report the detected operating system (often including the version) and system uptime where available.
TCP services A table of available TCP services and relevant further information is produced. Netcraft’s tests identify the network service on each port — in particular, standard network services running on non-standard ports are identified and fully tested.
UDP services A table of UDP ports which are believed to be open, and any information obtained from them. Note that due to the design of the UDP protocol, false positives are common in identifying active UDP ports, especially if firewalls are filtering content from these ports. If filtering is in place, our test suite switches to an alternative technique, sending well-formed packets to standard UDP services, and using other means (like RPC portmapper) to find listening services.
Vulnerabilities Our test suite contains tests for a large range of vulnerabilities in standard network services. Where applicable, available network services are tested for the presence of published well-known vulnerabilities.
SSL/TLS Services Netcraft’s tests against standard services, like HTTP, SMTP and IMAP, are all applied against the corresponding SSL/TLS services too. This is particularly important for highly secure services, where the main website may be very simple with only a few pages, but the HTTPS site offers a dynamic web application, using different software and a wider range of server modules. This is also important for sites using IDS systems, as the behaviour they experience over an SSL service may be different to the unwrapped service on the same machine. TLS-wrapped services are also tested.
Web Content When analysing the security of web servers, a web crawl is performed. This process begins with your site’s start page or other specified custom starting points and we then follow links to other pages on your site, limited to a certain depth and number of pages. This analysis is used to find web server technologies used by your site, like active server pages, CGI scripts, etc. If these technologies are discovered, further tests are performed, looking for misconfigurations and vulnerabilities, such as obtaining the source of server side scripts.

Stealth scans and spoofing attacks are not part of this automated test, since they require a more individual approach. Bespoke applications and server side scripts are also not covered by this automated scan. These are best covered by an in-depth web application test.

False Positives

Some vulnerabilities cannot be directly tested without disrupting your server. Denial of service attacks would obviously cause disruption if your system was vulnerable. Buffer overflow attacks are too dangerous to run against a live server. Therefore Netcraft avoids testing such vulnerabilities directly unless it can be done safely.

Netcraft often identifies such vulnerabilities via indirect methods: fingerprinting the operating system, the software installed, and the configuration of that software gives enough information to determine whether the server may be vulnerable to an exploit. If vulnerable software, or software in a vulnerable configuration, is found, then the vulnerability is reported as a “possible” vulnerability. It is also possible, however, that you are using a product that was vulnerable, for which you have applied a patch or configuration change that cannot be detected by our test. In this case the vulnerability will continue to be reported even though you are not vulnerable, because the patch cannot be safely tested. This is called a false positive.

Once a false positive has been investigated, Netcraft provides a facility to mark vulnerabilities that are false positives on your report. On future reports, a mark will appear in the “false positive” column for that vulnerability and the viewer can see when it was investigated and by whom. A count of the number of false positives is shown at the top of the report, just under the number of vulnerabilities – this allows you to see how many vulnerabilities have already been investigated, and how many remain.

The false positive approval process is only available to Audited by Netcraft seal customers. The seal shows the date of the last clean scan and cannot be updated until any false positives have been marked as such.

CVE Names

Netcraft provides CVE references, where possible, for each vulnerability. These allow cross-referencing with other sources of information about the vulnerabilities discovered during our tests.

CVE (Common Vulnerabilities and Exposures) is a dictionary of standardised names for vulnerabilities and other information security exposures, which has been adopted by a large number of organisations throughout the computer security industry. CVE names are often quoted in security advisories.

The CVE name for each vulnerability is included as part of the information for each vulnerability in the report.

Please note that not all vulnerabilities that are tested during a Network Examination will have an exact match to a CVE name. Usually this is because either the CVE entry is too specific or Netcraft’s tests include vulnerabilities for which no CVE entry exists. If there is no matching CVE entry then no corresponding CVE name is given in the report.

Netcraft retrieve new copies of the base CVE database on a regular basis. The version of the CVE database used for any given report is indicated at the end of the report.

CVE and the CVE logo are registered trademarks of The MITRE Corporation.

Audited Scan Interference

In order to give accurate results, the Audited by Netcraft service must be allowed to scan your network without interference from any active protection systems. An active protection system is defined as a system which modifies its behaviour in response to attack traffic.

“Non-attack traffic refers to potentially legitimate network traffic patterns that do not indicate malformed or malicious traffic, whereas attack traffic includes, for example, malicious network traffic patterns or patterns that match known attack signatures, malware, or packets exceeding the maximum permitted IP packet size.”
– PCI Approved Scanning Vendors Program Guide v2

Examples of interference which would cause inaccurate scan results include:

  • Blocking an IP address upon detection of a port scan.
  • Limiting traffic due to a Quality of Service (QoS) policy detecting anomalies in traffic volume (e.g. exceeding a preset threshold for DNS traffic)

Examples of acceptable behaviour that would not affect scan results include:

  • Blocking SQL injections, but allowing non-attack traffic from the same IP address.
  • Firewalls with consistent behaviour (i.e. always block traffic on particular ports and always allow traffic on other ports).

Errors and Omissions

If you find any serious inaccuracies in the report, or in the advisories referenced by the report, then please contact us at security@netcraft.com