Security Testing F.A.Q.

1. Why is certification a good idea?

Even if you already care strongly about security, and diligently test the security of your network, sites, and applications from both internal and external perspectives, few people outside your own organisation will know whether your network is secure or not.

The Audited by Netcraft seal is your opportunity to demonstrate the care and attention that you take in your internet security to your customers. This independent demonstration of your network’s security means that they can use your services with a much greater degree of assurance.

2. Could displaying a seal make me a more likely target for attackers?

The largest numbers of remote compromises come from very widespread attacks by worms or attackers trying a set of pre-made, publicly-available exploits across very large numbers of machines. Typically, these attackers will launch their attack against the whole of the internet, or a large IP address space, rather than a single site, and then review the resulting set of compromised machines.

Attackers targeting specific sites may infer from the seal that your organisation has paid particular attention to security, and is up to date against well-known vulnerabilities, and therefore move onto a target that has a less visible defensive stance instead.

3. Why Netcraft?

Netcraft has an excellent pedigree in security testing and network exploration: Netcraft has been providing network security services, including application and penetration testing, code reviews, and automated security scanning since 1994. Additionally, Netcraft has explored the internet providing research data and analysis on web servers, operating systems, hosting providers, ISPs, TLS certificates, electronic commerce, scripting languages and content technologies on the internet. This gives Netcraft a unique bird’s eye view of what is happening on the internet, and direct access to many of the world’s leading web technology organisations.

Netcraft has a cosmopolitan client list, spread through the UK, the USA, mainland Europe, the Middle East, Asia Pacific and Latin America. Clients include BNP Paribas, British Telecom, Capita, Cisco, Datapipe, Intel, Kaspersky, MercadoLibre, Microsoft, the 2010 and 2012 Olympic Games, Rackspace, Skype, Symantec, and IBM/Softlayer.

4. What does “Audited by Netcraft” actually mean?

“Audited by Netcraft” does not mean that the site or network is impregnable, but shows that the site is actively maintaining its security against remote compromise from the internet. Each site or network displaying the “Audited by Netcraft” seal is keeping the audited site or network address space free from well known remote attacks, and has an external, independent perspective with which to manage its external security. The “Audited by Netcraft” seal is served dynamically and shows the date of the last scan when no unmitigated, serious well-known vulnerabilities that could permit remote compromise were detected.

5. We already have a certificate from a certificate authority. Why should we certify our site twice?

Using an TLS certificate means that the traffic to and from your site using TLS services is encrypted and private from eavesdroppers. It says nothing about the underlying security of the site, network or its web applications. Many TLS sites have serious vulnerabilities that could be identified and resolved by using the regular scan reports provided with “Audited by Netcraft” service. In particular, libraries providing TLS support have previously been seriously affected by serious vulnerabilities themselves, most notably with the Heartbleed vulnerability that affected OpenSSL.

6. I run a shared hosting system. Is it possible to get the Netcraft seal not only for myself but also my customers?

Yes, it is. Please contact us with details of your system, specifically your network address space, and the hostnames of the virtual hosts you would like us to test. We can arrange a process for adding and removing virtual hosts from the test set.

7. I am a security expert. Couldn’t I test my own network?

We encourage clients to test their own networks and many of our clients have experienced internal security teams. However, relying on your own testing can be like marking your own examination paper. Most companies handling credit card data are required to be regularly scanned by an external vendor, under the Payment Card Industry Data Security Standard.

By using Netcraft’s services, you get access to people who test a plethora of different networks each year, not just one — a genuine internet’s eye view of your network from outside of your own firewall — and direct access to a professional second opinion.

8. We have a firewall. I thought that meant that we are secure?

A correctly configured firewall can eliminate attacks against services that that are not intended to be visible to the internet. However, many attacks exploit vulnerabilities critical network services such as HTTP, HTTPS, SMTP, and DNS, which must be permitted through your firewall to operate as intended.

Additionally, when you need to make changes to your firewall configuration, external independent testing will give you confidence that you have not inadvertently permitted any more services through the firewall than intended.

9. How does the Audited by Netcraft process work?

After signing up for the “Audited by Netcraft” service, we will scan the network address space or web site being audited. The time taken to scan an IP address can vary according to the number of discovered open ports, with a typical scan of a single address taking a few hours. An Audited By Netcraft scan begins with a full TCP and comprehensive UDP port scan to determine which services are available to the internet. Each service is tested for information leaks, configuration errors and outright vulnerabilities. The HTTP and HTTPS page trees where those services are present are inspected for attributes which may indicate risks. Netcraft tests multiple servers in parallel to reduce the risk of load on any one particular server, and accordingly the elapsed time for a network with 10 visible servers is not significantly greater than a single IP address.

Once the tests are complete we will email a summary containing a link to a web-accessible HTML report protected by both IP address restrictions and user accounts. For the vulnerabilities that have been found, links to advisories describing the problem and how to remedy are provided, so that you can fix the problems and retest your sites. Support by email and telephone is included within the service.

In some cases — for example in the case of a buffer overflow exploit or a denial of service attack, where we cannot directly test a vulnerability without risking crashing the server — we must rely on indirect tests that use version numbers and other metadata to determine if a vulnerability may be present. With these version-based tests, it is possible for a false positive to be generated, particularly on systems that use backports to apply security patches without incrementing displayed version numbers.

All customers can add annotations to their reports to provide evidence that the relevant patch has been applied. Those customers displaying a public Audited By Netcraft seal can request Netcraft confirm these are indeed false positives. Only once the report has no outstanding unmitigated serious vulnerabilities will the “Audited by Netcraft” security seal update to show the most recent passing scan date.

As time goes on, you will make changes to your configurations and new vulnerabilities in services you use will be discovered. When a change is discovered in your network or site’s internet profile, you will be alerted, and you can use the information in the advisory to fix the problem.

10. As a credit card merchant or service provider, we are required to have regular external scans by an approved vendor. Can Netcraft provide this?

Yes — Netcraft is an Approved Scanning Vendor in the PCI DSS scheme. Our reports will highlight vulnerabilities that need to be resolved to achieve PCI compliance, and we will produce quarterly reports to show your PCI compliance status. If you require scans for the purpose of PCI compliance, please mention this when asking about scans and we can help you determine the right networks and servers to be included in the scan to achieve PCI compliance.

11. How do you price Audited by Netcraft?

We price the “Audited by Netcraft” service based on the size of the IP address range we need to test, and the number of machines visible to the internet. We will confirm the IP address ranges with you, and quote a price on this basis.

12. Can you give 100% assurance that all security problems have been found?

No. By definition a testing service can only find vulnerabilities and cannot prove the absence of vulnerabilities. That said, our reports clearly show our methods and test scope, so a person with reasonable security experience can gauge the thoroughness of the tests. Netcraft has the custom of an impressive list of clients, with several well-known companies renewing their security testing contracts with us for over five years.

13. Could the tests crash my services?

It is unlikely as we take great care to avoid damage to the services we test. We are very experienced at testing business-critical live services. The load on the test site is typically low and should not disrupt other users. Denial of services exploits are detected by passive methods only, and buffer overflow exploits are not attempted.

14. Even if you don’t crash any services, what is the load likely to be, and how long will it last?

Netcraft’s testing is comprehensive, and a test of a single machine is likely to last at least 4 hours. This allows for a full TCP scan and a comprehensive UDP port scan, applying several thousand tests against all exposed network services, and retrieval of part of the web page tree from those services. We make the tests as well behaved as possible, by using generous timeouts on requests, and testing multiple machines in parallel to avoid hitting a single server intensively over too short a period of time.

15. Will reading all the reports and advisories and fixing the vulnerabilities be a lot of extra work?

No. The Netcraft auditing process will save a lot of time, as it removes the research effort required to find out about vulnerabilities and determine which are relevant to your own installation. The advisories collect information from multiple resources and give succinct instructions on patching, together with useful background information on each potential problem.

16. What about false positives?

Where it is not possible to confirm a potential vulnerability without risking disruption to your services, — such as Denial of Service attacks, or buffer overflows —vulnerabilities may be flagged on the basis of a software version number rather than generating the actual error condition.

This can lead to false positives, whereby a vulnerability that has already been fixed is reported on the basis of the software version.

We continually improve our tools to try and eliminate as many false positive issues as possible, and additionally make it possible for our customers who have a publicly-displayed Audited By Netcraft seal to mark vulnerabilities as patched. The person marking a vulnerability as a false positive effectively signs for this, making accountability possible within large organisations.

17. I have four servers behind one external load balanced address. How do you test them?

If we can only see one IP address externally, we treat that as one machine, and tests will be executed on whichever of the machines responds to our requests. If you want us to test all four, the simplest scenario is to make them individually visible to us on separate IP addresses. For larger configurations, we can arrange for an on-site testing point.

18. I have a reactive intrusion detection system, intrusion protection system or firewall. What are the implications?

Netcraft’s tests include comprehensive TCP and UDP port scans, and tests for large numbers of well known vulnerabilities which will trigger a reactive IDS, IPS or firewall system.

If Netcraft’s scans are blocked by your firewall we will be unable to give you accurate scan results. While this may be great for demonstrating that your firewall system works correctly we strongly recommend ensuring that our scanners are not blocked as this will help ensure our scan results are as good as possible.

Note that systems which only block attack traffic, rather than all traffic once an attack has been detected are acceptable as they will allow scanning to proceed. Similarly, any IDS which performs logging, or standard static firewalls which block ports that should not be internet accessible are perfectly acceptable, although you may wish to ensure that our scanners are whitelisted in any systems which are likely to send alerts to your staff!

PCI scans which are subject to blocking or other scan interference may be considered ‘inconclusive’. Please see the ASV program guide for full details.

If we believe that our scans are being blocked, we will let you know. We will also notify you of the IP address ranges we conduct our tests from in order to allow you to make any configuration changes.

19. So much for well known vulnerabilities. Will you also certify our own web applications?

Netcraft will certify web applications that it has tested, showing the date tested, and the number of days spent testing the application. Application testing is very important because even machines that are well administered, and correctly port filtered with no well known vulnerabilities, can be vulnerable to a direct attack on the application’s own functionality.

Whereas automated testing is good at finding common well-known (published) exploits, a consultancy audit can additionally find faults unique to your site, caused by application programming and design errors, as well as more complex configuration errors. A consultancy test can also interpret and exploit leaked information and give constructive advice on solutions.

For more information on web application security testing, see the Web Application Testing page.

20. Do you find vulnerabilities in third-party software, and what do you do if you find them?

Yes, often. If the vulnerability is new and not specific to your servers, and will affect others, then we work with you and the third-party vendor to find a solution before public announcement.

21. How do you charge for auditing web applications?

We charge for web application testing on a time and materials basis. Please contact security-sales@netcraft.com for further details.

22. Can you also provide performance monitoring and outage alerting for my web sites?

Yes. Netcraft has a network of performance monitoring points around Europe and North America and makes HTTP requests every fifteen minutes from each measurement point. Outage notifications are sent by email when all measurement points are unable to reach the site. Detailed performance graphs are made available with a 31-day data history.

For more information on performance monitoring, see the Performance Monitoring page.

23. Can you also provide testing for internal networks and extranets not generally visible to the internet?

Yes; these will normally involve site visits at the start of the project, but in other respects the process can be very similar. Testing machines can be positioned on internal networks, and provisioned with updates in the same way as our internet testing machines.