AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
We experimentally showed that ZMap is capable of scanning the public IPv4 address space on a single port in under 45 minutes, at 97% of the theoretical maximum speed for gigabit Ethernet and with an estimated 98% coverage of publicly available hosts
ZMap: fast internet-wide scanning and its security applications
USENIX Security, pp.605-620, (2013)
Internet-wide network scanning has numerous security applications, including exposing new vulnerabilities and tracking the adoption of defensive mechanisms, but probing the entire public address space with existing tools is both difficult and slow. We introduce ZMap, a modular, open-source network scanner specifically architected to perfo...More
PPT (Upload PPT)
- Recent studies have demonstrated that Internet-wide scanning can help reveal new kinds of vulnerabilities, monitor deployment of mitigations, and shed light on previously opaque distributed ecosystems [10, 12, 14, 15, 25, 27]
- This methodology has been more accessible to attackers than to legitimate researchers, who cannot employ stolen network access or spread self-replicating code.
- ZMap’s modular architecture can support many types of single-packet probes, including TCP SYN scans, ICMP echo request scans, and application-specific UDP scans, and it can interface with user-provided code to perform follow-up actions on discovered hosts, such as completing a protocol handshake
- Introduction and Roadmap
Internet-scale network surveys collect data by probing large subsets of the public IP address space
- While we have demonstrated that efficiently scanning the IPv4 address space at gigabit line speeds is possible, there remain several open questions related to performing network surveys over other protocols and at higher speeds
- To help researchers make the most of this window of opportunity, we developed ZMap, a network scanner architected for performing fast, comprehensive Internet-wide surveys
- We experimentally showed that ZMap is capable of scanning the public IPv4 address space on a single port in under 45 minutes, at 97% of the theoretical maximum speed for gigabit Ethernet and with an estimated 98% coverage of publicly available hosts
- We explored the security applications of high speed scanning, including the ability to track protocol adoption at Internet scale and to gain timely insight into opaque distributed systems such as the certificate authority ecosystem
- We further showed that high-speed scanning provides new attack vectors that we must consider when defending systems, including the ability to uncover hidden services, the potential to track users between IP addresses, and the risk of infection of vulnerable hosts en masse within minutes of a vulnerability’s discovery
- The authors estimate that ZMap achieves 98% network coverage using only a single probe per host, even at its maximum scanning speed.
- As shown in Table 2, the authors find significantly more TLS servers than previous work—78% more than Heninger et al and 196% more than the SSL Observatory—likely due to increased HTTPS deployment since those studies were conducted
- The authors explored the security applications of high speed scanning, including the ability to track protocol adoption at Internet scale and to gain timely insight into opaque distributed systems such as the certificate authority ecosystem.
- The authors further showed that high-speed scanning provides new attack vectors that the authors must consider when defending systems, including the ability to uncover hidden services, the potential to track users between IP addresses, and the risk of infection of vulnerable hosts en masse within minutes of a vulnerability’s discovery.
- The authors offer the recommendations the authors developed while performing the own scans as a starting point for further conversations about good scanning practice
- Table1: ZMap vs. Nmap Comparison — We scanned 1 million hosts on TCP port 443 using ZMap and Nmap and averaged over 10 trials. Despite running hundreds of times faster, ZMap finds more listening hosts than Nmap, due to Nmap’s low host timeout. Times for ZMap include a fixed 8 second delay to wait for responses after the final probe
- Table2: Comparison with Prior Internet-wide HTTPS Surveys — Due to growth in HTTPS deployment, ZMap finds almost three times as many TLS servers as the SSL Observatory did in late 2010, yet this process takes only 10 hours to complete from a single machine using a ZMap-based workflow, versus three months on three machines
- Table3: Top 10 Certificate Authorities — We used ZMap to perform regular comprehensive scans of HTTPS hosts in order gain visibility into the CA ecosystem. Ten organizations control 86% of browser trusted certificates
- Table4: Top 10 TCP ports — We scanned 2.15 million hosts on TCP ports 0–9175 and observed what fraction were listening on each port. We saw a surprising number of open ports associated with embedded devices, such as ports 7547 (CWMP) and 3479 (2-Wire RPC)
- Table5: Recommended Practices — We offer these suggestions for other researchers conducting fast Internetwide scans as guidelines for good Internet citizenship
- Table6: Responses by Entity Type — We classify the responses and complaints we received about our ongoing scans based on the type of entity that responded
- Many network scanning tools have been developed, the vast majority of which have been optimized to scan small network segments. The most popular and well respected is Nmap (“Network Mapper”) , a versatile, multipurpose tool that supports a wide variety of probing techniques. Unlike Nmap, ZMap is specifically designed for Internet-wide scanning, and it achieves much higher performance in this application.
Leonard and Loguinov introduced IRLscanner, an Internet–scale scanner with the demonstrated ability to probe the advertised IPv4 address space in approximately 24 hours, ultimately scanning at 24,421 packets per second . IRLscanner is able to perform scanning at this rate by utilizing a custom Windows network driver, IRLstack . However, IRLscanner does not process responses, requires a custom network driver and a complete routing table for each scan, and was never released to the research community. In comparison, we developed ZMap as a self-contained network scanner that requires no custom drivers, and we are releasing it to the community under an open source license. We find that ZMap can scan at 1.37 million packets per second, 56 times faster than IRLScanner was shown to operate.
- This work was supported in part by NSF grant CNS-1255153 and by an NSF Graduate Research Fellowship
- Anonymous. Internet census 2012. http://census2012.sourceforge.net/paper.html, March 2013.
- G. Bartlett, J. Heidemann, and C. Papadopoulos. Understanding passive and active service discovery. In 7th ACM SIGCOMM conference on Internet measurement (IMC), pages 57–70, 2007.
- L. Bello. DSA-1571-1 OpenSSL—Predictable random number generator, 2008. Debian Security Advisory. http://www.debian.org/security/2008/dsa-1571.
- D. J. Bernstein. SYN cookies. http://cr.yp.to/syncookies.html, 1996.
- J. Black, S. Halevi, H. Krawczyk, T. Krovetz, and P. Rogaway. UMAC: Fast and secure message authentication. In Advances in Cryptology—CRYPTO ’99, 1999.
- S. Blake-Wilson, M. Nystrom, D. Hopwood, J. Mikkelsen, and T. Wright. Transport Layer Security (TLS) Extensions. RFC 3546 (Proposed Standard), June 2003.
- T. Chown. IPv6 Implications for Network Scanning. RFC 5157 (Informational), March 2008.
- L. Deri. Improving passive packet capture: Beyond device polling. In 4th International System Administration and Network Engineering Conference (SANE), 2004.
- R. Dingledine. Research problems: Ten ways to discover Tor bridges. http://blog.torproject.org/blog/researchproblems-ten-ways-discover-tor-bridges, October 2011.
- P. Eckersley and J. Burns. An observatory for the SSLiverse. Talk at Defcon 18 (2010). https://www.eff.org/files/ DefconSSLiverse.pdf.
- S. Han, K. Jang, K. Park, and S. Moon. PacketShader: A GPU-accelerated software router. In ACM SIGCOMM, September 2010.
- J. Heidemann, Y. Pradkin, R. Govindan, C. Papadopoulos, G. Bartlett, and J. Bannister. Census and survey of the visible Internet. In 8th ACM SIGCOMM conference on Internet measurement (IMC), 2008.
- J. Heidemann, L. Quan, and Y. Pradkin. A preliminary analysis of network outages during hurricane sandy. Technical Report ISI-TR-2008-685b, USC/Information Sciences Institute, November 2012.
- N. Heninger, Z. Durumeric, E. Wustrow, and J. A. Halderman. Mining your Ps and Qs: Detection of widespread weak keys in network devices. In 21st USENIX Security Symposium, August 2012.
- R. Holz, L. Braun, N. Kammenhuber, and G. Carle. The SSL landscape: A thorough analysis of the X.509 PKI using active and passive measurements. In 11th ACM SIGCOMM conference on Internet measurement (IMC), pages 427–444, 2011.
- IANA. IPv4 address space registry. http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml.
- V. Jacobson, C. Leres, and S. McCanne. libpcap. Lawrence Berkeley National Laboratory, Berkeley, CA. Initial release June 1994.
- J. Kasten, E. Wustrow, and J. A. Halderman. Cage: Taming certificate authorities by inferring restricted scopes. In 17th International Conference on Financial Cryptography and Data Security (FC), 2013.
- M Koster. A standard for robot exclusion. http://www.robotstxt.org/orig.html, 1994.
- A. Langley. Enhancing digital certificate security. Google Online Security Blog, http://googleonlinesecurity.blogspot.com/2013/01/enhancing-digital-certificate-security.html, January 2013.
- E. Law. Understanding certificate name mismatches. http://blogs.msdn.com/b/ieinternals/archive/2009/12/07/certificate-name-mismatch-warnings-and-server-nameindication.aspx, December 2009.
- D. Leonard and D. Loguinov. Demystifying service discovery: Implementing an Internet-wide scanner. In 10th ACM SIGCOMM conference on Internet measurement (IMC), pages 109–122, 2010.
- Gordon Fyodor Lyon. Nmap Network Scanning: The Official Nmap Project Guide to Network Discovery and Security Scanning. Insecure, USA, 2009.
- N. Mathewson and N. Provos. libevent—An event notification library. http://libevent.org.
- HD Moore. Security flaws in universal plug and play. Unplug. Don’t Play, January 2013. http://community.rapid7.com/servlet/JiveServlet/download/2150-1-16596/SecurityFlawsUPnP.pdf.
- Netcraft, Ltd. Web server survey. http://news.netcraft.com/archives/2013/05/03/may-2013-web-server-survey.html, May 2013.
- N. Provos and P. Honeyman. ScanSSH: Scanning the Internet for SSH servers. In 16th USENIX Systems Administration Conference (LISA), 2001.
- Luigi Rizzo. netmap: A novel framework for fast packet I/O. In 2012 USENIX Annual Technical Conference, 2012.
- S. Sanfilippo and P. Noordhuis. Redis. http://redis.io.
- H. Scholz. SIP stack fingerprinting and stack difference attacks. Talk at Blackhat 2006. http://www.blackhat.com/presentations/bh-usa-06/BH-US-06-Scholz.pdf.
- A. Schulman and N. Spring. Pingin’ in the rain. In 11th ACM SIGCOMM conference on Internet measurement (IMC), pages 19–28, 2011.
- K. Sklower. A tree-based packet routing table for Berkeley Unix. In Winter USENIX Conference, 1991.
- M. Smith and D. Loguinov. Enabling high-performance Internet-wide measurements on Windows. In 11th International Conference on Passive and Active Measurement (PAM), pages 121–130.
- W. R. Stevens and G. R. Wright. TCP/IP Illustrated: The Implementation, volume 2. Addison-Wesley, 1995.
- Tor Project. Tor Bridges. https://www.torproject.org/docs/bridges, 2008.
- Tor Project. obfsproxy. https://www.torproject.org/projects/obfsproxy.html.en, 2012.
- J. Viega, M. Messier, and P. Chandra. Network Security with OpenSSL: Cryptography for Secure Communications. O’Reilly, 2002.
- T. Wilde. Great Firewall Tor probing. https://gist.github.com/twilde/da3c7a9af01d74cd7de7, 2012.