AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
We introduced a new system to generate blacklists for contributors to a large-scale security-log sharing infrastructure
Highly Predictive Blacklisting
;login:, no. 6 (2008): 107-122
The notion of blacklisting communication sources has been a well-established defensive measure since the ori- gins of the Internet community. In particular, the prac- tice of compiling and sharing lists of the worst offenders of unwanted traffic is a blacklisting strategy that has re- mained virtually unquestioned over many years. But do ...More
PPT (Upload PPT)
- The authors refer to the blacklists that are formulated by a large-scale alert repository and consist of the most prolific sources in the repository’s collection of data as the global worst offender list (GWOL)
- Another strategy for formulating network address blacklists is for an individual network to create a local blacklist based entirely on its own history of incoming communications.
- The authors call this blacklist scheme the local worst offender list (LWOL) method
- A network address blacklist represents a collection of source IP addresses that have been deemed undesirable, where typically these addresses have been involved in some previous illicit activities
- With more than 1700 contributing sources providing a daily stream of 30 million security log entries, such daily blacklists provide an informative view of those class C subnets that are among the bane of the Internet with respect to unwanted traffic
- We show that HPB analysis provides contributors a potential to predict more new attacks than global worst offender list (GWOL). (LWOL is not considered, since by definition it includes only attackers that are actively hitting the local worst offender list (LWOL) owner.) For each contributor, we construct two new HPB and GWOL lists with equal length of 1000 entries, such that no entries have been reported by the contributor during our training window
- We introduced a new system to generate blacklists for contributors to a large-scale security-log sharing infrastructure
- In April of 2007, we released a highly predictive blacklist service at DShield.org. We view this service as a first experimental step toward a new direction of high-quality blacklist generation. We believe that this service offers a new argument to help motivate the field of secure collaborative data sharing
- It demonstrates that people who collaborate in blacklist formulation can share a greater understanding of attack source histories, and thereby derive more informed filtering policies
- The authors created an experimental HPB blacklist formulation system. To evaluate the HPBs, the authors performed a battery of experiments using the DShield.org security firewall and IDS log repository.
- Since the relevance measure is based on correlations between contributors, HPB production is not applicable to contributors that have submitted very few reports (DShield has contributors that hand-select or sporadically contribute logs, providing very few alerts)
- The authors exclude those contributors that the authors find effectively have no correlation with the wider contributor pool or have too few alerts to produce meaningful results.
- The authors found that the authors could compute correlation relationships for about 700 contributors, or 41% of the DShield contributor pool
- The authors introduced a new system to generate blacklists for contributors to a large-scale security-log sharing infrastructure.
- The system employs a link analysis method similar to Google’s PageRank for blacklist formulation
- It integrates substantive log pre-.
- The authors believe that this service offers a new argument to help motivate the field of secure collaborative data sharing
- It demonstrates that people who collaborate in blacklist formulation can share a greater understanding of attack source histories, and thereby derive more informed filtering policies.
- The authors will continue to evolve the HPB blacklisting system as the experience grows through managing the blacklist service
- Table1: Sample Attack Table. Given this correlation matrix, we follow the aforem entioned intuition and calculate the relevance as ris = j∈T (s) W(i,j). This is to say that if the repository reports that source s has attacked contributor vj, this fact contributes a value of W(i,j) to the source’s relevance with respect to the victim vi. Written in vector form, it gives us rs = W · bs
- Table2: Summary of Relevance Model Notations the number seen by vj, and mij the number of common attack for vj wsohuilrecemms.ijjTshheorwatsiohommwiijimshpoowrtsanhtovwj important vi is is for vi. Since we want W(i,j) to reflect the strength of the connection between vi and
- Table3: Hit Number Comparison between HPB, LWOL and GWOL
- Table4: Hit Count Performance, HPB vs. (GWOL and LWOL), Length 1000 Entries
- Table5: Top 200 Contributors’ Hit Count Increases (Blacklist Length 1000)
- Network address and email blacklists have been around since the early development of the Internet . Today, sites such as DShield regularly compile and publish firewall-parsable filters of the most prolific attack sources reported to its website . DShield represents
17th USENIX Security Symposium a centralized approach to blacklist formulation, providing a daily perspective of the malicious background radiation that plagues the Internet [15, 20]. Other recent examples of computer and network blacklists include IP and DNS blacklists to help networks detect and block unwanted web content, SPAM producers, and phishing sites, to name a few [7, 8, 17, 18]. The HPB system presented here complements, but does not displace these resources or their blacklisting strategies. In addition, HPBs are only applicable to active log contributors (we hope as an incentive), not as generically publishable one-sizefits-all resources.
- This material is based upon work supported through the U.S Army Research Office under the Cyber-TA Research Grant No W911NF-06-1-0316
- ANAGNOSTAKIS, K. G., GREENWALD, M. B., IOANNIDIS, S., KEROMYTIS, A. D., AND LI, D. A cooperative immunization system for an untrusting Internet. In Proceedings of the 11th IEEE International Conference on Networks (ICON’03) (October 2003).
- BRIN, S., AND PAGE, L. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30, 1-7 (1998), 107–117.
- CAI, M., HWANG, K., KWOK, Y., SONG, S., AND CHEN, Y. Collaborative Internet worm containment. IEEE Security and Privacy Magazine 3, 3 (May/June 2005), 25–33.
- CHEN, Z., AND JI, C. Optimal worm-scanning method using vulnerable-host distributions. International Journal of Security and Networks (IJSN) Special Issue on Computer & Network Security 2, 1 (2007).
- COPPERSMITH, D., AND WINOGRAD, S. Matrix multiplication via arithmetic progressions. Journal of Symbolic Computation 9 (1990), 251–280.
- HUMPHRYS, M. The Internet in the 1980s. http://www.computing.dcu.ie/̃humphrys/net.80s.html, 2007.
- INCORPORATED, G.
- Internet/Abuse/Spam/Blacklist%s/, 2007.
-  INCORPORATED, G. Live-feed anti-phishing blacklist. http://sb.google.com/safebrowsing/update?version=goog-black-url:1:1, 2007.
-  JUNG, J., PAXSON, V., BERGER, A. W., AND BALAKRISHNAN, H. Fast portscan detection using sequential hypothesis testing. In IEEE Symposium on Security and Privacy 2004 (Oakland, CA, May 2004).
-  KATTI, S., KRISHNAMURTHY, B., AND KATABI, D. Collaborating against common enemies. In Proceedings of the ACM SIGCOMM/USENIX Internet Measurement Conference (October 2005).
-  KIM, H.-A., AND KARP, B. Autograph: Toward automated, distributed worm signature detection. In USENIX Security Symposium (2004), pp. 271–286.
-  LOCASTO, M., PAREKH, J., KEROMYTIS, A., AND STOLFO, S. Towards collaborative security and P2P intrusion detection. In Proceedings of the 2005 IEEE Workshop on Information Assurance and Security (June 2005).
-  M.GORI, AND PUCCI, A. Itemrank: A random-walk based scoring algorithm for recommender engines. In Proceedings of the International Joint Conference on Artificial Intelligence (January 2007).
-  PORRAS, P., BRIESEMEISTER, L., SKINNER, K., LEVITT, K., ROWE, J., AND TING, Y. A hybrid quarantine defense. In Proceedings of the 2004 ACM Workshop on Rapid Malcode (WORM) (October 2004).
-  RUOMING, P., YEGNESWARAN, V., BARFORD, P., PAXSON, V., AND PETERSON, L. Characteristics of internet background radiation. In Proceedings of ACM SIGCOMM/USENIX Internet Measurement Conference (October 2004).
-  THOMAS, R. Bogon dotted decimal list v3.9. http://www.cymru.com/Documents/bogon-dd.hml, October 2007.
-  ULLRICH, J. DShield global worst offender list. https://feeds.dshield.org/block.txt.
-  VIXIE, P., AND RAND, D. Mail abuse prevention system (MAPS). http://www.mail-abuse.com, 1997.
-  WISSNER-GROSS, A. D. Preparation of topical readings lists from the link structure of Wikipedia. In Proceedings of the IEEE International Conference on Advanced Learning Technology (July 2006).
-  YEGNESWARAN, V., BARFORD, P., AND ULLRICH, J. Internet intrusions: global characteristics and prevalence. In Proceedings of ACM SIGMETRICS (June 2003).
-  YEGNESWARAN, V., PORRAS, P., SAIDI, H., SHARIF, M., AND NARAYANAN, A. The Cyber-TA compendium honeynet page. http://www.cyber-ta.org/Honeynet.
-  ZHANG, J., PORRAS, P., AND ULLRICH, J. The DSHIELD highly predictive blacklisting service. http://www.dshield.org/hpbinfo.html.
-  ZHANG, J., PORRAS, P., AND ULLRICH, J. A new service for increasing the effectiveness of network address blacklists. In Proceedings of the 3rd Workshop of Steps to Reduce Unwanted Traffic on the Internet (June 2007).
-  ZHANG, J., PORRAS, P., AND ULLRICH, J. Gaussian process learning for cyber-attack early warning. to appear in Proceedings of SIAM Conference on data mining (2008).