Using OTX Threat Intelligence to Search PCAPs for Malicious Traffic

CapStar Forensics is an AlienVault Open Threat Exchange (OTX) participant. OTX is open to the public, and anyone can contribute to and download the threat data (which is called a “Pulse” in OTX).

So how can security professionals use this threat intelligence to help an organization defend against potential cyberattacks? In this blog, we show an example where CapStar used an OTX threat intel feed as source information to search a packet capture (pcap) for possible malicious traffic.

First, we downloaded all the OTX pulses and extracted the indicators of compromise (IoC) related to networking, to a file. There are 4 types: IPv4, hostname, domain and URL. In total, there are 19290 unique IoC's. Here is part of the file:

• domain qgqatfmbjrxwbk.cc
• domain rilmcycxjujewr.su
• domain lforanrwxevhyi.com
• domain vhngseuwuygrxy.net
• domain yhykhppwkxlfck.in
• domain lwvvefxffsiylo.me
• domain fqbassjbfsuthy.com
• domain bsikfiribqcudf.tw
• domain eoksneimvitpwo.net

Each line is a record in this file format. Each record has an IoC type followed by the IoC data.

We discussed what tool we could use to apply this threat intel to find the presence of malicious traffic in a pcap file? Many network and security professionals’ favorite tool is Wireshark for this purpose. However, due to the large number of IP addresses and hostnames, it's not practical to use Wireshark. The other good choice is to use an Intrusion Detection System (IDS). The problem with this approach is, one would have to create many IDS rules based on the specific intel, causing the investigation to slow down when the number of rules begins to be too large.

Fortunately, CapStar has a great tool for this scenario. A user can write a script that will read the threat intel in the above format and use it to match on packets very quickly.

Here is a CapStar script for just this. It consists of a few sections, each under a label.

• The “init” section reads the threat intel line by line, and will add each item of threat intel to the right category.
• The “pkt” section processes the packets individually. It will match on source or destination IP against the blacklisted IP in the intel, and in the case of DNS transactions, match the hostname or domain in the query against the blacklisted hostnames and domains.
• The “data” section processes the packets at data level. CapStar feeds the data packets in the right order, with respect to its session, to the script/logic in this section. CapStar does port-independent identification of the HTTP transactions, so the user logic for matching against a list of blacklisted URLs is pretty simple.
• The “end” section just summarizes the stats and displays them to the user.

One observation on the script is that it reuses the standard Wireshark display filter names, which are familiar to many network and security professionals. This is done to “extend” the Wireshark display filter so an investigator can implement arbitrary logic or expressions, and then perform a stateful pattern match that involves multiple packets.

We ran this script against a 1126MB malware pcap. Here is the partial output:

• number of blacklistedIPs: 4219
• number of blacklistDomains: 3611
• number of blacklistHosts: 5815
• number of blacklistUrls: 4986

contacted blacklisted IP:

• 195.3.96.72
• 64.14.68.53
• 79.170.40.37
• 74.54.133.186
• 66.133.129.5
• 69.16.175.10

contacted blacklisted Hosts:

• www.download.windowsupdate.com

contacted blacklisted domains:

• captainangry.net
• freeworldgo.com
• babkokohtybvcfreso.cf
• appleid-save.com
• bogev3ovaneu3dac3hnik.tk
• avdrygvovanemydak1.ga

CapStar is extremely fast in processing these packets. The entire run (from clicking the Run button to seeing the result) on this 1126 MB pcap file only takes 1.97 seconds. One of the more expensive parts of this operation is loading the IoC from file. After excluding that part, it took only 1.46 seconds to process, which is equivalent to processing 6.2Gbps of traffic. At this rate, CapStar is able to easily keep up with traffic from multiple Gigabit NIC.

CapStar is designed with maximum flexibility in mind. In the fight against cyber criminals, one has to be adaptive to the ever-changing environment and requirements. In the case of applying threat intel to analyzing network packets, a user can easily tweak the above script to do more specific scenarios and actions. For example, one can:

• Print the list of potentially compromised clients
• Create a smaller pcap file with the relevant packets to prove that a client is compromised
• Combine it with network behavior monitoring to detect network traffic to/from a non-blacklisted IP/host but that appears suspicious in some way.

If you are interested in giving CapStar a trial or have some challenging network scenarios you would like us to take a crack at, please send us an email info@capstarforensics.com.

About the Author

Dr. Jin Qian has worked in telecommunications as well as application and server performance for many years before diving deep into the challenging field of network security. In network security, he applies the same principle of making hard things easy and making technology more accessible for professionals of various backgrounds. His belief on fighting cyber criminals is to empower cyber warriors to be more adaptive and agile than the hackers, even if the hackers may be more experienced in programming.