Machine Learning Authors: Pat Romanski, Yeshim Deniz, Liz McMillan, William Schmarzo, Jason Bloomberg

Related Topics: Machine Learning , Microsoft Cloud, Cloud Security

Machine Learning : Tutorial

Intruder Detection with tcpdump

tcpdump tool

To capture, parse, and analyze traffic tcpdump is a very powerful tool. To begin a basic capture uses the following syntax.

tcpdump -n –i <interface> -s <snaplen>

-n      tells tcpdump to not resolve IP addresses to domain names and port numbers to service names.
-I       <interface> tells tcpdump which interface to use.
-s      <snaplen> tells tcpdump how much of the packet to record. I used 1515 but 1514 is sufficient for most cases. If you don’t specify a size then it will only capture the first 68 bytes of each packet. A snaplen value of 0 which will use the required length to catch whole packets can be used except for older versions of tcpdump.

Below is an example output of a dump, although it only contains a few lines it holds much information.

12:24:51.517451  IP >  54517 A? www.bluecoast.com.  (34)

12:24:51:517451                              represent the time                          Source address and port
>                                                          Traffic direction                                 Destination address and port
54517                                                 ID number that is shared by both the DNS server and
A?                                               asks a question regarding the A record for www.bluecoat.com
(34)                                                     The entire packet is 34 bytes long.

More tcpdump capture options

Here are some examples of options to use when capturing data and why to use them:

-I        specify an interface; this will ensure that you are sniffing where you expect to sniff.
-n       tells tcpdump not to resolve IP addresses to domain names and port numbers to service names
-nn    don’t resolve hostnames or port names
-X      Show packet’s contents in both hex and ASCII
-XX    Include Ethernet header
-v       Increase verbose –vv –vvv more info back
-c       Only get x number of packets and stop
-s       tell tcpdump how much of the packet to record
-S       print absolute sequence numbers
-e       get Ethernet header
-q       show less protocol info
-E       Decrypt IPSEC traffic by providing an encryption key

Packet, Segment, and Datagram
TCP accepts data from a data stream, segments it into chucks, and adds a TCP header creating a TCP segment. UDP sends messages referred to as a datagram to other hosts on an Internet Protocol (IP) network without requiring prior communications to set up special transmission channels or data paths. Internet Protocol then creates its own datagram out of what it receives from TCP or UDP. If the TCP segment or UDP datagram plus IP’s headers are small enough to send in a single package on the wire then IP creates a packet. If they are too large and exceed the maximum transmission unit (MTU) of the media, IP will fragment the datagram into smaller packets suitable to the MTU. The fragmented packets are then reassembled by the destination.

Tcpdump read and write to/from a file
Tcpdump allows you to write data to a file using the –w option and to read from a file with the –r option.

$ sudo tcpdump -i wlan0 -w dumpfile001

$ sudo tcpdump -r dumpfile.pcap

Some people like to see the files as they are captured and have them saved to a file. Use the following options: tcpdump –n –I eth1 –s 1515 –l | tee output.txt
This option tells tcpdump to make its output line-buffered, while piping the output to the tee utility sends output to the screen and the output.txt simultaneously. This command will display packets on the screen while writing data to an output file output.txt it will not be in binary libpcap format. The best way to do this is run a second instance of tcpdump.

When tcpdump captures packets in libpcap format, it adds a timestamp entry to the record in each packet in the capture file. We can augment that data with the –tttt flag, which adds a date to the timestamp (See Figure #1).

Figure 1

You can use the –tt flag to report the number of seconds and microseconds since the UNIX epoch of 00:00:00 UTC on January 1, 1970. If you are not sure you understand the time difference and need to be absolutely sure of time use the –tt option to show seconds and microseconds since the UNIX epoch (See Figure #2).

Figure 2

Useful syntax
Being able to cut the amount of traffic down to just what you are looking for is useful. Here are some useful expressions that can be helpful in tcpdump.

Net – This will capture the traffic on a block of IPs ex
# tcpdump net
Src, dst – This will only capture packets form a source or destination.
# tcpdump src
# tcpdump dst

Host – Capture only traffic based on the IP address
# tcpdump host
Proto – Capture works for tcp, udp, and icmp
# tcpdump tcp
Port – Capture packets coming from or going to a port.
# tcpdump port 21
Port ranges – capture packets
# tcpdump port 20-25
Using expressions such as AND [&&], OR [||], & EXCEPT [!]
# tcpdump –n –I eth1 host and host
# tcpdump –n –I eht1 src net and dst net or
# tcpdump –n –I eth1 src net and not icmp

Searching for info on packets with tcpdump
If you want to search for information in the packet you have to know where to look. Tcpdump starts counting bytes of header information at byte 0 and the 13th byte contains the TCP flags shown in Table #1


Talbe #1

Now looking at byte 13 and if the SYN and ACK are set then your binary value would be 00010010 which are the same as decimal 18. We can search for packets looking for this type of data inside byte 13 shown here.

# tcpdump –n –r dumpfile.lpc –c 10 ‘tcp[13] == 18’ and host

Here is a sample of what this command will return shown in Figure #3

Figure #3

When capturing data using tcpdump one way to ignore the arp traffic is to put in a filter like so.

# tcpdump –n –s 1515 –c 5 –I eth1 tcp or udp or icmp

This will catch only tcp, udp, or icmp.

If you want to find all the TCP packets with the SYN ACK flag set or other flags set take a look at Table #2 & tcpdump filter syntax shown below.

flag           Binary           Decimal
URG         00100000          32
ACK          00010000          16
PSH          00001000           8
RST          00000100           4
SYN          00000010           2
FIN            00000001           1
SYNACK  00010010         18

Table #2

Tcpdump filter syntax

Show all URGENT (URG) packets
# tcpdump ‘tcp[13] == 32’
Show all ACKNOWLEDGE (ACK) packets
# tcpdump ‘tcp[13] == 16’
Show all PUSH (PSH) packets
# tcpdump ‘tcp[13] == 8’
Show all RESET (RST) packets
# tcpdump ‘tcp[13] == 4’
Show all SYNCHRONIZE (SYN) packets
# tcpdump ‘tcp[13] ==2’
Show all FINISH (FIN) packets
# tcpdump ‘tcp[13] == 1’
# tcpdump ‘tcp[13] == 18’

Using tcpdump in Incident Response

When doing analysis on network traffic using a tool like tcpdump is critical. Below are some examples of using tcpdump to view a couple of different dump files to learn more about network problems or possible attack scenarios. The first is a binary dump file of a snort log and we are given the following information. The IP address of the Linux system is and an attacker got in using a WU-FTPD vulnerability and deployed a backdoor. What can we find out about how the attack happened and what he did?

First we will take a look at the file

# tcpdump –xX –r snort001.log
The log appears long at this point you may want to run the file in snort
# snort –r snort001.log –A full –c /etc/snort/snort.conf
This will give you some info like total packets processed, protocol breakdown, any alerts, etc. See Figure #4 & #5

Figure #4                                                                               Figure #5

Next extract the full snort log file for analysis

# tcpdump –nxX –s 1515 –r snort001.log > tcpdump-full.dat

This will give us a readable file to parse through. After looking through it we find ip-proto-11, which is Network Voice Protocol (NVP) traffic. Now we will search through the file looking for ip-proto-11.

# tcpdump –r snort001.log –w NVP-traffic.log proto 11
This command will read the snort001.log file and look for ‘log proto 11’ and writes the contents to the file NVP-traffic.log. Next we need to be able to view the file because it is a binary file.

# tcpdump –nxX –s 1515 –r NVP-traffic.log > nvp-traffic_log.dat
This will be a file of both hex and ASCII, which is nice but we just want the IP address. Try this.

# tcpdump –r NVP-traffic.log > nvp-traffic_log01.dat
This will give us a list of IP address that were communicating using the Network Voice Protocol (NVP) (See Figure #6).

Figure #6

Next we look at another snort dump file from a compromised windows box that was communicating with an IRC server. What IRC servers did the server at communicate with?

Look for TCP connections originating from the server toward the outside and we can use tcpdump with a filtering expression to capture SYN/ACK packets incoming from outside servers.

# tcpdump -n -nn -r snort_log 'tcp and dst host and tcp[13]==18'

This produces a long list of connections going from to outside connections. (see Figure #7).

Figure #7

Now we know that IRC communicate on port 6666 to 6669 so let’s add that and narrow down the search with the following command.
# tcpdump -n -nn -r snort_log 'tcp and dst host and tcp[13]==18' and portrange 6666-6669 (See output in Figure #8 below)

Figure #8

Now we have narrowed the list down to 3 IP’s that were communicating with the server using IRC.

Tcpdump is a wonderful, general-purpose packet sniffer and incident response tool that should be in your tool shed.

More Stories By David Dodd

David J. Dodd is currently in the United States and holds a current 'Top Secret' DoD Clearance and is available for consulting on various Information Assurance projects. A former U.S. Marine with Avionics background in Electronic Countermeasures Systems. David has given talks at the San Diego Regional Security Conference and SDISSA, is a member of InfraGard, and contributes to Secure our eCity http://securingourecity.org. He works for Xerox as Information Security Officer City of San Diego & pbnetworks Inc. http://pbnetworks.net a Service Disabled Veteran Owned Small Business (SDVOSB) located in San Diego, CA and can be contacted by emailing: dave at pbnetworks.net.

@CloudExpo Stories
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
Poor data quality and analytics drive down business value. In fact, Gartner estimated that the average financial impact of poor data quality on organizations is $9.7 million per year. But bad data is much more than a cost center. By eroding trust in information, analytics and the business decisions based on these, it is a serious impediment to digital transformation.
Predicting the future has never been more challenging - not because of the lack of data but because of the flood of ungoverned and risk laden information. Microsoft states that 2.5 exabytes of data are created every day. Expectations and reliance on data are being pushed to the limits, as demands around hybrid options continue to grow.
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
Evan Kirstel is an internationally recognized thought leader and social media influencer in IoT (#1 in 2017), Cloud, Data Security (2016), Health Tech (#9 in 2017), Digital Health (#6 in 2016), B2B Marketing (#5 in 2015), AI, Smart Home, Digital (2017), IIoT (#1 in 2017) and Telecom/Wireless/5G. His connections are a "Who's Who" in these technologies, He is in the top 10 most mentioned/re-tweeted by CMOs and CIOs (2016) and have been recently named 5th most influential B2B marketeer in the US. H...
The IoT Will Grow: In what might be the most obvious prediction of the decade, the IoT will continue to expand next year, with more and more devices coming online every single day. What isn’t so obvious about this prediction: where that growth will occur. The retail, healthcare, and industrial/supply chain industries will likely see the greatest growth. Forrester Research has predicted the IoT will become “the backbone” of customer value as it continues to grow. It is no surprise that retail is ...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of bus...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
DXWorldEXPO LLC announced today that "Miami Blockchain Event by FinTechEXPO" has announced that its Call for Papers is now open. The two-day event will present 20 top Blockchain experts. All speaking inquiries which covers the following information can be submitted by email to [email protected] Financial enterprises in New York City, London, Singapore, and other world financial capitals are embracing a new generation of smart, automated FinTech that eliminates many cumbersome, slow, and expe...
Cloud Expo | DXWorld Expo have announced the conference tracks for Cloud Expo 2018. Cloud Expo will be held June 5-7, 2018, at the Javits Center in New York City, and November 6-8, 2018, at the Santa Clara Convention Center, Santa Clara, CA. Digital Transformation (DX) is a major focus with the introduction of DX Expo within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive ov...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
@DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises - and delivering real results.
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
"We started a Master of Science in business analytics - that's the hot topic. We serve the business community around San Francisco so we educate the working professionals and this is where they all want to be," explained Judy Lee, Associate Professor and Department Chair at Golden Gate University, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
DXWorldEXPO LLC announced today that Dez Blanchfield joined the faculty of CloudEXPO's "10-Year Anniversary Event" which will take place on November 11-13, 2018 in New York City. Dez is a strategic leader in business and digital transformation with 25 years of experience in the IT and telecommunications industries developing strategies and implementing business initiatives. He has a breadth of expertise spanning technologies such as cloud computing, big data and analytics, cognitive computing, m...
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...