HowTo Geolocate Network Packet Capture Data: Difference between revisions

From MediaWiki
Jump to navigationJump to search
 
(25 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<center>'''<span style='color: red; font-size: 200%;'>***Note: Page Under Construction***</span>'''</center>
__TOC__
__TOC__
== '''Overview''' ==
== '''Overview''' ==
Line 7: Line 4:
This '''HowTo''' explains the procedure for geolocating '''IPv4 Address Conversations''' using the NST WUI and rendering the results on either a '''Mercator World Map''' projection or on a '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' '''Earth Browser''' such as '''[http://earth.google.com Google Earth]''', '''[http://maps.google.com Google Maps]''' or '''[http://edu.kde.org/marble Marble]'''.
This '''HowTo''' explains the procedure for geolocating '''IPv4 Address Conversations''' using the NST WUI and rendering the results on either a '''Mercator World Map''' projection or on a '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' '''Earth Browser''' such as '''[http://earth.google.com Google Earth]''', '''[http://maps.google.com Google Maps]''' or '''[http://edu.kde.org/marble Marble]'''.


There are a couple of items to consider prior to starting '''IPv4 Address Conversations''' geolocation. First, does the network packet capture make sense to use for geolocation. The list below are packet capture characteristics that would <u>not</u> be considered desirable for geolocation:
There are a couple of items to consider prior to geolocating '''IPv4 Address Conversations'''. First, does the network packet capture make sense to use for geolocation. The list below are packet capture characteristics that would <u>not</u> be considered desirable for geolocation:


* No '''IPv4 Addresses''' exist in the capture file. '''Results:''' No geolocations would be rendered.  
* No '''IPv4 Addresses''' exist in the capture file. '''Results:''' No geolocations would be rendered.  
Line 13: Line 10:
* All hosts are '''Private IP Addresses''' with no associated geolocation database information. '''Results:''' No geolocations would be rendered. '''Note:''' This condition can be corrected, see section: [[HowTo_Geolocate_Network_Packet_Capture_Data#IP_Geolocation_Database_.26_Adjustments | IP Geolocation Database & Adjustments]].
* All hosts are '''Private IP Addresses''' with no associated geolocation database information. '''Results:''' No geolocations would be rendered. '''Note:''' This condition can be corrected, see section: [[HowTo_Geolocate_Network_Packet_Capture_Data#IP_Geolocation_Database_.26_Adjustments | IP Geolocation Database & Adjustments]].


Secondly, has a geolocation database been configured for your NST probe. This includes the addition of configuring any '''Private IPv4 Addresses''' or '''Network''' geolocation information, see section: [[HowTo_Geolocate_Network_Packet_Capture_Data#IP_Geolocation_Database_.26_Adjustments | IP Geolocation Database & Adjustments]].
Secondly, has the geolocation database been setup and configured for your NST probe. This includes the addition of configuring any '''Private IPv4 Addresses''' or '''Network''' geolocation information. For details, see the section: [[HowTo_Geolocate_Network_Packet_Capture_Data#IP_Geolocation_Database_.26_Adjustments | IP Geolocation Database & Adjustments]].


== '''IP Geolocation Database & Adjustments''' ==
== '''IP Geolocation Database & Adjustments''' ==
Make sure that a geolocation database has been configured for your NST probe prior to attempt to use '''IPv4 Address Conversations''' geolocation. Use the ''''[[HowTo_Setup_The_NST_System_To_Geolocate_Data | IP Geolocate Configure]]'''' button shown below to manage the global geolocation policy for the NST system. This allows one to make latitude and longitude coordinate adjustments, configure '''Private IPv4 Addresses''' & '''Network''' coordinate locations and select a geolocation database source. In addition, one can also download and manage the '''[http://www.maxmind.com/ MaxMind]''' "'''GeoIP Country Edition'''", the enhanced "'''GeoIP Lite City Edition'''" and the "'''GeoIP AS Number Edition'''" data sets.
Make sure that a geolocation database has been configured for your NST system prior to geolocating '''IPv4 Address Conversations'''. Use the ''''[[HowTo_Setup_The_NST_System_To_Geolocate_Data | IP Geolocate Configure]]'''' button shown below to manage the global geolocation policy for the NST system. This allows one to make latitude and longitude coordinate adjustments, configure '''Private IPv4 Addresses''' & '''Network''' coordinate locations and select a geolocation database source. In addition, one can also download and manage the '''[http://www.maxmind.com/ MaxMind]''' "'''GeoIP Country Edition'''", the enhanced "'''GeoIP Lite City Edition'''" and the "'''GeoIP AS Number Edition'''" data sets.


== '''Geolocate IPv4 Address Conversations''' ==
== '''Geolocate IPv4 Address Conversations''' ==
=== '''Geolocation: Map & Network Packet Capture Decode Settings''' ===
=== '''Geolocation: Map & Network Packet Capture Decode Settings''' ===
Both the NST WUI '''Single-Tap''' and '''[[ Multi-Tap_Network_Packet_Capturing | Multi-Tap]]''' '''Network Packet Capture''' implementation support the ability to geolocate an '''IPv4 Address Conversation List''' derived from a '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark]''' decode. Once a capture file is available for decode analysis, you can then perform '''IPv4 Address Conversation List''' geolocations. The caption below shows the '''Text-Based Protocol Analyzer Decode''' section for the '''Single-Tap Network Packet Capture''' implementation with the '''Advanced Decode Protocol Analyzer Options''' folder ''expanded''. One can use the "'''Conversations - World Map'''" button to render '''IPv4 Address Conversations''' on a Mercator World Map bit image or the "'''Conversations - KML'''" button to render the '''IPv4 Address Conversations''' on an Earth Browser like '''[http://earth.google.com Google Earth]'''.
Both the NST WUI '''Single-Tap''' and '''[[ Multi-Tap_Network_Packet_Capturing | Multi-Tap]]''' '''Network Packet Capture''' implementation support the ability to geolocate an '''IPv4 Address Conversation List''' derived from output produced by the '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark]''' command. Once a packet capture file is available for decode analysis, you can then perform '''IPv4 Address Conversation List''' geolocations. The caption below shows the '''Text-Based Protocol Analyzer Decode''' section for the '''Single-Tap Network Packet Capture''' implementation with the '''Advanced Decode Protocol Analyzer Options''' folder ''expanded''. One can use the "'''Conversations - World Map'''" button to render '''IPv4 Address Conversations''' on a Mercator World Map bit image or the "'''Conversations - KML'''" button to render the '''IPv4 Address Conversations''' on an Earth Browser like '''[http://earth.google.com Google Earth]'''.


==== '''Geolocate Mercator World Map Conversation List Options''' ====
==== '''Geolocate Mercator World Map Conversation List Options''' ====
Line 47: Line 44:
* '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark] Display Filter Expression:''' Apply a display filter to isolate or limit the amount of network traffic to geolocate.
* '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark] Display Filter Expression:''' Apply a display filter to isolate or limit the amount of network traffic to geolocate.


* '''Advanced Decode Options:''' Use an advanced decode preference option like: "'''-o ip.use_geoip:TRUE'''" to allow geolocate display filter expressions (e.g., '''ip.geoip.lat > "23.5"''').
* '''Advanced Decode Options:''' Use an advanced decode preference option like: "'''-o ip.use_geoip:TRUE'''" to allow geolocate display filter expressions (e.g., '''ip.geoip.lat > "23.5"'''). However, be aware that geolocation filters applied by '''tshark''' use a different method for resolving locations and will not honor any custom locations you may have added to your geolocation database.


[[Image:Single_packet_capture_decode.png|center|frame|Single-Tap Network Packet Capture Text-Based Decode Section]]
[[Image:Single_packet_capture_decode.png|center|frame|Single-Tap Network Packet Capture Text-Based Decode Section]]
Line 73: Line 70:
[[Image:Conversations_wm_zoom.png|center|frame|Mercator World Map IPv4 Address Conversations: Albany, N.Y. To www.cnn.com - Zoomed In.]]
[[Image:Conversations_wm_zoom.png|center|frame|Mercator World Map IPv4 Address Conversations: Albany, N.Y. To www.cnn.com - Zoomed In.]]


The map below was ''generated'' using the same network packet capture but a '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark]''' display filter was used to limit the number of packets processed. Hover the mouse pointer over the "'''Information Icon'''" to reveal detailed capture file configuration settings, the '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark]''' command with display filter use to generate the '''IPv4 Address Conversation List''' and the geolocation results.
The map below was ''generated'' using the same network packet capture but a '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark]''' display filter was used to limit the number of packets processed. Hover the mouse pointer over the "'''Information Icon'''" to reveal detailed capture file configuration settings, the '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark]''' invocation including the display filter options used to generate the '''IPv4 Address Conversation List''' and the geolocation results.


<div class="centerBlock"><div class="noteMessage">'''Note:''' The '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark]''' display filter: ''frame.number >= 1 && frame.number <= 1000'' was used to limit the number of network packets processed for the results shown below. Notice that there are only "'''2'''" out of a total of "'''21'''" '''IPv4 Address Conversations''' geolocated.</div></div>
<div class="centerBlock"><div class="noteMessage">'''Note:''' The '''[http://www.wireshark.org/docs/man-pages/tshark.html tshark]''' frame display filter: ' <i>frame.number >= 1 && frame.number <= 1000 </i>' was used to limit the number of network packets processed for the results shown below. Notice that after applying the filter, only "'''2'''" of the original "'''21'''" '''IPv4 Address Conversations''' were geolocated.</div></div>


[[Image:Conversations_wm_filter_tooltip.png|center|frame|Mercator World Map IPv4 Address Conversations: Albany, N.Y. To www.cnn.com - With Display Filter and Information Tool Tip.]]
[[Image:Conversations_wm_filter_tooltip.png|center|frame|Mercator World Map IPv4 Address Conversations: Albany, N.Y. To www.cnn.com - With Display Filter and Information Tool Tip.]]


=== '''KML Document''' ===
=== '''KML Document (Google Earth)''' ===
Use the "'''Conversations - World Map'''" button to start the process of ''generating'' an '''IPv4 Address Conversations''' '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' document that can be rendered on a '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' '''Earth Browser''' such as '''[http://earth.google.com Google Earth]'''. A '''Progress''' page will first appear during the creation of the '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' document which can take some time depending on the number of conversations to geolocate. Once the document has completed, you can view it in '''[http://earth.google.com Google Earth]''' (See the HowTo document: [[HowTo_Setup_Your_Client_System_To_View_Geolocation_Data | HowTo Setup Your Client System To View Geolocation Data]]).
Use the "'''Conversations - World Map'''" button to start the process of ''generating'' an '''IPv4 Address Conversations''' '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' document that can be rendered on a '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' '''Earth Browser''' such as '''[http://earth.google.com Google Earth]'''. A '''Progress''' page will first appear during the creation of the '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' document which can take some time depending on the number of conversations to geolocate. Once the document has completed, you can view it in '''[http://earth.google.com Google Earth]''' (See the HowTo document: [[HowTo_Setup_Your_Client_System_To_View_Geolocation_Data | HowTo Setup Your Client System To View Geolocation Data]]).


Each '''IPv4 Address Conversation''' geolocated will appear as a '''[http://en.wikipedia.org/wiki/Great_circle Great Circle]''' between each conversation host endpoint. Each conversation '''[http://en.wikipedia.org/wiki/Great_circle Great Circle]''' contains a ''''Conversation Description'''' balloon depicting network traffic information. Click on a '''[http://en.wikipedia.org/wiki/Great_circle Great Circle]''' to reveal the ''''Conversation Description'''' balloon.
Conversations will be shown using the '''[http://en.wikipedia.org/wiki/Great_circle great-circle distance]''' connecting the two hosts. This represents the shortest distance between the two hosts while remaining on the surface of the planet ("as the crow flies"). Each line drawn has a ''''Conversation Description'''' balloon depicting network traffic information. Click on a line to reveal the ''''Conversation Description'''' balloon.
Additionally, each conversation host endpoint marker contains a ''''Host Description'''' balloon depicting network traffic information. Click on a host marker to reveal the ''''Host Description'''' balloon. Hyperlinks are also provided to one or more NST WUI ''''IP Tools'''' pages for additional network processing using the conversation host endpoint IP Addresses.
 
<div class="centerBlock"><div class="noteMessage">'''Note:''' Since each '''IPv4 Address Conversation''' is drawn as a '''[http://en.wikipedia.org/wiki/Great_circle great-circle distance]''', the "'''Hot Spot'''" to click on to reveal the ''''Conversation Description'''' balloon can be difficult to find. The "'''Hot Spot'''" may be significantly above or below the conversation line. Instead of trying to find the "'''Hot Spot'''", one can click on a conversation within the ''''Conversations'''' folder located under the '''Temporary Places''' on the left-hand sidebar. </div></div>
 
Additionally, each conversation host endpoint marker contains a ''''Host Description'''' balloon depicting network traffic information. Click on a host marker to reveal the ''''Host Description'''' balloon. Hyperlinks are provided to the NST WUI ''''IP Tools'''' page for the times when one wants to gather more information on a particular host.


{|cellpadding="5"
{|cellpadding="5"
Line 91: Line 91:
|}
|}


When using '''[http://earth.google.com Google Earth]''', one can also view the ''''Document Description'''' balloon by clicking on the generated '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' '''ntop Hosts''' place found under '''Temporary Places''' within the sidebar on the left-hand side. You can also expand the '''ntop Hosts''' place to ''explore'' all geolocated hosts and associated network statistics.
When using '''[http://earth.google.com Google Earth]''', one can also view the ''''Document Description'''' balloon by clicking on the generated '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' '''IPv4 Address Conversations''' place found under '''Temporary Places''' within the sidebar on the left-hand side. Hyperlinks are provided back to one or more NST WUI ''''Single-Tap'''' or ''''Multi-Tap Network Packet Capture'''' pages for further analyzing the network packet capture.
 
The ''''IPv4 Address Conversations - with Display Filter'''' example above shows how one can ''apply'' '''a Host Display Filter''' to focus your geolocation results on selective conversations.
 
You can also expand both the '''Conversations''' and '''Hosts''' folders (shown on the left side in the images above) to ''explore'' all geolocated conversations and hosts with associated network traffic information.


=== '''Working With Historical Generated IPv4 Address Conversations Maps''' ===
=== '''Working With Historical Generated IPv4 Address Conversations Maps''' ===
The generation of geolocated '''IPv4 Address Conversations''' maps and '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' documents is browser session based. Each time one is generated it is saved in a hierarchical time-stamped file layout for historical review and analysis. The "'''nstgeolocate Session Manager'''" page can be used to ''manage'' or ''view'' all created '''IPv4 Address Conversations''' maps and '''[http://en.wikipedia.org/wiki/Keyhole_Markup_Language KML]''' documents by the NST probe. See section '''[[HowTo_Automate_%26_Manage_NST_Geolocation_Results | HowTo Automate & Manage NST Geolocation Results]]''' for details.

Latest revision as of 11:37, 17 November 2010

Overview

This HowTo explains the procedure for geolocating IPv4 Address Conversations using the NST WUI and rendering the results on either a Mercator World Map projection or on a KML Earth Browser such as Google Earth, Google Maps or Marble.

There are a couple of items to consider prior to geolocating IPv4 Address Conversations. First, does the network packet capture make sense to use for geolocation. The list below are packet capture characteristics that would not be considered desirable for geolocation:

  • No IPv4 Addresses exist in the capture file. Results: No geolocations would be rendered.
  • All hosts in the capture file are located at the same physical location. Results: Geolocations would appear at a single point.
  • All hosts are Private IP Addresses with no associated geolocation database information. Results: No geolocations would be rendered. Note: This condition can be corrected, see section: IP Geolocation Database & Adjustments.

Secondly, has the geolocation database been setup and configured for your NST probe. This includes the addition of configuring any Private IPv4 Addresses or Network geolocation information. For details, see the section: IP Geolocation Database & Adjustments.

IP Geolocation Database & Adjustments

Make sure that a geolocation database has been configured for your NST system prior to geolocating IPv4 Address Conversations. Use the ' IP Geolocate Configure' button shown below to manage the global geolocation policy for the NST system. This allows one to make latitude and longitude coordinate adjustments, configure Private IPv4 Addresses & Network coordinate locations and select a geolocation database source. In addition, one can also download and manage the MaxMind "GeoIP Country Edition", the enhanced "GeoIP Lite City Edition" and the "GeoIP AS Number Edition" data sets.

Geolocate IPv4 Address Conversations

Geolocation: Map & Network Packet Capture Decode Settings

Both the NST WUI Single-Tap and Multi-Tap Network Packet Capture implementation support the ability to geolocate an IPv4 Address Conversation List derived from output produced by the tshark command. Once a packet capture file is available for decode analysis, you can then perform IPv4 Address Conversation List geolocations. The caption below shows the Text-Based Protocol Analyzer Decode section for the Single-Tap Network Packet Capture implementation with the Advanced Decode Protocol Analyzer Options folder expanded. One can use the "Conversations - World Map" button to render IPv4 Address Conversations on a Mercator World Map bit image or the "Conversations - KML" button to render the IPv4 Address Conversations on an Earth Browser like Google Earth.

Geolocate Mercator World Map Conversation List Options

The following options are available when generating an IPv4 Address Conversations Mercator World Map:

  • Mark Type: Select either a "point", "plus sign" or a "star" as the marker symbol.
  • Mark Color: Select the marker symbol color from a set of predefined colors.
  • Connect Conversation Lines: If selected, a line between each conversation host endpoint will be drawn. If unselected, just plot the hosts that can be geolocated from the network packet capture.
  • Conversations Annotation: Use this text field to describe the IPv4 Address Conversations Mercator World Map. If omitted, the capture note associated with the capture file will be used.

Geolocate KML Conversation List Options

The following options are available when generating an IPv4 Address Conversations KML document for rendering on Google Earth:

  • Conversation Line Width: Select a fixed conversation line width from 1 to 10 pixels. One can also select the value: "graduated" to set each conversation line width based on the total sent and received network traffic for the conversation.
  • Conversations Annotation: Use this text field to describe the IPv4 Address Conversations KML document. If omitted, the capture note associated with the capture file will be used.

Geolocate tshark Decode Conversation List Options

The following options are available when geolocating an IPv4 Address Conversation list using the tshark Protocol Analyzer.

  • Name Resolve: Provide network name resolution for each host in the capture file.
  • tshark Display Filter Expression: Apply a display filter to isolate or limit the amount of network traffic to geolocate.
  • Advanced Decode Options: Use an advanced decode preference option like: "-o ip.use_geoip:TRUE" to allow geolocate display filter expressions (e.g., ip.geoip.lat > "23.5"). However, be aware that geolocation filters applied by tshark use a different method for resolving locations and will not honor any custom locations you may have added to your geolocation database.
Single-Tap Network Packet Capture Text-Based Decode Section


The screen shot below shows the tool tip for the "Conversations - World Map" button located in the "Specialized tshark Decode Output Formats" section.

Single-Tap Network Packet Capture Text-Based Decode Section With The "Conversations - World Map" Tool Tip Shown

Mercator World Map

Use the "Conversations - World Map" button to start the process of generating IPv4 Address Conversations on a Mercator World Map bit image. A Progress Generation page similar to the one shown below will be displayed during the course of creating the entire bit image.

Note: One may need to allow pop-up window generation on your browser for the NST host so that the Progress Generation page can be displayed correctly.

One has some control during the generate process to "Hold", "Cancel" or "Continue". One can also use the "View Generate Log" to see all commands and log output for geolocating IPv4 Address Conversations on the Mercator World Map projection.

Mercator World Map IPv4 Address Conversations: Progress Generate Status.

Once completed, the Progress Generation page will be replaced with the final rendering of IPv4 Address Conversations on a Mercator World Map projection similar to one one shown below. A total of "21" IPv4 Address Conversations and "22" Hosts were geolocated.

Mercator World Map IPv4 Address Conversations: Albany, N.Y. To www.cnn.com.

Use the "Image Control Button Grid" to maneuver and/or zoom the map within your browser window (See page: World Map Image Control Button Grid for details). The map below was zoomed in and center positioned over the North American Continent to better illustrate the geolocated IPv4 Address Conversations.

Mercator World Map IPv4 Address Conversations: Albany, N.Y. To www.cnn.com - Zoomed In.

The map below was generated using the same network packet capture but a tshark display filter was used to limit the number of packets processed. Hover the mouse pointer over the "Information Icon" to reveal detailed capture file configuration settings, the tshark invocation including the display filter options used to generate the IPv4 Address Conversation List and the geolocation results.

Note: The tshark frame display filter: ' frame.number >= 1 && frame.number <= 1000 ' was used to limit the number of network packets processed for the results shown below. Notice that after applying the filter, only "2" of the original "21" IPv4 Address Conversations were geolocated.
Mercator World Map IPv4 Address Conversations: Albany, N.Y. To www.cnn.com - With Display Filter and Information Tool Tip.

KML Document (Google Earth)

Use the "Conversations - World Map" button to start the process of generating an IPv4 Address Conversations KML document that can be rendered on a KML Earth Browser such as Google Earth. A Progress page will first appear during the creation of the KML document which can take some time depending on the number of conversations to geolocate. Once the document has completed, you can view it in Google Earth (See the HowTo document: HowTo Setup Your Client System To View Geolocation Data).

Conversations will be shown using the great-circle distance connecting the two hosts. This represents the shortest distance between the two hosts while remaining on the surface of the planet ("as the crow flies"). Each line drawn has a 'Conversation Description' balloon depicting network traffic information. Click on a line to reveal the 'Conversation Description' balloon.

Note: Since each IPv4 Address Conversation is drawn as a great-circle distance, the "Hot Spot" to click on to reveal the 'Conversation Description' balloon can be difficult to find. The "Hot Spot" may be significantly above or below the conversation line. Instead of trying to find the "Hot Spot", one can click on a conversation within the 'Conversations' folder located under the Temporary Places on the left-hand sidebar.

Additionally, each conversation host endpoint marker contains a 'Host Description' balloon depicting network traffic information. Click on a host marker to reveal the 'Host Description' balloon. Hyperlinks are provided to the NST WUI 'IP Tools' page for the times when one wants to gather more information on a particular host.

IPv4 Address Conversations
IPv4 Address Conversations - with Conversation Balloon
IPv4 Address Conversations - with Display Filter

When using Google Earth, one can also view the 'Document Description' balloon by clicking on the generated KML IPv4 Address Conversations place found under Temporary Places within the sidebar on the left-hand side. Hyperlinks are provided back to one or more NST WUI 'Single-Tap' or 'Multi-Tap Network Packet Capture' pages for further analyzing the network packet capture.

The 'IPv4 Address Conversations - with Display Filter' example above shows how one can apply a Host Display Filter to focus your geolocation results on selective conversations.

You can also expand both the Conversations and Hosts folders (shown on the left side in the images above) to explore all geolocated conversations and hosts with associated network traffic information.

Working With Historical Generated IPv4 Address Conversations Maps

The generation of geolocated IPv4 Address Conversations maps and KML documents is browser session based. Each time one is generated it is saved in a hierarchical time-stamped file layout for historical review and analysis. The "nstgeolocate Session Manager" page can be used to manage or view all created IPv4 Address Conversations maps and KML documents by the NST probe. See section HowTo Automate & Manage NST Geolocation Results for details.