Previous Page
Next Page

VPN Client Troubleshooting

The remainder of this chapter will focus on troubleshooting some common problems when using the Cisco VPN Client software. The first topic I'll discuss is the logging abilities of the software client: the Log Viewer component of the VPN Client. Then I'll discuss some common problems you'll typically come across and what to look for on your computer (or concentrator) to troubleshoot these problems:

  • Authentication Problems

  • ISAKMP/IKE Policy Mismatch Issues

  • Address Assignment Troubleshooting

  • Split Tunneling Problems

  • Address Translation Problems

  • Fragmentation Issues

  • Microsoft Network Neighborhood Issues

Log Viewer

In the 3.x version of the VPN Client software, the Log viewer was a separate application; in 4.x, the Log application was integrated into the VPN Client software. To view the events in the log file, perform one of the following in the 4.x GUI:

  • Click the Log tab in the middle of the windowthis can be seen in Figure 12-22.

    Figure 12-22. Log Event Viewer

  • From the main menu you can choose Log > Log Window to pull up this window.

The information in the Log event viewer window can be used to help in troubleshooting client problems. You can control the amount of logging information, search the logging information, and view the logging information from this window.

Formatting of Logging Information

Events in the Log window have the following format in two or more lines:

event_# time date Sev=type/level event_class/message_ID
message_description

Each event is assigned a unique number in sequential order. Following this is the time and date of the event, and then the type and severity level of the event. After this is the event class and message ID. Table 12-7 lists the logging event classes. For every class, there are three severity types: fault (1: system failure or unrecoverable error), warning (23: possible system failure or major problem), and informational (46: events concerning connections and processes). Most events you'll see will have a severity level of 46.

Table 12-7. Logging Classes

Class Name

Description

CERT

Obtaining, validating, and renewing certificates

CLI

Interacting with the client CLI (not GUI)

CM

Using the connection manager to establish a secure session

CVPND

Initializing client services and controlling message processes and flows by the Cisco VPN Client daemon

GUI

Configuring the client software via the GUI and initiating and monitoring VPN sessions

FIREWALL

Using the CIC firewall

IKE

Building and managing ISAKMP/IKE Phase 1 SAs

IPSEC

Building and managing ISAKMP/IKE Phase 2 SAs, including applying rules to traffic that needs to be protected

PPP

Establishing VPN sessions via PPP dialup

XAUTH

Validating user account information for user authentication during ISAKMP/IKE Phase 1


You can control the capturing of these classes and levels by performing one of the following:

  • From the main menu, choose Log > Log Settings.

  • Click the Log tab and then click the Log Settings button.

This will pull up a separate window, shown in Figure 12-23. You can change the logging level by using the drop-down selector for each logging class. Supported logging levels include:

  • "Disabled" Turns on the logging for the specified class.

  • "Low" Turns on logging for severity levels 13; this is the default for all classes.

  • "Medium" Changes logging for the specified class to include levels 14, which includes the informational level 4 events.

  • "High" Changes logging for the specified class to include levels 16, which is all logging levels.

Figure 12-23. Log Settings


After making your changes, click the OK button to activate and save the changes. Your changes take effect immediately for any new logging information.

Tip

When troubleshooting problems, the default logging level will, in most cases, not display enough information for you to determine the exact problem. In this case, change the log level settings for the specific event(s) to assist in the troubleshooting process. For troubleshooting IPsec session problems, you would typically set the IKE and IPSEC logging classes to "High." I've also found that sometimes I need to troubleshoot the problem from the Easy VPN Server and VPN Client to resolve a specific problem.


Disabling the Logging Feature

A new log file is created every time you reboot your computer. Because of the amount of logging information created and the disk space issues that this poses, you might want to disable logging globally on the client. By default, logging is enabled. You can disable this function by performing one of the following:

  • From the main menu, choose Log > Disable

  • Click the Log tab and click the Disable button above the three tabs

Searching for Logging Information

Because you might have set the logging levels to 4 or higher, it sometimes might be difficult to find a particular message or event class type within the log file. You can search the log file for a certain string in it by choosing Log > Search Log from the main menu. A window will pop up where you can enter the string you want to search for; then click the Find Next button. This causes the VPN Client to search the log file, starting at the beginning, for the terms you entered. The first time the client finds a match, it highlights the term(s). Click the Find Next button again to find the next occurrence.

Note

Searches are case-insensitive and do not support a wildcard pattern matching feature.


Clearing Logging Information

You can clear the events in the Log window by performing one of the following:

  • From the main menu, choose Log > Clear.

  • Click the Log tab and then click the Clear button.

Note

Please note that the clear logging feature clears only the logging information in this windowit does not clear the log entries in the log file on the computer's hard drive.


Authentication Problems

Now that you have a basic understanding of the client's Log viewer application, the rest of the chapter will focus on using this feature to troubleshoot problems. This section will look at troubleshooting IPsec authentication issues when using the client.

Troubleshooting XAUTH (user authentication) issues is fairly easy because the user is re-prompted for username and password if these are incorrect; this is not true if you've misconfigured the pre-shared key for the group the user belongs to. If you've misconfigured the group's pre-shared key, you'll see output in the Log viewer something like that shown in Example 12-6 (this is with the default level settings of "Low"):

Example 12-6. Group Pre-Shared Key Authentication Problem
1      13:23:46.154  02/09/05  Sev=Warning/3 IKE/0xE3000056
The received HASH payload cannot be verified
2      13:23:46.154  02/09/05  Sev=Warning/2 IKE/0xE300007D
Hash verification failed... may be configured with invalid group
       password.
3      13:23:46.154  02/09/05  Sev=Warning/2 IKE/0xE3000099
Failed to authenticate peer (Navigator:904)
4      13:23:46.154  02/09/05  Sev=Warning/2 IKE/0xE30000A5
Unexpected SW error occurred while processing Aggressive Mode
       negotiator:(Navigator:2202)

In Example 12-6, there are four events. Events 1 and 2 indicate a problem with pre-shared key authentication with the group, and event 3 shows that device authentication has failed.

You also can use the Log viewer to troubleshoot problems related to obtaining and using certificates. Most of these problems probably will be associated with the validity date on a certificate. If this is the case and the client's own identity certificate has expired, a window will pop up indicating that the certificate is invalid because of its date. If the peer's certificate has expired, this will appear in the normal logging activity ("Low") in the client's Log window.

Tip

When you are experiencing a problem, I recommend clearing the log first and then re-attempting the session; then, the only log messages you'll have to decipher are those that relate to this particular problem, making the troubleshooting process easier. Also, I've experienced glitches in certain client versions where changing a logging level won't take effect until you exit the VPN Client and restart.


ISAKMP/IKE Policy Mismatch Issues

Sometimes you'll experience a problem where the appropriate ISAKMP/IKE Phase 1 transform, called an IKE Proposal on Cisco VPN 3000 concentrators, or the ISAKMP/ IKE Phase 2 transform doesn't match that found on the client. The Cisco Easy VPN clients, which include the Cisco VPN Client software, already have a list of predefined proposals and policies incorporated into their software. The Cisco Easy VPN Server must have a corresponding match for both the Phase 1 and Phase 2 transforms/policies. If a match is not found in Phase 1, no session is established. If no match is found in Phase 2, the ISAKMP/IKE Phase 1 management session will be built, but the data connection will failin this instance, the client will tear down the management connection also.

On the Cisco VPN Client, if there is not a matching transform/policy on the Easy VPN Server, you'll see in the message shown in Example 12-7 in the connection status pop-up window:

Example 12-7. Mismatched ISAKMP/IKE Phase 1 Policy with the Default Logging Level
Initializing the connection...
Contacting the security gateway at 192.1.1.1...
Secure VPN Connection terminated locally by the Client.
Reason 412: The remote peer is no longer responding.
Not connected.

The problem with the above message is that it doesn't illuminate the cause of the problem. On top of this, there are no events recorded in the Log view when the event class for "IKE" is set to "Low." In Example 12-8, I've changed the logging level for "IKE" to "Medium."

Example 12-8. Mismatched ISAKMP/IKE Phase 1 Policy With a "Medium" Logging Level
1      16:24:54.549  02/09/05  Sev=Info/4 IKE/0x63000013
SENDING >>> ISAKMP OAK AG (SA, KE, NON, ID, VID(Xauth), VID(dpd), VID(Nat-T),
  VID(Frag), VID(Unity)) to 192.1.1.1
2      16:24:59.906  02/09/05  Sev=Info/4 IKE/0x63000021
Retransmitting last packet!
3      16:24:59.906  02/09/05  Sev=Info/4 IKE/0x63000013
SENDING >>> ISAKMP OAK AG (Retransmission) to 192.1.1.1
4      16:25:04.914  02/09/05  Sev=Info/4 IKE/0x63000021
Retransmitting last packet!
5      16:25:04.914  02/09/05  Sev=Info/4 IKE/0x63000013
SENDING >>> ISAKMP OAK AG (Retransmission) to 192.1.1.1
6      16:25:09.921  02/09/05  Sev=Info/4 IKE/0x63000021
Retransmitting last packet!
7      16:25:09.921  02/09/05  Sev=Info/4 IKE/0x63000013
SENDING >>> ISAKMP OAK AG (Retransmission) to 192.1.1.1
8      16:25:14.928  02/09/05  Sev=Info/4 IKE/0x63000017
Marking IKE SA for deletion  (I_Cookie=92663B6F83385211
R_Cookie=0000000000000000) reason = DEL_REASON_PEER_NOT_RESPONDING
9      16:25:15.439  02/09/05  Sev=Info/4 IKE/0x6300004A
Discarding IKE SA negotiation (I_Cookie=92663B6F83385211
R_Cookie=0000000000000000) reason = DEL_REASON_PEER_NOT_RESPONDING
10     16:25:15.489  02/09/05  Sev=Info/4 IKE/0x63000001
IKE received signal to terminate VPN connection

Here you can see some additional information; for example, message #1 shows the ISAKMP policies being sent to the VPN gateway and messages #27 show the client resending this information. Finally, in message #8, the client gives up the session. Based on the information shown here, the appropriate ISAKMP/IKE Phase 1 policy hasn't been configured or enabled on the Easy VPN Server.

Note

On Cisco VPN 3000 concentrators, this shouldn't be an issue because some pre-defined policies for Cisco VPN clients are already enabled. Plus, troubleshooting the client connection on the concentrator is easier in this particular example.


Address Assignment Troubleshooting

One common problem with IPsec remote access sessions is that the Easy VPN Server administrator forgets to create an IP address pool for the group or all of the addresses in the pool are currently being used. When this happens, ISAKMP/IKE Phase 1 fails because the user device cannot obtain an internal IP address for the ESP tunnel connections.

When this occurs, you'll see the same message shown previously in Example 12-7. The only difference is that the reason code will be 427 instead of 412. There are two problems with this situation: this code doesn't tell you too much about the problem and the default logging level of the Log viewer of the client doesn't display the actual problem.

In this case you know that XAUTH works because you must supply the correct user name and password; otherwise you'll be reprompted for this information. Therefore the problem must be associated with IKE Mode Config or ISAKMP/IKE Phase 2. Example 12-9 shows the client Log with the IKE event class set to "HIGH" when this particular problem exists:

Example 12-9. Client Addressing Problem
1      15:03:39.734  02/10/05  Sev=Info/6 IKE/0x6300003B
Attempting to establish a connection with 192.1.1.1.
2      15:03:39.754  02/10/05  Sev=Info/4 IKE/0x63000013
SENDING >>> ISAKMP OAK AG (SA, KE, NON, ID, VID(Xauth), VID(dpd),
VID(Nat-T), VID(Frag), VID(Unity)) to 192.1.1.1
3      15:03:40.094  02/10/05  Sev=Info/5 IKE/0x6300002F
Received ISAKMP packet: peer = 192.1.1.1
4      15:03:40.094  02/10/05  Sev=Info/4 IKE/0x63000014
RECEIVING <<< ISAKMP OAK AG (SA, KE, NON, ID, HASH, VID(Unity),
VID(Xauth), VID(dpd), VID(Frag), VID(?), VID(?)) from 192.1.1.1
5      15:03:40.094  02/10/05  Sev=Info/5 IKE/0x63000001
Peer is a Cisco-Unity compliant peer
output omitted
14     15:03:40.134  02/10/05  Sev=Info/4 IKE/0x63000014
RECEIVING <<< ISAKMP OAK TRANS *(HASH, ATTR) from 192.1.1.1
15     15:03:50.559  02/10/05  Sev=Info/4 IKE/0x63000013
SENDING >>> ISAKMP OAK INFO *(HASH, NOTIFY:HEARTBEAT) to 192.1.1.1
16     15:03:50.559  02/10/05  Sev=Info/6 IKE/0x63000052
Sent a keepalive on the IKE SA
output omitted
26     15:03:51.541  02/10/05  Sev=Info/4 IKE/0x63000080
Delete Reason Code: 4 --> PEER_DELETE-IKE_DELETE_NO_ERROR.
27     15:03:51.541  02/10/05  Sev=Info/5 IKE/0x6300003C
Received a DELETE payload for IKE SA with Cookies:
I_Cookie=BDB5CBE8D8C747C2 R_Cookie=1E581493E9C2CBE9
28     15:03:51.541  02/10/05  Sev=Info/4 IKE/0x63000017
Marking IKE SA for deletion  (I_Cookie=BDB5CBE8D8C747C2
R_Cookie=1E581493E9C2CBE9) reason = PEER_DELETE-IKE_DELETE_NO_ERROR
29     15:03:52.061  02/10/05  Sev=Info/4 IKE/0x6300004A
Discarding IKE SA negotiation (I_Cookie=BDB5CBE8D8C747C2
R_Cookie=1E581493E9C2CBE9) reason = PEER_DELETE-IKE_DELETE_NO_ERROR
30     15:03:52.161  02/10/05  Sev=Info/4 IKE/0x63000001
IKE received signal to terminate VPN connection

Line 14 is where XAUTH begins (even though this isn't clear from the log output). Beginning with Line 26 you can see the VPN session being terminated. The problem with the above output is that there is nothing there (or in the pop-up window) that explains what the actual problem is. If you are able to obtain an address, you should see something like that shown in Example 12-10 in the clients' Log view:

Example 12-10. Client Addressing When Functioning Correctly
27     15:27:06.068  02/11/05  Sev=Info/5 IKE/0x63000010
MODE_CFG_REPLY: Attribute = INTERNAL_IPV4_ADDRESS: ,
value = 192.168.101.110
28     15:27:06.068  02/11/05  Sev=Info/5 IKE/0x63000010
MODE_CFG_REPLY: Attribute = INTERNAL_IPV4_NETMASK: ,
value = 255.255.255.0

One handy tool you can use is located at the Cisco site, called the "VPN Client Error Lookup Tool": http://www.cisco.com/en/US/products/sw/secursw/ps2308/products_tech_note09186a00801f253d.shtml. This tool is shown in Figure 12-24.

Figure 12-24. VPN Client Error Lookup Tool


At this point, you need to pull up the type of code, such as "Reason" and the number of the code (in this instance "427"). As you can see from the results, it says to check the peer device to see what the problem is. In this example, the Easy VPN Server is a VPN 3000 concentrator. Going to the Live Event Log on the concentrator, the output in Example 12-11 can be seen:

Example 12-11. Live Event Log: Client Addressing Problem
output omitted
378 02/10/2005 15:32:22.530 SEV=5 IKE/132 RPT=17 192.1.1.66
Group [Splitgroup] User [Richard]
Cannot obtain an IP address for remote peer - FAILED
380 02/10/2005 15:32:22.540 SEV=5 IKE/194 RPT=26 192.1.1.66
Group [Splitgroup] User [Richard]
Sending IKE Delete With Reason message: No Reason Provided.

As you can see from message 378, the user ("Richard") in the "Splitgroup" group cannot obtain an IP address. In this situation, make sure that an address pool exists for the group and that there are enough addresses in the pool for all of the users that need to connect at the same time. In message 380, the notification sent to the client is "No Reason Provided," which is what the client displayed.

Tip

In many instances, you'll need to troubleshoot the VPN problem from both ends of the tunnel to determine what the problem is.


Split Tunneling Problems

As you recall from Chapter 2, "VPN Technologies," split tunneling is sending some traffic protected to a VPN peer and other traffic unprotected to other devices. Typically this is used when you want to protect traffic from the remote access device to the corporate office, but don't want to overload the VPN session by sending the user's Internet traffic across this session also. You can use split tunneling to solve this problem.

However, if you don't set up split tunneling correctly, your user(s) will face connectivity problems. You can do a few things on the client end of the connection to troubleshoot this problem. First, determine if the issue is related to name resolution or a connection problem. The following two sections will discuss the additional steps you can take to solve each respective problem.

Connectivity Problems

Once you have established a VPN session to the Easy VPN Server, right-click the yellow padlock icon in the Windows taskbar and choose Statistics. Click the Route Details tab, showing you the window in Figure 12-25. There are two local LAN routes for the two NICs installed on this laptop. In the right-hand column ("Secured Routes"), all traffic is to be encrypted. Based on this, split tunneling for local LAN traffic has been set up and pushed down to the client during IKE Mode Config.

Figure 12-25. Route Details Window


As a further test, click the Tunnel Details tab in the window in Figure 12-25, changing the window contents to what you see in Figure 12-26. Then click the Reset button to reset the statistics. Next, open up a Windows command (DOS) prompt: Go to Start > Run, enter cmd.exe in the Open text box, and click the OK button. Ping a destination that is not located at the corporate site. In this example, you can see 5 under the Packets section for packets discarded, indicating that the destination is not across the tunnel or local LAN segments, and the client is discarding the packets.

Figure 12-26. Tunnel Details Window


Click the Reset button to reset the statistics. Next, ping the inside interface of the Easy VPN Server (the one connected to the corporate network). In the client's Statistics window, you should see the received and sent bytes incrementing in the Bytes section and the number of packets encrypted and decrypted incrementing in the Packets section.

Based on these results, you can access the corporate network, but nothing else. To allow the user to access other places, you'll need to configure split tunneling for the group or allow the user access to the Internet via the IPsec tunnel. This was discussed in Chapter 7, "Concentrator Remote Access Connections with IPsec."

Name Resolution Problems

You also might face problems with name, or DNS, resolution. Remember that for most remote access users, their ISPs assign them a DNS server address. Using an Easy VPN Server for remote access users, you also can push two DNS server addresses down to the client. Without split DNS, the client will use the assigned DNS server address(es) given to it during IKE Mode Config by the Easy VPN Server.

As an example of a problem with name resolution, you might have a DMZ with public devices on it, like a web server, e-mail server, and even a DNS server. These devices have been assigned private IP addresses, but the DMZ DNS server responds with public addresses to requesting devices; an address translation device also translates the public-to-private addresses, and vice versa, to allow for external communications to the DMZ services.

You also have an internal DNS server. Because the DMZ devices have private addresses assigned to their NICs, a DNS query to the internal DNS server comes back with a private address, allowing internal users to use the private addressing scheme to access the DMZ devices (no address translation is needed). When remote access clients access these services, you've set up split tunneling where this traffic is sent unprotected across the Internet; only corporate office internal traffic is protected by the tunnel. However, if you didn't set up split DNS, this would create connectivity problems. For remote access clients, when they performed the query to obtain the DMZ devices' IP addresses, the internal DNS server would respond with a private address, and the remote access clients need a public address (because this traffic isn't tunneled).

One option to solve the problem of incorrect name resolution might be to assign the IP address of the DMZ DNS server during IKE Mode Config to the clients; however, this server might not have internal resolutions for internal devices. Therefore, the best solution is to set up split DNS on the Easy VPN Server. Have two domains, like internal.cisco.com and external.cisco.com. Set up split DNS so that any resolutions for internal.cisco.com are resolved using the IKE Mode Config assigned DNS server and anything else resolved with the ISP's assigned DNS server. Another option is to hard-code the resolution in the user's local "host" file.

Tip

To determine if you are having resolution problems, use the nslookup.exe program (included with Windows) to perform DNS resolutions from the Windows command line. Based on the lookup results, you can determine what DNS server the client is using for the resolution, and the name-to-address resolution. Example 12-12 shows the use of this program. In this example, 4.2.2.1 is one of Verizon's DNS servers (a DNS server assigned by the user's ISP).


Example 12-12. nslookup Program Example
C:\> nslookup www.cisco.com
Server:  vnsc-pri.sys.gtei.net
Address:  4.2.2.1

Non-authoritative answer:
Name:    www.cisco.com
Address:  198.133.219.25

Address Translation Problems

One of the more common problems with remote access sessions deals with issues related to address translation. In many instances, a remote access user will be connecting via a public networkfor example, at a hotel or airportor will have a router for a SOHO session. In most of these instances, the user's sessions are being translated with PAT. As I discussed in Chapter 3, "IPsec," ESP is a Layer-3 protocol and doesn't work with PAT, which requires a Layer-4 transport. You can solve this problem by using NAT-T, IPsec over UDP, or IPsec over TCP; the former two encapsulate ESP in a UDP segment and the latter in a TCP segment. You really don't want to do either unless it's necessary, because the encapsulation process adds additional overhead to the connection.

Address translation issues can be difficult to troubleshoot. On a VPN Client device, it looks as if the connection is made and you see the closed yellow padlock icon in the taskbar. In reality, only the ISAKMP/IKE Phase 1 connection has completed, and the Phase 2 connection has completed but will not function. If the user was using a PIX security appliance for a connection to the ISP, the address translation table would look like that found in Example 12-13. As you can see from this example, only the ISAKMP/IKE Phase 1 port 500 connection has been established.

Example 12-13. PIX Firewall Translation Table with PAT Problem
pixfirewall# show xlate
1 in use, 1 most used
PAT Global 192.1.1.3(1) Local 191.1.1.66(500)

If you're using Dead Peer Detection (DPD, which is enabled by default), these are transported across the management connection, so the client and gateway assume the session is functioning correctly; however, if you would try, from the client, to ping the internal (private) interface of the Easy VPN Server, the ping would fail.

To solve address translation problems involving PAT, you need to use either NAT-T, IPsec over UDP, or IPsec over TCP; this will need to be configured on both the client and Easy VPN Server. On the client, you need to modify the properties of the remote access session. This window was shown previously in Figure 12-4. Make sure the Enable Transport Tunneling check box is checked (this is the default) and either the IPsec over UDP (NAT/PAT) or the IPsec over TCP radio box is checked. You'll then need to configure the Easy VPN Server. I discussed this process for the VPN 3000 concentrators in Chapter 7, "Concentrator Remote Access Connections with IPsec." On the concentrator, NAT-T and IPsec over TCP are configured globally while IPsec over UDP is configured within the group. On the client, if you choose IPsec over UDP (NAT/PAT) and you have both NAT-T and IPsec over UDP enabled on the Easy VPN Server, NAT-T takes precedence.

Example 12-14 shows an example of a PIX security appliance's translation table where NAT-T is used for the remote access session by the client and the PIX is the client's connection to the ISP, where PAT is being used for address translation. As you can see in this example, the first PAT translation is for the ISAKMP/IKE Phase 1 connection (port 500) and the second translation for NAT-T (UDP port 4,500), the encapsulated data connections.

Example 12-14. PIX Firewall Translation Table with PAT and NAT-T
pixfirewall# show xlate
2 in use, 4 most used
PAT Global 192.1.1.3(2) Local 191.1.1.66(500)
PAT Global 192.1.1.3(1024) Local 191.1.1.66(4500)

Fragmentation Issues

As mentioned in the last section, using NAT-T, IPsec over UDP, and IPsec over TCP adds additional overhead to an IPsec session. In many instances, the packets will have to be fragmented to be transmitted across the wire. Here's a quick illustration of this when using L2TP over IPsec over UDP: the L2TP header is 12 bytes (includes the PPP header), the UDP header is 8 bytes, and the IP header is 20 bytes. And if data sequencing is used (this is enabled on Cisco devices, by default), you'll need an additional 440 bytes. Adding these up totals 40 or more bytes for the encapsulation process; therefore, if you're transmitting this across an Ethernet medium, you might need 1,500 + 40 = 1,540 bytes. In this situation, the VPN device needs to fragment the packet to send it out the interface. The first fragment will contain 40 bytes of the L2TP encapsulation plus 1,460 bytes of the data, and the second fragment will contain the last 20 bytes of the IP header and the 40 remaining bytes of data (the L2TP header is only found in the first fragment).

Problems that Fragmentation Creates

There are a few problems that fragmentation can create:

  • Slower throughput

  • Data sometimes transmitted

  • Data never transmitted

One of the most common results of fragmentation is slower throughput. I've seen many administrators complain about throughput once they've set up a VPN implementation. At first they blame it on the encryption and decryption process; but in most cases, fragmentation is causing the problem. For example, if an IOS router has to handle the fragmentation and reassembly process, it must do this at the process level, which is very CPU-intensive. In many cases, I've seen throughput drop down to 1050 percent when fragmentation is occurring.

Another problem with fragmentation is that an intermediate device might need to perform the fragmentation to send the traffic across an intermediate link with a smaller MTU size, but the intermediate device isn't capable of handling fragmentation. In this instance, you'll see one of two scenarios. In the first one, sometimes things will work and sometimes they won't. Take an example of sending e-mail in this situation. If it's a small message (about 510 lines), it probably will be sent without fragmentation needed; but if you send an e-mail with an attachment, fragmentation probably will be needed and in this particular situation, it will fail. Or if you're always sending large data files, these will always fail because the intermediate device can't fragment the large packets. Typically, if you can ping a destination across a VPN, like an e-mail server, but can't send large e-mail messages to it, you're probably experiencing a fragmentation problem.

Misdiagnosing VPN Problems

One of the most common questions I'm asked is how much throughput a company will lose when deploying a VPN implementation such as IPsec. It really depends on a few factors, like how the information is transported between the two VPN peers. In a worse case example, you might have to deal with NAT-T, IPsec over UDP, or IPsec over TCP overhead in addition to PPPoE. Or you might have to deal with L2TP over IPsec over PPPoE, which is even worse! In most cases, though, you'll lose about 200 bytes. However, I commonly see administrators implement a VPN solution and then scratch their heads and give up when their throughput drops by 5090 percent. In most cases, this is not the packet overhead or the encryption/decryption processing that is causing the connection to slow down. The problem usually is fragmentation: either the remote side is fragmenting packets even before they hit the LAN or there is an intermediate device that won't perform any required fragmentation, and thus drops the packets. Diagnosing this problem is not very simple, but by tuning the MTU size for the users' connections, they probably won't even notice any difference between a connection to the corporate site in clear text and one going across a properly tuned VPN.


Looking for Fragmentation Problems

Before you begin implementing any solutions concerning fragmentation, first you should discover if you are having problems associated with fragmentation. On a VPN 3000 concentrator, you can go to the Monitoring > Statistics > MIB-II Stats > IP screen. Look at the bottom of the screen at the Fragments Needing Reassembly statistic. If this is incrementing, you have a device sending packets that are fragmented to the concentrator. Unfortunately, this won't tell you which client the fragments are coming from.

To determine if your Windows-based computer is performing the fragmentation, use the netstat -s command, shown in Example 12-15. Look at the bottom three lines under the IP Statistics section to determine if your remote access computer is creating fragments. In this example, five packets were fragmented into a total of 10 fragments.

Example 12-15. Using the netstat Program to Find Client Fragmentation
C:\> netstat -s
IP Statistics
  Packets Received                   = 91140
  Received Header Errors             = 0
  Received Address Errors            = 548
  Datagrams Forwarded                = 0
  Unknown Protocols Received         = 0
  Received Packets Discarded         = 0
  Received Packets Delivered         = 90709
  Output Requests                    = 92228
output omitted
  Reassembly Required                = 0
  Reassembly Successful              = 0
  Reassembly Failures                = 0
  Datagrams Successfully Fragmented  = 5
  Datagrams Failing Fragmentation    = 0
  Fragments Created                  = 10
output omitted

Of course, it might not be your remote access computer that's causing the problem, but an intermediate device. If your device's packets reach an intermediate device and the next-hop segment has a smaller MTU size, fragmentation is needed; however, the intermediate device might have disabled fragmentation, causing the packet to be dropped. Normally the intermediate device should send back an ICMP Destination Unreachable message with the "Don't Fragment" flag set. Some intelligent end system devices can then dynamically change their MTU size to meet the smaller MTU requirements. However, this doesn't always work correctly in the real world. Therefore, you might need to determine the smaller MTU size and change your device's configuration properly. To discover the router that has the MTU issue, sometimes referred to as the blackhole router, use the Windows ping command with the -f (sets the "do not fragment" flag) and the -1 (specifies the payload size of the ICMP packet) parameters:

ping -f -n #_of_echos -l payload_size_in_bytes destination_IP_address

Because Ethernet has an MTU size of 1,500 bytes and the Ethernet header is 8 bytes and the IP header is 20 bytes, set the payload size to 1,472 in the ping command initially. This will help determine if an intermediate router is having MTU issues with the size of packets you're sending through it. You might want to start with a smaller payload size to determine if the ICMP echos are successfully getting through, and then slowly increase the size to determine what the actual MTU size should be for your session. Example 12-16 shows an example of using the ping command on a Windows computer. The first ping test shows that no fragmentation occurred with a payload size of 1,472 bytes; however, as soon as I set the payload to 1,473 bytes (which will need to be fragmented), I received an ICMP message back in relation to this.

Example 12-16. Using the ping Program to Find an Optimal MTU Size
C:\> ping -f -n 1 -l 1472 192.1.1.1
 Pinging 192.1.1.1 with 1472 bytes of data:
 Reply from 192.1.1.1: bytes=1472 time<10ms TTL=128
 Ping statistics for 192.1.1.1:
     Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
 Approximate round trip times in milli-seconds:
     Minimum = 0ms, Maximum = 0ms, Average = 0ms
 C:\> ping -f -n 1 -l 1473 192.1.1.1
 Pinging 192.1.1.1 with 1473 bytes of data:
 Packet needs to be fragmented but DF set.
 Ping statistics for 192.1.1.1:
     Packets: Sent = 1, Received = 0, Lost = 1 (100% loss),
 Approximate round trip times in milli-seconds:
     Minimum = 0ms, Maximum =  0ms, Average =  0ms

The problem with using ping to find an optional size is that you have to tweak the payload size continually to discover the optimal size. If you have Windows NT or higher, you can use the Network Monitor program to view the optimal size, as shown in Example 12-17. In this example I used ping -f 1 1000 to create the output you see. In the output, the optimal size is 576 bytesusing this will not cause any fragmentation across the current path to 192.1.1.1.

Example 12-17. Using the Network Monitor Program to Find an Optimal MTU Size
+ FRAME: Base frame properties
 + ETHERNET: ETYPE = 0x0800 : Protocol = IP:  DOD Internet Protocol
 + IP: ID = 0x4401; Proto = ICMP; Len: 56
   ICMP: Destination Unreachable: 192.1.1.1   See frame 3
       ICMP: Packet Type = Destination Unreachable
       ICMP: Unreachable Code = Fragmentation Needed, DF Flag Set
       ICMP: Checksum = 0xA05B
       ICMP: Next Hop MTU = 576 (0x240)
       ICMP: Data: Number of data bytes remaining = 28 (0x001C)
     + ICMP: Description of original IP frame

On top of this, the size you find is optimal only for the current path the ICMP messages are traversing. This means that for a different Easy VPN Server, the size might be different; or if the traffic takes a different path to the same destination, your first optimal value still might cause fragmentation issues.

Note

You also can use the extended ping command on an IOS router to pinpoint the device performing fragmentation. The extended ping command allows you to specify the number of echo requests, the payload size, a sweep range of sizes, and many other parameters.


Fragmentation Solutions

There are two basic solutions you can use to solve fragmentation problems:

  • Use Path MTU (PMTU) discovery

  • Hard-code the MTU size for the adapter to a size that will always work

The preferable method is to use PMTU, which uses ICMP to discover the correct MTU setting for the current destination and automatically adjusts the MTU size. However, not all Windows systems support this or have it enabled. For XP and 2003, this is enabled by default. If you have Windows 2000, follow these steps to enable it:

1.
Start the editor for the registry (regedit.exe).

2.
Find this registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\tcpip\parameters.

3.
Choose Edit > New > DWORD Value.

4.
Change the name of the value to "EnablePMTUBHDetect," (without the quotes).

5.
Right-click the name of the value and choose Modify.

6.
Set the value to 1 and select the Decimal base radio button.

7.
Click the OK button.

8.
Exit the registry editor and reboot your computer for the results to take effect.

Your second option is to change the MTU size manually for your adapter. You can do this the hard way using Windows' registry editor (described here: http://support.microsoft.com/default.aspx?scid=kb;en-us;Q314825) or the easy way by using the Cisco SetMTU program. This program is included in the installation of the Cisco VPN Client software. During the installation of the 4.x client, the MTU size is adjusted automatically to 1,300 bytes for all NICs installed on the client's desktop. In 3.x, you must run this program manually to set the MTU size.

To access the SetMTU program, from the Windows desktop go to Start > Programs > Cisco Systems VPN Client > SetMTU, which displays the window shown in Figure 12-27. You can adjust the MTU size manually here. For LAN-based connections, typically 1,300 bytes will remove the likelihood of experiencing fragmentation problems; for dialup connections, typically 576 bytes will do the trick.

Figure 12-27. SetMTU Program


Tip

Please note that for the 4.x client, all adapters have their MTU adjusted to 1,300 bytes. However, many times this isn't very efficient: Cisco assumes the worse case of doing something like NAT-T with IPsec with PPPoE or L2TP over IPsec with PPPoE. Also, for WAN adapters, you'll want to adjust this to a typically much smaller value, like 576 bytes.

Therefore, if your computer doesn't support PMTU, use the ping command to find a more exact size for your client. Of course, if your client is continuously making connections from different destinations, I would use the 1,300- or 576-byte configuration depending on the adapter; otherwise, I would fine-tune the MTU size until I find the optimal one.


Note

I'll discuss fragmentation problems in more depth in Chapter 19, "Troubleshooting Router Connections."


Microsoft Network Neighborhood Issues

Besides address translation and fragmentation issues, the third biggest problem I commonly see network administrators deal with is Windows Network Neighborhood browsing and access issues when an IPsec session has been established. Some common problems might be that you can't ping a destination by its name, log in to a domain, browse the Network Neighborhood, or map a network drive. I'll discuss each of these problems, with their corresponding solutions, in the following subsections:

Cannot Log in to a Windows Domain

If you are having problems with logging in to a Windows domain, it probably is because you've misconfigured the Cisco VPN Client. Depending on the Windows platform you have, you'll do one of two things:

  • For Windows 95/98/ME: The client doesn't initially log in to the domain upon bootup. On the desktop, right-click the Network Neighborhood icon and select Properties; make sure that the Client for Microsoft Networks and File and Print Sharing are displayed. Then, on the VPN Client, click a connection profile and from the main menu choose Options > Windows Logon Properties. Make sure Enable start before logon is checked. If not, select it and exit the client. Log out of Windows and log back in and reconnectit is not necessary to reboot your PC.

  • For Windows NT/2000/XP: the client is prompted during bootup to log in to a domain. This is a catch-22 because the IPsec session hasn't been established. If you try to start up the IPsec session, you'll get this error message: "No Domain Controller could be found." Follow the instructions in the above bulleted item to ensure that Enable start before logon is checked for the connection profile in the VPN Client.

Cannot Ping Network Resources

First make sure that you ping the destination by its IP address: this indicates that you have network connectivity; if you are unable to, the problem is not a name resolution problem. If you can ping the destination by its address, but not by its name, the problem is associated with name resolution. Be sure that the client is assigned a DNS and/or WINS server address. On a VPN 3000 concentrator, this is configured on a group-by-group basis. If all groups use the same resolution servers, configure this in the base group and let the specific groups inherit this. If there is still a problem, see the following section.

To see if your client is assigned a DNS and/or WINS server during IKE Mode Config, set the IKE log level to "High" and then start up the IPsec session to the Easy VPN Server. Example 12-18 shows the output from the Log viewer. In this example, a DNS server was assigned, but a WINS server wasn't. Also, if you're using split DNS, make sure this was set up correctly on the Easy VPN Server.

Example 12-18. Determining if a DNS/WINS Server Address(es) Were Assigned to the Client
output omitted
27     15:27:06.068  02/11/05  Sev=Info/5 IKE/0x63000010
MODE_CFG_REPLY: Attribute = INTERNAL_IPV4_ADDRESS: ,
value = 192.168.101.110
28     15:27:06.068  02/11/05  Sev=Info/5 IKE/0x63000010
MODE_CFG_REPLY: Attribute = INTERNAL_IPV4_NETMASK: ,
value = 255.255.255.0
29     15:27:06.068  02/11/05  Sev=Info/5 IKE/0x63000010
MODE_CFG_REPLY: Attribute = INTERNAL_IPV4_DNS(1): ,
value = 192.168.101.99
output omitted
38     15:27:06.088  02/11/05  Sev=Info/4 IKE/0x63000055
Received a key request from Driver: Local IP = 192.168.101.110,
GW IP = 192.1.1.1, Remote IP = 0.0.0.0

Cannot Browse the Network or Map a Network Drive

One limitation of IPsec is that it requires IP as a transport. With Microsoft Networking, you might be configured to transport NetBIOS over NetBEUI, which can't be transported across an IPsec tunnel. This is a common problem I see when administrators are converting from Microsoft's L2TP/IPsec or PPTP clients to the Cisco client; Microsoft uses PPP as a transport, which can carry NetBEUI. Cisco uses IP, which won't.

If you have this problem, double-click the Network Neighborhood icon on the desktop and verify that the corporate resources are appearing. If they are not, then it is probably a WINS resolution problem; in that case, see the previous section. If you're using an LMHOSTS file for resolution, use the names listed in this file; you can view these by executing nbtstat -c from the Windows command prompt. Also make sure that NetBIOS over TCP is enabled for your network adapter!

Note

Browsing the Network Neighborhood is a Microsoft browsing function and has nothing to do with the Cisco VPN Clientmake sure that you have logged in to the Microsoft network first and are getting a WINS or DNS server address from the Easy VPN Server. Also make sure that no ACL is blocking the traffic between the Easy VPN Server and the corporate service.



Previous Page
Next Page