Multicast Troubleshooting

This section provides several general multicast troubleshooting tips to help you deploy your multicast applications. These tips are not applicable to unicast systems.

Verify Your Configuration

Before trying to resolve a complicated multicasting issue, follow these tips to eliminate or avoid basic problems:

Become familiar with the operation of your multicast application on a simple network before trying to make it work on a complicated one.

The "bottom-up" approach is generally best. First focus on getting the lowest layers of the network stack working.

Verify that the network has good unicast connectivity between the sender and all receivers before addressing multicast connectivity problems.

Ensure that multicast has been enabled across the test or production environment by checking that the switches and routers are multicast-enabled.

From a network point of view, try connecting your sending and receiving hosts to the same hub, not a switch or a router, and confirm that you have multicast connectivity. Once that works, move on to more complicated multicast networks.

From a SmartSockets point of view, try testing with two receiving clients that are on the same subnet as your RTgms process. Once that works, expand the test to other subnets.

Verify Your PGM Option Settings

Some common problems are caused because the options affecting PGM configuration are not set correctly. Check the value of an option by looking at how it is set in the applicable command (.cm) file.

Group_Names Option

Check that the receiving RTclients and the RTgms to which they are connecting use the same value for Group_Names. For more information on setting options for RTclients, see RTclient Options Summary. For more information on the option, see Group_Names.

Pgm_Udp_Encapsulation Option

Set in the $RTHOME/standard/mcast.cm file for your RTgms and RTclient processes.

Check that the receiving RTclients and the RTgms to which they are connecting use the same value for Pgm_Udp_Encapsulation. If the value is not present in those files, the default value is used and does match correctly.

If Pgm_Udp_Encapsulation is set to 0 and your operating system is UNIX, all receiving RTclients and the RTgms processes must be run as root. If Pgm_Udp_Encapsulation is set to 0 and your operating system is Windows 2000, all receiving RTclients and the RTgms processes must have administrator authority.

If Pgm_Udp_Encapsulation is set to 1 to use UDP encapsulation, no PGM subscribers can be run on the machine where RTgms is running. This is because NAKs from PGM might not be correctly processed by the UDP protocol when RTgms and a PGM subscriber are running on the same machine. This problem does not occur when PGM is using raw IP sockets (Pgm_Udp_Encapsulation set to 0).

Pgm_Source_Group_Ttl and Pgm_Receive_Nak_Ttl Options

Set in the $RTHOME/standard/mcast.cm file for your RTgms and RTclient processes.

These options must be set to the number of router hops a packet of data takes. The number is calculated starting from the source, the RTgms, and ends with the subscriber, the receiving RTclient. The default value of these options is 1, which does not allow the packet to go beyond the first hop, either a router or switch. For more information on these options, see Pgm_Source_Group_Ttl and Pgm_Receive_Nak_Ttl.

Tracing Problems to Their Source

Once you eliminate simple problems, tracing the source of a problem can be complicated in a multicast application. Here are some useful tips:

Make sure that multicast streams are being generated with a TTL adequate to reach their destination via the longest-possible path through the network. See Troubleshooting Multicast Problems with Cisco Systems Routers to check whether the TTL is too low.

Start at the source and trace your way through each switch and router to all receivers.

Use the SmartSockets PGM pthrpt command on the source to generate test streams. For example:

pthrpt -t 10 -r 10000 -m 224.13.13.13

generates a 10 Kbps stream with a TTL of 10 to group address 224.13.13.13. You can ask TIBCO Product Support for more information about PGM commands or how to get the SmartPGM User’s Guide.

Use the SmartSockets PGM pbw command or Cisco Systems IOS show ip mroute active command for monitoring the presence of test streams. See Troubleshooting Multicast Problems with Cisco Systems Routers.

Test receiver to last-hop router communication by running the SmartSockets Multicast pthrpt command on the receiver.

On UNIX, use the tcpdump command or the Solaris snoop command to monitor test streams.

If you see packet loss as multicast rates go up, look for routers or switches that are configured to limit the broadcast rate. These generally also limit the multicast rate. For example, Cisco Systems Catalyst 5000 series switches can be configured to limit the packet for each second or percentage of broadcast and multicast traffic with the set port broadcast command.

Try using UDP encapsulation to see if the network will pass UDP even if it is not passing PGM. This behavior has been seen in some networks with Catalyst 4000 series switches. See UDP Encapsulation of PGM.

Troubleshooting Multicast Problems with Cisco Systems Routers

A Cisco Systems router might refuse to forward multicast packets because their TTL (Time To Live) is too small. Use the IOS command show ip traffic and watch the number of bad hop count packets shown on the second line of output. This counter increments every time IOS throws a packet away because its TTL was too small. For example:

This shows that 59% (185555472/(322205025 - 8805271)) of all traffic that could have been forwarded was not because the TTL was too small.

The IOS command show ip mroute active is a quick way to see the most active multicast traffic passing through a router. For example:

The IOS command show ip mroute count is often helpful in determining if a router can see a multicast stream, how fast the stream is going, if the stream is being forwarded, and if not, why not. For example:

Router> show ip mroute count 
 IP Multicast Statistics 
 8 routes using 4024 bytes of memory 
 6 groups, 0.33 average sources per group 
 Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kilobits 
per second 
 Other counts: Total/RPF failed/Other drops(OIF-null, rate-limit 
etc) 
     
 Group: 225.0.11.66, Source count: 0, Group pkt count: 0 
 Group: 225.0.11.68, Source count: 0, Group pkt count: 0 
 Group: 224.0.1.40, Source count: 0, Group pkt count: 0 
 Group: 224.0.1.1, Source count: 1, Group pkt count: 1 
   Source: 216.0.13.9/32, Forwarding: 1/0/76/0, Other: 2/1/0 
 Group: 225.0.11.11, Source count: 0, Group pkt count: 0 
 Group: 224.13.13.13, Source count: 1, Group pkt count: 108 
   Source: 10.168.4.5/32, Forwarding: 108/3/537/12, Other: 108/0/0

Note the last group. It shows that the router is seeing 3 packets for each second and 12 Kbps to the group 224.13.13.13 from source 10.168.4.5.

See the Cisco Systems web site for excellent starting points for additional multicast troubleshooting information: