This week I had a very interesting question from one of the fellow blog readers.

Why do I see no traffic on a new FCIP tunnel when I already have an established tunnel between two Brocade 7800 switches?

Although I wasn’t provided with all the details I’ll try to describe some of the pitfalls that often occur when FCIP is configured.

Now be aware that creating an FCIP tunnel is nothing more then enabling FC over IP encapsulation in order to establish an ISL (or routed link via VEX ports) between two domains or fabrics. This means that all normal fibre-channel rules and regulations are still applicable. Fabric timeout values like E_D_TOV/ R_A_TOV, zone distribution, nameserver registrations, RSCN’s etc are still a thing to consider when configuring a fabric. Introducing FCIP does not mean you can ignore these.

With the 7800 switch Brocade introduced the concept of circuits. I’m not going into detail what a circuit is but basically its an IP connection between two GE ports on two 7800 switches. A simple configuration might look like this:

The tunnel consists of two circuits each connected to a 1Gb Ethernet interface. (Read carefully, just a 1Gb interface). This setup works without any issues. As mentioned the strange thing happens when a new tunnel with “the same” characteristics is configured.

You would expect that when this is created the DPS and DLS settings on the switch would balance out the number of ingress ports over the two tunnels. This didn’t happen.

As I mentioned I wasn’t given all details but one of the reasons this happening might be that the configured bandwidth (ie commit rate) is not equal which could result in a different link-cost. Brocade uses the following algorithm :

• If the bandwidth is greater than or equal to 2 Gbps, the link cost is 500.
• If the bandwidth is less than 2 Gbps, but greater than or equal to 1 Gbps, the link cost is 1,000,000 divided by the bandwidth in Mbps.
• If the bandwidth is less than 1 Gbps, the link cost is 2000 minus the bandwidth in Mbps.

In a case where you then have this configuration:

The FSPF link cost of the top tunnel would be 500 whereas the link cost of the bottom tunnel would be 1.000.000/1200=833. Be reminded that the used value is the lower band of the configured commit rate. Since in the above case the link cost of the bottom FSPF tunnel is higher then the top one the bottom one will not be used at all.

A better option would have been to add the two circuits into the same FCIP tunnel and adjust the overall aggregate commit rate to 3200Mb/s. This would then also allow for the FCIP trunking features (like resending IP packets over a different circuit without affecting the fibre-channel flow) to take effect.

The result would then be something like this:

I hope this explains at least one of the “gotchas” which could interfere with your FCIP setup.

Regards,

Erwin van Londen