Tag Archives: fibre channel

Enabling Verbose Logging on Linux with Emulex Host Bus Adapters

Where did my disks go?

So now and then you may run into an issue which cannot be explained properly by just looking at the standard events that show up in “/var/log/messages“.

Issues such as

Oct 7 18:24:20 centos8 kernel: lpfc 0000:81:00.0: 0:1305 Link Down Event xc received Data: xc x20 x800110 x0 x0
Oct 7 18:24:24 centos8 kernel: rport-11:0-4: blocked FC remote port time out: removing target and saving binding
Oct 7 18:24:24 centos8 kernel: lpfc 0000:81:00.0: 0:(0):0203 Devloss timeout on WWPN 50:06:0e:80:07:c3:70:00 NPort x01ee40 Data: x0 x8 x2

are fairly common and the above simply shows a Link Down event. These are the most easy to troubleshoot when the remote switchlog tell you

Continue reading

Initiator, Target, Both or None ??

Whenever you’ve encountered an output of a nameserver entry you may have come across the phenomenon that the fabric has no clue what the attached device is. For a FC switch an attached device (or N-port in technical terms) is not more than a source or destination where frames originate from or can be sent to. As soon as smarter functions are required it may be helpful (or required) to be able to obtain more information from that device.

Continue reading

Why FDMI is compulsory (or should be!!)

FDMI stands for Fabric Device Management Interface and is such an enormously cool feature and unfortunately one of the least used. From an operational management perspective FDMI provides a wealth of information to the fabric regarding the attached devices. The thing that flabbergasted me is that almost no device (HBA/Array) has this turned on of even has the functionality embedded.

Continue reading

Fillwords IDLE vs ARBff (one last time)

I’ve written about fillwords a lot (see here, here, and here) but I didn’t show you much about the different symptoms an incorrect fillword setting may incur.

As you’ve seen fillwords are a very nifty way of maintaining bit and word-sync on a serial transmission link when no actual frames are sent. Furthermore they also are replaceable with other primitive signals (Like R_RDY, VC_RDY etc) to utilize a very simple instruction method between two ports without interfering with actual frames. That means that fillwords are ALWAYS squeezed in between frames.

SQUEEZE

Continue reading

Energy Efficient Fibre Channel and related cost savings

For years many storage environments have used both active-active and active-passive multipath (MPIO) access mechanisms to access storage arrays in a dispersed or linear method. On enterprise class storage arrays with global caches the active-active method is most often used while on modular arrays you’ll see the active-passive scenario often applied. Inherently this means that during absence of IO, whether being the passive path or due to total non-IO operations (ie. there is no application or operating system sending or receiving any data), the actual fibre-channel links are only sending IDLE or ARB(ff) fillwords to maintain bit- and word synchronization. This also means that both the sender and receiver are always up and thus use the same amount of power as where they transmitting data at full line-rate. Obviously this is a waste of scarce resources and this is what has been addressed in the new FC standards that are coming up. The FC framing and signalling standard will be enhanced to have traffic diagnostics determine if an SFP should be in full power operating power or in a power reduced mode. Below are the details including some cost-savings calculations.

Continue reading

Time with and without NTP on FC switches

I’ve been writing about troubleshooting issues for a while now and one of the things that is very difficult and most time consuming is correlating events between host systems, switches and storage arrays in the even of storage related errors. My advice has always been the same. Hook everything up to NTP systems, make sure that time and date settings, including time-zones and DST settings do fall within the drift values of the NTP client and that little nifty piece of software will make sure time is equal on all systems. (See below how to accomplish this.)
There are however some issues when this is not fully followed through and virtual switches are used.

Continue reading

Brocade 65xx series and airflow

This is important!! Now, you would think that when you look at the weather channel and the person tells you there is a wind coming from the North the direction is from North to South. This could be interpreted somewhat different when he or she tells you there is a “northerly” wind. Does the later mean it’s coming from the North or is it going to the North. The same is now true for Brocade 65xx series switches. The 6505/6510 have a different interpretation of how the wind blows than the 6520 series. Is it moving forward or reverse, is it coming from the front or back and on which side is the intake and/or exhaust and which spare-part do you need to order in case one fails, xx-F or xx-R. If you order the wrong one the switch will tell your this after which it will shut down. Whoops. The documentation also shows some confusion around the PS acronym. Is it now related to the airflow, in which case it stands for Port-Side, or does it mean Power Supply when a FRU is meant?

Continue reading

FC Frame header, or is there more to it

The FC frame header has not changed since its inception back in the late 80-ies. This shows the absolute rock-solid backward compatibility towards previous generation platforms and vendors. Obviously the the FC protocol itself has grown and evolved with marked demands to provide an extremely flexible transport mechanism for the most demanding storage environments. When you start to design a new protocol the concept of future proofing is a must and sits very high on the top of the design list. Not only does it require insight into current markets and requirements but also a huge set of extremely smart brains to come up with possible scenario’s of what might be required in years to come. You don’t want it to become obsolete in 5 years time because there were inherent flaws in the protocol. Next to that your have to facilitate options which allows for flexible expansion of functions and features of which none of the above brains had ever thought of. (Did somebody know about VMware back in 1990?) The concepts of NPIV where not on the radar back then. So what is the structure of this protocol that makes it so flexible.

Continue reading