Brocade FOS version 8 and 32G hardware

8-ball

If you’ve been laying low during Christmas last year and have overlooked that Brocade announced its first to market with “Gen 6” (32G) hardware and FOS code 8, you’re forgiven. FOS 8 was mainly released to support the new G620 hardware but a lot of functions and features did either not work or were not supported yet. FOS 8 also dropped support for a lot of hardware which is a good thing IMHO. 8G equipment like the DCX4S, DCX as well as the single unit switches like the 5100 and 5300 were dropped.

Since the 7800 has already been superseded for a while by the 7840 it was removed from the supported list as well.

As a last nail in the FabricWatch coffin support for “bottleneck” monitoring as a single use was also removed. That feature is now incorporated into FlowMonitor and MAPS.

As the FOS 8.0.0 release was rather limited Brocade certainly made up for it with 8.0.1(a) where a whole new shebang of hardware was introduced as well as full support for for all the goodies that were already in FOS 7.4 (see here). Not only that but the successor to my favorite tool in FOS (which used to be FabricWatch) MAPS and FlowVision has been greatly enhanced. The FabricVision license should be on the top of your list of “gottahaves”.

When you start to use FOS 8 from the CLI you’ll notice that a lot of commands are either not there or have been modified. This is obviously the result of change in support of the different hardware platforms. As the FCoE10-24 blade is dropped from support the related commands and code also have been removed .

One other rather important change is that support for “Admin Domains” will be depreciated. The functionality is still there but it will be removed from future releases. I think the feature was not used much as I’ve only seen a handful of fabrics around the world where it was actually implemented. The introduction of virtual fabrics and the capability to tie both the virtual environment as well as the userbase together as a single security entity had made the”Admin Domain” feature obsolete.

Upgrade

I don’t want to sum up all changes as these are very well explained in the release notes but I do want to highlight the upgrade paths as these sometimes are interpreted incorrectly.

  • Upgrade from 8.0.0 to 8.0.1(a) = non-disruptive
  • Upgrade from 7.4.x to 8.0.1(a) = non-disruptive
  • Upgrade from 7.3.x (or older) to 8.0.1(a) = disruptive
  • All FCIP links on FX8-24 blades will bounce. Planning is needed.

Upgrade from 7.3 to 8.0.1 is the only two-stage upgrade that is allowed albeit disruptive and you need to use the “-s” parameter (single CP at a time).

Fabric Compatibility

Here’s an important one. If you have ANY equipment running FOS < 7.3.1 you CANNOT implement a Gen6 switch running FOS 8.0.1 in this existing fabric.

All 4G equipment is not allowed to have direct E-port connectivity to a switch running FOS 8. You must use FCR. The same is true for DCX(-4S) with an FA4-18 blade. I applaud Brocade for making this decision as I think that any 4G or 8G switch should be isolated from this new generation of hardware.

Migration

Depending on which revision of hardware and software you currently have there are basically two options.

  1. Migrating from 4G or older is not a wise move. Building a new storage network including cabling is highly recommended as 16G and 32G signal rated require a vastly improved optical real-estate than 4G (and even 8G) needed. If you have a 4G based storage network the chances are that the majority of HBA’s and target devices also sit in that speed bracket and simply hooking these up to a new 16G or 32G switch will most certainly provide you headaches with performance (or rather the lack thereof) and wide-spread fabric congestion and latency issues. At some stage, sooner rather then later, it will cause significant problems.
  2. If your current estate is already optimized for 16G speeds (OM4 cabling, HBA’s and storage devices on 8G and 16G) plus the FOS code levels on the switches is up-to-date you should be good to go.

As for overall fabric design be aware that traffic flow is highly depending on slow(er) equipment. As speeds have increased exponentially now up to 32Gb/s be aware that congestion and latency is linearly following. This means that any device holding up IO will the impact any place in the fabric much sooner causing IO issues on places you least expect it. Implementing MAPS with FPI rules will therefore be of the utmost importance to keep ahead of the game. Further MAPS actions like SDDQ and/or port-fencing help reduce the overall impact to the fabric but you have to configure it!!!!!

Fabric Vision

As FabricWatch has received its marching orders with FOS 7.2 and later all references and functions are now removed with this release. The good thing is that a vastly more cohesive set of integrated tools now work together and simplify the configuration and operations of your storage network. (No I don’t work for a marketing department but I do know what storage admins need over the course of 20 odd years in the SAN business) You receive a huge bag of tools when you buy a FabricVision license. FlowVision and MAPS have become an integral part of FOS in such a way that many critical functions which monitor the bare-bone health of a switch are now usable without a license. These mainly fall into the resources and statuses of the different parts in a switch (like FRU’s, CPU and memory usage) but also FPI (Fabric Performance Impact) monitoring. Most of the default MAPS settings can be viewed with the [“command” –show] like “mapsdb –show”. In order to fully utilize its potential though I strongly urge you to get the FabricVision license as this will give you a leading edge in day-to-day san operations plus a huge bonus of other tools in troubleshooting scenario’s.

As the 8.0.0 release was mainly to support the G620 no OEM vendor has supported the version. The 8.0.1a release is now adopted by most OEM’s as a supported version across the board.

Ficon

At the time of this writing IBM has not yet qualified the Gen6 equipment. The latest FOS version that is supported for zSeries is 7.4.1d (as per august 18th 2016) on the director class  8G and 16G switches. Most vendors supporting mainframe FICON connectivity follow IBM’s qualification path unless they find something which might impact the interoperability or stability of that release. Read your OEM’s support-statements before making a decision.

Defects

FOS 8 is not immune to defects. The engineers have fixed a large amount of outstanding ones still lingering from the 7.x releases but, as with many .0 releases, a fair amount of new ones have been found as well. I suspect the majority of these which are applicable to the new platforms will be resolved pretty quickly in subsequent releases.

Summary

With version 8 Brocade has made a large step in cleaning up FOS and provide a solid base for future use. Especially with the advance of flash and NMVe based storage environments the consolidation, or rather migration of numerous disparate performance tools, which were hardly linked to alerting mechanisms, into FabricVision gives a good road-map and a safe investment protection for the foreseeable future.

Let me know if you have any nice/useful additions or questions.

Kind regards

Erwin

Print Friendly, PDF & Email

Subscribe to our newsletter to receive updates on products, services and general information around Linux, Storage and Cybersecurity.

The Cybersecurity option is an OPT-OUT selection due to the importance of the category. Modify your choice if needed.

Select list(s):

4 responses on “Brocade FOS version 8 and 32G hardware

  1. nobody

    I like your style of writing and summary of FOS from 8.0.0 to 8.0.1!

    I’d enjoy reading an update of FOS 8.0.1 to 8.1, now that it is out.

  2. jaymike

    Good stuff, Erwin.

    Can you elaborate on this section a bit?

    “…simply hooking these [4G/8G devices] up to a new 16G or 32G switch will most certainly provide you headaches with performance (or rather the lack thereof) and wide-spread fabric congestion and latency issues. At some stage, sooner rather then later, it will cause significant problems.”

    Is the implication that if you mix these in with newer 16G/32G N_Port devices that you’ll run into trouble? Or is this more a statement about possible issues with cable type/quality? Just looking for a little clarification.

    I used to support fabrics from back in the 1G days until just after the 16G stuff came out, but have been out of the game for awhile. I still work on fibre target devices though, so I like to try to keep up. Thanks.

    1. Erwin van Londen

      Hi Jay,

      The problem is two fold. First the change in physical cabling requirements. 16G and 32G speeds simply require OM3 or better OM4 cabling. No questions asked. If you want to drive a Ferrari to go 250mph you need a good road. Same goes for these high speed links.

      Secondly the chance of credit back-pressure when mapping 4G devices to each other which traverse ISL’s or switch back-end links who also need to push frames from brand new kit will most certainly happen. You will run into scenarios like I described here https://erwinvanlonden.net/2015/04/cross-fabric-collateral-damage/

      The basic rule of thumb I always use is to have a maximum on 1 speed generation difference between any piece of equipment. Remember that speed increases are exponential in the FC world and so will the amount of performance problems if mixing old and new equipment.

      Hope this helps.

      Cheers,
      Erwin