Data Corruption in Storage Area Networks

If there is one thing that gets people relatively annoyed is that their data is not available for a certain amount of time. You can’t get to your sales data, orders cannot be fulfilled, your tracking and tracing options for your goods on their way to your customers does not show up etc. That makes you really crunching your teeth whilst waiting for this stuff to come back. Take that feeling into your head and extrapolate it to the point you start to realize this will not return, ever…….got it? That is when data-corruption has made its mark.

This content is for Troubleshooting members only.
Log In Register
Fibre Channel, Troubleshooting , , , ,

Host based mirroring kills your storage network!!

System administrators are very inventive and lazy. I know, I used to be one of them. 🙂 Everything that can be done to make ones life easier is about to be scripted, configured, designed etc.  If you are responsible for an overall environment from Apps to servers to networks and storage you can make very informed decisions on how you want to set up each different aspect of your environment. The last time I had this opportunity was back in 1995. Since then I have not come across an environment where a single person/team was responsible for each technology aspect of the infrastructure. As environments grow these teams grow as well. Business decisions like splits, acquisitions, outsourcing etc etc have enormous impacts not only on the business itself but also on people who are now forced to work with other people/teams who may have different mind-sets, processes and procedures and even completely different technologies. In many such instances strange things will happen and result in a very unpredictable behaviour of compute, network and storage systems. Below I’ll give you such an example where decisions from a systems-level perspective results in massive problems on a storage network.

This content is for Config Guide and Troubleshooting members only.
Log In Register
Brocade Technical, Config Guide, Storage Networking, Troubleshooting , , , , , , , , ,

Bro(ad)c(a|o)(de|m) – Brocade to Broadcom What’s next?

A relatively simple regular expression is all that needed to change all marketing material and documentation but the consequences for customers are significant. In the short term everything will be business as usual. The product portfolio currently designed, manufactured, sold and supported by Brocade will be heavily scrutinized. Broadcom has already mentioned that the entire IP networking portfolio will be sold and that mainly the SAN business will be kept.

This basically means that all previous Foundry customers who have invested heavily in the ICX, MLX and SLX equipment as well as current VDX owners will soon get a letter (or tweet, depending how it is handled) showing that their equipment will be maintained by another party. When looking at this from a business perspective I can’t see anything wrong with this as Broadcom has a huge OEM base and the current Brocade IP will most likely land with one of these companies. The one who is able to purchase that business is seriously well equipped with a huge knowledge and engineering asset with great gains for their current and new portfolio.

So why does Broadcom wants to buy Brocade? To me first and foremost this seems to be a money deal with the only thing wanted by Broadcom is getting a direct foothold in the large datacentres where Brocade primarily operates. As Avago bought Emulex a few months ago it seems Broadcom want to expand their business to other market segments via the SAN technology to level the playing field from a competitive view. I wouldn’t be surprised if an anouncement is made by Broadcom and Qlogic for a similar merger.

But the SAN business is declining” you might argue. Well, yes and no. Ethernet and IP storage networking still don’t cut it in the real storage world. I know, I get a huge amount of issues on my plate relating to to IP storage and FCoE which all suddenly seem to disappear when the same equipment is hooked up to FC interconnects so from a reliability point of view FC will be around for quite a while.

Secondly Broadcom has a vast OEM base which have likely “requested” (to put it mildly) that extending the portfolio on which they can build is a requirement for growth. One of the pathways is obviously storage networking. Competition is fierce in the ethernet world and new differentiation options need to be created in order to compete in this world.

One example is that there is a huge amount of engineering smarts in Brocade’s FC ASICS’s (Condor 1/2/3/4). It is very likely that Broadcom wants to create some sort of hybrid ASIC capable of doing both Ethernet as well as direct FC both on the switching side as well as the adapter side. The consolidation customers will then be able to achieve is vastly more compelling than using complex protocols like FCoE or relatively “slow” protocols like TCPIP.

Hooking up an adapter identifying itself to a switch as being a NIC which then triggers the switch to act as an Ethernet entity or the adapter acting as an HBA then flipping the switch to “talk” Fibre-Channel is obviously far more beneficial from an administrative and OPEX point of view than fooling around with creating complex configurations prone to errors like CEE maps and setting ETS and PFC QoS policies. The quantities of scale therefore make it easier to mass produce such equipment which inevitably brings down costs significantly. This is a real game-changer of which Broadcom now holds the key to the kingdom. It is up to them to play it smart and disrupt the current status quo massively.


I’ve invested a significant portion of my time and efforts of becoming a Brocade Distinguished Architect. (One of just around 200 in the world). Brocade just recently released an updated certification track with new courses, updated exams and a few adjustments in their overall curriculum. It seems these efforts have been in vain as 50% of the tracks are related to Ethernet/IP based tracks which will be no longer valid. I’m not saying these are not useful to have as these do require you to have a fair amount of knowledge around these protocol technologies but the part specifically related to the equipment is no longer a Brocade based asset.

Here’s the unsettling part: Broadcom does not have a certification program so as soon as BRCD as a ticker symbol ceases to exist so will the value of your certification.

I am very interested to see how this pans out. Personally I’m a bit sad to see another great company from whom I learned so much fall apart into a cut-up piece of meat as has happened with so many other great companies with a technological heart. (Tandem, DEC, Compaq, SUN, McData etc.) It seems money always talks the talk where tech requires to walk the walk.

Thank you Brocade for all the support and great technology you’ve given me over the last 20 years. I’ve made many Brocade friends all over the world whilst working with their products and technologies and I hope they will all find a good spot to land whether this is at Broadcom or somewhere else.


Brocade , ,

Why frames get dropped at your storage array

Arrays from all vendors have significant troubleshooting capabilities but in most cases these are scambled in the most obscure way you can imagine. You need special tools to be able to have a look at the log-dumps that come out of these systems. The nice part is that they contain a wealth of information including what is going on at the FC layer.

This content is for Config Guide and Troubleshooting members only.
Log In Register
Config Guide, Fibre Channel, Troubleshooting , ,

Ficon Dynamic Routing (FIDR)

In the storage world there mainly have been two environments : 1 FCP and 2. SBCC which stands for Fibre Channel – Protocol and Single Byte Command Code Set.

read on

Read more »

Storage Networking , , , ,

Brocade FOS version 8 and 32G hardware

If you’ve been laying low during Christmas last year and have overlooked that Brocade announced its first to market with “Gen 6” (32G) hardware and FOS code 8, you’re forgiven. FOS 8 was mainly released to support the new G620 hardware but a lot of functions and features did either not work or were not supported yet. FOS 8 also dropped support for a lot of hardware which is a good thing IMHO. 8G equipment like the DCX4S, DCX as well as the single unit switches like the 5100 and 5300 were dropped.

Read more »

Brocade , , , ,

Target Driven Zoning – Any demand??

Brocade , , , ,

Reset the Zoning Configuration and Prevent Mistakes.

There are occasions where you need to remove an entire zoning configuration from a switch. One of them is if you need to add a switch to an existing fabric and it still has a configuration in it. If these two conflict the switch will simply segment and the ISL’s get disabled. Another reason might be in case of configuration conflicts where and administrator had made zone changes whilst one switch was not participating in the fabric. Depending on how the switch is set up and the actual fabric state you need to follow different procedures.

This content is for Config Guide and Troubleshooting members only.
Log In Register
Brocade Technical, Config Guide, Troubleshooting , , , ,

Appalling state of Storage Networks

In the IT world a panic is most often related to a operating systems kernel to run out of resources or some unknown state where it cannot recover and stops. Depending on the configuration it might dump its memory contents to a file or a remote system. This allows developers to check this file and look for the reason why it happened.

A fibre-channel switch running Linux as OS is no different but the consequences can be far more severe.

This content is for Config Guide and Troubleshooting members only.
Log In Register
Brocade, Config Guide, Troubleshooting , , , , , , , ,

CiscoLive! Melbourne – 2016

Having to miss last years edition I was fortunate enough to be able to attend this years CiscoLive! conference in my hometown Melbourne again. Venue was excellent as usual. MCEC provides perfect facilities and is in the heart of the city so no dramas here.

The opening keynote already showed to direction where Cisco has steered the ship. Datacentre agility to cope with the ever increasing demand for business flexibility and be able to adapt quickly to changes in market conditions.

Cisco is acknowledging that it needs to adjust their products and services portfolio in such a way business can still use its entire catalog but by opening up the methodologies in which these products are used it gives more control and choice to the people who have to design, implement and operate these infrastructures. This way they don’t need to align their internal processes to the ever more diverse product-set but the other way around.

One example is NX-OS. The operating system running the core of Cisco’s networking products. With the introduction of NX-OS 7 on the Nexus 7000 and the new 9000, the operating system has “rebranded” itself to “Open NX-OS”. This doesn’t mean Cisco has put the source on Github but it gives a more accessible method of interacting with the switches, or any other supported platform for that matter. By providing a rich REST-full API it’s easy to create programs, tools and scripts which can automate deployment and day-to-day operations.

Evidence of this is the fact that Cisco had set up an entire track for developers in the form of a, so called “DevNet zone”. An area at the conference where developers, or anyone interested, could learn more about developing scripts and tools to better interact with the wide portfolio Cisco has. This was not only restricted to the conference. Cisco has ramped  up a large online devnet area including a impressive sandbox where developers can test and validate their programs towards the majority of Cisco technology. As you may have seen this is not new and the methodology is more or less adopted from the OpenSource communities around the worlds where this has been created. Distributed development of software by coding, peer review, testing, building and deploying has been done for over two decades and even long before that in the educational institutions.

A lot of Cisco teams and individuals contribute in the form of code, documentation, examples and a myriad of hints and tips to get you started.

Obviously Cisco is no philanthropic organization so don’t expect any non-cisco technologies to be covered. Even when you think you see Cisco’s involvement in a non-cisco product or code, like for example OpenStack, a lot of code is contributed in the form of Cisco enablement in that product. No problem here, all vendors do that and it increases the usability and experience of the overall product.

I must applaud Cisco for embracing the way IT is used these days and give customers more control in the way they want to use Cisco products. This really is the way forward and all vendors should adopt this methodology and provide resources in the form of knowledge, product resources and people. Other companies like EMC with {code} and IBM’s Developerworks provide a similar experience

CiscoLive! 2016 covered a lot more topics like security, wireless, communications etc. etc.

Another great feat was I got to see a few friend again. Dr J Metz finally made it to Australia and David Jansen who’s sessions I always enjoy for its pragmatic approach and technical depth. Great to see you guys again.

All in all another excellent conference where I learned a lot and provide me with a significant bag of information to keep me busy for another year.

Thanks Cisco.



Cisco , , , , ,