Storage networks have been used for disaster recovery and business continuity purposes for as long as fibre-channel existed. The moment arrays could communicate to other arrays over either short or long distances replication software has been developed and incorporated in firmware so that data could be copied to remote sites.
There are two methodologies how and where this can be done. Either array based or fabric based. Fibre-Channel as a protocol has always had the option to incorporate copy services on the FC 3 layer in the protocol stack however this has never been adopted by any vendor. The reasoning for this was primarily competition as that would have made it extremely simple to move and migrate from one vendor to another. Another reason is that in the end it is the array manufacturer who will be ultimately responsible for storing your data and guarantee its integrity and availability. If vendor A array sends a frame to vendor B array and some form of inconsistency, for whatever reason occurs, it is impossible for either of the three parties to warrant the status of your data. Anyway, so far some history.
The distance connectivity options are summarised as followed:
- Direct Connect
- Distance extension
- FCIP
The direct connect option is to most simple one. Basically an ISL forms over a dark fibre cable and additional capabilities in the switch ASIC in the form of buffer credits enables the correct utilisation of that link. Based on the ASIC and licenses the distance limits are set. When connecting switches via a long distance dark fibre-cable you obviously also need to obtain the correct optics to be able to reach the remote site.
Distance extension uses a different carrier like DWDM, SONET, SDH and alike. In these solutions separate equipment either utilises different optical technologies or adjust carrier frequencies to higher rates so multiple protocol and frame type can be transported over the same physical link.
Last but not least FCIP. This is the most used option in medium to long distance extensions as it provides the capability to lease external, IP based, network bandwidth in order to traverse storage traffic. This is not only often the most cost effective one but also provides a fair amount of flexibility. You don’t have to dig you own cables as with dark-fibre nor do you have to purchase expensive optical equipment when using DWDM. The drawback is that you are heavily depending on the capabilities and SLA’s your network-provider is able to offer.
Each option has its benefits and disadvantages. Direct connect over dark fibre is the most secure and best performing as well as least depending on other technologies but is very expensive especially when comparing to utilisation rates of FCIP and DWDM.
DWDM is relatively expensive from a CAPEX perspective but also very depending on options supported by the different vendors. If DWDM equipment is using optical filters and therefore using different wavelengths there is no interference on the FC1 and FC2 layer. If however the DWDM solution is using transponders and/or TDM the FC stream needs to be opened on the FC word boundary in order to be able to adjust signal rates. This then heavily influences the interoperability and features you can use on a FC level. I will come back to that later.
I will dive into each of the 3 option in a separate post.
- Direct Connect
- Distance Extention
- FCIP (to-be-published)
