Well, almost… 32GFC is fixed and at the standard has been ratified and the physical specs are moulded in concrete. In addition to this there are functions that enable to combine 4 of these into a single 128Gb link. Yes 128Gb/s. That’s fast….Ohhh wait… thats only ONE WAY which means a single link can push 256Gb/s in total. That allows you to push 25600MB per second through a link. In order to achieve these transmission speeds there has been a somewhat restructuring on the individual requirements w.r.t. quality of the actual link.

As FC-PI-6 shows the BER (Bit Error Rate) has been increased to 10^6 from 10^12 in previous specs. This means the quality can be less but the encoding mechanism of 256/257 ( a combination of 4*64/66 plus a Reed-Solomon parity check-symbol. I’ll spare you the details.) plus using  FEC (Forward Error Correction) in effect will make the BER after FEC almost undetectable. The Encoding/Decoding to a 64/66 transmissions word in addition to the scrambling which was introduced on the 16G (FC-PI-5) link speed has not changed.

Signal rating

In the same way 16GFC uses the 64/66 encoding technique which cleared the pathway for lower signal rates on the actual link in ratio to the bitrate, 32GFC does the same. 16GFC uses 14025MBd (Megabaud) and 32GFC uses 28050MBd.

FEC

FEC, or Forward Error Correction is not new. The telecom world has been using this for ages since by nature they are traversing much longer distances with numerous uncontrollable link quality flaws. Basically what it does it creates a set of blocks and creates a cross-calculated parity over those set of blocks which is then sent with the data to the remote side. The remote side first checks on transmission character mishaps on the encoding/decoding layer. If an issue is detected there it can have the FEC algorithm reverse calculate the parity therefore correcting the faults into the original bitstream before it is pushed further upstream. As I mentioned on 128GFC FEC is mandatory.

Transmitter training.

With the introduction of higher speeds and feeds with added functionality (either implementation specific by vendor or compulsory by standard) it is imperative that receivers and transmitters are aware of each others capabilities. Scrambling feeds and FEC are two of these options. One other (very cool option) is the reporting of the quality of the link. This implies that equalization and/or gain on the receiving side can be adjusted to accommodate optimum link characteristics. This will significantly reduce the change of encoding/decoding and CRC errors. The training “sequence”  consists of a training frame and a training pattern. The training frame starts with a frame marker (a bit like the well known K28.5 to make a short comparison) of 32 bits followed by a control field and a status field of each 16 bits. The control field holds all parameters on which a transmitter could adjust transmission characteristics based upon values it receives from the remote side. So the receiving side of the port checks the control field and acts upon instructions in that field. If that control field contains instructions to adjust certain parameters the transmitter side of that port makes the required adjustments and changes the respective bits in the status field so the remote side is able to check if these adjustments have been sufficient to allow for optimal link quality. If it has been determined that the current link characteristics fall withing the specifications of the equipment the transmitter can turn on bit15 in the status field to indicate that transmitter training from that side is completed. If the remote side does the same the link itself operates in optimum mode.

Now, this doesn’t mean that it can fix real physical issues. If you have cable and/or connector problems the transmitter training will indicate that it is unable to achieve optimum values and fall back to default/predetermined values. On an analyser you will see bit 12 in the status field being flipped to 1 and the link may or may not come up in a degraded or lower speed. Irrespective of all options the engineers in the standards bodies think of to improve transmission characteristics and fault-tolerance they cannot fix broken cables and dirty connectors. This is REALLY your responsibility.

128GFC

The 128GFC is a method of combining 4 * 32G links into a single logical link. FEC is mandatory and the way it is done is that on each 514th byte of the 4096th FEC codeword a so-called alignment marker is inserted. This is a fixed value on each lane per interval. This allows the receiver to map the bitstream on the correct lane. It is then pushed through a Reed-Solomon encoder to be able to add an error-correcting layer upon the bitstream. The push to the physical layer is then done according to  802.3bj standards. (IEEE 100Gb Ethernet, check symbol distribution and bit ordering)

The hard stuff

Cabling, and more the characteristics of these cables, determine if you will actually achieve these speeds. 32GFC REQUIRES at least OM3 cabling but OM4 is highly recommended. Anything less in the physical path is useless and you might save yourself the embarrassment of finding out afterwards you have older cables hooked up after a week of troubleshooting.  OM3 will give you around 50 to 70 meters tops and OM4 will reach to around 100 meters. Again these are maximum values and depending on link loss budgets in your cable plant it will be detemined if you’re able to get to 32GB/s. Be extremely vigilant in designing, planning and installing cable plants since this will determine your operational ability w.r.t. storage. One bad cable, connector or patchpanel can seriously impact your storage operations.

There are more guidelines being imposed and these go from launch power to receiver sensitivity, eye-pattern diagrams, reflection, return loss etc etc both on a optical and electrical perspective but I leave these for a possible next post.

As for when this is going to hit the market obviously depends on the vendors and from experience I wouldn’t be surprised that Brocade already has some kit working in their engineering labs. Cisco is with the 9710 also ready to adopt the 32GFC and 128GFC specs but I don’t have any insight in their release schedule. My suspicion is that around the of 2015 or early 2016 we’ll see the first products hit the market.

As you can see, Fibre Channel is still the leading transport mechanism for storage and is more than capable of future-proofing your investments.

Kind regards,

Erwin van Londen