SoE, SCSI over Ethernet.

Print Friendly, PDF & Email

It may come as no surprise that I’m not a fan of FCoE. Although I have nothing against the underlying thought of converged networking I do feel that the method of encapsulating multiple protocols in yet another frame is overkill, adds complexity, requires additional skills, training and operating methods and introduces risk so as far as I’m concerned it shouldn’t be needed. The main reason FCoE is invented is to have the ability to traverse traffic from Fibre Channel environments through gateways (called FCF’s) to an Ethernet connected Converged Network Adapter in order to save on some cabling. Yeah, yeah I know many say you’ll save a lot more but I’m not convinced.
After staring at some ads from numerous vendors I still wonder why they never came up with the ability to directly map the SCSI protocol on Ethernet in the same way they do with IP. After all with the introduction of 10G Ethernet all issues of reliability appear to have gone (have they??) so it shouldn’t be such a problem to directly address this. This was the main reason why Fibre Channel was invented in the first place. I think from a development perspective this should be an evenly amount of effort to have SCSI directly transported on Ethernet compared to Fibre Channel.From an interface perspective it shouldn’t be such a problem as well. I think storage would be as happy to shove in an Ethernet port in addition to FC. They wouldn’t need to use any difficult FCoE or iSCSI mechanisms.

Since all, or at least a lot, development efforts these days seem to have shifted to Ethernet why still invest in Fibre Channel. Ethernet still has a 7 layer OSI stack but you should be able to just use three, the physical, datalink, and networking layer. This should be enough to shove frames back and forth in a flat Ethernet network (or Ethernet Fabric as Brocade calls it).For other protocol like TCP/IP this is no problem since they already use the same stack but just travel a bit higher up. This then allows you to have a routable iSCSI environment (over IP) as well as a native SCSI protocol running on the same network. The biggest problem is then security. If SCSI runs on a flat Ethernet network there is no way (yet) to secure SCSI packets arriving at all ports in that particular network segment. This would be the same as having no zoning active as well as disabling all LUN masking on the arrays. The only way to circumvent this is to invent some sort of “Ethernet Firewall” mechanism. (I’m not aware of a product/vendor who provides this but I’ve never heard of it.) I’ts pretty easy to spoof a MAC address so that’s no good as a security precaution. 

As usual this should then also have all the other security features like authentication, authorisation etc etc. Fibre Channel already provides authentication based on DH-CHAP which is specified in the FC-SP standard. Although DH-CHAP exists in the Ethernet world it is strictly tied to higher layers like TCP. It would be good though to see this functionality on the lower layers as well.

I’m not an expert on Ethernet so I would welcome comments that would provide some more insight of the options and possibilities.

Food for thought.

Regards,
Erwin

About Erwin van Londen

Master Technical Analyst at Hitachi Data Systems
Fibre Channel, Storage Networking , , ,
  • Hello Raj,

    The feature you mention are IP based technologies. Only VLAN’s could be an option on the Ethernet layer.

    Then again, do you want to burn such a massive number of VLAN’s for initiator/target segregation? Network admins already think the current limitation is hampering their network growth. All in all it’s a bad idea.

    But that’s just my opinion. 🙂

    Regards,
    Erwin

  • Hi Erwin,

    You can use VLANS similar to that of FC Zones.
    Use MACADDRESS of the Initiator for LUN masking on the target..
    You can implement end-to-end encryption something similar to SSL/TLS. Exchange a session key on discovering the LUN by the initiator and use it for encryption/decryption for the life of that session..
    What do you think?

  • one can use vlan tagging to separate the traffic at level2. The switches normally won’t pass packets tagged for vlan A to vlan B

  • Anonymous
  • Hi David, thanks for your comment.

    As for AoE its a very lean and mean protocol. Coraid is one of the vendors using it. The problem is that you need a flat layer 2 Ethernet network since nothing is route-able and on large scale storage architectures it might be limited. Personally I never used it.

    You might want to check out the Tech Field day website (http://techfieldday.com/2012/coraid-presents-storage-field-day-1/) where Coraid presented. They also had a presentation on the protocol differences. Be aware that presentations and Q&A are always biased 🙂

    As with iSCSI, AoE also can have its place in the market but currently it looks like a very niche one to me.

  • StorageFreak

    Just curious… I stumbled across your article due to hearing about ATA-over-Ethernet (AoE) that’s been built into the Linux kernel for a few years now.

    What would your take be on AoE? I assume same security concerns– I’m not really sure I see an advantage of AoE over iSCSI.

    Your take?

    David

  • Anonymous

    Hi Erwin,

    The use of FCID forces host CNA to have to directly connect to FCF to acquire FCID allocated by the FCF (FCF maintains centralized SNS database), at least for current FC-BB-5 standard. Maybe FC-BB-6 has way to assign FCID w/o directly connecting CNA to FCF but that could add great deal of complexity, for example, FC-BB-6 introduces distributed FDF.

    I maybe wrong but just my 2 cents.

    Thanks.

    Colin

  • Anonymous,

    I don’t think the addressing itself is the problem but more the complexity it introduces.

    Thanks for you comments.

    Kind regards,
    Erwin

  • Anonymous

    I think you are right on. The problem with FCoE is that it is building a networking protocol (FC) over another networking protocol (Ethernet). FC uses 24-bit FCID for addressing but Ethernet uses 48-bit MAC for addressing, that is unnecessary and can only complicate design and cause more inter-op issues. It is absolutely possible to design a light-weight transport protocol to directly transport SCSI command/data over Ethernet.