7 – Fabric Security

This topic is hardly ever touched when fabric designs are developed and discussed among storage engineers but for me this always sits on my TODO list before hooking up any HBA or array port. It is as important in the storage world as it has been in the IP networking sector for decades. Historically the reasoning to not pay attention to this topic was that the SAN was always deeply embedded in tightly controlled data-centres with strict access policies. Additionally the use of fibre-optics and relatively complex architectures to the storage un-inaugurated even more, unfairly, devalued the necessity of implementing security policies.

Let me make one thing clear: Being able to gain access to a storage infrastructures is like finding the holy grail for archaeologists. If no storage infrastructure security is implemented it will allow you to obtain ALL data for good or bad purposes but even worse it also allows the non-invited guest to corrupt and destroy it. With this chapter I will outline some of the procedures I consider a MUST and some which you REALLY should take a good look at and if possible implement them.

There are quite a few books around that describe storage security in general. These touch on guidelines and legal implications plus some ways to implement these. Two good references are:

This post focuses on the fabric wide implementation which, in a Brocade environment, is done via so called policies. More on that later.

Basics

To begin with each Brocade switch comes by default with 4 users configured by default.

  1. root
  2. admin
  3. factory
  4. user

All the default passwords for these accounts have been same for years and are widely available in google so the first thing to do is to change them. (AND REALLY DO THIS !!!!!) The switch will alert you when you log in with one of these accounts who’s password has not been changed. For the root account create a really strong password, print it out, and lock it in a safe. One of the methods I use is on a Linux system to execute something like “date | sha256sum” which results in a string which looks like this

“9f33113d3ffc66ee89870cdbb8d2c586d28b438a58b85f2a11c1700692b23e94”

(When pasting this into http://howsecureismypassword.net it comes back with “It would take a desktop PC about 31 sesvigintillion years to crack your password”) Strong enough I presume. Still if this conflicts with your password policy where you need one or more upper-case letters and/or symbols you might not be able to use this. I’ll leave the policy up to you.

Do NOT use the “root” account for administrative purposes unless you’ve been told so by your OEM support organization or Brocade. You really can wreck the entire switch and if your maintenance has just expired you really are out of luck. Putting it on E-bay will likely get you scrap metal value for the box.

FIPS

For use in high-security areas such as Defence, Intelligence agencies and high impact financial systems you may see the requirement for FIPS compliance. This is the highest level of security that a switch state can be configured to. I really need to refer you to the FOS Admin Guide (up to FOS 7.2) or the “FIPS Configuration Guide” from FOS 7.3 onward. Be aware that enabling FIPS is one-way. All settings as well as shared secrets, private keys used in SSL, TLS and system login will be zeroized. The switch will then reboot, clearing memory and when it comes back with POST it will launch a KAT (Known Answer Test). A lot of restrictions will be enforced. Insecure communications on the management port like telnet, http and snmp v1 will be disabled. All firmware upgrades will need to be accompanied with signature validation etc etc. READ THAT GUIDE and plan well ahead !!

Remote authentication

If possible link the switch to an Active Directory or RADIUS authentication method. Not only is this more convenient for administrators but also allows for centralized password policies across all systems in the entire environment. Always make sure you at least two authentication methods active plus when using LDAP, RADIUS or TACACS+ configure at least two authentication systems. If one method (ie LDAP) fails you can use the other (ie local) or if one authentication server fails it can use the other.

As an example you can enable LDAP and use local for failback with the following command:

aaaconfig –authspec “ldap;local”

This will try to authenticate with the LDAP system first and if that fails use local. The flaw in this is that if the authentication fails because someone used incorrect credentials it will still fall back to the local database and will try there. To prevent this you may want to add the “–backup” parameter at the end. This will make sure it will only fail back if the LDAP server(s) are not available. If incorrect credentials are used and LDAP returns a “denied” message the switch will not fail back to local. Also make sure you type in the command exactly as above. If you forget the ” (quotes) it will only enable LDAP (because in Linux/Unix the ; is a command separator. FOS supports OpenLDAP and LDAP on Microsoft Active Directory.

LDAP attribute dependencies.

When using LDAP you need to make sure your triangular dependencies between users, groups and roles are setup correctly. In LDAP or AD a user is part of one or more groups. These group names need to coincide with the switch roles configured on the switches which bring me to the next subject.

Roles and Responsibilities

RBAC (Role Based Access Control) is the methodology Brocade provides for operational authorizations. Although it is not used very often I think a lot of problems could be prevented by assigning roles and responsibilities. I’ve seen a huge number of problems whereby, how do I say this politically correct, less senior administrators introduced massive issues by changing switch or fabric attributes causing large scale outages. I’m not blaming anyone for this however in the IT world there is an unwritten rule which more or less says “If you don’t know what the button is for, don NOT touch it“. But sometimes its like talking to a toddler in a candy store and telling him/her to not touch the licorice.

In the occasions where I had to set up this scenario there is a starting point which differentiates three roles.

  1. Admins – Overall Fabric-wide administrators responsible of the entire fabric operations. This includes everything that has to do with end-to-end connectivity, performance analytics, fabric parameters etc etc.
  2. Operators – Mostly tasked with provisioning and responsible for hooking up servers, arrays and tape kit and making sure that luns and tape drives become visible to hosts.
  3. NOCs or Datacentre monitoring personnel. These are often external companies or separate departments in an organization who keep a 24×7 helicopter view on your infrastructure but have very few operational administrative tasks.

The people who are primarily responsible for the overall status of the fabric (#1) most often have administrative access and are assigned the Admin role. If you are just taking take care of provisioning (#2) you might need to obtain zoning and maybe enable/disable port privileges and obviously for NOC’s you’ll most often see just the User (Observe) role assigned to the account. By default FOS ships with 8 roles but you can create your own and assign privileges as needed.

Admin Domains

An Admin Domain has nothing to do with storage operations itself. It is a logical authorization container which creates a boundary of responsibility. Combined with the RBAC methodology described above you can create separate environments in fabrics and assign the respective required authorization to these environments and users/admins. Admin domains and Virtual Fabrics are mutually exclusive so it either Admin Domains or Virtual Fabric but not both. This feature is depreciated as of FOS 8 so even if you thought of starting to implement it I would not start implementing it.

Virtual Fabrics

One of the most useful tools introduced with FOS 7 was the concept of virtual fabrics. This is not only a great option for segregating fabrics but also provides the possibility of creating a sandbox environment without having an effect on other, production, environments. That being said it is still not a good idea to start and perform all sorts of high impact storage traffic like stress tests, massive backup/restore activities and those kind of activities. Even though the fabric processes are separated the physical paths through a switch are not. This implicitly means that frame will traverse the same paths on the back-end even though they might be logically in a different fabric. The concept of using XISL’s (shared ISL’s across two or more virtual fabrics) between two physical switches even increases this phenomenon but that is beside the scope of this post.

To connect or not connect

Piggy backing on Shakespeare the best security of all is preventing anything to connect to the switch. By default all FC ports (licensing aside) come in an “Enabled” state by default. this means that whatever you connect to it will be able to use the resources provides by the switch. The issue with this is that even though you have set a fairly strict policy these will not propagate by default to other switches. So in case someone plug in a “plain vanilla” switch the entire fabric configuration is propagated to that switch but without the userdatabase and password policies. Obviously you will get events logged but you may already be too late to act on these.

To prevent this from happening (either by mistake or a malicious intent) by default persistently disable all ports.

portcfgpersistentdisable <SLOT>/<PORT> (on chassis based systems)

or portcfgpersistentdisable <PORT> (on fixed port based switches)

Only re-enable them again when you need to.

Locks on the door

Even though you have disabled all ports as described above this still doesn’t provide you with the granularity to control what can connect to certain ports. If you implemented a user accounts policy as I described earlier and someone tasked with provisioning authority accidentally hooks up a switch to a wrong port you might be in for a nasty surprise. To prevent this from happening Brocade has build in an option called ACL policies. These can be split in 4 features:

  1. FCS – Fabric Configuration Server
  2. DCC – Device Connection Control
  3. SCC – Switch Connection Control
  4. IPfilter – for management port access control (I will not describe this in this post but is similar to the normal netfilter firewall in Linux systems)

Each of the first three features are stored in a local database and depending on the distribution setting these can be propagated throughout the entire fabric. More on that later.

Fabric Configuration Server

The FCS is a setting which depicts which switch in the fabric is allowed to modify the fabric configuration. You can have a primary and one or more backup FCS switches. If you don’t configure a backup FCS switch and the primary fails you will not be able to make any configurations in the fabric. Only if the primary FCS switch is unavailable (ie not actively participating in a fabric) the backup FCS switches will allow modify commands. Some commands which only require “Observe/View” privileges are still possible.

Especially in environments where FC switches are located in different locations connected to a fair share of network segments who might be shielded by a multitude of different VLAN’s, IP firewall rules and router ACL’s you might not have a good insight who is able to connect to which switch. To have only a few switches which can actually modify settings allows you to tie down the possibility of a security leak.

Device Connection Control

A DCC policy configures some sort of control which devices are able to connect to a fabric. It basically is a list of WWN’s tied to a list of domain ID’s and port-numbers to which these devices can connect to. So if you have created a DCC policy with:

secpolicycreate “DCC_POLICY_xyz”,”10:00:00:00:01:01:01:01;2(3-15)”

that WWN is able to connect to domain 2 on ports 3 to 15. No other WWN’s will be able to connect to these ports unless you create more DCC policies with different WWN’s and also incorporate this domain and ports. Be aware that if this is the only DCC policy active on that switch the device with that WWN can still connect to all other domains including domain 2, port 0-2,16-<max port on that switch>. A DCC policy is not an “exclude all unless known” option. In order to achieve this you’ll need to create a DCC policy with a dummy WWN which includes all ports on that switch. Only by adding additional DCC policies with the real WWN’s you’ll be able to set an allow/prohibit matrix (not to confuse with the FICON option).

There are some caveats when you start using FA-PWWN (Fabric Assigned Port WWN’s. Refer to the admin guide for that.

Switch Connection Control

Switch connection control is used when a restriction is required of which switches can join a fabric. If a list of switch WWN’s in the SCC_POLICY does not match the one trying to join the fabric the fabric will segment that switch and disable the port. The SCC policy will need to be adjust to include the switch WWN. After activating the new policy and re-enabling the port the new switch will be allowed to connect and normal E-port/fabric configuration will take place. An SCC policy does accept switch WWN’s, domain IDs and switch names. I’d rather prefer the switch WWN. In contrast of DCC  where you can have more than one policy active the SCC policy is only constructed of a single list.

secpolicycreate  \ “SCC_POLICY”,”10:00:00:05:e5:01:99:fb;10:00:00:05:c3:d2:66:7a”

In the above examples only these two switches are able to construct a fabric. All other switches are denied. (Be aware that the order of the WWN’s are important. A policy of “SCC_POLICY”,”ABC;123″ is different than “SCC_POLICY”,”123;ABC” (See later on distribution).

Distribution of Policies

In line with the usual networking firewall and load-balancing clusters you do not configure these policies on each switch. Instead you configure distribution of these policies. This can be done manually or automatically. The manual method is by using the “distribute” command. The automatic method is by configuring the distribution by utilising the “fddcfg” command

First all switches need to be set to accept the distribution of policies:

fddcfg –localaccept

The policy databases that are most often distributed are the SCC, DCC and FCS policies. The password policy is most often hooked up via LDAP in a corporate determine security schema. The IPfilter policy may be distributed however this is rarely seen especially if FC switches in a single fabric are connected tow different VLAN’s, IP segments and/or security boundaries. The latter two (IPfilter and password) can only be distributed manually.

When the switches have been configured to accept the policy distribution you can (from the FCS primary switch if configured) the distribution via:

fddcfg –fabwideset “SCC;DCC;FCS”

There are two sorts of fabric-wide consistency policies: Tolerant and Strict. The “tolerant” form is shown above. When executing the same command with  “:S” behind the policy it means that one is now set to “Strict”.

fddcfg –fabwideset “SCC:S;DCC;FCS”

If the policy on two switches is different but not set as “Strict” the switches will merge to form a fabric however the policy distribution is blocked. If the content of the different policies are different (ie different switch WWN’s) the switches will NOT merge.

If you have a cascaded FICON (Mainframe) environment the SCC distribution has to be set to “Strict”.

Obviously there are numerous resources around the web available who provide guidelines around network access security, business policy controls, legal information etc. This post is just an introduction into what is currently possible on a Brocade FC environment.

Print Friendly, PDF & Email

Subscribe to our newsletter to receive updates on products, services and general information around Linux, Storage and Cybersecurity.

The Cybersecurity option is an OPT-OUT selection due to the importance of the category. Modify your choice if needed.

Select list(s):