Understanding FC (and FCoE) fabric configuration in 5 minutes or less.

Forget what you’ve heard, it isn’t that hard.  Assuming you’re setting up a basic fibre channel SAN for VMware or a cluster filesystem like Apple’s Xsan the process is pretty straight-forward.

For detail on FCoE there’s a great paper on it from Brocade available here but the basics of configuring a fabric which I’m about to cover applies to both traditional Fibre Channel (FC) switches and the newer Fibre-Channel-over-Ethernet (FCoE) switches.

Terminology

Fabric – A Fibre Channel fabric is simply a network consisting of one or more Fibre Channel switches.  If you have just one switch that’s ok it’s still a fabric.  In larger networks you would interconnect many switches in a topology to provide higher throughput and availability in the event one or more switches fail.

Initiator – An initiator is the consumer of storage, typically a server with an adapter card in it called a Host Bus Adapter (HBA).  The initiator “initiates” a connection over the fabric to one or more ports on your storage system which are called target ports.

Target – Target ports are the ports on your storage system which deliver storage volumes (called target devices or LUNs) to the initiators

WWN – In Fibre Channel there’s a special term for a port identifier, specifically WWN or World Wide Name.  Just as every network card as a unique MAC address for each NIC port, every single FC port has a unique 64bit port WWN.  For example, a FC port WWN might look like this: 20:00:00:81:23:45:ac:01 and though I’ve used the colon (:) as a separator/delimiter it’s just as common to see the 16 hex digits separated by hyphens (-).

Note that initiators do not talk to initiators and targets do not talk to targets.  It’s generally bad practice to have initiators on one server connected to initiators on another.  Why?  Initiators are chatty.  When it connects to a fabric the initiator will talk to every single target port (and every single target device exposed on those target ports) that it has been allowed access to and it can also send out bus resets which can confuse the other initiators.

Basic Components of a SAN

Zoning

So to keep those chatty initiators from communicating with all the target ports on your fabric you’ve got to put them into what are called zones, and that’s the heart of it all. Fabric management is all about restricting access to groups of initiator and target ports through zoning.

A zone is simply a list of IDs containing an initiator port ID and one or more target port IDs.  It’s like setting up virtual cabling or paths so that there’s a data path from the initiators to the targets.  Delete the zone for a given group and those paths which that zone allowed are deleted and the associated traffic for those paths stops.  If no zones exist, no traffic is allowed to flow through the fabric.

Zonesets

The zoneset is just a group of zones which you want to make as an active configuration.  So when you make changes to your zones you have to reactivate the zoneset that they’re in for the configuration changes to become active in the fabric.  You can have multiple zonesets but only one can be active at time.  In this way you can reconfigure the switch, make new zones and zonesets and then make the configuration changes active at a later date.

At this point you’ve got all the basics, the rest of the stuff from here out is icing.

Zone Aliases

Note that when you create a zone you are simply putting a list of WWNs in it which includes at least one initiator WWN and one or more target WWNs.  Well, trying to remember a bunch of 64 bit numbers for your initiators and targets is a mess and a headache and error prone so wouldn’t it be nice if you could give each port WWN a nice name so it’s easy to keep track of what’s what?  This is where aliases come in, specifically what are called zone aliases.  As noted above WWNs look like this 20:00:00:81:23:45:ac:01 so continuing the example you might have a zone called zone-winserver123 with one initiator and two target ports which might look like this:

zone-winserver123 {20:00:00:81:23:45:ac:01, 50:00:00:51:63:25:dd:ef, 50:00:00:51:63:25:dd:ee }

As you can see it’s difficult to make heads or tails of what’s going on here because the numbers are too big and it’s hard to remember which port is an initiator, which is a target and from which server or storage system.  This is where zone aliases come in.  A zone alias is a name you can apply to any WWN or group of WWNs so you don’t have to go cross-eyed looking at 64bit WWN numbers.  It’s very important that you set up your zoning correctly so the first thing you should do anytime you have a new fabric to configure is to connect the initiator and target ports to the switch and then set up zone aliases for everything.  I usually group together all the target port WWNs from a given storage system into one zone alias and then create a zone alias for each initiator port.  I also like to put za- at the front of the alias name so that I can clearly see the difference between zone aliases and zones in switch web management interface but it’s not necessary.  So here’s an example of what your aliases might look like:

za-emc-clariion-001 { 50:00:00:51:63:25:dd:ef, 50:00:00:51:63:25:dd:ee }

za-winserver123-port1 { 20:00:00:81:23:45:ac:01 }

za-winserver123-port2 { 20:00:00:81:23:45:ac:00 }

Now when you create a zone as in the example above it’s all made clear:

zone-winserver123 {za-emc-clariion-001, za-winserver123-port1, za-winserver123-port2}

Now anyone that looks at the switch configuration can see that the purpose of this zone is to provide a windows server (winserver123) with access to target ports on your EMC Clariion system 001.

That’s really all there is to it: initiators, targets, zone aliases, zones, and zone sets.

Final Notes

Yes, there’s more terminology (N_Port, F_Port, E_Port) you might familiarize yourself with and what we’ve gone over here is what’s called WWN zoning which allows you to plug your cables into pretty much any port you want (as opposed to Port zoning).  Also note, it doesn’t matter if you’re configuring a Brocade or a Qlogic FC switch, all the terminology is the same as this was worked out by SNIA long ago.

No, there are no LUNs to think about, switches don’t care what devices (targets) are being accessed, they’re simply focused on which initiator ports can communicate with which target ports.  Storage provisioning and controlling which initiator can access to which specific LUNs / storage volumes in a system is controlled and configured within your storage system or appliance and is outside of the scope of the fabric configuration. That process of configuring access to storage volumes is called “LUN Masking” which is a mapping of storage volumes (aka LUNs) to initiator port WWNs.  It says “these initiators (servers) are allowed to access these storage volumes / virtual disks.”  But that’s for another article.

Last, I’m going to leave you with a diagram of what a simple fabric configuration looks like.

Hope you enjoyed the article,

Best,

-Steve



Categories: Storage Appliance Hardware

Tags: , , , , , , ,

34 replies

  1. Hi there, You’ve done an excellent job. I will definitely digg it and personally suggest to my friends. I’m confident they’ll be benefited from this website.

  2. best article to understand basics of SAN switch

  3. Nice defenition. Thanks

  4. Simple and easy ….zillion likes

  5. Beautifully explained!

  6. Very clear… and good to start with…
    Thanks for the work.

  7. Very nicely explained…thanks

  8. good, do you have any video demonstrating the zoning

  9. Great! It is what I’ve been looking for.
    thank you.

  10. more than nice. succinct. ad perfectum.

  11. Superb. V Easy to Understand Complicated SAN Topic………

  12. Beautifully concise and understandable. Kudos!

  13. Good article for a me a zoning novice. Well articulated. Thanx

  14. Very easy to understand SAN. Now SAN concept has been cleared. Thanks

  15. Nicely explained. Good piece of information about FC switch.

  16. Simple to understand,Thank you.

  17. Excellent post !!! i am really benefited from your post. You have awesome explanation skill.

  18. Thanks a lot for the work done here. It helps a lot to have the basics so clearly explained. Thanks.

  19. Wonderful! Learned a lot.

  20. Thanks! But I’m still confused…should I create a zone for each initiator port to the target group alias? In other words, if an initiator has multiple initiator ports, should each port be a separate zone?

    • Hi John, in short, yes. That advice came from a Brocade engineer years ago. Technically, you are not likely to ever have problems grouping the initiator ports for an HBA into an alias. That said, pretty much all modern FC HBAs can be put in Target mode, Initiator mode and both at the same time. So it is cleaner to do the zone per initiator port route..

  21. Steve this was an excellent explanation. I have two new guys I’m training on Backups and storage. I read this blog and it’s AWESOME! I feel this will help me to succeed in bringing these guys up to speed quickly. They’re totally new to SAN. Thanks Steve

  22. Hands down the best 5-minutes spent!

  23. Hi you have a user friendly site It was very easy to post I enjoyed your site

  24. Amazing synthesis, thanks a million, now I’m ready to get into the scary fabric

  25. Very well explained. Thank you

Trackbacks

  1. Resumo sobre FC e FCoE para administradores de rede local (LAN) –

Leave a reply to Nonylus Cancel reply