Update on George Ou’s status

I will be presenting a keynote in front of at least 700 Verizon managers plus additional industry partners at the Verizon Technology Forum this Friday on 4/11/2008 starting at 8:20 AM.  Note that my title was written before news of my ZDNet departure so it’s not updated yet.  Here’s an excerpt from EarthTimes.org:


“PALATINE, Ill. – (Business Wire) Verizon and OSP® Magazine today announced a stellar speaker lineup that will participate at the second annual exclusive Technology Forum to be held April 10-11, 2008, at the Hershey Lodge in Hershey, Pennsylvania. Speakers include: Day One Keynote by Claire Beth Nogay, SVP & Chief Network Officer, Verizon Telecom; Day Two Keynote ÔOur Industry Ten Years and Beyond,’ by David Carnevale, VP, Multimedia Content and Distribution, iSuppli Corporation and ÔThe Battle of the HD,’ by George Ou, CISSP, ZDNet Editor at Large (CNET Networks).”

Next week on Thursday 4/17/2008, I will appear on a panel before the FCC at Stanford University in Palo Alto California.  I will be on the panel called “Network Management and Consumer Expectations”.

The following week after that on the morning of 4/25/2008, I will be on another panel at the Center for Business and Public Policy at Georgetown University titledSpectrum Policy: From the Foundations to the Future.

My job hunt looks pretty good with a lot of good potential leads.  I’ll keep everyone posted here.

George Ou’s departure from ZDNet

This is a note to my valuable readers.  CNET Networks, owner of ZDNet, laid off 10% of its North American work force on Wednesday March 26th 2008 and I was caught up in this round’s layoffs.  I was one of two permanent editors on staff at ZDNet but despite being one of the top draws with ~1 million page views a month, CNET/ZDNet let me go.  I provided valuable insight to ZDNet while I was there with in-depth analysis and insight from an IT Engineer’s point of view and I enjoyed stimulating your minds and entertaining you.  CNET is in some financial trouble and they took some desperate measures and they made the decision that I am expendable.  While I may disagree with that decision, it isn’t mine to make and I’m going to move on to greener pastures.

So far, several potential news organizations have shown interest but they’re checking for an opening.  But if I can’t get a solid offer, then I’ll go back to the IT Consulting business which I was perfectly happy doing and maybe do some blogging on the side.  My IT skill set hasn’t declined because I spent the last 2 years analyzing and researching technology and I acquired my CISSP certification during that time.  However, I may also explore other careers but it’s too early to say at this point since it’s only been 2 days since my departure from ZDNet.

At this point in time, this site is a static HTML site only but I’ll convert it in to some sort of IT/Technology portal with dynamic HTML, RSS, and a full talkback system.  Sorry there are no RSS feeds on this site right now but I’ll keep this front door updated with any additional news so please check back manually for the time being.  Thank you for your support.
 

Sincerely,
George Ou

Implementing VLAN trunking

Contents

  • Introduction
  • Cisco switch configurations
  • Cisco router configurations
  • Windows configuration with Intel Pro Series adapters

Introduction
In my last article “Introduction to VLAN trunking”, I wetted your appetite for a hot new technology that is revolutionizing the way network topology are being designed and interconnected. In this piece, I will show you how to actually implement this new technology in the three most common types of equipment you will come across. The three types of equipment are Cisco switches, routers, and Servers or Workstations running the Windows operating system. The only prerequisite for this article is a basic working knowledge of Switches, Routers, and PCs running Windows for their respective sections. Click here for a network diagram of the lab environment created in this article. Note that the examples I use are on based on the 802.1q standard.

Cisco switch configurations
Cisco switches primarily come in two flavors, CatOS (Catalyst OS) and IOS (Internetworking OS). Although Cisco is trying to migrate almost everything to the IOS type operating system on their equipment, there is still a large install base for CatOS switches. Cisco’s flag ship 6500 series switch can actually run CatOS or IOS, but most people I know run CatOS on the 6500s. Smaller switches like the 2950 and the 3550 all run IOS. Then there is the odd ball 2948g-L3 that really is more of a router than a switch (2948 without the “L3” is a normal IOS switch) and you should refer to the next section on routers for it’s configuration.

Note: In many ways, I personally love the CatOS over IOS for it’s UI’s (User Interface) superior method of entering system configuration. For example, if you ever need to apply a common configuration to 48 Ethernet ports on module 4, you simply need to apply a command to “4/1-48”. On the IOS UI, you would need to enter each interface for all 48 ports and apply 96 individual commands vs. one command on the CatOS! Viewing the configuration on IOS is equally bloated.

Here is a breakdown of trunking support for the various Cisco switches

IOS CatOS
2900 Series (on some IOS versions) 2980 (Same IOS image as the 4000)
2948 (Non L3) 4000 Series
2950 Series 5000 and 5500 Series
3548 6000 and 6500 Series
3550 Series
6500 running IOS

To set up the CatOS or IOS on Cisco Switches, the port that needs to trunked must be configured for the right kind of VLAN trunking (Note that not every module and interface on a switch support trunking and will give you an error message if you try to set it for trunking, this may require you look up the port capabilities for each port).

Here is the configuration guide for both IOS and CatOS.

Configuring and locking down IOS switches:

IOS Command Description
Enable Switch to enable mode
Configure Terminal Enter global configuration mode
Interface FastEthernet0/1 Entering interface configuration for port 0/1. This is where you pick the port you want to trunk.
Switchport mode trunk Set port to trunking mode.
Switchport trunk encapsulation dot1q Set trunk type to 802.1q. If your switch only supports either ISL or 802.1q, this command does not exist because there is nothing to specify. This command only works when you can choose between the two.
Switchport trunk allow vlan 10-15,20 Allow only VLANs 10 through 15 and VLAN 20. It is important that you restrict the VLANs to only the ones you need for security best practices.
Exit Exit interface
Exit Exit global configuration
Write memory Commit changes to NVRAM

Locking down CatOS for security:

CatOS Command Description
Enable Switch to enable mode
Clear trunk 1/1-2 1-1005
Clear trunk 2/1-2 1-1005
Clear trunk 3/1-24 1-1005
…fill in the pieces…
Clear trunk 12/1-24 1-1005
Set trunk 1/1-2 off
Set trunk 2/1-2 off
Set trunk 3/1-24 off
Set trunk 4/1-24 off
…fill in the pieces…
Set trunk 9/1-24 off
This is an example of how to lock down a Cisco 6500 switch. First it clears VLANs from all ports on a 6500 switch, and then it explicitly disables trunking from every single port. Whether you intend to use trunking on your CatOS switch or not, you would be very wise to implement this lock down on all of your CatOS switches. Otherwise, a hacker can bypass all Layer 3 (firewall) security by simply hopping VLANs. I included this section before the “Configuring CatOS” section because the lockdown needs to be done before any custom configuration is entered.

Although this section is not really mandatory for trunking to function, I felt irresponsible not to include this layer 2 security lockdown procedure. Although the CatOS switch has a far more streamlined UI compared to the IOS switches, it is notoriously promiscuous with it’s default settings on VLAN trunking. The trunking auto-negotiation is equally alarming on both the IOS and CatOS switches, which if left default will automatically connect switches as fully enabled and wide open. You would be shocked to see the sloppy Layer 2 security on most networks. If left unchecked, you are not only opened to malicious hacks, but someone could accidentally plug in a Cisco switch with a VTP engine and accidentally nuke your network by changing your VLAN configuration.

Configuring CatOS switches:

CatOS Command Description
Enable Switch to enable mode
Set trunk 1/1 on dot1q 10-15,20 The “on” switch enables trunking on this port. “Dot1q” sets the port to 802.1q mode. “10-15,20” enables VLAN 10-15 and 20 to be supported on this trunking interface.

You may find it funny that it was so much work to lock down your switch while it only took one command to enable trunking. If you didn’t bother to follow the lockdown procedure shown above, specifying the “10-15, 20” VLAN IDs are useless because it simply adds them to the existing 1-1005 pool which remains wide open. This behavior of the CatOS is very annoying and insecure by default. The IOS switches on the other hand only permit the VLANs you enter last, which also has it’s user friendliness downside. On an IOS switch, if you enter “10-15,20” with your “allow VLAN” statement, it nullifies any other allowed VLAN out side of 10-15 and 20. The big plus to this is default security.

Cisco router configurations
Cisco router configuration for trunking is fundamentally different from Cisco Switch configuration. A router encapsulates traffic to be carried on the switch infrastructure and behaves as a multi-home node on the network just like a Server, Workstation, or Firewall. A switch performs as the infrastructure to carry traffic for VLANs (for those that are allowed) on the Layer 2 infrastructure as the VLAN traffic director where as the router performs a higher layer function as a network gateway that can route Layer 3 traffic. You can basically configure a router with number of desired virtual interfaces (AKA sub-interface) from a single interface and designate the VLAN you want those interfaces to be switched to. The switch determines where the traffic from that router’s virtual interface will wind up based on the VLAN ID portion of the 802.1q tag that was inserted in to the Ethernet frame header by the router.

Configuring Cisco Routers:

IOS Command Description
Enable Switch to enable mode
Configure terminal Switch to global configuration mode
Interface FastEthernet0/0.1 Creates first sub-interface for FastEthernet0/0
Encapsulation dot1q 10 Injects 802.1q tag with VLAN ID 10 into every frame coming from first sub-interface.
IP address 10.1.1.1 255.255.255.0 Defines IP/mask for this first sub-interface
Exit Exits first sub-interface
Interface FastEthernet0/0.2 Creates second sub-interface for FastEthernet0/0
Encapsulation dot1q 11 Injects 802.1q tag with VLAN ID 11 into every frame coming from second sub-interface.
IP address 10.1.2.1 255.255.255.0 Defines IP/mask for this second sub-interface
Exit Exits second sub-interface
Exit Exit global config
Write memory Commits changes to NVRAM

You can continue to add any number of sub-interfaces you need. Once FastEthernet0/0 is connected to a switched port configured for 802.1q trunking as shown in the above switch examples, all the sub-interfaces of FastEthernet0/0 becomes a routable node (can be default gateway) on the subnets that correspond to their VLAN.

Windows configuration with Intel Pro Series adapters
Conceptually, trunking a Windows workstation or server to a switch is the same a trunking a router to a switch. The only difference is the procedure, and a much easier one I might add. The ubiquitous Intel Pro Series adapters provide a simple to use graphical tool called PROSet that any one can learn within a minute or even someone who is just winging it. Note that the same Intel adapters with the ANS drivers can provide similar capabilities on Linux. You can get more information on Linux here from Intel.

To get started, simply invoke the Intel PROSet or PROSet II utility (assuming PROSet is installed). This can be done by simply double clicking the PROSet icon in the system tray on the lower right hand corner of the desktop. The following utility should come up.

Next we must add a VLAN interface. Simply right click on Intel adapter with the PCI Card icon and click “Add VLAN”. Note in the following screen capture, the virtual interface for VLAN 100 is already there and we are adding an additional one.

The “Add New VLAN” window comes up. Enter the VLAN ID you want this interface to trunk in to in the ID field, then give it a name that describes the VLAN function. In this case, we will be adding VLAN 69 labeled the Wireless LAB.

Once this is completed and you click “OK”, simply click “Apply” and “OK” on the PROSet window to commit the changes and get out of the PROSet utility. The next step is to configure the virtual interfaces. Simply open up the “Network Connections” window and begin configuring the virtual interface as you would any other physical interface. Note that the interface names already correspond to the names of the VLAN interfaces you added. However, auto-naming only works in Windows XP. Windows 2000 just gives them generic names, so you must add one interface at a time and rename the interface under “Network Connections” before you add another VLAN interface. If you don’t do that, it is impossible to tell which Interface goes to which VLAN without some tedious trial and error. One other very important thing to note, the physical interface it self “Local Area Connection” is not bound to anything except for the “Intel Advance Network Services Protocol”. It is not used for anything else and only serves as a host for all of the virtual interfaces and it does not have it’s own IP address or VLAN.

Just remember that only your primary interface is registered with internal Dynamic DNS and WINS and is the only interface that can have a default gateway. This is the same as when you have multiple physical network interfaces. In both cases whether there are multiple physical or virtual interfaces, you must set manual routes to take advantage of the other non-primary interfaces. This is why in the TCP/IP configuration window above, I deliberately left the Default gateway and DNS settings blank because those settings went on to the VLAN 100 interface. If you put a default gateway on the VLAN 69 interface, it will take over and the default gateway for the VLAN 100 interface will disappear. All the default gateway means is the route for 0.0.0.0 network with mask 0.0.0.0 (which really just means any IP destination) will route to the default gateway. You can easily tell this with the “route print” command.

From this point on, you may add as many VLANs as you need using the example above. The only other thing you should be aware of when dealing with these VLAN Interfaces is that you should not “Disable or Enable” them from the “Network Connections” folder, and instead you should deal with the Interface from the PROSet tool. Doing so will cause you to encounter some strange behaviors.

An introduction to VLAN Trunking

Contents

  • Introduction
  • Applications of VLAN Trunking
  • VLAN encapsulation types
  • Trunking requirements

Introduction:
There are many Network Devices in the Data Center that require multi-homing (multiple network adapters) to tie in to multiple network segments.  As the number of those systems increase, it becomes more and more difficult to provide the network infrastructure (due to the sheer number of Ethernet connections that need to be provided) from the perspective of cost, space, and wire management.  A technology called VLAN (Virtual LAN broadcast domains logically segmented on an Ethernet switch) trunking that was once primarily the domain of network switches has now trickled down to the rest of the Data Center to address these issues.  Now it is possible for these multi-homing devices to be multi-homing in function without the need for multiple physical network adapters and the additional infrastructure associated with them.  VLAN trunking allows a single network adapter to behave as “n” number of virtual network adapters, where ”n” has a theoretical upper limit of 4096 but is typically limited to 1000 VLAN network segments.  In the case where a single gigabit Ethernet adapter is trunked in place of using multiple FastEthernet adapters, higher performance at a lower cost while increasing flexibility can be achieved.  This really is the best of all worlds.  In this article, I will give you an overview of VLAN trunking, how it works what it is used for.

Applications of VLAN Trunking:
Here are some common examples of Network Devices that benefit from VLAN trunking:

  • Routers
  • Firewalls (software or hardware)
  • Transparent proxy servers
  • VMWare hosts
  • Wireless Access Points

Routers can become infinitely more useful once they are trunked in to the enterprise switch infrastructure.  Once trunked, they become omnipresent and can provide routing services to any subnet in any corner of the enterprise network.  This is in essence what a routing module in a high-end core or distribution L3 (Layer 3) switch provides.  This technique can be a poor man’s substitute for a high-end routing module on a switch, or it can complement the high-end L3 switch by providing additional isolated routed zones for test labs, guest networks, and any other network segment that requires isolation.

Firewalls are another device that can greatly benefit from VLAN trunking now that all the big players like Cisco, Nokia/CheckPoint, and NetScreen support it.  In today’s high stakes environment where security concerns are ever increasing, the more firewall zones (subnets connected by separate virtual or physical network adapters) a firewall provides the better.  With the exception of NetScreen firewalls, firewalls can only block potentially hazardous traffic between zones and not traffic within the same zone.  Therefore, the more you separate devices like routers and servers by logical function and security level, the better off you are since you can limit unnecessary traffic and mitigate many security threats.  Since VLAN trunking provides a nearly unlimited number of virtual network connections at a lower cost and higher performance, it is the perfect addition to firewalls.  You can read more on this in:

Understand how to design a secure firewall policy

Increase firewall protection with a better network topology

Transparent proxy servers such as a Windows server running Microsoft ISA or a Linux server running Squid can now be built with a single gigabit Ethernet adapter costing as little as $40.  A traditional proxy server can be built with a single network connection, but a transparent proxy server usually cannot.  Since transparent proxy servers can be implemented with zero client deployment or SOCKS compliance; they are an extremely attractive new technology.  Trunking just makes it that much simpler and cheaper to implement.

VMWare hosts are servers that host multiple virtual servers for the purpose of server virtualization or system modeling for laboratory testing and research.  Although VMWare already provides the ability to have multiple VLANs within the VMWare host, it’s ability to connect those VLANs to physical VLANs is limited to the number of network adapters on the VMWare host.  A VMWare host can provide up to 3 network connections to each virtual machine.  Since applications cannot tell the difference between a virtual adapter and a physical one, a VMWare host armed with a trunked interface is significantly more flexible and simpler to manage.

One of the hottest new applications of VLAN trunking is wireless networking.  The new Cisco AP 1200 for example can behave as 16 virtual Wireless LAN infrastructures.  Some VLANs can be used for low security guest Internet access, others for minimum security enterprise users, and administrators can be put on a high security VLAN with enhanced firewall permissions.  All this can be achieved using a single Wi-Fi infrastructure to emulate up to 16 Wi-Fi infrastructures.  The Cisco AP 1200 does this by assigning each of the 16 VLANs it’s own Wi-Fi SSID, so when you look at it from NetSumbler (free wireless sniffer), you will think you are looking at up to 16 different wireless networks.  Those 16 VLANs are then trunked over the AP 1200’s FastEthernet port.  This offers wireless nirvana in Wireless LAN capabilities.

VLAN encapsulation types:

There are several types of VLAN encapsulation.  The two most common types are Cisco’s proprietary ISL (Inter Switch Link) and the IEEE 802.1q specification.  ISL is an older standard that Cisco was using to connect it’s switches and routers, but now that 802.1q is ratified, all of the newer Cisco gear either support both ISL and 802.1q or only 802.1q.  Older Cisco equipment may only support ISL trunking, so you must look up the individual specifications of your gear before attempting to connect them.

The 802.1q standard works by injecting a 32 bit VLAN tag into the Ethernet frame of all network traffic in which 12 of those bits define the VLAN ID.  The VLAN ID simply declares what VLAN the Ethernet frame belongs to, and the switch uses that ID to sort out and place the frames in their proper VLANs.  Once a frame reaches the end of the line or hits a non-trunked port, the VLAN tag is stripped from the frame because it no longer needs it.  This also means that if you attempt to trunk a host to a non-trunked port, it obviously will not work because that non-trunked port will strip the VLAN tags upon entry.  Note that there are very serious security implications of using VLAN technology; I will elaborate on that in a future article on VLAN Layer 2 security.

Given that a VLAN tag must be inserted into each and every Ethernet frame, it does mean that there is a little overhead in terms of slightly increased frame sizes and some CPU over head required to inject the tags.  Because of this, separate physical network adapters will always perform better than virtual network adapters on a single adapter of the same speed.  But remember, this performance deficiency is quickly reversed if a single gigabit Ethernet adapter is used in place of multiple FastEthernet adapters.  Given all the rewards of VLAN trunking, the small overhead is more than justified.

Trunking requirements:
VLAN Trunking requires that the network switch, the network adapter, and the drivers for the operating system all support VLAN tagging in order for them to trunk.  Almost any enterprise grade switch made by Cisco, Extreme, Foundry, and others support 802.1q.  A few examples of this on the smaller scale are the Cisco’s 2950 series and Netgear’s FSM726.   Most high end client adapters support VLAN trunking, but one of the most common ones you will find is the Intel Pro/100 and Pro/1000 adapters because it is included on almost every server manufacture’s motherboard.  For those without an integrated Intel adapter, a separate Pro/1000 PCI card can be bought for as little $40.  Drivers support on the Intel adapters are excellent and covers almost everything from BSD to Linux to Windows client and server operating systems.  My follow up article on how to actually implement VLAN trunking will focus on Cisco and Intel equipment.

Building a Redundant and Manageable DHCP infrastructure

Table of contents

  • Introduction
  • Designing your TCP/IP network
    • Performance considerations and the proper sizing of Subnets
    • Designing a clean “Binary nice” subnet site
    • Routing of Subnets with Layer 3 switches
  • DHCP Relay
    • Using an NT or Win2k DHCP relay server in each subnet
    • Using a Layer 3 switch to relay DHCP for all VLANs
  • DHCP Redundancy and Configuration
    • Setting up two non-overlapping DHCP servers for maximum availability
    • Building DHCP scopes and setting the scope and server options
  • Using MAC address “security” on DHCP
  • Advanced Switches with 802.1x Port Based Access Control and EAP
  • Conclusion

Introduction

This document is meant as an introduction and overview on how to build a redundant and easy to manage DHCP infrastructure with modern technology.  DHCP is a critical service that needs to be thoroughly integrated in to a good network design for a practical and functional network infrastructure.  Because it is impossible to talk about DHCP without talking about network infrastructure, I will start off by covering some basic TCP/IP network design.  Although there is a prerequisite for a basic understanding of TCP/IP networking concepts and the Cisco layer 3 switching configuration to fully comprehend all of the material, you can still read it on a high level to get a good basic understanding of this technology.  Doing so will help you work better with professional networking consultants.

Designing your TCP/IP network

Performance and Sizing of Subnets:

In order to design a high performance and low congestion network, we must understand what the enemies of network performance are.  The biggest enemy of network performance in the past was data collisions with the use of Ethernet Repeaters (AKA Hubs).  Any time any data is transmitted by one computer to another, the data is repeated to every single port of the Hub which causes congestion for every one.  In today’s network, this is a thing of the past because Ethernet switches have reached such a high economy of scale and are so affordable that it would almost be silly to continue to purchase Ethernet repeater technology.  Data collisions have all but become moot on modern Ethernet networks.  Ethernet Switches isolate traffic between two computers while keeping all other ports clear and open for all the other computers on the switch.  Because of this, the new king of congestion is the broadcast storm.  Computers (especially the ones running NetBEUI) have a nasty habit of calling out or announcing to the entire TCP/IP subnet forcing the Ethernet Switch that normally likes to keep traffic isolated to send that data stream to every port on the switch on the same subnet.  Even worst, sometimes every computer on that subnet has to respond to the sender causing the original broadcast to be amplified a thousand times.  Unfortunately, this sometimes puts us back into the same predicament that Ethernet Hubs had to constantly deal with.  The only way to combat this is to keep the number of hosts on a single broadcast domain to a minimum.  That means probably no more than 128 computers on a single TCP/IP subnet.  I have seen sites with thousands of computers on a single subnet and I can tell you it is not pretty when monitoring the broadcast storms.  In fact, it was so bad that it was enough to kick people out of their terminal server sessions a dozen times a day because of network instability!

Designing a clean “Binary nice” subnet site:

We will start with the premise that we have single LAN site on a single campus.  While it is possible to run DHCP over Wide Area Networks, it is not considered best practice so we will stick to a single LAN in this paper.  The site will have up to 1000 users with 1000 computers broken down in to 256-host sized VLANs (Virtual Local Area Networks created by logically segmenting a network with a managed layer 2 or layer 3 switch) with no more than 100 users per VLAN with room to spare.  This means we will require a minimum of 10 VLANs on this site.  Additionally, because we want to be able to summarize this site in to a single supernet when routing, we will round up to the next “nice” binary number 16.  We will use the private class A scheme of 10.x.x.x for our company, so for this site, we will run the entire site under the network ID of 10.0.0.0/20.  For those of you new to this terminology, this is the abbreviated terminology for the Network ID of 10.0.0.0 with subnet mask of 255.255.240.0 which defines all IP addresses ranging from 10.0.0.0 to 10.0.15.255.  By using “binary nice” numbers like 2, 4, 8, 16, 32, and so on, I am able to define the entire subnet by the single network ID of 10.0.0.0/20.  The reason for this is not solely ascetic, it greatly simplifies routing and security rules because I can define the entire network with a single statement.  This not only simplifies management, but also improves performance and reduces the chance of mistakes.  Some of you at this point may be balking at the idea of running 10 separate subnets for “only” 1000 users, but bear with me, it is not that difficult to handle if you use the right technology.  Also keep in mind that there are 65,536 256-host sized subnets in the 10.0.0.0/8 class A private network.  This means that you can have 4096 of these sites with 16 subnets each.  Obviously, the next campus LANs of similar size will be defined as 10.0.16.0/20, 10.0.32.0/20, 10.0.48.0/20, and so on.

Routing of Subnets with Layer 3 switches

Now that we have the basic network laid out, we must build it.  The best way to handle this is with a managed Ethernet Layer 3 switch such as a Cisco Catalyst 6500 series with MSFC but a Cisco 3550-12G can be used instead for smaller networks or tighter budgets (Note that Cisco is not the only company than can fill this job, but for the purposes of this paper, I will use the Cisco example.  Additionally, the 3550-12G makes for a great poor man’s core/distribution layer switch at 1/10th the cost).  Both of these switches can act as the core, or core and distribution layers of the network.  Then we can proceed to connect Access layer switches such as the Cisco 2980 switches (you can use cheaper unmanaged switches for this too but understand that you can’t break them up into additional VLANs or have trunking support) to the 6500 via gigabit Ethernet uplinks.  Then distribute these access layer switches around the campus so that the actual Cat5e or Cat6 copper runs to the clients are kept to a minimum length vastly reducing cabling cost in material and labor while increasing signal reliability.  Once this 2 or 3 tier design is in place, we can proceed to configuring the switches.  The Cisco 2980 access layer switch has VLAN or Bridge Group capabilities, but has no routing capabilities of it’s own.  For that, it can connect or trunk into the core switch using 802.1q trunking over the gigabit uplink via Cat6 copper or full duplex Fiber.  The core switch using the 6500 MSFC or the 3550-12G can act as a massive VLAN router to handle all routing requests and act as the default gateway for every VLAN on all tiers by configuring a single static routing table and/or protocol such as EIGRP, RIP, or OSPF.  Additionally, it can also act as the DHCP relay agent for all the VLANs as well and is definitely easier and cheaper than setting up at least 10 separate Windows or Linux boxes to act as DHCP relay agents.

Example with six VLANs using 6 2980 L2 switches and a 3550 12G as core/distribution layer switch:

DHCP relay:
A DHCP relay agent sits in place of an actual DHCP server in a TCP/IP subnet.  It basically extends the reach of the DHCP server without the need for multiple DHCP servers on each subnet by acting as the DHCP server’s helper agent in a remote subnet.  DHCP relay does not manage IP addresses itself, but relays the DHCP request to the DHCP server on behalf of the client, obtains the IP address, and then hands out the IP addresses to the asking client on behalf of the DHCP server.  The only other option is a single DHCP server with multiple Ethernet ports sitting on each VLAN but that has some serious limitations in scalability.  On Cisco Layer 3 switches, DHCP relay can easily be achieved with a single command of ip helper-address 10.0.14.255 entered in to each VLAN interface as shown below.  10.0.14.255 will be the broadcast address of the VLAN that will home my DHCP servers.  You can use a specific IP address here instead of a broadcast address, but that would mean only having one active DHCP server or you must cluster two or more DHCP servers on a single IP address.  For our example, the following are configuration examples with VLAN definitions (AKA Bridge Group), default gateways, and DHCP relay configurations for Cisco or IEEE standard configurations.

IEEE standard configuration on a Cisco 2948-L3 switch used as a Core/Distribution layer switch:

Bridge group declarations

bridge 1 protocol ieee

bridge 1 route ip

bridge 2 protocol ieee

bridge 2 route ip

…all the way through…

bridge 15 protocol ieee

bridge 15 route ip

Declares VLAN 1

Enables routing in VLAN 1

Declares VLAN 2

Enables routing in VLAN 2

… Declare VLANs 3 – 14 your self …

Declares VLAN 15

Enables routing in VLAN 15

Interface configurations

interface BVI1

ip address 10.0.1.1 255.255.255.0

ip helper-address 10.0.14.255

no ip directed-broadcast

interface BVI2

ip address 10.0.2.1 255.255.255.0

ip helper-address 10.0.14.255

no ip directed-broadcast

…all the way through…

interface BVI15

ip address 10.0.15.1 255.255.255.0

ip helper-address 10.0.14.255

no ip directed-broadcast

Defines VLAN 1

Sets the default gateway listener for VLAN 1 as 10.0.1.1

Sets the DHCP relay agent to forward to 10.0.14.255

Defines VLAN 2

Sets the default gateway listener for VLAN 2 as 10.0.2.1

Sets the DHCP relay agent to forward to 10.0.14.255

… Fill in VLANs 3 – 14 your self …

Defines VLAN 15

Sets the default gateway listener for VLAN 15 as 10.0.15.1

Sets the DHCP relay agent to forward to 10.0.14.255

Port configurations

interface FastEthernet1

no ip address

no ip directed-broadcast

bridge-group 1

interface FastEthernet2

no ip address

no ip directed-broadcast

bridge-group 2

…all the way through…
interface FastEthernet15

no ip address

no ip directed-broadcast

bridge-group 15

Defines Fast Ethernet Interface 1
Binds Interface 1 to VLAN 1
Defines Fast Ethernet Interface 2
Binds Interface 2 to VLAN 2

… Fill in Interfaces 3 – 14 your self …

Defines Fast Ethernet Interface 15

Binds Interface 15 to VLAN 15

(Note that VLAN 14 is the only bridge group interface that does not need the helper-address because it contains the DHCP server it self.  Also note that VLAN 11 through 15 will be used for spare, DMZ, or server farms.  I didn’t have a 3550-12G with Gigabit Ethernet handy so I used a 2948-L3 with Fast Ethernet instead for this example as the core switch, but it is the same ideal.)

Additionally, some of the Cisco L3 switches use a different type of command line interface.  The following is an example with a Cisco 6509 MSFC L3 module:

interface Vlan1

description Subnet 1

ip address 10.0.1.1 255.255.255.0

ip helper-address 10.0.14.255

no ip redirects

no ip directed-broadcast

This looks quite a bit different than the 2948-L3, but still uses the same DHCP relay command.  The VLAN command accomplishes the same thing as the BVI command, but it is a little easier with the 6509 type CLI (command line interface) because you don’t need to declare the IEEE bridge protocol.  The Cisco 2948-L3 CLI must manage the routing as well as the switching and port configurations.  The 6509 MSFC module is more of a dedicated routing and management module with the physical switch ports handled by a separate CLI.  You can consult your switch manual or Cisco’s web site for more information on your particular hardware.

While it is possible to use a Windows or Linux server as a DHCP relay agent, it would seen to be over kill to dedicate 15 or more separate machines to do the job of a single command on your Layer 3 switch.  Note that without this technique of using the L3 Switch, it would be extremely impractical to implement this degree of TCP/IP segmentation on a LAN.  You would also need 15 separate servers for DHCP relay agents and 15 traditional routers to join the 15 VLANs all of which would be absurd.  The point is, take the easy route and use a Layer 3 switch at the heart of your network.  It opens up all sorts of possibilities.

DHCP redundancy and configuration:

Setting up two non-overlapping DHCP servers for maximum availability
As I mentioned earlier, the DHCP servers will reside in the subnet of 10.0.14.0/24 along with many of your other servers.  Since DHCP is an extremely low activity service, my recommendation for this is that you may host the DHCP servers on your Windows NT or Windows 2000 Domain controllers or File servers along with other services like DNS, WINS, and other common services.  You only need to find two servers for a home.  Once you do, simply install the DHCP service and proceed to configure each server to serve only half the subnet with non-overlapping scopes (it is also good to cluster your DHCP servers, but that requires windows advanced server which may not be an option for everyone).  The first DHCP server will be configured with a scope of host numbers 10-109, and the second DHCP server will host 110 to 219.  This leaves hosts 1-10 reserved and 220-254 for static IP addresses for things like printers.  This is what is called a 50/50 configuration and you may also hear recommendations for an 80/20 configuration where IP addresses are a bit scarcer.  In this architecture how ever, we are leaving so much breathing room that only 50% of the total subnet is more than enough for all DHCP clients.  I also recommend not using DHCP reservations because this makes the management of DHCP servers extremely messy by fragmenting the scopes.  I would much rather assign people addresses in the 220-254 range (make this range as large as you need) rather than letting other system administrators or users pick their favorite number.  Because the DHCP relay agent is forwarding to the broadcast address where these two DHCP servers reside, it is basically a first respond first serve environment.  But it doesn’t matter since all of our users can fit in a single DHCP server with tons of room to spare.  Statistically if the two computers have equal load and are equal in speed, users will end up half and half on each DHCP server.

Building DHCP scopes and setting the scope and server options

For our example, we will use Windows 2000.  On these DHCP servers, you will need to create 10 new scopes using the create scope wizard.  During the creation of these scopes, simply name them VLAN1 through VLAN10 and enter the corresponding IP ranges.  Be sure to only enter the default gateway for each scope and don’t enter any other DHCP options.  This is explained by the differentiation between scope options and global (server) options and I often see people confuse the two.  It is possible to put any type of DHCP attributes like default gateway, DNS server, WINS, and such on either Scope or Server options, but there is only one proper way to do it.  The default gateway should always be put under scope options as you have already done during the creation of the scopes, all other standard attributes like WINS and DNS should be placed under server options (formerly known as global options under NT4).  Then the server options will automatically be inherited into all of the scopes saving you a lot of manual entry and possibility for errors.  To configure the server options, simply right click on server options and hit “configure options” to get the following window:

Set 006 for your DNS servers

Set 015 for your default domain suffix

Set 044 for your WINS servers

Set 046 for 0x8 for your WINS/NBT Node type

Now imagine doing this 10 times for each scope, it would be silly.  Putting these additional settings under Server Options is the best way to go.  Then repeat this procedure on the second DHCP server doing everything the same.  The only difference is that the host range will be 110-219 instead of 10-109 on the first DHCP server.  Some of you astute readers at this point may be wondering how to actually bind all the different scopes to their respective network IDs.  The answer is surprisingly simple, nothing!  When you created the scopes, you had to define the separate IP ranges of all the corresponding scopes it should operate in.  That alone is enough configuration to match up the scopes with the subnets they will serve.  When the DHCP server receives the DHCP forwarded request from the DHCP relay agent (or IP Helper), it simply examines the source IP of the DHCP relay agent that forwarded the request, then matches it up to the scope that serves the subnet of the DHCP relay agent and grants an IP-configuration-set back to the relay agent.  Then that IP-configuration-set is passed on by the DHCP relay agent to the original client that made the DHCP request in the first place.

Then finally after all that, be sure you “activate” your DHCP servers and authorize them by right clicking on the DHCP server and choosing “authorize”.  Once authorized and activated, you have just set up a two DHCP servers to serve 10 separate subnets with the aid of a single layer 3 switch.  Note that this type of infrastructure is extremely scalable and could just as easily serve 1000 scopes if needed.  A DHCP server only has to do one transaction per user per week so even 1000 scopes is not a lot of work for a slow 486 computer.

Be aware that this type of architecture absolutely mandates a good DNS and WINS infrastructure.  You cannot rely on the old broadcast discovery techniques like you could under a flat subnet where everyone lived.  But that is a great performance advantage and puts less reliance on luck when using broadcast and prey for a response.  But rest assure that having a disciplined TCP/IP name resolution infrastructure will pay great dividends when all the inconsistencies and mysteries of legacy style Windows networking disappears.

Using MAC address “security” on DHCP:

You can set up casual “security” for your network by only issuing IP addresses with pre-reserved MAC addresses.  The reason I say that sarcastically is because it is security through obscurity.  It can only be used for casual security because it is based on the honor system.  MAC addresses can be changed on any network adapter within seconds.  Your MAC address is what you declare it to be.  This is the same reason why MAC addresses can’t really secure wireless networks because it is so easy to forge.  The other problem with this “Security” scheme is that even if you don’t assign some one an IP address, that doesn’t mean they can’t just simply type in an IP address manually and still participate on your network.  Additionally, maintaining a 12 digit hex number gets to be quite cumbersome for a thousand users.  This technique keeps the non technical person out, but it has no security capabilities beyond that.  Real security needs to be handled at the switch level with 802.1x and EAP.

Advanced Switches with 802.1x Port Based Access Control and EAP:

Some advanced switches like the Cisco Catalyst 6500 supports 802.1x port based access control and extensible authentication protocol.  Basically, this means no authentication no access.  The Ethernet port remains closed until you authenticated successfully over EAP.  Unlike the previous method discussed using MAC reservations on the DHCP server, you can’t just forge the MAC address or even manually enter an IP address.  Either hack is useless when 802.1x/EAP is employed on the access switch.  When 802.1x is employed, a client connecting to a port on the switch must support the 802.1x protocol.  Currently, Windows XP is the only operating system that natively supports 802.1x but Microsoft is promising 802.1x support for legacy operating systems like 98, NT, and 2000 by the end of 2002.  Basically, when the 802.1x capable client connects to the switch, it must send EAP credentials to the switch.  The switch then forwards the EAP message to the RADIUS (Remote Authentication Dial-In User Service) server.  If the RADIUS server accepts the credentials, it will respond with an EAP success message to the Switch.  Only then will the Switch transition the port to an open state and then permit DHCP requests and full network participation.  Additionally, this same RADIUS infrastructure can be used to provide enterprise grade wireless security.

For more information on 802.1x and Cisco Switches, see this Cisco configuration guide for port based authentication.

Conclusion:

All the old concepts and ideas on hubs, switches, routers, and DHCP servers have been revolutionized by this new approach.  Not only are we able to create a more manageable and robust DHCP and Network infrastructure, we are able to do it with less money, equipment, and time.  It is simply a matter of taking advantage of what new technology has to offer.

Because technology isn't just for geeks