Build Your Own Layer Cake

hether you are planning to replace an MPLS network that you deployed ten years ago or building a multi-site, private network from scratch, evaluating a software-defined, wide-area-network (SD-WAN) is likely going to be of interest to you. SD-WAN can be described as a next-generation or advanced feature, virtual-private-network (VPN) platform. Examples include support for: dynamically creating multiple VPN tunnels across one or more network connections to other, allowed nodes or sites; sending/receiving packets across multiple links; automated failover between connections; packet duplication across multiple links to reduce packet loss or errors; software-based quality of service (QoS); direct local internet access for trusted partner communications (e.g., private cloud); centralized management; zero-touch provisioning (ZTP); cellular out-of-band and cellular path of last resort; and the list goes on. What qualifies a technology to be named an SD-WAN platform can also be a point of debate.

There are several WAN solution elements, a layer cake of sorts, that should be considered in the quest to design and deploy a platform that will fulfill each organization’s unique needs. As you continue to read, you will also learn that understanding each layer is relevant to the overall design. These layers are summarized below (the layer cake):

(If figure 1 does not directly follow, list of bullets that match the table should be added)

(Each section that follows should be titled, either with a snip of the layer cake or text-based title)

Secure WAN

This is the SD-WAN platform. There are scores of companies offering solutions, including VeloCloud, Viptella, Aruba, SilverPeak, Maraki, and Fat Pipe. Requesting demos is an essential part of building knowledge of features and identifying those that may be important to your organization. This can be done at tradeshows or directly by sales teams from your existing or new value-added resellers (VAR). Build a checklist of features that may qualify or disqualify each vendor. A vendor short-list will likely emerge. Consider features that may become important in the future. Completing a proof-of-concept (POC), with the vanilla offering from multiple SD-WAN suppliers, is a quick way

to kick the tires. I recommend as many as three, and one will likely stand out to you. Run the POCs sequentially as they are typically time-bound, e.g., four weeks. Insist on engineering resources from the supplier, so you have the proper guidance. Tip: The vanilla offering, direct from the SD-WAN supplier, is often the fastest to kick-off a proof-of-concept. Other purchase channels (e.g., telco bundled offering) can be more complex to navigate for proofs-of-concept. If you need more time, don’t be shy to ask for it. Be aware that some vendors may charge POC fees or ask for an agreement that will automatically result in an equipment purchase of the POC hardware, beyond an agreed POC end date. This should not be cause for alarm, and it will be money well spent, even if the result is ruling the platform out. For each platform, document your findings, as this will be the basis for discussions to compare platforms. Plan for two internet connections, in the case of a basic POC, or four,in order to evaluate the features that SD-WAN platforms are so well known for. You may be able to get creative with the connections you already have. Consider ordering new links for the POC, as a permanent lab is something I highly recommend. Depending on your scale, conclude vendor selection after piloting at a sampling of sites that are representative of your business. A hybrid of multiple solutions may be necessary in some environments due to M&A or cost. E.g., SD-WAN nodes plus traditional hardware VPN nodes.

Hardware

You might think this is obvious, but I beg to differ. There are some options.

1. Hardware from the SD-WAN software provider (vanilla offering). Understanding system capacities is about the only decision driver. Pros: Simple. Cons: Proprietary/dedicated hardware. Tip: Some managed service providers will resell the hardware from the SD-WAN software provider, under their own brand, and some platforms, at specific hardware levels, support limited, network function virtualization (NFV). E.g., Palo Alto virtual wire on VeloCloud.

2. Managed service provider hardware, universal Customer Premise Equipment (MSP uCPE). An example from AT&T is FlexWare. This is a proprietary, container/app platform with an app store. Essentially a closed hypervisor for NFVs. Your selected SD-WAN, firewall software, and other network functions can be deployed on a single piece of hardware. 

NFVs can be added and subtracted using the app menu, e.g., Cisco router (IOS). Pros: Flexible. Cons: Locks you to the MSP, potentially higher cost.

3. Whitebox uCPE. Imagine a room full of the same model of spare. As your needs or software solution template changes, just swap out NFVs, without the need to change to a different hardware platform. In short, current-generation x86 CPU (e.g., Intel Atom C3000 and Xenon D-2100 which both have support for AES-NI, PCI pass through, SR-IOV), enough RAM (e.g., 32GB), combined with multiple network ports, SSD storage, and a license-free or license paid, open hypervisor. Pros: Total flexibility. Cons: In-house build. Examples of specific hardware: Super Micro 5019A-FTN10P and 1019D-FRN8TP. Tip: Understanding SSD endurance is vital. This can be addressed by selecting a small MLC drive (250GB) or a large TLC drive (1TB). Even though you may only use a mere 25GB, the larger the drive, the longer it will last and is vital for life expectancy (e.g., ten years). Terabytes Written (TBW) is the measure of endurance on SSD. Plan on running your SSD at less than 10% used storage to get the full benefit from wear leveling. Consider keeping the boot device separate, e.g., 128GB SATA-DOM. This simplifies hypervisor rebuilds when the NFV datastore resides on a physically separate drive. Raid is unnecessary. When multiple NFVs reside on the same uCPE, reduce the risk of this single point of failure and deploy two uCPE with redundant NFVs. No need for things like ESXI vMotion and traditional full backups. Simple configuration backups will suffice, just like any piece of network equipment. The choice of hypervisor is relevant. Document all the platforms in your environment that support virtualization, as well as those that might occur in the future. Build a matrix of hypervisors they support. Consider the hypervisors and knowledge already in your environment. If your network vendor does not publish their platform virtualized, contact the head of the product and request it.

Synergy with secure LAN

Do you have internal network segmentation requirements? If so, are they based on firewalling alone (e.g., Palo Alto, Fortinet), firewalling between Virtual Routing and Forwarding instances (VRFs), or driven by policy on a fully integrated, Network-Access Control (NAC) platform? e.g., Aruba. A well-defined NAC, SD-WAN and firewall strategy, may drive platform selection, and the organization of those resources supporting it.

Internet Access

This might seem obvious, but it needs to be considered. Even though you are using internet connections to stitch together a secure and private WAN for your organization, the internet egress point for traffic will need to be defined, and sometimes with a high degree of precision, including deliberate variances for specific scenarios. This can be important as you overlay a web-content filtering solution on SD-WAN or consider the other companies you do business with, e.g., web-site access restrictions based on source IP address, and guest/BYOD egress. As previously mentioned, directly egressing to trusted partner platforms (e.g., private cloud) can unburden other security platforms, filtering traffic that does not need to be filtered. Internet DNS provider selection is a consideration that should be explored in detail.

SD-WAN can be described as a next-generation or advanced feature, virtual-private-network (VPN) platform

Connectivity

SD-WAN is nothing without network connections. In developed countries, you can assume that fiber-based Direct Internet Access (DIA) connections, which offer symmetric bandwidth, at a lower cost per Megabit per second (Mbps) than MPLS, will offer the same low latency as MPLS. This is because both offerings will typically traverse a single provider backbone until it hops to a destination not served by that provider. Selecting a single Internet Service Provider (ISP) offers the additional benefit that traffic avoids internet peering points, between providers, where performance may suffer. Broadband (e.g., coaxial cable internet) should be considered, as it is generally cheaper per Mbps than DIA. It offers asymmetric bandwidth and can provide a level of physical and provider diversity and redundancy when combined with DIA. Use cases for broadband include a single, primary connection for smaller sites, or at larger sites, as supplemental bandwidth, and for added resiliency. Broadband can also be configured as the only path for guest and BYOD traffic. Fixed Wireless (e.g., microwave) may be necessary for locations where traditional wired solutions are not available or reliable, both in developed and less developed countries. Cellular is often viable for out-of-band (OOB) management or path of last resort. Wireless Broadband services (4G, 5G), which is cellular with “all you can eat” data and pre-determined bandwidth (e.g., 8Mbps), will offer viable primary, secondary or tertiary connectivity, depending on the latency and bandwidth required. In less developed countries, depending on the facilities available, you may still require MPLS or MPLS over Fixed Wireless, which then presents the challenge that one or more of the other sites on your network will also need to maintain MPLS. Considering the multitude of internet-based connections, on average, due to the expected lower cost per Mbps overall, you should expect bandwidth increases in the range of 10x to 20x over MPLS.

Purchase channels

Last but certainly not least, how do you make these purchases?

1. Direct from the connectivity provider (multi-vendor). Pros: Easy to establish contact with. Cons: Each provider does business differently.

2. Wholesaler/Agent. Pros: Consolidated relationship and purchasing power. Cons: Minimum volume may be required.

3. MSP. Depending on the scale of your organization, there may be a value proposition to consider a one-stop shop MSP as an SD-WAN platform supplier and connectivity provider. Some MSP will offer a co-managed platform model. Bench strength within your organization may vary over time, and the opportunity to lean on the MSP can be a distinct advantage, be it planned or unplanned. Pros: Turn-key. Cons: Minimum volume may be required, locked to a single vendor for all services. Tip: MSP comes in all shapes and sizes. Do your research and conduct interviews.

As you evaluate each element in your solution, quote, and forecast every aspect before making a final decision. The total cost, when compared to your existing platform, will vary (e.g., 25% less expensive). You might have a hump year as you navigate your way between platforms. I recommend a three to five year of expenditure analysis in order to understand this. Keep in mind that the offerings from each provider, in each layer, will change over time. Understanding the roadmaps and planned features can be vital as you form these new partnerships, and evolve your solution over time.

In summary, SD-WAN is a technology that will allow you to take a layered cake approach to design a solution that fits your organization’s needs. It offers levels of flexibility that will safeguard and improve overall WAN performance and resiliency. It will enable you to decouple many attributes of the WAN solution that were not possible with those technologies that came before it. The opportunity to select the hardware, connectivity, and purchase channels independently of the other, combined with the advanced features of the selected SD-WAN platform, will likely be a key technology enabler for your enterprise.