Cloud – Page 2 – Chris Cormack

Foundational DevOps

Benefits of Infrastructure-as-Code and Cloud Economics

As I see customers adopt Amazon Web Services, one of the first benefits they quickly realise is the ability to create and bootstrap environments at a time that suits them. This is a great benefit that helps to: (1) manage costs; and, (2) enable experimentation of new ideas. It appeals from both a financial perspective and an engineering perspective. With this foundational capability in hand, an organisation can build on it to gain further benefits. For example, accelerating product development to gain a competitive advantage.

Environments in Traditional Data Centres

In a traditional data centre we would typically see a dev | test | prod | dr type approach to defining non-production (development and test) and production (prod and disaster recovery) environments. The infrastructure for these environments would be purchased at a high cost. Then it would often be written down, for example over a typical 3-5 year hardware refresh cycle. Guesses would be made to estimate capacity in advance of equipment purchase, and proof-of-concept work would typically occur just-in-time of purchase. Proof-of-concept in a hardware refresh cycle might trial and prove new application architectures at that time, perhaps not to be revisited until the next refresh.

Environments in AWS Cloud

Thank goodness we’re no longer confined to traditional data centres! With Amazon Web Services, you can create infrastructure and services without paying any upfront purchase costs. You pay for what you use, when you use it. What’s more (and even better), when you are finished you can destroy the infrastructure and services you provisioned and no further costs are incurred. (Note of course I’m not suggesting you destroy your production environments here, but highlighting the lifecycle capability of provisioning environments in cloud).

TRG Talk - Cloud - The Economics of Cloud Computing

Run a proof-of-concept whenever you want! Trial adoption of database-as-a-service like Amazon Relational Database Service (RDS) to reduce your database administration costs and improve service availablity! Introduce high-availability and self-healing compute infrastructure, with Amazon Elastic Load Balancing across Availability Zones and EC2 Auto Scaling!

Why Does It Matter?

Cloud providers such as Amazon Web Services have heralded changes that are nothing short of revolutionary. These changes contribute to the widely acknowledged current technological revolution – the Fourth Industrial Revolution. Globally we have seen the concept of cloud economics introduced to organisations and rapidly adopted. There’s now a more level playing field between smaller organisations and larger ones, which is accelerating innovation, disruptive ideas and products.

Underlying digital agility, innovation and productivity is IaC. Infrastructure-as-Code. IaC is a foundational capability of agile digital organisations. Using IaC you write the programming code to create your infrastructure and services. Once the code is written, the process is effectively automated.

Amazon Web Services provides CloudFormation and the Cloud Development Kit (CDK) for IaC.

Why use a human to do dumb, repetitive tasks? Automate them and boost your operational efficiency. Once you have your infrastructure code in hand, build a DevOps pipeline to manage the process of provisioning.

Foundational DevOps relies on IaC.

Amazon Virtual Private Cloud

Amazon Virtual Private Cloud (VPC) is an abstract network service that allows you to create a virtual network of your own. Back when first introduced in 2009, it was a revolutionary concept that enabled the creation of a network of your very own – without you needing to own any IT hardware.

At present time of writing a VPC enables you to create a network address space using any IPv4 address range, including RFC 1918 or publicly routable IP ranges. The network can be between 16 and 65,536 IPv4 addresses in size. IPv6 is also supported.

The architecture of AWS Global Infrastructure means that your VPC spans multiple Availability Zones. It spans all Availability Zones in the AWS Region. Unlike many technology infrastructure providers, every AWS Region has 3 or more Availability Zones (AZ). AZs are geographically separated locations within an AWS region, connected by redundant fast fibre-optic data links.

You can learn more about the AWS Global Network here: AWS re:Invent 2016: Amazon Global Network Overview with James Hamilton

Within your VPC, you define subnets in an Availability Zone. This means whilst your VPC spans all AZs, your subnets will not.

To manage and secure network traffic flow you use route tables. A VPC is created with a main route table. Each subnet you create must be associated with a custom route table or the main route table. The route table defines routing for your subnet, indicating how network data should flow.

To further secure your subnets, Network Access Control Lists (NACLs) can be defined. A NACL can be used to explicitly Allow or Deny network data to cross the boundary into or out of your subnet. Each subnet must be associated with a NACL – either the default NACL (provisioned when your VPC is first created) or a custom NACL.

One more security feature for capturing network traffic flows is VPC Flow Logs. This allows you to capture the traffic that flows to and from the network interfaces in your VPC or subnet.

There is much more to VPCs than this but these are the fundamentals. You can create an AWS account and create and destroy VPCs either through a management console or programmatically.

There is some further reading here exploring options to extend your data centres to include VPCs: AWS Whitepaper: Extend Your IT Infrastructure with Amazon Virtual Private Cloud

PrivateLink – It’s a Kind of Magic

AWS PrivateLink is an interesting way to create an endpoint by which you can provide services to other AWS accounts. You can do this without the need to run requests through the Internet and without peering or otherwise “connecting” VPCs. What is this particular type of AWS magic, I hear you say?

Says the Amazon web site, “AWS PrivateLink provides private connectivity between VPCs and services hosted on AWS or on-premises, securely on the Amazon network. By providing a private endpoint to access your services, AWS PrivateLink ensures your traffic is not exposed to the public internet. AWS PrivateLink makes it easy to connect services across different accounts and VPCs to significantly simplify your network architecture.”

So this is a particularly interesting AWS magic trick in that we can provide services to other consumer VPCs, through the Amazon backbone. We simply do two things to make his happen:

Create an Endpoint Service, in our serivce provider VPC
Create an Interface Endpoint (linked to our Endpoint Service), in our service consumer VPC

Where this gets really interesting is that it avoids all the unruliness of network address spaces and having to deal with Network Address Tranlsation (NAT). Routing works through a network interface to the endpoint service and you don’t have to worry about the network addresses. And if the endpoint service is unavailable in one Availability Zone, well that’s not a problem because your endpoint service will load balance across multiple Availability Zones.

Not to put too finer point on it, but to get the engineering and provisioning underlying all that without lifting a finger? That’s a kind of magic.