The changes 2 weeks can bring … AWS July 2015 service updates

Since my AWS Architecture Overview post a couple weeks ago there have been some significant new capabilities introduced on July 9th at the AWS Summit in New York.  Here is an update to the architecture overview diagram which highlights those changes.

AWS Arch Overview July 2015 tlFigure – AWS Architecture Overview with Development and Application Hosting

This version of the diagram highlights Development capabilities and application hosting as well as catalogs in the white areas.  Labels in red depict services released this week.

A lot of the announcements enrich the DevOps space and make application development and management much simpler.  To expand the CodeDeploy distribution mechanism previously available, AWS has added CodeCommit to standardize the management of GIT version control libraries as well as CodePipeline to assist in the automation of end to end release management workflows.

To further drive componentization of services and modular programming practices, AWS announced the API Gateway feature which makes it much easier to define, instantiate, scale and manage web services interfaces to your application components.

The Service Catalog feature went into production so that you can define and host your own catalog of services for users in your enterprise.  You can present exactly the subset of features you want, grouped and labelled for your industry and user roles.  By giving you control over which services are presented to which entitled users, you can govern the types of services your users have available to pick from.

There was some discussion of the Elastic File System (EFS) feature which is still operating as a Preview.  This service offers a distributed and dynamically scaleable file system to support the many kinds of applications that require this infrastructure capability.

Great to see the rich platform AWS offers continuing to expand and improve.  Looking forward to helping my clients to exploit some of these features in the coming months.

AWS Cloud Architecture Overview

On this blog I will eventually touch on a number of different cloud environments, but my current focus and the focus of many of my clients is certainly on Amazon Web Services (AWS).  As such, AWS will be the primary proof point and foundation for many of the cloud concepts demonstrated here.  The following architecture overview diagram depicts the main services available in AWS in a layered fashion.

AWS Arch Overview Analytics tl

Figure – AWS Architecture Overview

If you start at the bottom and build up towards the top, each box or icon is a critical service and concept in AWS certainly up through the Foundation Services layer.  In the Application Services layer you can get a sense of the diverse set of platforms and technologies you can easily deploy and manage with AWS.

The concepts of Region, Availability Zone and Edge Locations are critical to the design of distributed, resilient applications.  See the AWS Global Infrastructure page to get a sense of where these services are available.

Over the next couple months, I’ll share concepts and best practices related to each of these components.  In the meantime the AWS website and Document Library have great documentation on every one of these services.

Start with the basics under

and you’ll be on your way.  I’ll link in more blog entries here as they become available.

What is this ‘Cloud’ you speak of and why do I need one?

Cloud computing, according to Wikipedia ‘refers to the practice of transitioning computer services such as computation or data storage to multiple redundant offsite locations available on the Internet, which allows application software to be operated using internet-enabled devices. Clouds can be classified as public, private, and hybrid.’

That is a fine starting point.  Cloud computing is simply the ‘As a Service’ (XaaS) approach to providing compute, storage, networking and all sorts of application and services over public and private networks.  The cheapest, easiest version to access and utilize is the Public Cloud where shared datacenters and servers deliver the capability, primarily over the public Internet.

In many ways, cloud computing is no different than basic web hosting approaches Managed Service Providers (MSPs) have been delivering since the nineties.  When MSPs are leveraging cloud technologies which virtually all now are, they can be referred to as Cloud Service Providers (CSPs).  CSPs may deliver many levels of service from the infrastructure level (IaaS) up to managed platforms as a service (PaaS) for things such as relational databases (RDS in AWS) and finally true Software as a Service (SaaS) where you can purchase whole software suites and functions as a service as in, SAP and Oracle cloud services.

On the other hand, everything is different.  The level of automation, virtualization and speed of provisioning is like legacy infrastructure on steroids.  The way you procure and manage it is totally different and much more self-service.  To truly  benefit from the cost savings aspects of cloud, you need to fundamentally change how and when you spin up and shutdown environments, tieing those events much more closely to demand and consumption rather than following the ‘if we build it they will come’ deployment model of days gone by.  There is no excuse any more for leaving idle instances running for ever wasting space, electricity and money.

Automation and speed of provisioning lead to a related concept you are going to hear a lot about.  Elasticity.  A lot of the offering or capability names in AWS have elastic in the name (elastic compute or EC2, elastic block storage or EBS, elastic load balancers or ELBs,…) because it is an important property.  It means you can scale capacity up and down depending on what you use, and only pay for that amount.  This is where the cloud gets radically different and beneficial.

Most of these concepts of automation, virtualization and elasticity can be built in private, dedicated, onsite data centers.  That is the concept of a Private Cloud.

Most enterprises today are using heterogeneous combinations of onsite infrastructure combined with multiple offsite shared clouds at the same time (AWS, Office365, ServiceNow as three very common examples). Integrating all these disparate capabilities into a seamless cloud infrastructure is where the concept of Hybrid Cloud computing becomes important.  The critical thing about private cloud is to have modular, automated deployment practices that allow you to quickly and easily target multiple cloud hosting environments with common hypervisors or virtualization engines such as Xen and VMWare.  To provide a common interface and management layer whole consortiums have evolved such as OpenStack and tools have become common for developing and deploying via scripted cookbooks such as Chef.  With modular, automated workloads, and a plethora of public clouds becoming available, your deployment options become as limitless as your imagination.

All this freedom is not easy or free of course.  The complexity of securing and managing hybrid cloud requires some engineering discipline and IT knowledge.  That is part of what I hope to share through this blog.

So that is what cloud is.  To answer the final question in the title.  Do you need one?

If you need a compute environment that has the following characteristics, you need one.  Would you like to:

  1. Trade capital expense for variable expense – Only pay when you consume computing resources, and only pay for how much you consume.
  2. Have fewer up front investments – Benefit from massive economies of scale.  Thousands of cloud customers leveraging cloud services such as AWS translates into lower cost per unit than dedicated data centers.
  3. Stop guessing capacity – Avoid expensive idle resources and unacceptable capacity limitations. Access as much or as little as you need, and scale up and down as required with very little notice
  4. Increase speed and agility – Cloud provisioning makes resources available to your developers in minutes instead of weeks.
  5. Dramatically decrease the cost and time it takes to experiment and develop
  6. Stop spending money on running and maintaining data centers
  7. Focus on projects that differentiate your business and on your own customers –   Leave the heavy lifting of racking, stacking and powering servers to the service provider
    Go global in minutes – Regional points of presence near your users means you can provide lower latency and a better user experience without the need to build out your own global infrastructure

If that sounds like an AWS sales pitch, that is ok, it basically is.  The benefits of cloud are very real though for any business and application owner/developer willing to make the leap and adapt to cloud deployment and management practices.

What are you waiting for?  Dive in!

Cloud Building Blocks

Lego BlocksAs an IT Architect, the way I understand complex topics is by breaking them down into simpler ones.  Think of the common Lego building block example.  Each IT function needed to build an IT solution is a building block.  Some architecture methods actually refer to Architecture and Solution Building Blocks.  The ways that you combine and layer those building blocks can be captured and depicted through Reference Architectures and Patterns.

Here is a picture depicting some common building blocks utilized as part of cloud computing.

Cloud building blocks - generalThe left side of the diagram focuses on technologies used in the building and running of applications.

The right side of the diagram focuses on technologies used in the deployment and operation of infrastructure and applications.

DevOps is an interesting area of IT focused on the integration of Application Development and Infrastructure Operations through common process and tooling to achieve continuous and agile deployment of solutions.  We’ll explore more on how to achieve this in upcoming posts.

In future posts we’ll explore technologies behind some of these building blocks and how to combine them through patterns and use cases into useful solutions.  Because there are many platforms and cloud hosting environments available for use today, I’ll show specific examples mapped to actual technology and providers.  For example Amazon Web Services as a common and leading edge cloud service provider.

Cloud builidng blocks - AWSLet’s break out those building blocks and get building!