Cloud Computing for busy people
The number of cloud services is growing constantly and I dare say trying to keep count of all of them will be a futile exercise. Yet, often times we expect solutions architect or engineer to know and understand them all. Certainly, this can be done, but it takes good amount of time and practice. You have your day to day tasks to do, and might find it very difficult to allocate time to learn them all. So you might ask, what are the top 3 things I need to know to do my job right?
In this article, we will explore where you should be spending your focus on, ones that gives you the best return of investment of your precious time.
Before we go too far, please keep in mind that it is also virtually impossible to pick the top 3 that fits all use cases. In reality, how one use a Cloud Computing platform really depends on one’s roles.
A software developer would have different priorities to an infrastructure engineer, a QA engineer would have different priorities to an applications architect, a DevOps engineer would have different priorities to a data scientist, and so on… you get the picture.
This article looks at the top 3 from a broad, generalist perspective. Before looking at what these are, we need to understand what factors were taken into consideration to arrive at these choices. The context is as follows: although not always, in many cases people and/or organisations leverage the Cloud Computing platform to run applications that support their business. They want to be able to do this quickly, cost-effectively and be able to scale. They don’t want to have to deal with either the upfront cost and ongoing maintenance of procuring IT infrastructure such as servers, data-centres, networking appliances and so on. In other words, they need a place to run code but without the extra maintenance. This will need the following as a bare minimum, and along with them are the basic building block of any Cloud-native solutions:
- Compute: AWS EC2 (Elastic Compute Cloud) and Azure VMs (Virtual Machines)
- Object-level Storage: AWS S3 (Simple Storage Service) and Azure Blob Storage
- Security: AWS IAM Roles and Azure RBAC
Again, the services above are by no means solution to all problems. Depending on the nature of the problem and use cases, the solution architecture that solves it will be very different. However, if you want to start designing solutions that runs in the Cloud, it would be safe to say that inevitably you will come across the 3 above. So let’s take a look at what these are at a very high level, and the key concepts you need to get a firm grasp on for each of these.
AWS EC2 (Elastic Compute Cloud) and Azure VMs (Virtual Machines)
In the compute PaaS (Platform as a Service) space, I think Microsoft nailed the naming on this one. The fundamental compute service in Azure is called Azure VM (Virtual Machine). On AWS it is called the EC2, which stands for Elastic Compute Cloud. Basically, these are services that allow you to run Virtual Machines on Microsoft and Amazon data centres around the globe.
With AWS, these data centres are grouped into regions, within regions we have availability zones. Azure has geography, region and availability zones. They also expand although not very rapidly. For AWS, check this link out and this one for Azure.
When you want to launch a VM instance, you can choose from a predefined selection of machine specifications (think combination of CPU types, CPU cores, amount of memory, storage volume type, size, etc.), then you choose your operating system (typically Linux or Windows, but AWS recently announced the availability of EC2 MacOS instances), network settings, and finally launch the instance.
See below for a general idea of some of the available configurations made available when creating a VM in Azure. The configurations are not too different in AWS EC2, although there are a few different concepts when you go deeper into the details.
Once the instance is running, you can then connect to it (typically via SSH for Linux instances or RDP for Windows instances) and operate it just like how you would operate a computer on-premise. You can place your application or code and its runtime on these VMs. One thing to note here is that because these are VMs, remember that unlike containers, they are deployed on top of hypervisors. Although recently, AWS made an interesting move by introducing the Nitro system, that uses Amazon proprietary Nitro chips, essentially offloading the hypervisor to dedicated hardware.
Also while we are here talking about compute, it would be wise to run through the basics of serverless compute: AWS Lambda and Azure Functions. These are your FaaS (Function as a service), basically all you need to do is write some code and not have to worry about the underlying infrastructure, scale-out, scale-in as these are all managed for you.
Amazon S3 (Simple Storage Service) and Azure Blob Storage
Next: object storage. Unlike file storage or block storage, object storage manages data as objects. Think of it as a container that allows you to store a piece of BLOB (binary large object) and some metadata. You can have virtually unlimited number of blobs, but you have a limit of how big a blob can be. You typically pay for the amount of data you store and data transfers.
Official documentation by Microsoft mentions a few sample use cases for object level storage, such as the following:
- Serving images or documents directly to a browser.
- Storing files for distributed access.
- Streaming video and audio.
- Writing to log files.
- Storing data for backup and restore, disaster recovery, and archiving.
- Storing data for analysis by an on-premises or Cloud-hosted service.
Now note this, once you start using more Cloud services, S3 and Blob storage are fundamentally indispensable as the Platform itself leverages a lot of it. One of the many examples of this on AWS side is when you take a snapshot of your EBS volume which typically is attached to EC2 instance, AWS stores the EBS snapshot in S3. Therefore it is crucial to understand this service early on.
See below for a general idea of how it looks like on Amazon S3. On S3, storage is organized into S3 buckets. We can store anything in an S3 bucket, like seen below, where I have uploaded a csv file, a text file and xml file. S3 supports folders as well (which are not real folders by the way, they are simply there for the sake of organizational simplicity).
Personally, I find S3 a bit simpler to work with than Azure blob storage to start with but they are equivalently good and capable. With AWS, I simply had to go to my AWS account > S3 and create a bucket, then I can put blobs in my bucket. With Azure, the terminologies were just a bit confusing at first as there are quite a few nesting. With Azure, you’d have to create storage account within a resource group, then create container in the blob containers. However, I do think the resource group feature of Azure comes really handy. More on that next time.
AWS IAM Roles and Azure RBAC
I was tempted to put this one as the #1 of the list, but it can be quite an abstract concept to get a grasp on, so it’s probably more logical to cover this one as the third item on the top 3, because if you have experimented a bit with compute and object storage, it means you now have some resources in the Cloud. The natural next question is: how will you secure these resources? You wouldn’t want to open everything up to the entire world.
Therefore, one should not overlook the importance of security in the Cloud. Principle of least privilege should be applied everywhere, including in the Cloud and security in the Cloud is shared responsibility of the customer and Cloud provider. Luckily, both AWS and Microsoft already helped us to some extent in this regard. By default, nobody can access any resource on your AWS or Azure account, unless you specifically grant permission to it.
This is important to understand, because if you have your application or code running in AWS EC2 or Azure VM and it needs to access other Cloud resources like read an object in S3 or Blob storage, it won’t be able to do so, unless you grant the right permission to it, well that is unless if you make your object public.
The wrong way to do this is to grant permission to a user account or a service account more than what the account really needs, even worse if the security credential is hardcoded somewhere in the code or config, or access is granted permanently.
Essentially, you will want to grant specific security principal (be it a person or a machine) access to only the specific resource (e.g. an object store container or bucket) and be specific on what the security principal can do (e.g. read-only, or read & write but not delete).
Enter Role Based Access Control (RBAC). AWS and Azure handles this a bit differently and the terms aren’t exactly equivalent or 100% transferable, but understand this: Role is the one that bridges ones who needs access to the actual resource that needs to be accessed.
The AWS definition of IAM Role is as follows:
A role is intended to be assumable by anyone who needs it. Also, a role does not have standard long-term credentials such as a password or access keys associated with it. Instead, when you assume a role, it provides you with temporary security credentials for your role session. You can use roles to delegate access to users, applications, or services that don’t normally have access to your AWS resources.
For the busy people, check this 43 second video by Microsoft Azure about RBAC:
I also consider he following documentations must-read:
- What is Azure RBAC: https://docs.microsoft.com/en-us/azure/role-based-access-control/overview
- AWS IAM Roles terms and concepts: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html
