Serverless Architectures on AWS
With examples using AWS Lambda
Peter Sbarski
  • MEAP began April 2016
  • Publication in Summer 2017 (estimated)
  • ISBN 9781617293825
  • 425 pages (estimated)
  • printed in black & white

There's a shift underway toward serverless cloud architectures. With the release of serverless compute technologies, such as AWS Lambda, developers are now building entirely serverless platforms at scale. In these new architectures, traditional back-end servers are replaced with cloud functions acting as discrete single-purpose services. By composing and combining these serverless cloud functions together in a loose orchestration, and adopting useful third-party services, you can build powerful yet easy to understand applications. Serverless architecture is about building rich, scalable, high-performing, and cost-effective systems without having to worry about traditional compute infrastructure, having more time to focus on code, and moving quickly.

Serverless Architectures on AWS teaches you how to build, secure and manage serverless architectures that can power the most demanding web and mobile apps. You'll get going quickly with this book's ready-made and real-world examples, code snippets, diagrams, and descriptions of architectures that can be readily applied. This book describes a traditional application and its back end concerns and then shows how to solve these same problems with a serverless approach. You'll begin with a high-level overview of what serverless is all about, start creating your own media transcoding system, and learn more about AWS. Next, you'll go in depth and learn about Lambda, API Gateway and other important serverless technologies. This section will teach you how to compose Lambda functions and discuss important considerations when it comes to building serverless systems. The third part of the book focuses on more advanced topics as your architecture grows. By the end, you'll be able to reason about serverless systems and be able to compose your own systems by applying these ideas and examples.

Table of Contents detailed table of contents

First Steps

1. Going serverless

1.1. How we got to where we are

1.1.1. Service-oriented architecture & microservices

1.1.2. Software design

1.2. Principles of serverless architectures

1.2.1. Use a compute service to execute code on demand

1.2.2. Write single-purpose stateless functions

1.2.3. Design push-based, event-driven pipelines

1.2.4. Create thicker, more powerful front ends

1.2.5. Embrace third-party services

1.3. Transitioning from a server to services

1.4. Serverless pros and cons

1.4.1. Decision drivers

1.4.2. When to use serverless

1.5. Summary

2. Architectures and patterns

2.1. Use cases

2.1.1. Application back end

2.1.2. Data processing and manipulation

2.1.3. Real-time analytics

2.1.4. Legacy API wrapper

2.1.5. Scheduled services

2.1.6. Bots and skills

2.2. Architectures

2.2.1. Compute as back end

2.2.2. Legacy API wrapper

2.2.3. Hybrid

2.2.4. GraphQL

2.2.5. Compute as glue

2.2.6. Real-time processing

2.3. Patterns

2.3.1. Command pattern

2.3.2. Messaging pattern

2.3.3. Priority queue pattern

2.3.4. Fan-out pattern

2.3.5. Pipes and filters pattern

2.4. Summary

3. Building a serverless application

3.1. 24-Hour Video

3.1.1. General requirements

3.1.2. Amazon Web Services

3.1.3. Creating your first Lambda function

3.1.4. Naming Lambda

3.1.5. Testing locally

3.1.6. Deploying to AWS

3.1.7. Connecting S3 to Lambda

3.1.8. Testing in AWS

3.1.9. Looking at logs

3.2. Configuring Simple Notification Service

3.2.1. Connecting SNS to S3

3.2.2. Getting email from SNS

3.2.3. Testing SNS

3.3. Setting video permissions

3.3.1. Creating the second function

3.3.2. Configuring and securing

3.3.3. Testing the second function

3.4. Generating metadata

3.4.1. Creating the third function and FFprobe

3.5. Finishing touches

3.6. Exercises

3.7. Summary

4. Setting up your cloud

4.1. Security model and identity management

4.1.1. Creating and managing IAM users

4.1.2. Creating groups

4.1.3. Creating roles

4.1.4. Resources

4.1.5. Permissions and policies

4.2. Logging and alerting

4.2.1. Setting up logging

4.2.2. Log retention

4.2.3. Filters, metrics, and alarms

4.2.4. Searching log data

4.2.5. S3 and logging

4.2.6. More on alarms

4.2.7. CloudTrail

4.3. Costs

4.3.1. Creating billing alerts

4.3.2. Monitoring and optimizing costs

4.3.3. Using the Simple Monthly Calculator

4.3.4. Calculating Lambda and API Gateway Costs

4.4. Exercises

4.5. Summary

Part 2: Core ideas

5. Authentication and authorization

5.1. Authentication in a serverless environment

5.1.1. A serverless approach

5.1.2. Amazon Cognito

5.1.3. Auth0

5.2. Adding authentication to 24-Hour Video

5.2.1. The plan

5.2.2. Invoking Lambda directly

5.2.3. 24-Hour Video website

5.2.4. Auth0 configuration

5.2.5. Adding Auth0 to the website

5.2.6. Testing Auth0 integration

5.3. Integration with AWS

5.3.1. User Profile Lambda

5.3.2. API Gateway

5.3.3. Mappings

5.3.4. Invoking Lambda via API Gateway

5.3.5. Custom Authorizer

5.4. Delegation Tokens

5.4.1. Real-world examples

5.4.2. Provisioning delegation tokens

5.5. Exercises

5.6. Summary

6. Lambda the orchestrator

6.1. Inside Lambda

6.1.1. Event models and sources

6.1.2. Push and pull event models

6.1.3. Concurrent executions

6.1.4. Container reuse

6.1.5. Cold and warm Lambda

6.2. Programming model

6.2.1. Function handler

6.2.2. Event object

6.2.3. Context object

6.2.4. Callback function

6.2.5. Logging

6.3. Versioning, aliases, and environment variables

6.3.1. Versioning

6.3.2. Aliases

6.3.3. Environment Variables

6.4. Using the CLI

6.4.1. Invoking commands

6.4.2. Creating and deploying functions

6.5. Lambda patterns

6.5.1. Async waterfall

6.5.2. Series and parallel

6.5.3. Using libraries

6.5.4. Move logic to another file

6.6. Testing Lambda functions

6.6.1. Testing locally

6.6.2. Writing tests

6.6.3. Testing in AWS

6.7. Exercises

6.8. Summary

7. API Gateway

7.1. API Gateway as the interface

7.1.1. Integration with AWS services

7.1.2. Caching, throttling, and logging

7.1.3. Staging and versioning

7.1.4. Scripting

7.2. Working with the API Gateway

7.2.1. The plan

7.2.2. Creating the resource and method

7.2.3. Configuring method execution

7.2.4. The Lambda function

7.2.5. Updating the website

7.3. Optimizing the Gateway

7.3.1. Throttling

7.3.2. Logging

7.3.3. Caching

7.4. Stages and versions

7.4.1. Creating a stage variable

7.4.2. Using stage variables

7.4.3. Versions

7.5. Exercises

7.6. Summary

Part 3: Growing your architecture

8. Storage

8.1. Smarter Storage

8.1.1. Versioning

8.1.2. Hosting a static website

8.1.3. Storage classes

8.1.4. Object lifecycle management

8.1.5. Transfer acceleration

8.1.6. Requestor Pays buckets

8.1.7. Event notifications

8.2. Secure Upload

8.2.1. Architecture

8.2.2. Upload Policy Lambda

8.2.3. S3 CORS Configuration

8.2.4. Uploading from the website

8.3. Restricting access to files

8.3.1. Removing public access

8.3.2. Generating pre-signed URLs

8.4. Exercises

8.5. Summary

9. Database

9.1. Introduction to Firebase

9.1.1. Data Structure

9.1.2. Security Rules

9.2. Adding Firebase to 24-Hour Video

9.2.1. Architecture

9.2.2. Setting up Firebase

9.2.3. Modifying Transcode Video Lambda

9.2.4. Transcode Video Firebase Update

9.2.5. Connecting Lambda

9.2.6. Website

9.2.7. End-to-end testing

9.3. Securing Access to Files

9.3.1. Signed URL Lambda

9.3.2. API Gateway Settings

9.3.3. Updating the Website again

9.3.4. Improving performance

9.3.5. Improving Firebase Security

9.4. Exercises

9.5. Summary

10. Going the last mile

10.1. Deployment and Frameworks

10.2. Toward better microservices

10.3. Step Functions

10.3.1. Image Processing Example

10.4. AWS Marketplace

10.5. Where from here


Appendix A: Services for your serverless architecture

A.1. API Gateway

A.2. Simple Notification Service

A.3. Simple Storage Service

A.4. Simple Queue Service

A.5. Simple Email Service

A.6. Relational Database Service and DynamoDB

A.7. CloudSearch

A.8. Elastic Transcoder

A.9. Kinesis Streams

A.10. Cognito

A.11. Auth0

A.12. Firebase

A.13. Other services

Appendix B: Installation and setup

B.1. Preparing your system

B.2. Setting up an IAM user and CLI

B.3. Setting user permissions

B.4. Making new S3 buckets

B.5. Creating an IAM role

B.6. Preparing for Lambda

B.7. Configuring Elastic Transcoder

B.8. Setting up npm

Appendix C: Authentication and authorization

C.1. Basics of authentication and authorization

C.2. JSON Web Token

Appendix D: Lambda insider

D.1. Execution environment

D.2. Limitations

D.3. Working with older runtimes

D.3.1. Succeed

D.3.2. Fail

D.3.3. Done

Appendix E: Models and mapping

E.1. Get Video List

E.1.1. GET Method

E.1.2. Handling errors

E.1.3. Deploying API Gateway

Appendix F: S3 event message structure

F.1. S3 structure

Appendix G: Serverless Framework and SAM

G.1. Serverless Framework

G.1.1. Installation

G.1.2. Beginning Serverless Framework

G.1.3. Using Serverless Framework

G.1.4. Packaging

G.1.5. Testing

G.1.6. Plugins

G.1.7. Examples

G.2. Serverless Application Model

====Getting started ==== Example with SAM === Summary

What's inside

  • Creating a serverless back end
  • Using Lambda and the API Gateway
  • Connecting multiple services
  • Authorization and authentication in a serverless environment
  • Securely communicating with third-party services
  • Interacting with a database from the front end
  • Setting up continuous integration and deployment
  • Building high-performance systems using messaging and eventing
  • Using AWS to your advantage

About the reader

This book is for all software developers interested in back end technologies. Experience with JavaScript (node.js) and AWS is useful but not required.

About the author

Dr. Peter Sbarski is CTO at Ephemeral Pty Ltd and VP of Engineering at A Cloud Guru. He's had an extensive career working with prominent Australian IT companies and has led teams across large enterprise solutions, with a focus on web and AWS cloud technologies.

Manning Early Access Program (MEAP) Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
MEAP combo $44.99 pBook + eBook
MEAP eBook $39.99 pdf + ePub + kindle

FREE domestic shipping on three or more pBooks