Serverless Gardens

IoT + Serverless

twitter.com/@johncmckim

johncmckim.me

medium.com/@johncmckim

Software Engineer at A Cloud Guru

John McKim

@johncmckim

Contribute to Serverless Framework

https://acloud.guru

Serverless Framework

https://serverless.com

Agenda

• What is Serverless
• Why I built this project
• Overall Architecture
• Design of each Microservice
• GraphQL + Lambda
• What I learnt
• Questions

What is Serverless?

Serverless

FaaS + The Herd

What is Serverless?

A Serverless Architecture is an event driven system that utilises FaaS  and other fully managed services for logic and persistence.

Why choose Serverless?

Benefits

• Easier Operations Mangement
• Reduced Operational Cost
• Reduced Development Time / Cost
• Highly Scalable
• Loosely Coupled systems

Why build this?

For fun and learning

The Problem

Caring for my Garden

Serverless Garden

IoT Service

AWS IoT Service

How It works

Device Gatway

Protocols

• MQTT  - devices
• MQTT over Web Sockets  - browsers
• HTTP  - last resort

Device Gatway

Authentication

• X.509 Certificates - Mutual TLS
• IAM - Signed Requests
• Cognito - tokens

Device

Fake Device

                            const awsIot = require('aws-iot-device-sdk');

const device = awsIot.device({
  'keyPath': './certificates/private.pem.key',
  'certPath': './certificates/certificate.pem.crt',
  'caPath': './certificates/verisign-ca.pem',
  'clientId': 'garden-aid-client-test-js',
  'region': 'ap-southeast-2'
});

device
  .on('connect', function() {
    const topic = 'garden/soil/moisture';
    const message = JSON.stringify({
      DeviceId: 'test-js-device',
      Recorded: (new Date()).toISOString(),
      Level: level
    });

    device.publish(topic, message, {});
  });

Demo

Fake Device

Rules Engine

Message Selection & Transformation

Rules Engine

Actions

• Lambda

• DynamoDB

• ElasticSearch

• SNS

• SQS

• Kinesis

• CloudWatch

• Republish to another MQTT topic.

Rules Engine

IoT Rule in serverless.yml

                            SensorThingRule:
  Type: AWS::IoT::TopicRule
  Properties:
    TopicRulePayload:
      RuleDisabled: false
      Sql: "SELECT DeviceId, Recorded, Level FROM '$/garden/soil/moisture'"
      Actions:
        -
          DynamoDB:
            TableName: { Ref: MoistureData }
            HashKeyField: "ClientId"
            HashKeyValue: "${clientId()}"
            RangeKeyField: "Timestamp"
            RangeKeyValue: "${timestamp()}"
            PayloadField: "Data"
            RoleArn: { Fn::GetAtt: [ IotThingRole, Arn ] }
        -
          Lambda:
            FunctionArn: { Fn::GetAtt: [ checkMoistureLevel, Arn ] }

Notifications Service

• Single purpose functions
• High cohesion
• Loose coupling

Messaging Options

Amazon Simple Queue Service (SQS)

Fully Managed message queuing service.

Messaging Options

Amazon Kinesis Streams

Capture and store streaming data.

Messaging Options

Amazon Simple Notification Service (SNS) 

Full managed messaging and Pub/Sub service

Notification Service

Check Level

                            const AWS = require('aws-sdk');
const sns = new AWS.SNS();

const publish = (msg, topicArn, cb) => {
  sns.publish({
    Message: JSON.stringify({
      message: msg
    }),
    TopicArn: topicArn
  }, cb);
};

module.exports.checkLevel = (event, context, cb) => {
  if(event.Level < 2.5) {
    const msg = 'Moisture level has dropped to ' + event.Level;

    const topicArn = process.env.mositureNotifyTopic;

    publish(msg, topicArn, cb);
    cb(null, { message: msg, event: event });
    return;
  }

  cb(null, { message: 'No message to publish', event: event });
}

Notifications Service

Slack Notifier

                            const BbPromise = require('bluebird');
const rp = require('request-promise');
const util = require('util');

const notify = (msg) => {
  return rp({
    method: 'POST',
    uri: process.env.slackWebHookUrl,
    json: true,
    body: {
      text: msg,
    },
  });
}

module.exports.notify = (event, context, cb) =>  {
  console.log(util.inspect(event, false, 5));

  const promises = [];

  event.Records.forEach(function(record) {
    if(record.EventSource !== 'aws:sns') {
      console.warn('Recieved non sns event: ', record);
      return;
    }

    const notification = JSON.parse(record.Sns.Message);
    promises.push(notify(notification.message));
  });

  return BbPromise.all(promises)
          .then(() => cb(null, { message: 'success' }))
          .catch(cb);
};

Demo

Slack Notifications

Web Services

Web Services

API Gateway

What is it?

• HTTP Endpoint as a Service
• Integrates with Lambda
• Convert HTTP Request to Event
• Can delegate Authorization

Web Services

Auth0 Authentication

Web Services

Authentication with GraphQL

                            const networkInterface = createNetworkInterface(GRAPHQL_URL);

networkInterface.use([{

  applyMiddleware(req, next) {
    if (!req.options.headers) {
      req.options.headers = {}; // Create the header object if needed.
    }

    // get the authentication token from local storage if it exists
    const idToken = localStorage.getItem('idToken') || null;
    if (idToken) {
      req.options.headers.Authorization = `Bearer ${idToken}`;
    }
    next();
  },
}]);

Web Services

Custom Authorizer

                            const utils = require('./auth/utils');
const auth0 = require('./auth/auth0');
const AuthenticationClient = require('auth0').AuthenticationClient;

const authClient = new AuthenticationClient({
  domain: process.env.AUTH0_DOMAIN,
  clientId: process.env.AUTH0_CLIENT_ID,
});

module.exports.handler = (event, context, cb) => {
  console.log('Received event', event);

  const token = utils.getToken(event.authorizationToken);

  if (!token) {
    return cb('Missing token from event');
  }

  const authInfo = utils.getAuthInfo(event.methodArn);

  return authClient.tokens.getInfo(token)
    .then((userInfo) => {
      if (!userInfo || !userInfo.user_id) {
        throw new Error('No user_id returned from Auth0');
      }

      console.log(`Building policy for ${userInfo.user_id} with: `, authInfo);

      const policy = new AuthPolicy(userInfo.user_id, authInfo.accountId, authInfo);

      policy.allowMethod(AuthPolicy.HttpVerb.POST, '/graphql');

      const result = policy.build();
      console.log('Returning auth result: ', result, result.policyDocument.Statement);

      return result;
    })
    .catch((err) => {
      console.log(err);
      return 'Unauthorized';
    });
};

Demo

Dashboard

What is GraphQL?

Why GraphQL?

• One endpoint (per service) to access your data
• The client chooses the response format
• No versioning *

                            import gql from 'graphql-tag';
import { connect } from 'react-apollo';
import MoistureChart from '../../pres/Moisture/Chart';

export default connect({
  mapQueriesToProps({ ownProps, state }) {
    return {
      moisture: {
        query: gql`{
          moisture(hours: ${ownProps.hours}, clientId: "${ownProps.clientId}") {
            date, moisture
          }
        }`,
        variables: {},
        pollInterval: 1000 * 30, // 30 seconds
      },
    };
  },
})(MoistureChart);

GraphQL Query

GraphQL Schema

                            const graphql = require('graphql');
const tablesFactory = require('./dynamodb/tables');
const MoistureService    = require('./services/moisture');

const tables = tablesFactory();
const moistureService = MoistureService({ moistureTable: tables.Moisture });

const MoistureType = new graphql.GraphQLObjectType({
  name: 'MoistureType',
  fields: {
    date: { type: graphql.GraphQLString },
    moisture: { type: graphql.GraphQLFloat },
  }
});

const schema = new graphql.GraphQLSchema({
  query: new graphql.GraphQLObjectType({
    name: 'Root',
    description: 'Root of the Schema',
    fields: {
      moisture:
        name: 'MoistureQuery',
        description: 'Retrieve moisture levels',
        type: new graphql.GraphQLList(MoistureType),
        args: {
          clientId: {
            type: graphql.GraphQLString,
          },
          hours: {
            type: graphql.GraphQLInt,
            defaultValue: 1
          },
        },
        resolve: (source, args, root, ast) => {
          const hours = args.hours > 0 ? args.hours : 1;
          return moistureService.getLastHours(args.clientId, hours);
        }
      }
    }
  })
});

module.exports = schema;

AWS Lambda

                            const graphql = require('graphql');

const schema = require('./schema');

module.exports.handler = function(event, context, cb) {
  console.log('Received event', event);

  const query = event.body.query;

  return graphql.query(schema, event.body.query)
    .then((response) => {
      cb(null, response)
    })
    .catch((error) => {
      cb(error)
    });
}

Demo

GraphQL Query

GraphQL on AWS Lambda

Single Lambda Design

GraphQL on AWS Lambda

Lambda Tree Design

Summary

Serverless + IoT

My Experiences

• No server operations
• Cost - $0
• Use *aaS services
• Focus on developing functionality
• Iterate quickly & scale

Alternative Options

IoT Service

Device Shadows

• Stores Device State
• Get current state
• Track state

Alternative Options

Notifications Service

• Monolithic Notification Lambda
• Other notification services
  • Facebook Messenger
  • Sms - Twillio, Nexmo

Alternative Options

Web Services

• Front-end Framework
  • Angular
  • Vue
• Elastic Search instead of DynamoDB 
• Web Service own Data Store

What did I learn?

• Know your services well
• Know what services exist
• Selecting Boundaries is hard
• Automation is always worth it
• GraphQL is awesome

Many things

Resources

Code + Reading

• github.com/garden-aid
• serverless.zone

Frameworks & Tools

• serverless.com
• AWS
• Firebase
• Auth0

Thanks for Listening!

Questions?

twitter.com/@johncmckim

johncmckim.me

medium.com/@johncmckim