Websocket guide (wip)

Datomic Cloud
1

Here are some incomplete notes on setting up WebSockets with Datomic Cloud. Assuming version 884-9095 or higher. I’ll fill this and revise it as I have time.

At a high level, were going to create an API Gateway to handle the WebSocket Handshake, which is responsible for upgrading from the HTTP protocol to WebSockets. It also provides you with a connection-id that can be used to track the connections. Typically persisted in a database.

There are two supported ways API Gateway types that work to do this forwarding: HTTP and lambda.

HTTP

In the HTTP setup, you’re using Datomic’s http-direct feature, which means your can curl your HTTP handler as discussed here Ions Tutorial | Datomic.

  1. To do this you this Choose: Websocket API for your API Type
  2. Give it a name and just use the default route selection expression (will come back to this)
  3. Add the three predined routes, there likely all you will need, but you can come back to this later.
  4. For all routes, set Integration Type set to HTTP. Method POST for all of them because you will be forwarding information in the body. Set Integration URL to your IonApiGatewayEndpoint (see datomic docs but its found at CloudFormation > Stacks > outputs tab > IonApiGatewayEndpoint (don’t accidentally get the ClientAPI…).
  5. Set your stage name
  6. Create and deploy

Ok, so there is some more to do here.

If you select routes > connect you will see your Route setup. HTTP Proxy Integration basically just passes all the webapi information through to your handler. For reasons unknown to me, this option didn’t seem to work for me. So I did the following:

  1. Un select “use HTTP proxy integration”. click save You will know get a prompt. at the bottom called “Request Templates” These just select data API gateway creates so its passed to your handlers.
  2. pick a template selection expression like default. Then create a Template Key by the same name (so its selected). Here is an example template, you will want the connection id and event Type (MESSAGE, CONNECT, DISCONNECT) at a minimal (or maybe the routekey depending). You can see the full list of websocket options at the aws docs 
{"connection-id": "$context.connectionId",
  "event-type": "$context.eventType"}
  1. Click Save.
  2. Click Actions > deploy api, takes about 5 min. note the websocket protocol url your given.
  3. Install Wscat so you can test it from the command line
  4. Run Wscat -c <websocket-url> ;; (the c is for connect)
  5. Have a webhandler setup. My webhandler useful for debugging might look like this:
(defn http-handler [request]
  (cast/event {:msg "WebSocketRequestHTTP" ::request request :str (pr-str request) :body (slurp (:body request))})
  ...)

and your ion-config has a :http-direct key whose value is handler.

cast is datomic.ion.cast/event. You might be tempted to use cast/dev but that doesn’t post to CloudWatch.

pr-str is useful for inspection because the format (capitalization, snack-case->CamelCase) has changed. That and, the request body is a InputBuffer, so you can slurp it to inspect it. Your application logs are

  1. view your logs by visiting cloudwatch > log groups > datomic-
  2. There are lots of ways to query, but its sufficient (though probably slow) to just look for something in the cast event e.g “WebSocketRquest”. there is also a cli logging option I haven’t tried yet.

TODO: other routes e.g $default and $disconnect

Next up i’ll talk about the lambda handler.

Lambda

  1. Your ion config will need to reference the handler :lambdas {:some-name {:fn }}
  2. Your handler will be different because it receives input in a different way e.g

(defn lambda-handler [{:keys [input] :as request}]
  (let [{{connection-id :connectionId
          route-key     :routeKey} :requestContext} (json/read-str input :key-fn keyword)]
    (cast/event {:msg "WebSocketRequestLambda" ::request request ::input input ::connection-id connection-id})
    ...
    ))
  1. The setup at API gateway is similar steps
  2. choose integration type = lambda
  3. choose your - fn
  4. choose stage
  5. create
  6. I choose to use the proxy on my lambda handler, which works, otherwise you will have to setup the request Template as we did with HTTP. I assume the same template could be used.

Some questions I still have

  • As datomic cloud users is there a reason to prefer the HTTP vs Lambda option?
  • Is there a reason to prefer Proxy over non proxy?
  • For the HTTP Type, is content handling “passthrough” ideal or should I consider “convert to binary” or “convert to text”. The latter two seem to result in problems, e.g it seemed like my request didnt’ even make it to the app.
  • Why do i end up with a InputStream Buffer when i use HTTP but a json when i use lambda. Not a big deal either way, but it’s annoying not knowing why.
  • How would i do all this setup/ops in code or data?
2 Likes
2

Thanks, these instructions seem very useful!

I’ll answer the few questions that I think I can.

Http option will send requests directly to your app without you waiting for a Lambda to cold-start when it’s been scaled down (or you having to pay to provision it permanently).

And you won’t pay in money and in time for two steps before reaching your app (first: Gateway, second: Lambda, third: your app), you’ll pay just one intermediate step (first: Gateway, second: your app).

This seems like standard behavior for an HTTP endpoint, whereas Lambdas make more assumptions for you.

1 Like