`:endpoint` for `d/client` on Datomic Cloud 884-9095

Datomic Cloud
1

Do I understand well, that the current recommendation is to always use the randomly assigned ClientApiGatewayEndpoint as the value of the :endpoint, even when the code is running on the compute nodes?

Former ion-starter example was using a stable, internal URL as the :endpoint value: http://entry.mbrainz-stu.us-east-1.datomic.net:8182, which was only resolvable within the Datomic VPC.

As I see, that is still available and still works.

Does AWS do any network access optimization, when accessing the ClientApiGatewayEndpoint, as opposed to the DispatchAddress?

The Datomic Architecture | Datomic diagram suggests that the ion application should use the fixed, internal DNS name for the ALB.

The latest documentation at Ions Reference | Datomic shows a public address as the :endpoint value:

(require '[datomic.client.api :as d])

(def cfg {:server-type :ion
          :region "us-east-1"
          :system "inventory-dev"
          :endpoint "https://ljfrt3pr18.execute-api.us-east-1.amazonaws.com"})

(def client (d/client cfg))

The ion-starter project has a similar config:

{:server-type :ion
 :region "us-east-1"
 :system "mbrainz-stu"
 ;ClientApiGatewayEndpoint output from cloudformation stack
 :endpoint "https://a2id5latdd.execute-api.us-east-1.amazonaws.com/"}

These 2 examples suggests that we are supposed to use this randomly created ClientApiGatewayEndpoint, when we want to connect to our Datomic Cloud System (DCS further on), regardless of accessing it from the Datomic EC2 instances or through the internet.

This however, assumes that we provision our DCS first, then we either include this randomly named endpoint into the artifact, we `clojure -M:ion-dev ‘{:op :deploy}’, or push it somewhere as a configuration for the ion app to pick up during its bootup time.

If that’s the recommended approach, I would like to have some guidance on how to determine the :endpoint address, if it depends on the development stage?

2

For now, we have attached custom domains to the ClientApiGateway and also create our own API gateway for the HTTP ions, because we are using a JWT authorizer on certain paths.

Here is the complete CloudFormation template we are using to tailor a stock DCS to our needs:

AWSTemplateFormatVersion: '2010-09-09'
Description: |
  API Gateway for exposing the `api` <product-code-name> web service.

  It REQUIRES the Datomic Cloud app to be deployed, before provisioning this
  stack, to ensure that the following outputs of a compute stack are available:
  * VpcLinkId
  * HttpDirectListener
  * ClientApiGatewayId
Parameters:
  Env:
    Type: String
    Default: "dev"
  ProductCodeName:
    Type: String
    Default: "product-code-name"
  SystemName:
    Description: System name that this stack will serve. This is the name you gave
      the Storage when you created it.
    Type: String
  ApiName:
    Type: String
    Description: |
      Name of the API, eg. `api`.
  DatomicDeploymentGroup:
    Type: String
    Description: |
      Name of the compute-group, where the API is deployed to.
  Domain:
    Type: String
    Description: |
      The domain name with wildcard SSL cert (for the region),
      e.g. `product-code-name.company.com`
  SubDomain:
    Type: String
    Description: |
      The subdomain name, e.g. `api`
  DomainWildcardCertificate:
    Type: String
    Description: |
      The domain's wildcard SSL cert ARN
  GoogleAuthorizerAudiences:
    Type: CommaDelimitedList
    Description: |
      List of Client ID from google that are authorized to call the
      "protected" endpoints on API2 by a google user JWT

Resources:
  Api:
    Type: AWS::ApiGatewayV2::Api
    Properties:
      Description: Gateway to the API, proxied by an ion lambda function
      Name: !Sub "${ProductCodeName}-${Env}-${ApiName}"
      ProtocolType: HTTP

  ApiLogs:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub "/aws/apigw/${ProductCodeName}-${Env}-${ApiName}"
      RetentionInDays: 30

  DefaultStage:
    Type: AWS::ApiGatewayV2::Stage
    Properties:
      StageName: "$default"
      ApiId: !Ref Api
      AutoDeploy: true
      AccessLogSettings:
        DestinationArn: !GetAtt ApiLogs.Arn
        Format: >-
          $context.requestId [$context.requestTime]
          $context.protocol $context.status
          "$context.httpMethod $context.path"
          $context.error.messageString
          $context.authorizer.error
          $context.integration.error.message
          $context.identity.sourceIp

  HttpDirectApiIntegration:
    Type: AWS::ApiGatewayV2::Integration
    Properties:
      Description: !Sub 'Datomic web ion access through VPC Link'
      ApiId: !Ref Api
      IntegrationMethod: ANY
      IntegrationType: HTTP_PROXY
      IntegrationUri: { Fn::ImportValue: !Sub '${SystemName}-HttpDirectListener' }
      ConnectionType: VPC_LINK
      ConnectionId: { Fn::ImportValue: !Sub '${SystemName}-VpcLinkId' }
      PayloadFormatVersion: '1.0'

  GoogleIdentityAuthorizer:
    Type: AWS::ApiGatewayV2::Authorizer
    Properties:
      ApiId: !Ref Api
      AuthorizerType: JWT
      IdentitySource:
        - '$request.header.Authorization'
      JwtConfiguration:
        Audience: !Ref GoogleAuthorizerAudiences
        Issuer: "https://accounts.google.com"
      Name: !Sub "${Api}-authorizer"

  CatchAllRoute:
    Type: AWS::ApiGatewayV2::Route
    Properties:
      ApiId: !Ref Api
      RouteKey: "ANY /{proxy+}"
      Target: !Sub "integrations/${HttpDirectApiIntegration}"

  AuthenticatedRoute:
    Type: AWS::ApiGatewayV2::Route
    Properties:
      ApiId: !Ref Api
      RouteKey: "ANY /u/{proxy+}"
      Target: !Sub "integrations/${HttpDirectApiIntegration}"
      AuthorizationType: JWT
      AuthorizerId: !Ref GoogleIdentityAuthorizer

  ApiDomain:
    Type: AWS::ApiGatewayV2::DomainName
    Properties:
      DomainName: !Sub '${SubDomain}.${Domain}'
      DomainNameConfigurations:
        - CertificateArn: !Ref DomainWildcardCertificate

  ApiMapping:
    Type: AWS::ApiGatewayV2::ApiMapping
    Properties:
      DomainName: !Ref ApiDomain
      ApiId: !Ref Api
      Stage: !Ref DefaultStage

  ApiDns:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneName: !Sub "${Domain}."
      Name: !Ref ApiDomain
      Type: A
      AliasTarget:
        DNSName: !GetAtt ApiDomain.RegionalDomainName
        HostedZoneId: !GetAtt ApiDomain.RegionalHostedZoneId

  DatomicClientApiDomain:
    Type: AWS::ApiGatewayV2::DomainName
    Properties:
      DomainName: !Sub 'datomic.${Domain}'
      DomainNameConfigurations:
        - CertificateArn: !Ref DomainWildcardCertificate

  DatomicClientApiMapping:
    Type: AWS::ApiGatewayV2::ApiMapping
    Properties:
      DomainName: !Ref DatomicClientApiDomain
      ApiId: { Fn::ImportValue: !Sub '${SystemName}-ClientApiGatewayId' }
      Stage: "$default"

  DatomicClientApiDns:
    Type: AWS::Route53::RecordSet
    Properties:
      HostedZoneName: !Sub "${Domain}."
      Name: !Ref DatomicClientApiDomain
      Type: A
      AliasTarget:
        DNSName: !GetAtt DatomicClientApiDomain.RegionalDomainName
        HostedZoneId: !GetAtt DatomicClientApiDomain.RegionalHostedZoneId

Outputs:
  DefaultApiURL:
    Value: !GetAtt Api.ApiEndpoint

  ApiURL:
    Value: !Sub "https://${ApiDns}"

  DatomicClientApiURL:
    Value: !Sub "https://${DatomicClientApiDns}"

  ApiLogs:
    Value: !Ref ApiLogs
2 Likes