Trouble connecting to Heroku Transactor

Datomic Pro Troubleshooting
1

I’m trying to run a transactor on Heroku. The transactor is up and running with Heroku Postgres and my peer can connect to storage to retrieve the host and alt-host values, but it cannot connect to the transactor. The values in the datomic_kvs table are being set correctly, or at least to the values I have provided: the port is set dynamically from Heroku’s $PORT env var, and I’m using sed to put it in the transactor.properties. The host is set to 0.0.0.0 and the alt-host is set to the Heroku-provided hostname of my application.

I get this error (this from my local REPL):

ActiveMQNotConnectedException AMQ119007: Cannot connect to server(s). Tried with all available servers.  org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory (ServerLocatorImpl.java:799)
                                                                  clojure.core/eval                core.clj: 3206
                                                                                ...                              
                                                                      user/eval2432               REPL Input     
                                                        datomic.api/create-database                 api.clj:   19
                                                        datomic.Peer.createDatabase               Peer.java:  117
                                                                                ...                              
                                                       datomic.peer/create-database                peer.clj:  764
                                                       datomic.peer/create-database                peer.clj:  772
                                                    datomic.peer/send-admin-request                peer.clj:  752
                                                 datomic.peer/send-admin-request/fn                peer.clj:  760
                               datomic.connector/create-transactor-hornet-connector           connector.clj:  320
                               datomic.connector/create-transactor-hornet-connector           connector.clj:  322
                                            datomic.connector/create-hornet-factory           connector.clj:  142
                                               datomic.connector/try-hornet-connect           connector.clj:  110
                                      datomic.artemis-client/create-session-factory      artemis_client.clj:  114
org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory  ServerLocatorImpl.java:  799
org.apache.activemq.artemis.api.core.ActiveMQNotConnectedException: AMQ119007: Cannot connect to server(s). Tried with all available servers.
    type: #object[org.apache.activemq.artemis.api.core.ActiveMQExceptionType$3 0x5d70be8d "NOT_CONNECTED"]
                                        clojure.lang.ExceptionInfo: Error communicating with HOST 0.0.0.0 or ALT_HOST ef-datomic.herokuapp.com on PORT 3206
           alt-host: "ef-datomic.herokuapp.com"
    encrypt-channel: true
               host: "0.0.0.0"
           password: <redacted>
       peer-version: 2
               port: 3206
          timestamp: 1515588947255
           username: <redacted>
            version: "0.9.5561.50"

Can anybody help with suggestions of possible reasons that neither my peer (in Heroku) or my REPL (on my laptop) can connect to this running transactor? The error shows that I can connect to storage fine.

2

I’m also experiencing the same issue with a transactor running on AWS, backed with Dynamo and using the CloudFormation template. I think there’s something missing from my understanding.

3

When you are trying on AWS with Dynamo, are you using the steps outlined here?

Have you verified that your transactor is up and healthy (i.e. reporting heartbeat to CloudWatch)?

If you’re attempting to connect to the system from a REPL on your local machine, you’ll need to allow traffic from your local system to the transactor’s security group.

4

I have now managed to connect to the AWS one, but I need to run it on Heroku and still no dice.

5

Has anyone heard a story of a transactor being used successfully on Heroku?

6

Conan,

A Datomic user wrote about his experiences using Datomic on Heroku some time ago:

http://blog.opengrail.com/datomic/heroku/postgres/2015/11/19/datomic-heroku-spaces.html

7

Thanks, but that requires Private Spaces, which are currently only available to Heroku Enterprise customers (it costs $1000/month). Do you know if there’s a part of Datomic’s networking requirements that are met by Private Spaces but not by standard Heroku?

8

I’m really hoping there’s a way around this problem, Datomic is so difficult to set up compared to other datastores and a shortcut like this would be hugely helpful.

9

I believe the issue with Heroku is that the transactor requires a non-HTTP port, which is only supported in Private Spaces.

Many users successfully run their transactor on AWS and their peers in Heroku - perhaps that is an option for you?

10

Yes, I’m coming to the conclusion that might be the best option. If I’m putting the effort into running the transactor on AWS then it seems a shame to have to expose it to the internet, so I’m in two minds! I’ll write a blog post about it all when I’m done.