What does this ST tell me is wrong?

Greetings, our aws service is failing to deploy and I would like some help understanding the stack trace. (at bottom)

To me, it’s clear roughly where the problem happens in our app, it’s when we interact with datomic, either creating the database or maybe establishing the connection

(datomic.api/create-database "datomic:ddb://us-east-1/centriq-web-staging/datomic-database")

But what’s not clear to me, is why that doesn’t work, as i can run that locally and nothing is thrown. e.g i running this works for me locally

  (def conn
    (do
      (d/create-database uri)
      (d/connect uri)))
  (def db (d/db conn))
  (d/q'[some-query!] db)

So my only hope is that there is some more information in this stack trace that will help narrow it down between works for me and fails in aws. I feel like the :cause being com.amazonaws.retry.RetryMode should tell me something. Maybe a network issue?

I have to guess it’s an networking issue… its always a networking issue.

The docs for it RetryMode (AWS SDK for Java - 1.12.367) just point out that its thrown when something hits a retry limit, but what was getting retried that failed?

Something deep in datomic I’m guessing. Maybe the best i can do is understand that datomic.api/connect failed and go based off that?

Thanks a lot for any pointers, even just a casual glance or offhand comment sometimes helps a ton. Here is the stacktrace:

UncaughtException
error: Error in component :datomic-db in system com.stuartsierra.component.SystemMap calling #'com.stuartsierra.component/start
Open CloudWatch - https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/cacapi-staging/log-events$3FfilterPattern$3D$257B$2524.Id$253D$252225674b3c-6fe7-4e57-a9e5-daca982c3700$2522$257D$26start$3D1671069011899$26end$3D1671072611899
{:msg "UncaughtException", :ex #error {
:cause "com.amazonaws.retry.RetryMode"
:via
[{:type clojure.lang.ExceptionInfo
  :message "Error in component :datomic-db in system com.stuartsierra.component.SystemMap calling #'com.stuartsierra.component/start"
  :data {:reason :com.stuartsierra.component/component-function-threw-exception, :function #'com.stuartsierra.component/start, :system-key :datomic-db, :component #centriq_web.datomic.DatomicDB{:uri "datomic:ddb://us-east-1/centriq-web-staging/datomic-database", :conn nil, :running? nil, :init-fn nil}, :system #<SystemMap>}
  :at [clojure.core$ex_info invokeStatic "core.clj" 4617]}
 {:type java.lang.ExceptionInInitializerError
  :message nil
  :at [datomic.aws__init load nil 4]}
 {:type java.lang.ClassNotFoundException
  :message "com.amazonaws.retry.RetryMode"
  :at [java.net.URLClassLoader findClass "URLClassLoader.java" 382]}]
:trace
[[java.net.URLClassLoader findClass "URLClassLoader.java" 382]
 [java.lang.ClassLoader loadClass "ClassLoader.java" 424]
 [sun.misc.Launcher$AppClassLoader loadClass "Launcher.java" 349]
 [java.lang.ClassLoader loadClass "ClassLoader.java" 357]
 [java.lang.Class forName0 "Class.java" -2]
 [java.lang.Class forName "Class.java" 348]
 [clojure.lang.RT classForName "RT.java" 2168]
 [clojure.lang.RT classForName "RT.java" 2177]
 [datomic.aws$fn__1182 <clinit> "aws.clj" 54]
 [datomic.aws__init load nil 4]
 [datomic.aws__init <clinit> nil -1]
 [java.lang.Class forName0 "Class.java" -2]
 [java.lang.Class forName "Class.java" 348]
 [clojure.lang.RT classForName "RT.java" 2168]
 [clojure.lang.RT classForName "RT.java" 2177]
 [clojure.lang.RT loadClassForName "RT.java" 2196]
 [clojure.lang.RT load "RT.java" 443]
 [clojure.lang.RT load "RT.java" 419]
 [clojure.core$load$fn__5677 invoke "core.clj" 5893]
 [clojure.core$load invokeStatic "core.clj" 5892]
 [clojure.core$load doInvoke "core.clj" 5876]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5697]
 [clojure.core$load_one invoke "core.clj" 5692]
 [clojure.core$load_lib$fn__5626 invoke "core.clj" 5737]
 [clojure.core$load_lib invokeStatic "core.clj" 5736]
 [clojure.core$load_lib doInvoke "core.clj" 5717]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 648]
 [clojure.core$load_libs invokeStatic "core.clj" 5774]
 [clojure.core$load_libs doInvoke "core.clj" 5758]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 648]
 [clojure.core$require invokeStatic "core.clj" 5796]
 [clojure.core$require doInvoke "core.clj" 5796]
 [clojure.lang.RestFn invoke "RestFn.java" 457]
 [datomic.ddb$loading__5569__auto____2373 invoke "ddb.clj" 4]
 [datomic.ddb__init load nil 4]
 [datomic.ddb__init <clinit> nil -1]
 [java.lang.Class forName0 "Class.java" -2]
 [java.lang.Class forName "Class.java" 348]
 [clojure.lang.RT classForName "RT.java" 2168]
 [clojure.lang.RT classForName "RT.java" 2177]
 [clojure.lang.RT loadClassForName "RT.java" 2196]
 [clojure.lang.RT load "RT.java" 443]
 [clojure.lang.RT load "RT.java" 419]
 [clojure.core$load$fn__5677 invoke "core.clj" 5893]
 [clojure.core$load invokeStatic "core.clj" 5892]
 [clojure.core$load doInvoke "core.clj" 5876]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5697]
 [clojure.core$load_one invoke "core.clj" 5692]
 [clojure.core$load_lib$fn__5626 invoke "core.clj" 5737]
 [clojure.core$load_lib invokeStatic "core.clj" 5736]
 [clojure.core$load_lib doInvoke "core.clj" 5717]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 648]
 [clojure.core$load_libs invokeStatic "core.clj" 5774]
 [clojure.core$load_libs doInvoke "core.clj" 5758]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 648]
 [clojure.core$require invokeStatic "core.clj" 5796]
 [clojure.core$require doInvoke "core.clj" 5796]
 [clojure.lang.RestFn invoke "RestFn.java" 619]
 [datomic.ddb_cluster$loading__5569__auto____13256 invoke "ddb_cluster.clj" 4]
 [datomic.ddb_cluster__init load nil 4]
 [datomic.ddb_cluster__init <clinit> nil -1]
 [java.lang.Class forName0 "Class.java" -2]
 [java.lang.Class forName "Class.java" 348]
 [clojure.lang.RT classForName "RT.java" 2168]
 [clojure.lang.RT classForName "RT.java" 2177]
 [clojure.lang.RT loadClassForName "RT.java" 2196]
 [clojure.lang.RT load "RT.java" 443]
 [clojure.lang.RT load "RT.java" 419]
 [clojure.core$load$fn__5677 invoke "core.clj" 5893]
 [clojure.core$load invokeStatic "core.clj" 5892]
 [clojure.core$load doInvoke "core.clj" 5876]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [clojure.core$load_one invokeStatic "core.clj" 5697]
 [clojure.core$load_one invoke "core.clj" 5692]
 [clojure.core$load_lib$fn__5626 invoke "core.clj" 5737]
 [clojure.core$load_lib invokeStatic "core.clj" 5736]
 [clojure.core$load_lib doInvoke "core.clj" 5717]
 [clojure.lang.RestFn applyTo "RestFn.java" 142]
 [clojure.core$apply invokeStatic "core.clj" 648]
 [clojure.core$load_libs invokeStatic "core.clj" 5774]
 [clojure.core$load_libs doInvoke "core.clj" 5758]
 [clojure.lang.RestFn applyTo "RestFn.java" 137]
 [clojure.core$apply invokeStatic "core.clj" 648]
 [clojure.core$require invokeStatic "core.clj" 5796]
 [clojure.core$require doInvoke "core.clj" 5796]
 [clojure.lang.RestFn invoke "RestFn.java" 408]
 [datomic.require$require_and_run invokeStatic "require.clj" 21]
 [datomic.require$require_and_run doInvoke "require.clj" 17]
 [clojure.lang.RestFn invoke "RestFn.java" 423]
 [datomic.coordination_ext$fn__9769 invokeStatic "coordination_ext.clj" 15]
 [datomic.coordination_ext$fn__9769 invoke "coordination_ext.clj" 12]
 [clojure.lang.MultiFn invoke "MultiFn.java" 229]
 [datomic.coordination$create_system_cluster invokeStatic "coordination.clj" 94]
 [datomic.coordination$create_system_cluster invoke "coordination.clj" 90]
 [datomic.peer$send_admin_request$fn__10971 invoke "peer.clj" 798]
 [datomic.peer$send_admin_request invokeStatic "peer.clj" 792]
 [datomic.peer$send_admin_request invoke "peer.clj" 790]
 [datomic.peer$create_database invokeStatic "peer.clj" 812]
 [datomic.peer$create_database invoke "peer.clj" 802]
 [datomic.peer$create_database invokeStatic "peer.clj" 804]
 [datomic.peer$create_database invoke "peer.clj" 802]
 [clojure.lang.Var invoke "Var.java" 379]
 [datomic.Peer createDatabase "Peer.java" 115]
 [datomic.api$create_database invokeStatic "api.clj" 24]
 [datomic.api$create_database invoke "api.clj" 22]
 [centriq_web.datomic.DatomicDB start "datomic.clj" 57]
 [com.stuartsierra.component$fn__12874$G__12868__12876 invoke "component.cljc" 5]
 [com.stuartsierra.component$fn__12874$G__12867__12879 invoke "component.cljc" 5]
 [clojure.lang.Var invoke "Var.java" 379]
 [clojure.lang.AFn applyToHelper "AFn.java" 154]
 [clojure.lang.Var applyTo "Var.java" 700]
 [clojure.core$apply invokeStatic "core.clj" 648]
 [clojure.core$apply invoke "core.clj" 641]
 [com.stuartsierra.component$try_action invokeStatic "component.cljc" 117]
 [com.stuartsierra.component$try_action invoke "component.cljc" 116]
 [com.stuartsierra.component$update_system$fn__12925 invoke "component.cljc" 139]
 [clojure.lang.ArraySeq reduce "ArraySeq.java" 114]
 [clojure.core$reduce invokeStatic "core.clj" 6544]
 [clojure.core$reduce invoke "core.clj" 6527]
 [com.stuartsierra.component$update_system invokeStatic "component.cljc" 135]
 [com.stuartsierra.component$update_system doInvoke "component.cljc" 129]
 [clojure.lang.RestFn invoke "RestFn.java" 445]
 [com.stuartsierra.component$start_system invokeStatic "component.cljc" 163]
 [com.stuartsierra.component$start_system invoke "component.cljc" 155]
 [com.stuartsierra.component$start_system invokeStatic "component.cljc" 161]
 [com.stuartsierra.component$sta…

I’m know under the impression the stack trace is just telling me it can’t load a class. Though why that would be, given the changes i’m aware of, is a mystery.

It’s probably not datomic related then, though having a list of things we could change on our end concerning datomic that could result in this error, would be ideal.

I assume the it would be just deps in our application so the datomic-pro version. Is that right?