Ion-dev 1.0.356 [Datomic Cloud]

This message covers changes in this release. For more details, see Datomic Cloud Releases | Datomic

2026/07/01 - 1.0.356 - Ion-Dev

  • Feature: IMDSv2 is now supported. Push a new revision using ion-dev 1.0.356 then :deploy it before enforcing IMDSv2 on your Primary Compute Group.

Can you point us into some docs about this IMDSv2 support, please?
Why do we need this?
I’d guess it’s more secure.
But then what should we set that new ImdsHttpToken CF param, mentioned in Compute Templates | Datomic ?

I haven’t set the ImdsHttpToken, so it’s probably unrelated, but

  1. I’ve done a successful deployment to both our dev-env transactor and query group.
  2. Storage stack upgrade and primary compute group upgrades were both successful
  3. but the query group update rolled back after an hour:
|----------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------|
|      LogicalResourceId     |                ResourceStatus                |                                                                                  ResourceStatusReason                                                                                 |      Timestamp      |
|----------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------|
| ---                        | ---                                          | ---                                                                                                                                                                                   | ---                 |
| <query-group-stack>        | UPDATE_ROLLBACK_COMPLETE                     |                                                                                                                                                                                       | 2026-07-06 17:10:01 |
| <query-group-stack>        | UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS |                                                                                                                                                                                       | 2026-07-06 17:10:00 |
| QueryGroupAutoScalingGroup | UPDATE_COMPLETE                              |                                                                                                                                                                                       | 2026-07-06 17:09:58 |
| QueryGroupAutoScalingGroup | UPDATE_IN_PROGRESS                           |                                                                                                                                                                                       | 2026-07-06 17:09:42 |
| QueryGroupLaunchTemplate   | UPDATE_COMPLETE                              |                                                                                                                                                                                       | 2026-07-06 17:09:42 |
| QueryGroupLaunchTemplate   | UPDATE_IN_PROGRESS                           |                                                                                                                                                                                       | 2026-07-06 17:09:27 |
| <query-group-stack>        | UPDATE_ROLLBACK_IN_PROGRESS                  | The following resource(s) failed to update: [QueryGroupAutoScalingGroup].                                                                                                             | 2026-07-06 17:09:19 |
| QueryGroupAutoScalingGroup | UPDATE_FAILED                                | Group did not stabilize. {current/minSize/maxSize} group size = {1/2/3}. Failed Scaling Activity: One or more target groups not found. Validating load balancer configuration failed. | 2026-07-06 17:09:19 |
| QueryGroupAutoScalingGroup | UPDATE_IN_PROGRESS                           | Batch size: 1. Waiting for 1 instances in this batch to reach InService before continuing. Time Elapsed: 1 hour. Time remaining for Stabilization: 0 minutes.                         | 2026-07-06 17:09:18 |
| QueryGroupAutoScalingGroup | UPDATE_IN_PROGRESS                           | Batch size: 1. Waiting for 1 instances in this batch to reach InService before continuing. Time Elapsed: 30 minutes. Time remaining for Stabilization: 30 minutes.                    | 2026-07-06 16:39:11 |
| QueryGroupAutoScalingGroup | UPDATE_IN_PROGRESS                           | Batch size: 1. Waiting for 1 instances in this batch to reach InService before continuing. Time Elapsed: 0 minutes. Time remaining for Stabilization: 1 hour.                         | 2026-07-06 16:08:15 |
| QueryGroupAutoScalingGroup | UPDATE_IN_PROGRESS                           | Temporarily setting autoscaling group MinSize and DesiredCapacity to 2.                                                                                                               | 2026-07-06 16:08:13 |
| QueryGroupAutoScalingGroup | UPDATE_IN_PROGRESS                           | Rolling update initiated. Terminating 1 obsolete instance(s) in batches of 1, while keeping at least 1 instance(s) in service.                                                        | 2026-07-06 16:08:13 |
| QueryGroupAutoScalingGroup | UPDATE_IN_PROGRESS                           |                                                                                                                                                                                       | 2026-07-06 16:08:10 |
| QueryGroupLaunchTemplate   | UPDATE_COMPLETE                              |                                                                                                                                                                                       | 2026-07-06 16:08:08 |
| QueryGroupLaunchTemplate   | UPDATE_IN_PROGRESS                           |                                                                                                                                                                                       | 2026-07-06 16:08:06 |
| <query-group-stack>        | UPDATE_IN_PROGRESS                           | User Initiated                                                                                                                                                                        | 2026-07-06 16:07:55 |
| <query-group-stack>        | UPDATE_COMPLETE                              |                                                                                                                                                                                       | 2026-02-24 13:55:14 |
| <query-group-stack>        | UPDATE_COMPLETE_CLEANUP_IN_PROGRESS          |                                                                                                                                                                                       | 2026-02-24 13:55:12 |
| QueryGroupAutoScalingGroup | UPDATE_COMPLETE                              |                                                                                                                                                                                       | 2026-02-24 13:55:07 |
|----------------------------+----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------|

and ideas what to do?

Never mind. We had a stale target group attached to the auto-scaling group and that resulted in this failure:

Group did not stabilize. {current/minSize/maxSize} group size = {1/2/3}. Failed Scaling Activity: One or more target groups not found. Validating load balancer configuration failed.

Regarding the ImdsHttpToken I would be interested in how connections from development Clojure process to an ion database would be affected by this change? Would d/connect just handle it transparently?

datomic/ion/dev/cloud-deps.edn doesn’t seem to be in sync with the actual deps on the ion servers:

~/vm/ion-dev via ☕ v21.0.2 on ☁  (ap-southeast-1) 
❯ export -f function ion_dev_deps () {
    local ver="$1";                   
    local jar="${HOME}/.m2/repository/com/datomic/ion-dev/$ver/ion-dev-$ver.jar";
    unzip -p $jar datomic/ion/dev/cloud-deps.edn |
    clojure -M -e "(->> *in* slurp read-string :deps (mapv (juxt key (comp :mvn/version val))) sort (run! (partial apply prn)))" |
    tee ion-dev-${ver}-deps.edn
}

~/vm/ion-dev via ☕ v21.0.2 on ☁  (ap-southeast-1) 
❯ ion_dev_deps 1.0.356 | grep jetty   
org.eclipse.jetty/jetty-http "9.4.53.v20231009"
org.eclipse.jetty/jetty-io "9.4.53.v20231009"
org.eclipse.jetty/jetty-security "9.4.53.v20231009"
org.eclipse.jetty/jetty-server "9.4.53.v20231009"
org.eclipse.jetty/jetty-servlet "9.4.53.v20231009"
org.eclipse.jetty/jetty-util "9.4.53.v20231009"
org.eclipse.jetty/jetty-util-ajax "9.4.53.v20231009"

deps on the ion servers:

ion-repl> (-> (System/getProperty "java.class.path") (str/split #":") (->> (filter #(re-find #"jetty" %))))
("lib/jetty-ee11-servlet-12.1.10.jar"
 "lib/jetty-http-12.1.10.jar"
 "lib/jetty-io-12.1.10.jar"
 "lib/jetty-security-12.1.10.jar"
 "lib/jetty-server-12.1.10.jar"
 "lib/jetty-session-12.1.10.jar"
 "lib/jetty-util-12.1.10.jar"
 "/home/datomic/.m2/repository/org/eclipse/jetty/jetty-servlet/9.4.53.v20231009/jetty-servlet-9.4.53.v20231009.jar"
 "/home/datomic/.m2/repository/org/eclipse/jetty/jetty-util-ajax/9.4.53.v20231009/jetty-util-ajax-9.4.53.v20231009.jar")

i’ve ended up with a mixture of jetty library versions, since i can’t reproduce the jetty 12 deps locally, which are hardwired on the ion servers:

~/vm/ion-dev via ☕ v21.0.2 on ☁  (ap-southeast-1) 
❯ cat deps.edn 
{:aliases
 {:ion-dev
  {:deps {com.datomic/ion-dev {:mvn/version "1.0.356"}}
   :main-opts ["-m" "datomic.ion.dev"]}}
  :mvn/repos
  {"datomic-cloud" {:url "s3://datomic-releases-1fc2183a/maven/releases"}}}

~/vm/ion-dev via ☕ v21.0.2 on ☁  (ap-southeast-1) 
❯ clj -X:deps tree :aliases '[:ion-dev]' | grep jetty
    . org.eclipse.jetty/jetty-http 9.4.56.v20240826
      . org.eclipse.jetty/jetty-util 9.4.56.v20240826
      . org.eclipse.jetty/jetty-io 9.4.56.v20240826
        . org.eclipse.jetty/jetty-util 9.4.56.v20240826
    . org.eclipse.jetty/jetty-client 9.4.56.v20240826
      . org.eclipse.jetty/jetty-http 9.4.56.v20240826
      . org.eclipse.jetty/jetty-io 9.4.56.v20240826

last time we discussed this, you advised to ignore the conflicts im seeing on :op :push, but im getting more and more uneasy about this situation. we keep experiencing that mysterious “Loading database…” exception on d/connect or d/transact calls and i’m worried that such unsupported mixture of versions might be behind the root cause, for example…