Were trying to sync part of datomic to datacript, and we ran into an issue that datomic id’s produced are too big for datascript.
I believe the best solution likely involves syncing around a uuid rather then the db/id. This fixes the issue with the servers db/id being too long on the client and it generically fixes the issue of uniquely identifying an entity across multiple databases.
One downside is that we have to add a UUID to every entity in the db. And when we transact data to datomic,that we then need to ship to datascript, we would need a way to get that uuid (as it wouldn’t be in the tx-data log sense it wasn’t touched). However, that could just be done with a lookup, probably not too slow.
Alternatively, if we don’t want to add any facts to the Datomic, we could sync over an in memory unique.value client side attribute called :server/id, that is the :db/id from the server.
One common issue in both approaches is modifying the transaction log to add information (the uuid or server/id). The only idea i have is to do a none side effecting transaction using d/db-after and get the tx-data form that which always seems to be a log as opposed to the nested maps that can be submitted to transact! I just assume it will be hard to properly parse nested map tx’s so using the log will be far easier.
The code to change the data script client tx-data log to add this server/id currently taking this shape:
(reduce-kv
(fn [tx-data idx [too-long-server-db-id & rest-of-datom]]
(let [temp-id (str idx)]
(conj tx-data
(vec (conj rest-of-datom temp-id :db/add)) ;; we replace the server id which is too long with a temporary id based on the idx of the log
[:db/add temp-id :server/id (str too-long-server-db-id)])))
#{}
(vec (take 1 tx-data)))
;; => #{[:db/add "0" :server/id "316659348816869"]
;; [:db/add
;; "0"
;; :internal-team/id
;; #uuid "5caf6e45-f54d-4c0e-9658-33f63c069569"
;; 536870913
;; true]}
If you have any thoughts, let me know (yes, I see that fact has a UUID, but they aren’t on everything!).
- Here is a slack discussion on the topic that some might find useful
- datasync does this, maybe i could pull inspiration from there, i wonder if it predates unique.value in datascript though, or at least why they wouldn’t use that (i don’t see it mentioned.