# Syncing to datascript **URL:** \ **Category:** Datomic Pro\ **Created:** [August 30, 2022, 7:01pm UTC](https://forum.datomic.com/t/syncing-to-datascript/2121 "2022-08-30T19:01:04Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![drewverlee](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/drewverlee/32/308_2.png) [@drewverlee](https://forum.datomic.com/u/drewverlee)\ **Post date:** [August 30, 2022, 7:01pm UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/1 "2022-08-30T19:01:04Z")
Were trying to sync part of datomic to datacript, and we ran into an [issue](https://clojurians.slack.com/archives/C07V8N22C/p1660528936955479) that datomic id’s produced are too big for datascript. I believe the best solution likely involves syncing around a uuid rather then the db/id. This fixes the issue with the servers db/id being too long on the client and it generically fixes the issue of uniquely identifying an entity across multiple databases. One downside is that we have to add a UUID to every entity in the db. And when we transact data to datomic,that we then need to ship to datascript, we would need a way to get that uuid (as it wouldn’t be in the tx-data log sense it wasn’t touched). However, that could just be done with a lookup, probably not too slow. Alternatively, if we don’t want to add any facts to the Datomic, we could sync over an in memory unique.value client side attribute called :server/id, that is the :db/id from the server. One common issue in both approaches is modifying the transaction log to add information (the uuid or server/id). The only idea i have is to do a none side effecting transaction using d/db-after and get the tx-data form that which always seems to be a log as opposed to the nested maps that can be submitted to transact! I just assume it will be hard to properly parse nested map tx’s so using the log will be far easier. The code to change the data script client tx-data log to add this server/id currently taking this shape: ```auto (reduce-kv (fn [tx-data idx [too-long-server-db-id & rest-of-datom]] (let [temp-id (str idx)] (conj tx-data (vec (conj rest-of-datom temp-id :db/add)) ;; we replace the server id which is too long with a temporary id based on the idx of the log [:db/add temp-id :server/id (str too-long-server-db-id)]))) #{} (vec (take 1 tx-data))) ;; => #{[:db/add "0" :server/id "316659348816869"] ;; [:db/add ;; "0" ;; :internal-team/id ;; #uuid "5caf6e45-f54d-4c0e-9658-33f63c069569" ;; 536870913 ;; true]} ``` If you have any thoughts, let me know 🙂 (yes, I see that fact has a UUID, but they aren’t on everything!). - Here is a [slack discussion](https://clojurians.slack.com/archives/C07V8N22C/p1660528936955479) on the topic that some might find useful - [datasync](https://github.com/metasoarous/datsync) does this, maybe i could pull inspiration from there, i wonder if it predates unique.value in datascript though, or at least why they wouldn’t use that (i don’t see it mentioned. ---
**Author:** ![drewverlee](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/drewverlee/32/308_2.png) [@drewverlee](https://forum.datomic.com/u/drewverlee)\ **Post date:** [September 11, 2022, 3:41am UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/2 "2022-09-11T03:41:22Z")
I ended up forking datascript and bumping the emax number, as it was just a perf hit. ---
**Author:** ![danbunea](https://avatars.discourse-cdn.com/v4/letter/d/e47774/32.png) [@danbunea](https://forum.datomic.com/u/danbunea)\ **Post date:** [April 13, 2023, 9:17pm UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/3 "2023-04-13T21:17:48Z")
The way I did it, was to to a map of the server (datomic ids)-\>Datascript ids which I use every time I transact to the server and back. My hope is that this will allow me to have a Datascript database that reads data from several Datomic (and or DataHike) databases and be able to combine it locally. ---
**Author:** ![Leon](https://avatars.discourse-cdn.com/v4/letter/l/dc4da7/32.png) [@Leon](https://forum.datomic.com/u/Leon)\ **Post date:** [April 30, 2023, 9:05pm UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/4 "2023-04-30T21:05:59Z")
Consider using a custom (unique identity) field in Datascript that represents the :db/id of the Datomic instance. Use type string to avoid integer limits. ```auto ;; Datomic {:db/id 213891283192 :db/doc "My Entity"} ;; Datascript {:db/id 8 :db/doc "My Entity" :datomic/id "213891283192"} ``` ---
**Author:** ![ttallman](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/ttallman/32/528_2.png) [@ttallman](https://forum.datomic.com/u/ttallman)\ **Post date:** [May 10, 2023, 2:20pm UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/5 "2023-05-10T14:20:46Z")
> [@drewverlee](#): > > emax If you use datascript in clojurescript raising emax will probably cause you an issue. That is the max representable integer in javascript. Every thing above that will function as a double. This will result in many phantom bugs because mast emax some numbers will round to the same floating point representation. ---
**Author:** ![drewverlee](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/drewverlee/32/308_2.png) [@drewverlee](https://forum.datomic.com/u/drewverlee)\ **Post date:** [May 12, 2023, 3:49pm UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/6 "2023-05-12T15:49:28Z")
Thanks for the input @ttallman ! To make sure we’re on the page, I’m going to add some details I think are missing, namely, what we increased emax to, and why I think it will allow our system to do correct number comparisons. If in the end, you’re not convinced our solution is reasonable, I would be extremely interested to know why, as this is a critical part of our code. I would be happy to dig up any more information you need as well. What I understand is that emax is a var in the [datascript code base](https://github.com/tonsky/datascript/blob/045655499b079997d1df9fcd7446ea8243efc66a/src/datascript/db.cljc#L26) ```auto (def ^:const emax 0x7FFFFFFF) ``` Which is a hexadecimal representation of 2147483647. ```auto parseInt("0x7FFFFFFF", 16) 2147483647 ``` Which, as you imply, is to keep eid’s from becoming too large to be compared. ```auto (if (> eid emax) (raise "Highest supported entity id is " emax ", got " eid {:error :entity-id :value eid}) eid) ``` However, it’s far less than the largest safe number, which is the [Number.MAX\_SAFE\_INTEGER](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER#description) ```auto Number.MAX_SAFE_INTEGER 9007199254740991 ``` Which Mozilla describes as > Safe" in this context refers to the ability to represent integers exactly and to compare them correctly. For example, `Number.MAX_SAFE_INTEGER + 1 === Number.MAX_SAFE_INTEGER + 2` will evaluate to true, which is mathematically incorrect. See [`Number.isSafeInteger()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/isSafeInteger) for more information. I took this to mean that datascript’s emax was overly conservative, so I set emax to the max safe integer. Here is a javascript console sandbox session illustrating if the numbers are safe: ```auto Number.isSafeInteger(parseInt("0x7FFFFFFF", 16)) true Number.isSafeInteger(Number.MAX_SAFE_INTEGER) true Number.isSafeInteger(Number.MAX_SAFE_INTEGER + 1) false Number.MAX_SAFE_INTEGER + 1 === Number.MAX_SAFE_INTEGER + 2 true ``` So given emax is still under the max safe integer, we should be safe! [Here is a conversation](https://clojurians.slack.com/archives/C07V8N22C/p1662582290388149) where i talked it over in the datascript channel, i think it jumps around a bit, i haven’t read it myself in a while, but it might contain more information i have forgotten. ---
**Author:** ![ttallman](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/ttallman/32/528_2.png) [@ttallman](https://forum.datomic.com/u/ttallman)\ **Post date:** [May 12, 2023, 5:27pm UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/7 "2023-05-12T17:27:54Z")
Your are correct I wrongly assumed 0x7FFFFFFF was MAX\_SAFE\_INTEGER ---
**Author:** ![dakra](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/dakra/32/349_2.png) [@dakra](https://forum.datomic.com/u/dakra)\ **Post date:** [May 19, 2023, 11:27am UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/8 "2023-05-19T11:27:38Z")
Would it make sense to increase `emax` then upstream? The way I understood this conversation is that it basically has no downsides and makes the DataScript db/ids compatible with Datomic?! ---
**Author:** ![oscarlinusericsson](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/oscarlinusericsson/32/376_2.png) [@oscarlinusericsson](https://forum.datomic.com/u/oscarlinusericsson)\ **Post date:** [June 15, 2023, 5:33pm UTC](https://forum.datomic.com/t/syncing-to-datascript/2121/9 "2023-06-15T17:33:59Z")
In older javascript, bit operations on numbers converts them to 32-bit integers, which I think would not be compatible with how Datascript uses the eid to access the the datoms internally. In more recent javascript, there seems to (finally) be a [BigInt](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt), availiable which can store 64 bit integers and more. But BigInts can, for instance, not be serialized into JSON and cannot be intermixed with Number in mathematical expressions (which seems sound). So I think the update needed is more extensive than it looks, and I’m not sure of the performance characteristics of BigInts compared to Number/sometimes 32bit integer.