Shared Datomic DB stores

Got a situation where my Ops guy has stood up a single datomic instance containing two separate DBs for two databases. When he imports one, he also overwrites the other, which is not desired. (We have some relatively fixed/template data, and then more active stuff.)

How do people handle this, usually?

Within one storage (instance) we allow for the “db name” part of the datomic URI to be variable within our app. Using the datomic tools for backup/restore requires that you have a coherent timeline. Good for disaster recovery, not so great for rollback or experimentation. When you restore a backup and don’t want to overwrite, pick a new (globally unique) db name and then reconfigure your app to read from the new location.

Adam—

Thank you! That is very helpful.

This kind of raises a related question:

I found errors in our data, so REPLed in and changed them with transactions. And I (very cleverly, I thought) noted these transactions down because I knew I’d have to do them again with the production data.

But since the transactions are all based on references, it seems to me there’s a very good chance that the reference numbers are all different. In fact, I don’t see how they could be the same.

I think this would depend on how truly identical your two environments are. AFAIK the entity ids are deterministic, but if there are different interleaving of transactions, different schema, different partitioning, different (implicit) txInstant values etc. then they may still vary. For this reason, as you suggest, you shouldn’t consider the entity ids to be portable between two databases.

The correct way to handle this would be to define a primary key in your domain model and have an attribute model that PK for each entity type (I seem to recall that it is a best practice not to just use a single id attribute across heterogeneous entities…probably due to index performance concerns, but I’m not sure this applies now that adaptive indexing is available).

Then instead of writing your transactions against an “e” like 123 you can write it against a lookup-ref like [:my/id "abc"] … lookup refs work pretty much anywhere an e is accepted and in the case of transactions they are resolved by the transactor. https://docs.datomic.com/on-prem/identity.html#lookup-refs

Another option if you don’t know for sure that the thing you are referencing even exists yet, but are sure that if it does exist it has a certain ID attribute, would be to do something similar but with upserts. Include an “upsert” for every referenced entity in your transaction and use the temp-id elsewhere in the same transaction to reference it: https://docs.datomic.com/on-prem/transactions.html#temp-id-resolution

1 Like

Great! Thanks again, Adam!