Hi there.
My current cenario is that we’re using datomic cloud in one of our major services and it is around 60M entities/3.5B datoms and some particular queries are under performatic.
As we plan to grow some orders of magnitude, I was exploring alternatives to escalate both our writes and reads.
From my understanding so far, given that I’m able to scale the number of processors to serve my dbs, and transactors don’t compete for resources among those dbs, I started experimenting with the following:
1 Have my service write in parallel to multiple dbs (let’s say db0 db1 db2 all with the same schema), ensuring that the same entity always end up in the correct db so I don’t end up with partial data split across my databases
2 When querying, I issue them in parallel, then merge the results in my application, something like
(pcalls query-for-satellites0 query-for-satellites1 query-for-satellites2)
So far, this parallel read/write cenario has proven to be really performatic
Now my question to you guys is if I’m missing on something, or are there any achitectural gotchas that would make this a bad idea?