Overwhelmed query group, so can I issue transactions directly to the compute group?

A batch processing job ran into an issue where it overwhelmed it’s query group. It was processing large queries with an occasional transact on too many threads. The datomic instance was pegged to 100% CPU usage, so it’s pretty clear what happened. Oddly and repeatably, the queries never timeout, but the transacts do eventually give a :cognitect.anomalies/unavailable.

We ultimately solve the problem by reducing the number of threads our batch job is running. However, we’d like to know if we can also avoid the transact problem by issuing the transacts directly to the primary compute group rather than through the query-group (which I assume just forwards them anyway)? The primary computer group is under practically no load. However, we’re not sure if the datomic implementation would cause and read-after-write consistency errors, as we often issue a query directly after the transact to fetch back the same information we just transacted.

You can certainly issue transactions directly to a query group. But I would have to know more about your specific issue to provide the best advice.

Generally, it is advisable to issue all transactions to a query group as the query group will forward any transactions to the primary compute group. And this pattern allows for you to effectively filter any query groups for the query group and forward transactions onto primary.

My concern is if you alter your batch job you will ultimately overwhelm your primary compute group instead of being in a situation where you only affected your query group and your primary is still available to transact.

I’d be happy to dive into the details on a support case (support@Datomic.com) and then circle back here when we have the details for any advice that could be useful to others.