# Where are queries being cached?

**URL:** <https://forum.datomic.com/t/where-are-queries-being-cached/762>\
**Category:** General\
**Created:** [December 25, 2018, 1:03am UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762 "2018-12-25T01:03:16Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![timofeytt007](https://avatars.discourse-cdn.com/v4/letter/t/dc4da7/32.png) [@timofeytt007](https://forum.datomic.com/u/timofeytt007)\
**Post date:** [December 25, 2018, 1:03am UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/1 "2018-12-25T01:03:16Z")

</div>

What memory setting affects how many queries can be cached?  
[https://docs.datomic.com/on-prem/capacity.html](https://docs.datomic.com/on-prem/capacity.html)  
Object cache?

---

<div class="post-metadata">

**Author:** ![marshall](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/marshall/32/48_2.png) [@marshall](https://forum.datomic.com/u/marshall)\
**Post date:** [December 26, 2018, 6:25pm UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/2 "2018-12-26T18:25:36Z")

</div>

Are you referring to the results of the query being cached on your peer? If so, that will (indirectly) be controlled by the Object Cache, as you suggest

Datomic also caches processed query representations ([https://docs.datomic.com/on-prem/query.html#query-caching](https://docs.datomic.com/on-prem/query.html#query-caching)). The size of the parsed query cache is not user-configurable.

---

<div class="post-metadata">

**Author:** ![timofeytt007](https://avatars.discourse-cdn.com/v4/letter/t/dc4da7/32.png) [@timofeytt007](https://forum.datomic.com/u/timofeytt007)\
**Post date:** [December 27, 2018, 5:16am UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/3 "2018-12-27T05:16:55Z")

</div>

Well, query results must go somewhere? Can I tune it somehow indirectly with -Xmx?

Is a processed query representation a query with an argument?

---

<div class="post-metadata">

**Author:** ![marshall](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/marshall/32/48_2.png) [@marshall](https://forum.datomic.com/u/marshall)\
**Post date:** [December 27, 2018, 3:19pm UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/4 "2018-12-27T15:19:47Z")

</div>

The final result of a query is returned to your application and isn’t cached ‘as is’ by Datomic.

However, the Object Cache holds segments of the Datomic database in local memory. The Object Cache is an LRU cache of raw index segments and its size can be tuned with a command line argument to your peer process ([https://docs.datomic.com/on-prem/system-properties.html](https://docs.datomic.com/on-prem/system-properties.html)).

The query engine will first look in the Object Cache when it needs a database segment, so if you have sufficient memory on your machine and have configured a large Object Cache, you can often serve a large proportion of your query requirements via the in-memory local cache, instead of having to read segments from a remote location (i.e. storage).

---

<div class="post-metadata">

**Author:** ![timofeytt007](https://avatars.discourse-cdn.com/v4/letter/t/dc4da7/32.png) [@timofeytt007](https://forum.datomic.com/u/timofeytt007)\
**Post date:** [December 28, 2018, 3:25pm UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/5 "2018-12-28T15:25:43Z")

</div>

[https://docs.datomic.com/on-prem/best-practices.html#parameterize-queries](https://docs.datomic.com/on-prem/best-practices.html#parameterize-queries)  
Does this mean that query representations are cached, but query results are not!?

---

<div class="post-metadata">

**Author:** ![marshall](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/marshall/32/48_2.png) [@marshall](https://forum.datomic.com/u/marshall)\
**Post date:** [December 28, 2018, 4:57pm UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/6 "2018-12-28T16:57:20Z")

</div>

The result of a query is returned to your calling application code.  
The segments used during that query’s execution will likely be cached in your local object cache (assuming it’s large enough to hold them all).

The recommendation to parameterize queries is strictly about performance enhancement at the level of query parsing and “compilation”.

---

<div class="post-metadata">

**Author:** ![timofeytt007](https://avatars.discourse-cdn.com/v4/letter/t/dc4da7/32.png) [@timofeytt007](https://forum.datomic.com/u/timofeytt007)\
**Post date:** [December 29, 2018, 7:09am UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/7 "2018-12-29T07:09:26Z")

</div>

Thank you for the clarification. Do you think it would be useful to implement query result caching? It would be tied to the timestamp of the query execution.

---

<div class="post-metadata">

**Author:** ![adam](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@adam](https://forum.datomic.com/u/adam)\
**Post date:** [January 4, 2019, 2:33pm UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/8 "2019-01-04T14:33:53Z")

</div>

The query execution timestamp should be irrelevant in terms of the query results, since the database “value” passed to query is immutable. You would only need to key your result cache on query inputs (in the case of `Database` inputs, its id and `basisT` would represent this adequately).

I’ve found that for the most part the results of a query are an intermediate form…there might be multiple queries, serialization for clients, etc. that happens in your application and it is better to cache the end result. Since datomic doesn’t know anything of these details, “built in” query result caching would add complexity and consume resources for very little benefit. As Marshall mentioned, the segments _are_ cached. Segments are expensive to retrieve (since they must traverse the network) and can be reused across many different queries, so there is more benefit in caching them.

If you really do need to serve the same query result again and again, adding this caching layer is nearly trivial for the application. Use the list of query inputs as your cache key.

---

<div class="post-metadata">

**Author:** ![dustingetz](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/dustingetz/32/56_2.png) [@dustingetz](https://forum.datomic.com/u/dustingetz)\
**Post date:** [January 4, 2019, 2:56pm UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/9 "2019-01-04T14:56:12Z")

</div>

I think the basis-t advances too quickly in many apps for a result cache to be useful. We would need an incremental query engine for this to be useful

---

<div class="post-metadata">

**Author:** ![adam](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@adam](https://forum.datomic.com/u/adam)\
**Post date:** [January 4, 2019, 3:26pm UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/10 "2019-01-04T15:26:24Z")

</div>

Indeed, I completely agree. Sometimes the application can annotate transactions with additional attributes to establish coarser grains than basisT, still nothing datomic knows about.

In terms of reactive query, are you referring to something like this? [http://2018.clojure-conj.org/nikolas-gobel/](http://2018.clojure-conj.org/nikolas-gobel/)

---

<div class="post-metadata">

**Author:** ![dustingetz](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.datomic.com/dustingetz/32/56_2.png) [@dustingetz](https://forum.datomic.com/u/dustingetz)\
**Post date:** [January 4, 2019, 5:17pm UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/11 "2019-01-04T17:17:03Z")

</div>

yes, Niko’s research is awesome

---

<div class="post-metadata">

**Author:** ![timofeytt007](https://avatars.discourse-cdn.com/v4/letter/t/dc4da7/32.png) [@timofeytt007](https://forum.datomic.com/u/timofeytt007)\
**Post date:** [January 14, 2019, 1:13am UTC](https://forum.datomic.com/t/where-are-queries-being-cached/762/12 "2019-01-14T01:13:24Z")

</div>

I meant some kind of a timestamp that is tied to the query result. If this is basisT in Datomic, then that’s it.  
I don’t get your rationale why caching wouldn’t be useful in Datomic. Materialized views in an accumulate-only database like Datomic are more useful than in a regular database. You could even get correct results instead of cached results:

1. get cached results from materialized view
2. fix them based on recent transaction log changes
