Total number of attributes

Datomic Applications
1

I am designing an app that will ingest and persist many different datasets (e.g. CSV files or intermediate computational results). For each column I am creating a new attribute. Is this approach feasible? What do I need to consider when growing a set of attributes exponentially?

Datomic enforces the total number of schema elements to be fewer than 2^20, which includes schema attributes, value types and partitions. Attempts to create more than 2^20 schema elements will fail.

— Schema | Datomic

Part of the reason for the limit of schema elements is that all of the idents are stored in memory in every transactor and peer. The more attributes you create the more expensive the cost of storing these in memory.

As long as you keep the 2^20 schema limit in mind you can use Datomic to dynamically create attributes with generated IDs and meet this desired use case.

2

Here is what I found in the docs: “Idents are designed to be extremely fast and always available. All idents associated with a database are stored in memory in every Datomic transactor and peer.” I assume that is only true for the Datoms [attribute-eid :db/ident :attr/foo], but not for other Datoms associated with the attribute-eid (like :db/doc). Correct?

3

As indicated in the documentation, “idents … are stored in memory”. Specifically, the idents are stored, datoms are not.