History db indexes sorted by Op-T?

General
1

When using a history database I found that indexes are ordered by the boolean asserted/retracted datom value before the T value. First by retractions, then assertions, or false-T, then true-T. I was surprised that my T’s were not in order by themselves. I haven’t been able to find this behaviour mentioned in the documentation and wonder what the rationale for it is?

Thanks,
Marc

2

Marc,

Which index(es) specifically did you investigate (and I’m assuming you’re doing this investigation via the datoms API directly)?

I don’t believe any of the indexes make specific guarantees about the relative ordering of datoms based on the Op value.

3

Marshall,

I’ll try demonstrating by doing two transactions where the first one produces entity id 17592186045445 and the second updates that entity:

tx1
[[:db/add, #db/id[:db.part/user -1000097], :ns/int, 1]]

tx2
[[:db/add, 17592186045445, :ns/int, 2]]

Then when I call the datoms API on a history db:

conn.db().history().datoms(datomic.Database.EAVT, 17592186045445.asInstanceOf[Object]).forEach(d =>
  println(s"[${d.e}   ${d.a}   ${d.v}   ${d.tx}   ${d.added()}]")
)

I get the following 3 datoms:

[17592186045445   64   1   13194139534342   false]
[17592186045445   64   1   13194139534340   true]
[17592186045445   64   2   13194139534342   true]

whereas I would have expected the following order where the datoms are ordered by T (tx id) too:

[17592186045445   64   1   13194139534340   true]
[17592186045445   64   1   13194139534342   false]
[17592186045445   64   2   13194139534342   true]
4

Hi Marc,

The T is sorted in descending order.

This is a fairly “rare” case to see (i.e. you’d have to be using datoms on a history database), but I will see if I can add a description of this to the documentation.

5

Marc,

I’ve updated the docs:
https://docs.datomic.com/on-prem/indexes.html#basics

Thanks for pointing it out.

6

Thanks for the clarification and docs update, Marshall.

If you allow me to be super-nitty-picky, then you might even consider making an addition to clarify that Datoms of the log (also an Index although not “one of four”) are sorted with T ascending. Maybe the first sentence of the log paragraph could be expanded to something like

“The log indexes datoms by time with transaction ids/T in ascending order.”

The Log feels naturally progressing in time for each transaction iterated whereas the descending order of Ts in the other four indexes (when using the history database) feels a bit like “three steps forward (E/A/V) and one step backward (T)”. From a user perspective I think it would be nice if Ts of the four indexes were instead sorted ascending to semantically align with the log.