Upsert behavior with composite tuple key

Although the composite tuple key does get added on the second transaction, the third transaction fails. A transacted entity won’t actually resolve to an existing entity unless I specify the :foo+bar key explicitly, which seems like a bug (or at least undesirable behavior). So this means that if you want upsert behavior, you have to include both the composite attribute and the individual attributes whenever you transact an entity.

user=> 
(let [db (d/with-db system/conn)
        txes [[#:db{:ident :foo,
                    :valueType :db.type/string,
                    :cardinality :db.cardinality/one}
               #:db{:ident :bar,
                    :valueType :db.type/string,
                    :cardinality :db.cardinality/one}
               #:db{:ident :baz,
                    :valueType :db.type/string,
                    :cardinality :db.cardinality/one}
               {:db/ident :foo+bar,
                :db/cardinality :db.cardinality/one
                :db/unique :db.unique/identity
                :db/valueType :db.type/tuple
                :db/tupleAttrs [:foo :bar]}]
              [{:foo "foo"
                :bar "bar"}]
              ; doesn't work
              [{:foo "foo"
                :bar "bar"
                :baz "baz"}]
              ; works
              [{:foo "foo"
                :bar "bar"
                :foo+bar ["foo" "bar"]
                :baz "baz"}]]]
    (reduce (fn [db tx]
              (try
                (let [db (:db-after (d/with db {:tx-data tx}))]
                  (println "transaction successful")
                  db)
                (catch Exception e
                  (println "transaction failed:" (.getMessage e))
                  db)))
            db txes))
transaction successful
transaction successful
transaction failed: Unique conflict: :foo+bar, value: ["foo" "bar"] already held by: 51263630133428781 asserted for: 4631142976193070
transaction successful

(Also, I noticed that the schema data reference still says:

Datomic does not provide a mechanism to declare composite uniqueness constraints; however, you can implement them (or any arbitrary functional constraint) via transaction functions.

This should probably be updated to mention composite tuples.)

Update: upsert is completely broken when the key tuple includes a false boolean value. Example from my application:

(d/transact sys/conn
            {:tx-data [{:rec.oc/track 9895604649984119
                        :rec.oc/user 28666467159376431
                        :rec.oc/hit? false
                        :rec.oc/key [28666467159376431 9895604649984119 false]}]})
Unique conflict: :rec.oc/key, value: [28666467159376431 9895604649984119 false] already held by: 18159534044352573 asserted for: 4925812092440654

I’m assuming this is related to this. Notes:

  • False values only showed up as nil when using d/with, not when doing a real transaction (note value: [28666467159376431 9895604649984119 false] in the example above).
  • The above example works when using d/with, it seems that upsert-for-keys-with-false-values is only completely broken for real transactions (at least when using d/with it’s only partially broken :wink: ).

For now I’m thinking I’ll have to just use another attribute in place of booleans, e.g. a keyword attribute with :true and :false as the values.

Edit: did some more investigating.

  (d/create-database client {:db-name "test"})
  (let [conn (d/connect client {:db-name "test"})
        txes [[#:db{:ident :foo,
                    :valueType :db.type/string,
                    :cardinality :db.cardinality/one}
               #:db{:ident :bar,
                    :valueType :db.type/boolean,
                    :cardinality :db.cardinality/one}
               #:db{:ident :baz,
                    :valueType :db.type/string,
                    :cardinality :db.cardinality/one}
               {:db/ident :foo+bar,
                :db/cardinality :db.cardinality/one
                :db/unique :db.unique/identity
                :db/valueType :db.type/tuple
                :db/tupleAttrs [:foo :bar]}]

              ; The value of :foo+bar after this transaction will be ["a" nil]:
              [{:foo "a"
                :bar false}]

              ; And somehow that causes this to fail:
              [{:foo "a"
                :bar false}]

              ; Upsert will work on the first time if we set :foo+bar explicitly.
              ; Also notice that :foo+bar is actually set to ["a" false] after this transaction.
              [{:foo "a"
                :bar false
                :foo+bar ["a" false]}]

              ; But on the second time, it'll create a new entity with :foo+bar set to ["a" nil]
              [{:foo "a"
                :bar false
                :foo+bar ["a" false]}]

              ; If we now try to do it again, changing the second entity to ["a" false] will fail
              ; because one already exists:
              [{:foo "a"
                :bar false
                :foo+bar ["a" false]}]]]
    (doseq [tx txes]
      (try
        (let [db (:db-after (d/transact conn {:tx-data tx}))]
          (println "transaction:")
          (pprint tx)
          (println "result:")
          (pprint
            (d/q '[:find (pull ?e [*]) :where [?e :foo]]
                 db))
          (println))
        (catch Exception e
          (println "transaction failed:" (.getMessage e))
          (println)))))
transaction:                                                                                                                                                                                                                         
[#:db{:ident :foo,
      :valueType :db.type/string,
      :cardinality :db.cardinality/one}
 #:db{:ident :bar,
      :valueType :db.type/boolean,
      :cardinality :db.cardinality/one}
 #:db{:ident :baz,
      :valueType :db.type/string,
      :cardinality :db.cardinality/one}
 #:db{:ident :foo+bar,
      :cardinality :db.cardinality/one,
      :unique :db.unique/identity,
      :valueType :db.type/tuple,
      :tupleAttrs [:foo :bar]}]
result:
[]

transaction:
[{:foo "a", :bar false}]
result:
[[{:db/id 53137197947158605, :foo "a", :bar false, :foo+bar ["a" nil]}]]

transaction failed: Unique conflict: :foo+bar, value: ["a" nil] already held by: 53137197947158605 asserted for: 69475940735909966

transaction:
[{:foo "a", :bar false, :foo+bar ["a" false]}]
result:
[[{:db/id 53137197947158605,
   :foo "a",
   :bar false,
   :foo+bar ["a" false]}]]

transaction:
[{:foo "a", :bar false, :foo+bar ["a" false]}]
result:
[[{:db/id 53137197947158605,
   :foo "a",
   :bar false,
   :foo+bar ["a" false]}]
 [{:db/id 19624083532546126, :foo "a", :bar false, :foo+bar ["a" nil]}]]

transaction failed: Unique conflict: :foo+bar, value: ["a" false] already held by: 53137197947158605 asserted for: 19624083532546126

So, in any case, it does seem that composite tuple handling of false values is badly broken and they (i.e., composite tuples with boolean values) shouldn’t be used until this is fixed.