Db pull and lookup refs issue

According to the docs lookup refs can be used in pull function

No expected result is returned if use pull with lookup refs, steps to reproduce:

(require '[datomic.client.api :as d])
(def cfg {:server-type :peer-server
:access-key "datomic"
:secret "datomic"
:endpoint "localhost:8998"
:validate-hostnames false})
(def client (d/client cfg))
(def conn (d/connect client {:db-name "database"}))

(def user-schema [
                   {
                       :db/ident :User/name
                       :db/valueType :db.type/string
                       :db/cardinality :db.cardinality/one
                       :db/unique :db.unique/value
                   }
                   ])
(d/transact conn {:tx-data user-schema})

(def user-1-name "User")
(def user-data [
  {
    :User/name user-1-name
  }
])
(d/transact conn {:tx-data user-data})

(def db (d/db conn))
(d/pull db '[*]' [:User/name user-1-name] )

The result is:

#:db{:id nil}

The same actions executed using peer API work well

Env:
Datomic 1.0.6202

1 Like

Hey @nikolayandr I reproduced this behavior and it looks like a bug to me. I am logging a story for us to evaluate a fix.

Actually, @nikolayandr I spent some more time looking at this and I don’t think it’s the bug I thought it was. You have a typo in your example adding a ' after the [*]. I created a gist here testing on various Datomic versions and confirmed that it works to pull without that extra '. I think we should probably behave a little better and throw a warning or something because I don’t think that syntax is actually valid. So I am going to follow up on that.

user=> (d/pull db '[*]' [:User/name user-1-name] )
#:db{:id nil}
user=> (d/pull db '[*] [:User/name user-1-name] )
{:db/id 17592186045418, :User/name "User"}
user=>

Gist work:

Thank you for helping, works for me

You have a typo in your example adding a ' after the [*]

But this ‘typo’ works when entity id is used to identify an entity :slight_smile:
I mean if instead of [:User/name user-1-name] to use the id of entity (d/pull db '[*]' 123324) then the expected result is returned

@jaret Could you please tell me, if the function db always calls the peer server to get the last available state of DB? Depending on your answer I still might have an issue with pulling data

Maybe my question should sound like the next:
How to get DB available on the peer server? Should connection always be created or is it enough to call db function?

If I call db function, can I be sure the last state of DB available on the peer server is returned by the function?

1 Like

I noticed the next behavior: calling of db function in forever cycle makes network communication with peer server only each ~2secs and I see basis t of connection is updated(in case newer DB is available on peer server) when network communication has happened

1 Like

Another interesting behavior: if to execute some function that requires network communication with peer server(e.g. pull) then basis t of connection is updated(in case newer DB is available on peer server)

Looks like the main idea is clear: to reduce network communication with the peer server but it leads to some issues with getting data from peer server. E.g. we do not know basis t of the last available DB on peer server but we know that the data is present, so we do pull

(def db (d/db conn)
(d/pull db someId pattern)

But since we can’t make sure (d/db conn) returns the newest DB from the peer server we need to do some workaround

(def db (d/db conn) # even peer server has db with basis-t e.g. 1002 this call returns db with basis t 1001
(d/pull db someId pattern) # this call returns nil but when this call is executed we see conn has updated with the newest basis-t so the next call returns necessary data
(d/pull db someId pattern)

I believe would be a good API for getting DB from connection would allow passing some boolean flag which allows forcing getting of the last available DB from the peer server