Db pull and lookup refs issue

Datomic Cloud Troubleshooting
1

According to the docs lookup refs can be used in pull function

No expected result is returned if use pull with lookup refs, steps to reproduce:

(require '[datomic.client.api :as d])
(def cfg {:server-type :peer-server
:access-key "datomic"
:secret "datomic"
:endpoint "localhost:8998"
:validate-hostnames false})
(def client (d/client cfg))
(def conn (d/connect client {:db-name "database"}))

(def user-schema [
                   {
                       :db/ident :User/name
                       :db/valueType :db.type/string
                       :db/cardinality :db.cardinality/one
                       :db/unique :db.unique/value
                   }
                   ])
(d/transact conn {:tx-data user-schema})

(def user-1-name "User")
(def user-data [
  {
    :User/name user-1-name
  }
])
(d/transact conn {:tx-data user-data})

(def db (d/db conn))
(d/pull db '[*]' [:User/name user-1-name] )

The result is:

#:db{:id nil}

The same actions executed using peer API work well

Env:
Datomic 1.0.6202

1 Like
2

Hey @nikolayandr I reproduced this behavior and it looks like a bug to me. I am logging a story for us to evaluate a fix.

3

Actually, @nikolayandr I spent some more time looking at this and I don’t think it’s the bug I thought it was. You have a typo in your example adding a ' after the [*]. I created a gist here testing on various Datomic versions and confirmed that it works to pull without that extra '. I think we should probably behave a little better and throw a warning or something because I don’t think that syntax is actually valid. So I am going to follow up on that.

user=> (d/pull db '[*]' [:User/name user-1-name] )
#:db{:id nil}
user=> (d/pull db '[*] [:User/name user-1-name] )
{:db/id 17592186045418, :User/name "User"}
user=>

Gist work:

4

Thank you for helping, works for me

5

You have a typo in your example adding a ' after the [*]

But this ‘typo’ works when entity id is used to identify an entity :slight_smile:
I mean if instead of [:User/name user-1-name] to use the id of entity (d/pull db '[*]' 123324) then the expected result is returned

6

@jaret Could you please tell me, if the function db always calls the peer server to get the last available state of DB? Depending on your answer I still might have an issue with pulling data

7

Maybe my question should sound like the next:
How to get DB available on the peer server? Should connection always be created or is it enough to call db function?

If I call db function, can I be sure the last state of DB available on the peer server is returned by the function?

1 Like
8

I noticed the next behavior: calling of db function in forever cycle makes network communication with peer server only each ~2secs and I see basis t of connection is updated(in case newer DB is available on peer server) when network communication has happened

1 Like
9

Another interesting behavior: if to execute some function that requires network communication with peer server(e.g. pull) then basis t of connection is updated(in case newer DB is available on peer server)

10

Looks like the main idea is clear: to reduce network communication with the peer server but it leads to some issues with getting data from peer server. E.g. we do not know basis t of the last available DB on peer server but we know that the data is present, so we do pull

(def db (d/db conn)
(d/pull db someId pattern)

But since we can’t make sure (d/db conn) returns the newest DB from the peer server we need to do some workaround

(def db (d/db conn) # even peer server has db with basis-t e.g. 1002 this call returns db with basis t 1001
(d/pull db someId pattern) # this call returns nil but when this call is executed we see conn has updated with the newest basis-t so the next call returns necessary data
(d/pull db someId pattern)

I believe would be a good API for getting DB from connection would allow passing some boolean flag which allows forcing getting of the last available DB from the peer server