Dev and test locally with dev-local

With Datomic dev-local you can develop and test applications with minimal connectivity and setup. Get the dev-local library, add it to your classpath, and you have access to the full Client API (both synchronous and asynchronous flavors). This allows you to:

Please give dev-local a try and let us know what you think!
The Datomic Team

5 Likes

Let’s suppose that I a developer want to develop and test a FOSS library like datomock or hyperfiddle

This license allow me to setup a CI system for my library?

If yes, how to I download this zip/jar into my CI system/container?

If not, there is plans from cognitect team to support this developers? Or there is no interested in libraries around datomic-cloud?

Hi souenzzo,

Supporting CI is one of the key objectives of dev-local. So yes, you can setup CI with it. How you do that is up to you. You could copy the dev-local library directly onto a CI box, or into a script that dynamically builds an ephemeral cloud CI box, or into a private maven repository used by a CI box, or into a private git repository, etc. As a long-time consumer of maven repositories, I personally prefer a private maven repository, as that helps me have audit control over all my dependencies. But YMMV.

Does this clarify things?

2 Likes

This is amazing. Changing from cloud to dev-local sped up some tests by two orders of magnitude.

Just a note about the configuration, I had to add com.cognitect/hmac-authn {:mvn/version "0.1.195"} to my project’s dependencies. Maybe it should be declared as a dependency of com.datomic/dev-local in an accompanying pom.xml file or perhaps the docs could make it more clear that it is needed.

Anyway, big thanks to the datomic team.

Hi Datomic Team,

this is great news, both for testing and development. Until now we used internal library similar to datomic-client-memdb. Now I am looking to move to standard dev-local solution.

I have read documentation page about dev-local but I have not found anything related to custom Clojure functions. Does dev-local support Clojure functions (as a a transaction function, query function or entity and attribute predicate)?

Thank you.

Hi @rafael.ferreira

Could you share a trace.edn file for your project? We’d like to understand if the exclusion in dev-local is affecting something that needs hmac-authn.

clj -Strace

Feel free to e-mail this to support@cognitect.com as sharing here on the forum might not be the best place for this sort of thing.

Thanks,
Jaret

Hi @skardan, Yes. dev-local runs in your process and you can call any function that is in your process classpath. This is analogous to locally testing an ion before you deploy your ion project.

https://docs.datomic.com/cloud/ions/ions-reference.html#develop

Thanks,
Jaret

Hi @jaret, I can’t reproduce the issue anymore. I think the only thing that changed in my setup was that I ran clj with -Sforce on the projects, but I can’t be sure. Now everything works fine without depending on com.cognitect/hmac-authn.

Hi @rafael.ferreira and anyone else who encounters an issue with hmac-authn after including dev-local in your project. We have addressed this issue with a new version of clj. Please upgrade with:

brew upgrade clojure/tools/clojure

Verify you’re on 1.10.1.561 with clojure -Sdescribe .

Cheers,
Jaret

2 Likes

Is there a more up-to-date guide on how to setup a maven repo in a git repo?

I found this article on the topic, but it seems a bit outdated: https://cemerick.com/2010/08/24/hosting-maven-repos-on-github/

Currently I’ve resorted to using a small installer script, which pulls the dev-local jar from S3:

#!/usr/bin/env bash
cd "$(dirname "$0")" || exit 1

DEV_LOCAL_VERSION="0.9.183"

aws s3 cp "s3://<company-bucket>/datomic/dev-local/dev-local-${DEV_LOCAL_VERSION}.jar" .

mvn org.apache.maven.plugins:maven-install-plugin:3.0.0-M1:install-file \
  -Dfile="dev-local-${DEV_LOCAL_VERSION}.jar"

but then I also need a script, which uploads the new versions:

DEV_LOCAL_VERSION="0.9.183"; aws s3 cp ~/Downloads/"datomic-dev-local-${DEV_LOCAL_VERSION}/dev-local-${DEV_LOCAL_VERSION}.jar" "s3://<company-bucket>/datomic/dev-local/dev-local-${DEV_LOCAL_VERSION}.jar"

The need for that extra maven plugin is also a nuisance. That command doesn’t exactly roll off the tongue and it also causes quite a lot of extra dependencies to download :slight_smile:

What’s the recommended way to use in-memory databases - for automated tests - via the Datomic Client API?

Is that something I could/should do with dev-local?

So far I was using the Peer API for that purpose, but if I want to transition to using the Client API, then I can’t use that approach anymore, can I?

I’m aware of the https://github.com/ComputeSoftware/datomic-client-memdb library, which has been deprecated in favour of dev-local recently, but I don’t see how dev-local serves the in-memory use-case.

According to https://docs.datomic.com/on-prem/peer-server.html, we can start a peer-server, which exposes an in-memory db, but it doesn’t allow my automated test suite to isolate my tests into different, independent in-memory DBs.

I also haven’t found any official way to directly start a temporary peer-server from my application’s process, without shelling out.

I did look into what happens when I bin/run -m datomic.peer-server ..., so I could start a peer-server, but I have no idea how to shut it down.

I’ve added these to deps.edn (where CURRENT is a link to datomic-pro-1.0.6165/):

 :aliases
 {:datomic-peer
  {:extra-paths
   ["CURRENT/resources"
    "CURRENT/datomic-transactor-pro-1.0.6165.jar"
    "CURRENT/lib/*"
    "CURRENT/bin"]}

then I could do the following:

;; Start a Peer Server for convenience
(comment
  ;; Based on
  ;; https://docs.datomic.com/on-prem/getting-started/connect-to-a-database.html
  ;;   bin/run -m peer-server -h localhost -p 8998 \
  ;;           -a myaccesskey,mysecret -d hello,datomic:mem://hello

  (require '[datomic.peer-server :as peer-server]
           '[clojure.tools.cli :as cli])

  @(def peer-server-config
     (-> ["-h" "localhost"
          "-p" "8998"
          "-a" "myaccesskey,mysecret"
          "-d" "hello,datomic:mem://hello"]
         (cli/parse-opts datomic.peer-server/cli-options)))

  @(def peer-server
     (datomic.peer-server/create (:options peer-server-config)))

  )

Hi @onetom,

For on-prem Datomic projects you should continue to use :mem protocol and the tools available in on-prem. Dev-local is intended for use with Datomic Cloud projects.

Thanks,
Jaret

1 Like

Hi all. My company has started using dev-local since the first release and it has been excellent. I cannot thank the Datomic team enough for working on this :heart:

In hopes that it might help others, I have written a short blog post on how we’ve been using dev-local (new library: dev-local-tu). It also answers @onetom’s question of how to get dev-local into a S3 Maven repo (another new lib: s3-mvn-upload).

Curious to hear feedback on this approach or approaches others have taken.

1 Like