After spending more time looking into the issue I (finally) figured out it the root cause of the problem.
I set the AWS credentials manually in the terminal session via a resource file. That allowed the aws s3 sp command to work. But when starting the clojure REPL, it does not use the terminal session environment variable, but different ones set in an old, non-obvious resource file in my root folder.
After removing out the overriding environment variables, every worked.