API Gateway internal server error


#1

This kind of looks like a cold start issue, since it only happens after I make a request against API gateway routes after not using them for a little while. The first request always returns an internal server error to me.

From the associated lambda logs, I see this error:

java.io.IOException: Connection reset by peer: datomic.ion.lambda.handler.exceptions.Fault
datomic.ion.lambda.handler.exceptions.Fault: java.io.IOException: Connection reset by peer
at datomic.ion.lambda.handler$throw_anomaly.invokeStatic(handler.clj:24)
at datomic.ion.lambda.handler$throw_anomaly.invoke(handler.clj:20)
at datomic.ion.lambda.handler.Handler.on_anomaly(handler.clj:139)
at datomic.ion.lambda.handler.Handler.handle_request(handler.clj:155)
at datomic.ion.lambda.handler$fn__4075$G__4011__4080.invoke(handler.clj:70)
at datomic.ion.lambda.handler$fn__4075$G__4010__4086.invoke(handler.clj:70)
at clojure.lang.Var.invoke(Var.java:396)
at datomic.ion.lambda.handler.Thunk.handleRequest(Thunk.java:35)

It takes about 5 seconds for the error to happen. The lambda timeout is set to 3 minutes, and the API Gateway timeout is the default.

Subsequent requests (with the exact same request parameters) work without issue. This also doesn’t seem to be 100% reproducible, since sometimes I get a response, with the ~5 second pause before the response arrives.


#2

I have also hit this exception, so I opened a ticket with the Datomic team. Their response was: retry the request. I added retry to my code and haven’t hit the exception yet. IMO, the exception should be fixed rather than worked around.


#3

I’m unsure what I could even retry in this case. I just have API Gateway pointing at the lambda, and my code literally returns a constant value.