Message Listeners in Ions


#1

If I have an SQS message listener that I need to run for async processing, is there any way I can start this from within the nodes on a compute group, or do I always have to invoke a lambda from the outside to do such a thing?

I understand that it may not be a good idea to run asynchronous processes inside my compute group, but I’d like to get the benefit of “effective in-memory access to datoms” for the purposes of processing these messages.


#2

You can certainly have a background thread that runs on your Datomic node(s) and handles async processing.

See Kafka Consumer as an Ion?

The main point I should make here is that your process should be coordinated via ion calls, but you should not make individual ion calls that are long-lived (i.e. don’t make run-my-whole-import-job a lambda-invoked ion).


#3

Good to know. That still leaves me wondering how to ensure that the message listener is started after every redeploy.

I’m still not completely familiar with the internals of ions, so the crux of my question is really where in the code I should put the bootstrap of such listeners. Normally, I’d do something like this in an application’s “main” entry point and bootstrap the entire system (à la component, mount, integrant, etc.), but I couldn’t figure out how to declare such an entry point with an ion deployment.

Triggering something from the outside for each member of the ASG seems like it would be needlessly complicated.