Rotating AWS access key

Datomic Pro Troubleshooting
1

Hello,
We are rotating the AWS access keys for all the users in our AWS account. I’ve created a new key for the user account whose access key id/secret key we have configured in our peer application servers, and I updated our peer app server configs to use the new key.

I still have both the old key and the new key active, and we’re trying to find and eliminate all usages of the old key, so we can safely delete it.

Based on the Last used message in the IAM console, I’m seeing activity using the new key, like accessing dynamo db, so I think the peers working with the new key.

However, we are still seeing some activity from the old access key - something is accessing S3 about every 15-30 minutes (not sure the exact interval) - e.g. a Last used message like: 2019-07-15 22:15 MDT with s3 in us-east-1.

I’m wondering if it’s the datomic transactors that are still accessing S3 for logging? Would they be using a stored access key/secret key to access S3? I’ve scoured AWS for any hints as to what’s using that key, but haven’t found anything. We are setup using a CloudFormation template, running against dynamodb, and I believe we are using the role-based configuration.

I’m not sure what information is relevant to paste to help anyone troubleshoot this. In our CloudFormation template, I see this line:

"AWS_ACCESS_KEY_ID=\"${DATOMIC_READ_DEPLOY_ACCESS_KEY_ID}\" AWS_SECRET_ACCESS_KEY=\"${DATOMIC_READ_DEPLOY_AWS_SECRET_KEY}\" aws s3 cp \"s3:\/\/${DATOMIC_DEPLOY_BUCKET}\/${DATOMIC_VERSION}\/startup.sh\" startup.sh",

but I’m not sure where these DATOMIC_READ_DEPLOY_ACCESS_KEY_ID/DATOMIC_READ_DEPLOY_AWS_SECRET_KEY env vars might be coming from - is there a config bundle that it’s accessing for those?

If the transactor is using an access key+secret key, what command do I use to update it? ensure-transactor? ensure-cf? create-cf-template and update the CF stack? something else?

Any help would be appreciated!

2

The AWS keys you identified in the CFT directly are only used internally by the AMI to fetch necessary resources to run Datomic, but they should not correspond to any keys you previously used from within your account.

To be sure you’ve replaced any AWS keys used by the system, you can run ensure-cf and create-cf-template again with your new AWS keys in your local environment then update your CF stack with the newly generated CFT.