Ion/get-params throws exception when returning more than 10 values

Datomic Cloud
1

We use ion/get-params to configure our Query Groups at runtime using values from our SSM Parameter Store, including which database to connect to based on the results of ion/get-env.

The AWS API returns pages of 10 Parameter Store values at a time, but unfortunately the ion/get-params function just throws an exception when passed a :path key that contains more than 10 values. So an otherwise healthy, running Ion application can be easily broken by adding an 11th parameter to any parameter path ready by a project.

For example, an Ion project could need these parameter values to operate:

  • /datomic-shared/prod/db-name <-- important for all queries
  • /datomic-shared/prod/key2
  • /datomic-shared/prod/key3
  • …
  • /datomic-shared/prod/key10

By adding an 11th parameter:

  • /datomic-shared/prod/key11

connections/clients/queries that rely on /datomic-shared/prod/db-name will fail, because ion/get-params will throw an exception.

Adding parameters to your AWS Parameter Store is seemingly “far” from Datomic and should normally be an innocuous event, but doing so can tank a healthy, production application. Sure, a team could have the disciple to organize their parameters so that path keys never reach more than 10 values, but that’s an important policy to remember and a harder one to enforce.

Is the best option just to avoid ion/get-params all together, and implement a custom solution which handles pagination?

1 Like
2

The get-params function is currently limited to 10 parameters, which the same as the GetParametersByPath API’s default MaxResults:

https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParametersByPath.html

MaxResults
The maximum number of items to return for this call. The call also returns a token that you can specify in a subsequent call to get the next set of results.

Type: Integer

Valid Range: Minimum value of 1. Maximum value of 10.

Required: No

get-params is a helper function. The AWS API has the ability to accept a NextToken for pagination of results, but we have not implemented that in our get-params helper function. However, you could write your own get-params using the GetParametersByPath API’s Next Token to return more than 10 params.

3

It looks like this topic was addressed in this release:
https://docs.datomic.com/cloud/releases.html#ion-dev-282

2 Likes