Skip to content

Conversation

@lewayne-aws
Copy link

Issue #, if available:
n/a

Description of changes:
If ECS starts before credentials-fetcher, the agent fails to connect to the credentials-fetcher socket. So, when installing credentials-fetcher.service, add a drop-in configuration for ecs.service, so that credentials-fetcher.service effectively becomes a component of ecs.service.

With this change, if ecs.service is started, stopped, or restarted, credentials-fetcher.service will also do the same, and ecs.service will wait for the socket file to be created when starting credentials-fetcher.service.

Since ecs.service becomes so tightly coupled with credentials-fetcher.service, we don't have to explicitly enable it for it to start on systems where the ECS agent is installed. Instead, ecs.service triggers the service to start with it as a dependency.

If this is installed on a system where ECS agent is not installed, there are no side effects. Although there would be a drop-in file for ecs.service, without the underlying unit file also present (provided by ECS agent), the drop-in will be ignored by systemd. Similarly, this approach does not clobber any of the files provided by ECS agent directly, and simply leverages existing systemd functionality to enforce our intent and requirement.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Startup order is important for this service to integrate with the ECS
agent. To account for this, create a systemd drop-in file for
`ecs.service` so that it depends on `credentials-fetcher.service` as
long as it is installed.

The drop-in also makes the agent wait for the credentials-fetcher socket
to be created.
/usr/bin/systemctl stop credentials-fetcher.service
/usr/bin/systemctl is-enabled --quiet ecs.service 2>/dev/null && /usr/bin/systemctl restart ecs.service || :
fi

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add to the changelog as well

@heathhey
Copy link

After a review of the pull request, I think if you want to use systemd to enforce ordering of startup and restart etc. then this is the best solution. I have not personally setup or used this solution, so I recommend testing this setup. with this configuration, if the credentials-fetcher service dies, or is stopped or is restarted, then the ecs service will follow suit, this is what the BindsTo option does.

I would highly recommend testing this thoroughly. Doing something like, making sure both packages are installed, and that the service is running, then force a restart or a stop and stare on credentials-fetcher and verifying that the ecs service follows along. Also, a test with a custom AMI with both installed to verify that it does solve the race condition and the both services start properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants