Everyone starts with one AWS account, but it is a sensible approach for a larger organization to split out operations into the use of multiple, even many accounts, all linked to one main billing account. Numerous benefits result from isolating functions or teams to their own accounts, and in some cases it is simply necessary. For example, Amazon isn't willing to raise some limits on usage, such as SES sending limits, but has no problem with a single organization opening multiple accounts to carry out the necessary activity. Also, beyond a few thousand instances or a few hundred of many other types of resource the AWS console starts to become sluggish to the point of making everyday usage a challenge. It is perfectly possible to reinvent the necessary wheels by building internal applications that run against the relevant APIs, but why take the time when you could just run ten different accounts? Last, but by no means least, splitting up company functions into multiple accounts limits the damage that can be result from a security breach, and allows for a solid separation of concerns for sensitive information like that related to billing.

Given potentially scores of accounts, how to manage monitoring and security-related tasks like granting and revoking access, however? This is a particular issue when it comes to purchase and use of reserved instances, since they are a billing device rather than a technical item. Reserved instances purchased in one of many linked accounts will be used in any of those accounts, so obtaining a useful picture requires access to all accounts. That would be an exceedingly tedious job to carry out manually, so some kind of tool is needed. That tool will need to run in one account and gather information via the AWS EC2 APIs from all of the other accounts. The naive approach to grant access would be to create an IAM user with appropriate permissions in each account, generate access keys, and deploy the access keys with the application. This is moderately terrible for all the obvious reasons: the use of keys should be minimized, as they represent a considerable security burden. Storing them safely such that they are unlikely to be accidentally published to a repository, or end up common knowledge, or otherwise become a potential threat, is not a trivial undertaking in any organization. Management of secrets is hard - better never to have the secrets that need managing in the first place.

The right way to go about this is through the use of IAM roles and policies, as is the case for pretty much everything that involves granting permissions to deployed applications in AWS. For an application that carries out its operations entirely within one account, this is simple. Create the role, add an appropriate policy, and assign the role to the instances when they launch. This is usually all done within a CloudFormation template, though common or larger roles tend to be managed elsewhere, referenced by ARN both for convenience and to reduce the size of templates. For an application that must carry out API operations for multiple accounts, while running in one account, a different role-based strategy is needed, however. An approach is outlined below, with examples to illustrate the steps.

Step 1: Create a Role in Each Account

Let us say that we are setting up an example application that counts running instances by type in all of the accounts for a company. That requires permission for the ec2:DescribeInstances action, so create a CrossAccountInstanceCounter role in each account and attach the following trivial policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "ec2:DescribeInstances",
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

Step 2: Allow the Application Account to Grant Rights to the CrossAccountInstanceCounter Roles

Each of the CrossAccountInstanceCounter roles must now be made accessible from the account in which the application is running, say, 111222333444. In the IAM console, open the Trust Relationships tab for each role and set the following policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::111222333444:root"
        ]
      }
    }
  ]
} 

Step 3: Allow the Application to Assume the CrossAccountInstanceCounter Role

The instance that the application is running on in account 111222333444 must be associated with a role and policy that allow it to use the sts:AssumeRole action on the CrossAccountInstanceCounter role in each of the other company accounts. That will look like this, where the wildcard is in the account ID position:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Resource": [
         "arn:aws:iam::*:role/CrossAccountInstanceCounter"
      ]
    }
  ]
} 

Step 4: Assume Roles in the Application

The following Javascript illustrates how things might work in an application accessing multiple accounts. An STS client with default permissions derived from instance metadata is used to obtain credentials associated with the CrossAccountInstanceCounter role in another account. An EC2 client is then created using these credentials, at which point the application can use the ec2.DescribeInstances action to list instances in that account.

// Core.
var util = require('util');
// NPM.
var AWS = require('aws-sdk');

/**
 * Run one or more describeInstances requests to get a count of instances.
 *
 * @param {AWS.EC2} ec2Client A client with suitable credentials.
 * @param {String} instanceType The instance type, e.g. 't2.medium'.
 * @param {Function} callback Of the form function (error, count).
 */
function describeInstances (ec2Client, instanceType, callback) {
  var instanceCount = 0;

  /**
   * Page through responses from the describeInstances endpoint for so long as 
   * there is a NextToken value.
   */
  function recurse (nextToken) {
    var params = {
      Filters: [
        {
          Name: 'instance-type',
          Values: [
            instanceType
          ]
        },
        {
          Name: 'instance-state-name',
          // Only count running instances.
          Values: [
            'running'
          ]
        }
      ]
    };

    if (nextToken) {
      params.NextToken = nextToken;
    }

    ec2Client.describeInstances(params, function(error, response) {
      if (error) {
        return callback(error);
      }

      _.each(response.Reservations, function (reservation) {
        instanceCount += reservation.Instances.length;
      });

      if (response.NextToken) {
        recurse(response.NextToken)
      }
      else {
        callback(null, instanceCount);
      }
    });
  }

  recurse();
}

/**
 * Obtain the running instance count for a particular type from another account.
 * 
 * @param {String} accountId The account to access, e.g. '555666777888'.
 * @param {String} instanceType The instance type, e.g. 't2.medium'.
 * @param {Function} callback Of the form function (error, count).
 */
function obtainInstanceCount (accountId, instanceType, callback) {
  // This client uses the default configuration read from the environment or 
  // instance metadata.
  var stsClient = new AWS.STS();

  // The role credentials will be good for the DurationSeconds value, so either
  // request new credentials for every usage, as here, or the better approach is
  // to cache the credentials or client instances until they expire.
  stsClient.assumeRole(
    {
      RoleArn: util.format(
        'arn:aws:iam::%s:role/CrossAccountInstanceCounter',
        accountId
      ),
      // Arbitrary string, used to keep track of which applications are making
      // use of the cross account role.
      RoleSessionName: 'instance-counter',
      DurationSeconds: 3600
    },
    function (error, response) {
      if (error) {
        return callback(error);
      }

      var ec2Client = new AWS.EC2({
        accessKeyId: response.Credentials.AccessKeyId,
        secretAccessKey: response.Credentials.SecretAccessKey,
        sessionToken: response.Credentials.SessionToken
      });

      describeInstances(ec2Client, instanceType, callback);
    }
);