Skip to content

Commit

Permalink
fix style, improve readme, and typos in experiment descriptions
Browse files Browse the repository at this point in the history
  • Loading branch information
adhorn committed Jan 4, 2022
1 parent efbf53b commit a38382a
Show file tree
Hide file tree
Showing 17 changed files with 3,940 additions and 1,346 deletions.
29 changes: 20 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,29 +18,29 @@ To use AWS FIS, you set up and run experiments that help you create the real-wor
This CDK package will deplay a bunch of stacks.
(1) the parent stack `FISPa`, (2) a stack for the IAM roles `FisRole`, (3) a stack for the stop-condition `StopCond` (CloudWatch alarm), (4) a stack for each FIS experiment group (`EC2API`, `AsgExp`, `EksExp`, `NaclExp`, `Ec2InstExp`), and (5) a stack dedicated to uploading SSM documents `FisSsmDocs`.

You can pick and choose which experiment group you want to deploy by simply commenting out the respective stacks [here](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/parent-stack-ts)
You can pick and choose which experiment group you want to deploy by simply commenting out the respective stacks [here](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/parent-stack.ts)

## 1 - The [IAM roles](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-role) required to run the experiments:
- The AWS FIS role with all necessary policies as described [here](https://docs.aws.amazon.com/fis/latest/userguide/getting-started-iam-service-role.html)
- SSM Automation document role for [faults using SSMA](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/documents).
- SSM Automation document role for [faults using SSMA](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-upload-ssm-docs/documents).

## 2 - A set of [AWS FIS experiments](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-experiments) to get you started:

### [EC2 Instance faults](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-experiments/ec2-instance-faults)
- Including:
- Stopping and restarting (after duration) all EC2 instances in a VPC, an AZ, and with particular tags.
- Injecting CPU stress on random EC2 instances in a VPC
- Injecting latency on random EC2 instances all EC2 instances in a VPC, an AZ, and with particular tags.
- Injecting latency on requets to particular domain (e.g. www.amazon.com) to all EC2 instances in a VPC, an AZ, and with particular tags.

### [EC2 Control Plane faults](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-experiments/ec2-control-plane-faults)
- Including:
- Injecting EC2 API Internal Error on the target IAM role
- Injecting EC2 API Throttle Error on the target IAM role
- Injecting EC2 API Unavailable Error on the target IAM role
- Injecting EC2 API Internal Error on a target IAM role
- Injecting EC2 API Throttle Error on a target IAM role
- Injecting EC2 API Unavailable Error on a target IAM role

### [Auto Scaling Group faults](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-experiments/asg-faults)
- Including:
- Stopping and restarting (after duration) all EC2 instances of a random AZ in a particular auto scaling group.
- Terminate all EC2 instances of a random AZ in a particular auto scaling group.
- Injecting CPU stress on All EC2 instances of a particular auto scaling group.

### [Network Access Control List faults](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-experiments/nacl-faults)
Expand All @@ -55,7 +55,7 @@ You can pick and choose which experiment group you want to deploy by simply comm
- Including:
- Changing a particular security group ingress rule (open SSH to 0.0.0.0/0) to verify remediation automation or monitoring. (Courtesy of Jonathan Rudge). Possible remediation automation (https://github.com/adhorn/ssh-restricted)

### [Iam Access faults](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-experiments/iam-accesss-faults)
### [Iam Access faults](https://github.com/adhorn/aws-fis-templates-cdk/tree/main/lib/fis-experiments/iam-access-faults)
- Including:
- Denying Access to an S3 Resource from any application/services by targeting its Iam Role. (Courtesy of Rudolph Wagner)

Expand Down Expand Up @@ -115,8 +115,19 @@ Finally, you can deploy these FIS experiments using the CDK as follows:

```bash
npm install
cdk deploy
cdk bootstrap
cdk deploy --all
```
During the creation of the different stacks, some will generate a security warning as follow:

````
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)
Do you wish to deploy these changes (y/n)?
````

Select `y` (yes).


### Other useful CDK commands:

Expand Down
13 changes: 5 additions & 8 deletions bin/aws-fis-cdk-templates.ts
Original file line number Diff line number Diff line change
@@ -1,21 +1,18 @@
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { FIS } from '../lib/parent-stack';
import "source-map-support/register";
import * as cdk from "aws-cdk-lib";
import { FIS } from "../lib/parent-stack";

const app = new cdk.App();
new FIS(app, 'FISPa', {
new FIS(app, "FISPa", {
/* If you don't specify 'env', this stack will be environment-agnostic.
* Account/Region-dependent features and context lookups will not work,
* but a single synthesized template can be deployed anywhere. */

/* Uncomment the next line to specialize this stack for the AWS Account
* and Region that are implied by the current CLI configuration. */
// env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },

/* Uncomment the next line if you know exactly what Account and Region you
* want to deploy the stack to. */
// env: { account: '123456789012', region: 'us-east-1' },

/* For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html */
});
});
4 changes: 2 additions & 2 deletions cdk.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"asg_name": "Test-FIS-ASG",
"eks_cluster_name": "test-cluster-chaos",
"security_group_id": "sg-022eb488dbd1655b3",
"target_role_name": "myrole",
"s3-bucket-to-deny": "mybucket/*"
"target_role_name": "SSM-Chaos",
"s3-bucket-to-deny": "chaos-ssm-documents/*"
}
}
168 changes: 89 additions & 79 deletions lib/fis-experiments/asg-faults/experiments-stack.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { StackProps, Stack } from 'aws-cdk-lib';
import { aws_fis as fis } from 'aws-cdk-lib';
import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import { StackProps, Stack } from "aws-cdk-lib";
import { aws_fis as fis } from "aws-cdk-lib";

export class AsgExperiments extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);

// Import FIS Role and Stop Condition
const importedFISRoleArn = cdk.Fn.importValue('FISIamRoleArn');
const importedStopConditionArn = cdk.Fn.importValue('StopConditionArn');
const importedFISRoleArn = cdk.Fn.importValue("FISIamRoleArn");
const importedStopConditionArn = cdk.Fn.importValue("StopConditionArn");

// Variables you may want to change based on your environment
// const asgName = new cdk.CfnParameter(this, 'asgName', {
Expand All @@ -20,111 +20,121 @@ export class AsgExperiments extends Stack {
//console.log('asgName: ', asgName.valueAsString);

// if asg_name parameter is in cdk.json us the below
const asgName = this.node.tryGetContext('asg_name');
const asgName = this.node.tryGetContext("asg_name");
const availabilityZones = Stack.of(this).availabilityZones;
const randomAvailabilityZone = availabilityZones[Math.floor(Math.random() * availabilityZones.length)];
const randomAvailabilityZone =
availabilityZones[Math.floor(Math.random() * availabilityZones.length)];
// const randomAvailabilityZone = 'us-east-1a'


// Targets
const TargetAllInstancesASG: fis.CfnExperimentTemplate.ExperimentTemplateTargetProperty = {
resourceType: 'aws:ec2:instance',
selectionMode: 'ALL',
resourceTags: {
'aws:autoscaling:groupName': asgName.toString()
},
filters: [
{
path: 'State.Name',
values: ['running']
}
]
}

const TargetAllInstancesASGAZ: fis.CfnExperimentTemplate.ExperimentTemplateTargetProperty = {
resourceType: 'aws:ec2:instance',
selectionMode: 'ALL',
resourceTags: {
'aws:autoscaling:groupName': asgName.toString()
},
filters: [
{
path: 'State.Name',
values: ['running']
const TargetAllInstancesASG: fis.CfnExperimentTemplate.ExperimentTemplateTargetProperty =
{
resourceType: "aws:ec2:instance",
selectionMode: "ALL",
resourceTags: {
"aws:autoscaling:groupName": asgName.toString(),
},
{
path: 'Placement.AvailabilityZone',
values: [randomAvailabilityZone]
filters: [
{
path: "State.Name",
values: ["running"],
},
],
};

const TargetAllInstancesASGAZ: fis.CfnExperimentTemplate.ExperimentTemplateTargetProperty =
{
resourceType: "aws:ec2:instance",
selectionMode: "ALL",
resourceTags: {
"aws:autoscaling:groupName": asgName.toString(),
},
]
}
filters: [
{
path: "State.Name",
values: ["running"],
},
{
path: "Placement.AvailabilityZone",
values: [randomAvailabilityZone],
},
],
};

// Actions
const stopInstanceAction: fis.CfnExperimentTemplate.ExperimentTemplateActionProperty = {
actionId: 'aws:ec2:terminate-instances',
parameters: {},
targets: {
Instances: 'instanceTargets'
}
}
const terminateInstanceAction: fis.CfnExperimentTemplate.ExperimentTemplateActionProperty =
{
actionId: "aws:ec2:terminate-instances",
parameters: {},
targets: {
Instances: "instanceTargets",
},
};

const cpuStressAction = {
actionId: 'aws:ssm:send-command',
description: 'CPU stress via SSM',
actionId: "aws:ssm:send-command",
description: "CPU stress via SSM",
parameters: {
documentArn: `arn:aws:ssm:${this.region}::document/AWSFIS-Run-CPU-Stress`,
documentParameters: JSON.stringify(
{
DurationSeconds: '120',
InstallDependencies: 'True',
CPU: '0'
}
),
duration: 'PT2M'
documentParameters: JSON.stringify({
DurationSeconds: "120",
InstallDependencies: "True",
CPU: "0",
}),
duration: "PT2M",
},
targets: { Instances: 'instanceTargets' }
}
targets: { Instances: "instanceTargets" },
};

// Experiments
const templateStopStartInstance = new fis.CfnExperimentTemplate(this, 'fis-template-stop-instances-in-asg-az',
const templateTerminateInstanceASGAZ = new fis.CfnExperimentTemplate(
this,
"fis-template-stop-instances-in-asg-az",
{
description: 'Stop and restart all instances of ASG in particular AZ',
description: "Terminate all instances of ASG in random AZ",
roleArn: importedFISRoleArn.toString(),
stopConditions: [{
source: 'aws:cloudwatch:alarm',
value: importedStopConditionArn.toString()
}],
stopConditions: [
{
source: "aws:cloudwatch:alarm",
value: importedStopConditionArn.toString(),
},
],
tags: {
Name: 'FIS Experiment',
Stackname: this.stackName
Name: "Terminate instances of ASG in random AZ",
Stackname: this.stackName,
},
actions: {
'instanceActions': stopInstanceAction
instanceActions: terminateInstanceAction,
},
targets: {
'instanceTargets': TargetAllInstancesASGAZ
}
instanceTargets: TargetAllInstancesASGAZ,
},
}
);

const templateCPUStress = new fis.CfnExperimentTemplate(this, 'fis-template-CPU-stress-random-instances-in-vpc',
const templateCPUStress = new fis.CfnExperimentTemplate(
this,
"fis-template-CPU-stress-random-instances-in-vpc",
{
description: 'Runs CPU stress on all instances of an ASG using the stress-ng tool. Uses the AWS FIS provided document - AWSFIS-Run-CPU-Stress',
description:
"Runs CPU stress on all instances of an ASG using the stress-ng tool. Uses the AWS FIS provided document - AWSFIS-Run-CPU-Stress",
roleArn: importedFISRoleArn.toString(),
stopConditions: [{
source: 'aws:cloudwatch:alarm',
value: importedStopConditionArn.toString()
}],
stopConditions: [
{
source: "aws:cloudwatch:alarm",
value: importedStopConditionArn.toString(),
},
],
tags: {
Name: 'FIS Experiment',
Stackname: this.stackName
Name: "CPU Stress to all instances of ASG",
Stackname: this.stackName,
},
actions: {
'instanceActions': cpuStressAction
instanceActions: cpuStressAction,
},
targets: {
'instanceTargets': TargetAllInstancesASG
}
instanceTargets: TargetAllInstancesASG,
},
}
);
}
Expand Down
Loading

0 comments on commit a38382a

Please sign in to comment.