web-development/strapi-elb
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Moving documentation

Browse Source
This commit is contained in:
Daniel Tomlinson
2020-07-28 23:01:36 +01:00
parent 3b18a9193d
commit 88d0cd0755
7 changed files with 72 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

72
documentation/jq.md Normal file
View File

@@ -0,0 +1,72 @@
# JQ
## Piping into jq
You can `cat` or `bat` a file and pipe it into `jq`.
You can also take a command that returns json and pipe it into `jq`.
## Returning data without "
To return data without `jq` wrapping results in `"` use the `-r` flag.
`jq -r`
## Filtering
### Get values from a key
Running `aws --profile admin cloudformation describe-stack-resources --stack-name strapi-vpc | jq` returns:
```json
{
"StackResources": [
{
"StackName": "strapi-vpc",
"StackId": "arn:aws:cloudformation:eu-west-1:745437999005:stack/strapi-vpc/a9e41430-8afc-11ea-bdaa-0a736ea8438a",
"LogicalResourceId": "InternetGateway",
"PhysicalResourceId": "igw-0e059db8e0795ac32",
"ResourceType": "AWS::EC2::InternetGateway",
"Timestamp": "2020-04-30T16:07:42.434Z",
"ResourceStatus": "CREATE_COMPLETE",
"DriftInformation": {
"StackResourceDriftStatus": "NOT_CHECKED"
}
},
{
"StackName": "strapi-vpc",
"StackId": "arn:aws:cloudformation:eu-west-1:745437999005:stack/strapi-vpc/a9e41430-8afc-11ea-bdaa-0a736ea8438a",
"LogicalResourceId": "InternetGatewayAttachment",
"PhysicalResourceId": "strap-Inter-1413K0IDR1L3N",
"ResourceType": "AWS::EC2::VPCGatewayAttachment",
"Timestamp": "2020-04-30T16:08:00.147Z",
"ResourceStatus": "CREATE_COMPLETE",
"DriftInformation": {
"StackResourceDriftStatus": "NOT_CHECKED"
}
},
```
We can then use `jq`'s filtering to return values.
We have a key of `StackResources` which contains a list: `.StackResources[]`
We can then pass in the key we want `.StackResources[].PhysicalResourceId`
`aws --profile admin cloudformation describe-stack-resources --stack-name strapi-vpc | jq -r '.StackResources[].PhysicalResourceId'` which gives:
```json
"igw-0e059db8e0795ac32"
"strap-Inter-1413K0IDR1L3N"
"strap-Publi-1TS82BV8W4UFD"
"rtb-0cf8d05f71a30ef03"
"subnet-051fe56dc37d8396d"
"rtbassoc-0f7ae2fbdfe6bf2a5"
"subnet-0ea9f2f165a57be27"
"rtbassoc-00a67937c3778e273"
"subnet-09b28d722f41b2dde"
"rtbassoc-0a0a6bd0f8ff641df"
"vpc-029d232726cbf591d"
```
`aws --profile admin cloudformation describe-stack-resources --stack-name strapi-vpc | jq -r '.StackResources[] | .ResourceType + ": " + .PhysicalResourceId'`

42
documentation/running_notes.md Normal file
View File

@@ -0,0 +1,42 @@
# Notes
## HTTPS
### With load balancer
HTTPS can terminate at the load balancer
Load balancer to EC2 can be HTTP
From the front end all is well as the connection is secure.
When terminating at the load balancer 08-loadbalancer.config shows the option setting
<https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/configuring-https-elb.html>
## Database
Connecting an external DB: <https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.RDS.html>
Configure the auto scaling group to use an additional scaling group that allows ingress to the RDS instance.
You can configure the RDS credentials either with environment variables in the ELB config file, or use S3: <https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/rds-external-credentials.html>.
To create your own RDS instance you will need to create:
- A VPC - for the RDS
- Subnets - for the RDS
- A subnet group
- A security group
Use `aws ec2 describe-availability-zones --region eu-west-1 --profile admin` to get a list of availability zones for the region.
VPC terraform will create
- A IGW
- A route table
- A security group
## AWS Networking
- A VPC is a network that you give a CIDR block to.
- You create subnets for a VPC. These subnets will be split evenly across availability zones (for redundancy) and private/local (whether they have internet access or not).
- Behind the scenes (if using TF), internet gateway, routing tables, attachments will all be created for you. If using CF you will need to create these yourself.
- A security group is a firewall that is _attached to an EC2 instance_. A security group belongs to a VPC. You can permit instances to talk to each other by setting the source and destination to be the security group itself. You can control ports/ips exactly on an instance basis using security groups.

571
documentation/runningnotes.md Normal file
View File

@@ -0,0 +1,571 @@
<!-- vscode-markdown-toc -->
* [Running strapi in different modes](#Runningstrapiindifferentmodes)
* [Strapi documentation](#Strapidocumentation)
* [API Examples using HTTPIE](#APIExamplesusingHTTPIE)
* [Authenticate with the API](#AuthenticatewiththeAPI)
* [Get a Single Content Type](#GetaSingleContentType)
* [Use query parameters to filter for Multiple Content Type](#UsequeryparameterstofilterforMultipleContentType)
* [S3 Upload Addon](#S3UploadAddon)
* [AWS Resources](#AWSResources)
* [Configuration](#Configuration)
* [Fix Version Numbers](#FixVersionNumbers)
* [Strapi in git](#Strapiingit)
* [Cloudformation](#Cloudformation)
* [Output naming convention](#Outputnamingconvention)
* [Creating templates](#Creatingtemplates)
* [Adding resources](#Addingresources)
* [Using parameters](#Usingparameters)
* [Using outputs](#Usingoutputs)
* [Using functions](#Usingfunctions)
* [Examples](#Examples)
* [Short form](#Shortform)
* [Outputs](#Outputs)
* [Referencing other resources internally.](#Referencingotherresourcesinternally.)
* [Pesudeo references](#Pesudeoreferences)
* [Referencing other resources from external templates](#Referencingotherresourcesfromexternaltemplates)
* [Deploy a stack/template](#Deployastacktemplate)
* [Passing in parameters](#Passinginparameters)
* [Tags](#Tags)
* [Updating stack](#Updatingstack)
* [Failure](#Failure)
* [Stacks](#Stacks)
* [Snippets](#Snippets)
* [Deploy a template/stack](#Deployatemplatestack)
* [Destroy a stack](#Destroyastack)
* [Tags](#Tags-1)
* [Cloudformation default tags](#Cloudformationdefaulttags)
<!-- vscode-markdown-toc-config
numbering=false
autoSave=true
/vscode-markdown-toc-config -->
<!-- /vscode-markdown-toc -->
# Running notes
deocument that the db has to be done from cli arg, but the configs can be done via files.
SSL? https://levelup.gitconnected.com/beginners-guide-to-aws-beanstalk-using-node-js-d061bb4b8755
If doesnt work, try installing yarn in the ELB instance
Create seperate sql database + VPC rules:
http://blog.blackninjadojo.com/aws/elastic-beanstalk/2019/01/28/adding-a-database-to-your-rails-application-on-elastic-beanstalk-using-rds.html
Tie this in with a cloudformation template + hooking it up
/opt/elasticbeanstalk/node-install/node-v12.16.1-linux-x64/bin
Try setting the database name using cloudformation template
## <a name='Runningstrapiindifferentmodes'></a>Running strapi in different modes
You should use development for developing strapi and then deploy it to production.
If you run strapi in production, you cannot edit content types. See this git issue for the thread.
If you're running Strapi in a multiple instance you should:
- Run strapi locally in develop mode.
- Create content types.
- Build strapi in production.
- Push to ELB.
If you're running a single instance, you can alternatively just run it in develop mode in ELB.
Strapi stores its models locally on the instance and not on the database.
<https://github.com/strapi/strapi/issues/4798>
```text
This is not a bug and is intended, as the CTB (Content-Type builder) saves model configurations to files doing so in production would require Strapi to restart and thus could potentially knock your production API offline. Along with the previous reason, strapi is also very much pushed as a scale able application which would mean these changes would not be replicated across any clustered configurations.
There is no current plans to allow for this, as well as no plans to move these model definitions into the database. The enforcement of using the proper environment for the proper task (Production, Staging, and Development) is something that has been pushed from day 1.
Due to the reasons I explained above I am going to mark this as closed but please do feel free to discuss.
```
## <a name='Strapidocumentation'></a>Strapi documentation
<https://strapi.io/blog/api-documentation-plugin>
You can install the strapi documentation plugin by running: `npm run strapi install documentation`.
You can then access it through the Strapi Admin panel.
You should change the production URL server url in the documentation settings.
Edit the file `./extensions/documentation/documentation/1.0.0/full_documentation.json` and change `YOUR_PRODUCTION_SERVER` to the ELB URL of your environment.
## <a name='APIExamplesusingHTTPIE'></a>API Examples using HTTPIE
### <a name='AuthenticatewiththeAPI'></a>Authenticate with the API
`http http://strapi-prod.eu-west-1.elasticbeanstalk.com/auth/local identifier=apiuser password=password`
### <a name='GetaSingleContentType'></a>Get a Single Content Type
`http http://strapi-prod.eu-west-1.elasticbeanstalk.com/tests Authorization:"Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MiwiaWF0IjoxNTg3ODY3NzQ4LCJleHAiOjE1OTA0NTk3NDh9.McAi1b-F3IT2Mw90652AprEMtknJrW66Aw5FGMBOTj0"`
### <a name='UsequeryparameterstofilterforMultipleContentType'></a>Use query parameters to filter for Multiple Content Type
You can use query parameters to filter requests made to the API.
<https://strapi.io/documentation/3.0.0-beta.x/content-api/parameters.html#parameters>
The syntax is `?field_operator=value`, e.g `?title_contains=test`, after the endpoint URL for the content type.
`http "http://strapi-prod.eu-west-1.elasticbeanstalk.com/tests?title_contains=test" Authorization:"Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwiaXNBZG1pbiI6dHJ1ZSwiaWF0IjoxNTg3ODY3NzMwLCJleHAiOjE1OTA0NTk3MzB9.XXdoZUk_GuOION2KlpeWZ7qwXAoEq9vTlIeD2XTnJxY"`
## <a name='S3UploadAddon'></a>S3 Upload Addon
You should add the `strapi-provider-upload-aws-s3` extension using NPM. Make sure you add the same version of Strapi you are using.
`npm i strapi-provider-upload-aws-s3@3.0.0-beta.20`
### <a name='AWSResources'></a>AWS Resources
You should have an S3 bucket with public access, and an AWS account that has a policy to access the bucket.
### <a name='Configuration'></a>Configuration
You should create a settings file at `./extensions/upload/config/settings.json`.
This file defines an S3 object as in: <https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#constructor-property>.
You can use Strapi dynamic config files to set environment variables:
- provider
- providerOptions
- accessKeyId
- secretAccessKey
- region
- params
- Bucket
```json
{
"provider": "aws-s3",
"providerOptions": {
"accessKeyId": "${ process.env.STRAPI_S3_ACCESS_KEY || 'AKIA23D4RF6OZWGDKV7W' }",
"secretAccessKey": "${ process.env.STRAPI_S3_SECRET_KEY || '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' }",
"region": "${ process.env.STRAPI_S3_REGION || 'eu-west-1' }",
"params": {
"Bucket": "${ process.env.STRAPI_S3_BUCKET || 'elb-example-bucket' }"
}
}
}
```
Alternatively if you want to use different options for different environments, you can use a settings.js file instead.
<https://strapi.io/documentation/3.0.0-beta.x/plugins/upload.html#using-a-provider>
```javascript
if (process.env.NODE_ENV === "production") {
module.exports = {
provider: "aws-s3",
providerOptions: {
accessKeyId: process.env.STRAPI_S3_ACCESS_KEY,
secretAccessKey: process.env.STRAPI_S3_SECRET_KEY,
region: process.env.STRAPI_S3_REGION,
params: {
Bucket: process.env.STRAPI_S3_BUCKET,
},
},
};
} else {
module.exports = {};
}
```
## <a name='FixVersionNumbers'></a>Fix Version Numbers
When using Strapi you should make sure the version numbers for **all** dependencies in `./package.json` are fixed for Strapi modules. You cannot mix and match and upgrade arbitrarily.
An example is:
```json
{
"dependencies": {
"knex": "<0.20.0",
"pg": "^8.0.3",
"sqlite3": "latest",
"strapi": "3.0.0-beta.20",
"strapi-admin": "3.0.0-beta.20",
"strapi-connector-bookshelf": "3.0.0-beta.20",
"strapi-plugin-content-manager": "3.0.0-beta.20",
"strapi-plugin-content-type-builder": "3.0.0-beta.20",
"strapi-plugin-documentation": "3.0.0-beta.20",
"strapi-plugin-email": "3.0.0-beta.20",
"strapi-plugin-upload": "3.0.0-beta.20",
"strapi-plugin-users-permissions": "3.0.0-beta.20",
"strapi-provider-upload-aws-s3": "3.0.0-beta.20",
"strapi-utils": "3.0.0-beta.20"
}
}
```
## <a name='Strapiingit'></a>Strapi in git
To have a strapi project in github you should remove the:
- `./build`
- `./node_modules`
folders.
When cloning from the repo you should then do a:
- `NODE_ENV=development npm install`
- `NODE_ENV=development npm run build`
You can then run Strapi with `npm run develop` or `NODE_ENV=production npm run start`.
## <a name='Cloudformation'></a>Cloudformation
<https://adamtheautomator.com/aws-cli-cloudformation/> (example of deploying an S3 bucket with static site `index.html`.)
### <a name='Outputnamingconvention'></a>Output naming convention
You should follow a standard naming convention for your CF outputs.
For example:
```yaml
Outputs:
PublicVPCOutput:
Description: The VPC ID.
Value: !Ref PublicVPC
Export:
Name: !Sub "${AWS::StackName}-EBStrapiPublicVPC"
```
Defines a VPC. We can then pass in the stackname to another CF template and it can reference this VPC. The VPC names are static between projects (they don't have to be but here they are).
### <a name='Creatingtemplates'></a>Creating templates
To create a cloudformation template you should create a `template.yaml`. This yaml file should have at the top:
```yaml
AWSTemplateFormatVersion: 2010-09-09
Description: A simple CloudFormation template
```
Then you should add a `Resources` key and populate this with all the infrastructure you need to provision.
### <a name='Addingresources'></a>Adding resources
Documentation for all AWS resources is: <https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html>.
A good approach is to use the GUI to create an object, and then lookup the cloudformation template as you go along.
### <a name='Usingparameters'></a>Using parameters
<https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html>
You can use parameters in your templates. This allows you to use names/resources from other templates, or specify them at creation on the CLI.
To use a parameter you should create a `Parameters` section in the yaml on the same level as a `Resources`.
```yaml
Parameters:
InstanceTypeParameter:
Type: String
Default: t2.micro
AllowedValues:
- t2.micro
- m1.small
- m1.large
Description: Enter t2.micro, m1.small, or m1.large. Default is t2.micro.
```
### <a name='Usingoutputs'></a>Using outputs
<https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html>
### <a name='Usingfunctions'></a>Using functions
A list of all Cloudformation functions is: <https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html>.
`Fn::Select` will select a single object from a list of objects by index.
`Fn::GetAZs` returns an array that lists all availability zones for a specified region.
`!Ref` returns the value of the specified parameter or resource.
#### <a name='Examples'></a>Examples
##### Select, GetAZs and Ref
Example of `Fn::Select`, `Fn::GetAZs` and `!Ref`:
```yaml
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: !Ref "AWS::Region"
```
##### GetAtt
`Fn::GetAtt` differs from `Ref` in that `!GetAtt` gets an attribute of a resource, whereas `Ref` will reference the actual resource itself. An attribute is a return value of a resource. For example, a VPC resource has a `DefaultSecurityGroup` as an attribute that you can access.
To see attributes that you can reference with `!GetAtt`, you should check the Cloudformation documentation for the resource in question and look at the "Return Values" header: <https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html#aws-resource-ec2-vpc-return-values>.
An example would be using `Fn::GetAtt` to export a return value for some object in a template:
```yaml
Outputs:
PublicVPCIDDefaultSecurityGroup:
Description: The VPC ID.
Value: !GetAtt PublicVPC.DefaultSecurityGroup
Export:
Name: !Sub "${AWS::StackName}-PublicVPCIDDefaultSecurityGroup"
```
Long syntax: `Fn::GetAtt: [ logicalNameOfResource, attributeName ]`
##### Sub
A really good resource for Cloudformation functions is: <https://www.fischco.org/technica/2017/cloud-formation-sub/>.
Using `Fn::Sub` allows you to substitue a variable into the string you are trying to create. You might want to substitue an input parameter in for example.
```yaml
AppDnsRecord:
Type: AWS::Route53::RecordSet
Properties:
HostedZoneId: !ImportValue HostedZone-zone-id
Name:
Fn::Sub:
- "myapp.${HostedZoneName}"
- HostedZoneName: !ImportValue HostedZone-zone-name
```
Here we have referenced `${HostedZoneName}` - this is a temporary parameter in the sub command. At this point it does not exist, which is why we create a map which defines this variable as the second argument to Sub. In this example it is using `Fn::ImportValue` to import a resource from another Cloudformation stack.
As this second argument is a map (denoted by the `:`), we can have multiple key,value pairs.
```yaml
Name:
Fn::Sub:
- "myapp.${SubDomain}.${HostedZoneName}"
- HostedZoneName: !ImportValue HostedZone-zone-name
SubDomain: !ImportValue HostedZone-subzone-name
```
Note here that the second definition of the key, value pair does not have a leading `-`. We don't want to pass another argument to the `Sub` command, rather, we want to define additional key,value pair to be substituted in.
If our import value name also depended on an input parameter (say our imported value name depeneded on a stack name) we would have to use nested sub functions. In the above example we are simply importing a static import value, the string is hardcoded, if we wanted this to be dynamic, and be populated from an input parameter, then we can use:
```yaml
Parameters:
Route53StackName:
Type: String
Resources:
AppDnsRecord:
Type: AWS::Route53::RecordSet
Properties:
HostedZoneName:
Fn::ImportValue: !Sub "${Route53StackName}-zone-name"
Name:
Fn::Sub:
- "myapp.${ZoneName}"
- ZoneName:
Fn::ImportValue: !Sub "${Route53StackName}-zone-name"
```
Pay attention to the double indentation after `ZoneName`!.
We have to use the long form of `Fn::ImportValue` here and not the shorthand - this is a Cloudformation restriction.
#### <a name='Shortform'></a>Short form
If you are writing templates in yaml there is a long and shortform available.
An example for the `Sub` function:
- Longform `Fn::Sub: String`
- Shortform `!Sub String`
### <a name='Outputs'></a>Outputs
You can use the `Outputs:` header in your Cloudformation templates to specify outputs to be used in other Cloudformation templates.
<https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html>
```yaml
Outputs:
PublicVPCID:
Description: The VPC ID.
Value: !Ref PublicVPC
Export:
Name: ELBStrapiPublicVPC
```
`Value` returns the value of the property by an `aws cloudformation describe-stacks` command. The value can contain literals, parameter references, pseudo-parameters, mapping values or functions.
`Name` goes under `Export:` and is used for cross-stack reference. This name should be unique within a region. You can use this name in other Cloudformation templates to reference the `Value` you have specified above. You can set content in other cloudformation templates this way.
You can refer to these in ELB `./config` files for example - allowing you to dynamically link to other AWS resources in your ELB environment.
### <a name='Referencingotherresourcesinternally.'></a>Referencing other resources internally.
You can reference other resources in the template. This is useful say if you want to define a VPC and a subnet and reference the VPC from the subnet.
To do this you should use the `!Ref` function:
```yaml
VpcId: !Ref PublicVPC
```
Note that this is a special syntax, it doesn't have `Fn::` in the long form and the short form, `!Ref` is actually longer than the long form in this case.
<https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html>
#### <a name='Pesudeoreferences'></a>Pesudeo references
You can also reference certain AWS references: <https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html>.
Examples include `AWS::AccountId` and `AWS::StackName` among others.
### <a name='Referencingotherresourcesfromexternaltemplates'></a>Referencing other resources from external templates
Say we have a Cloudformation template where we have created a VPC:
```yaml
Outputs:
PublicSubnet0ID:
Description: The ID of the subnet.
Value: !Ref PublicSubnet0
Export:
Name: !Sub "${AWS::StackName}-PublicSubnet0"
```
We want to be able to use this, dynamically, in another template.
To do this we can use the `Fn::Sub` and `Fn::ImportValue` functions.
```yaml
Parameters:
StackName:
Description: The stack name of another CloudFormation template. This is used
to prepend the name of other resources in other templates.
Type: String
Resources:
RDSSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: A subnet group for the RDS instance.
SubnetIds:
- Fn::ImportValue: !Sub "${StackName}-PublicSubnet0"
- Fn::ImportValue: !Sub "${StackName}-PublicSubnet1"
```
### <a name='Deployastacktemplate'></a>Deploy a stack/template
To deploy, you should run the command: `aws cloudformation deploy --template-file template.yaml --stack-name static-website`
### <a name='Passinginparameters'></a>Passing in parameters
You can define parameters in its own section in a Cloudformation template:
```yaml
Parameters:
StackName:
Description: The stack name of another CloudFormation template. This is used
to prepend the name of other resources in other templates.
Type: String
```
You can set a default value which will be used if no value is passed in.
To pass values in using the CLI you should use the `--parameter-overrides` argument and pass them in as key=value pairs seperated by a space:
```bash
--parameter-overrides StackName=temp-vpc
```
### <a name='Tags'></a>Tags
When setting tags you can set them on individual resources in the Cloudformation template:
```yaml
Tags:
- Key: git
Value: web-dev
- Key: owner
Value: home
- Key: project
Value: strapi-elb
- Key: test
Value: true
- Key: deployment
Value: cloudformation
```
Alternatively if you have many tags to be shared across all resources you can set them when you use the CLI to deploy: `--tags git=web-dev owner=home project=strapi-elb test=true deployment=cloudformation`
### <a name='Updatingstack'></a>Updating stack
To update a stack you can use `deploy`. Note that the default behaviour is to create the new resources side by side, then once successful remove the old ones. You may run into errors when updating certain resources (updating a VPC subnet will fail as it has to create the new subnet alongside the existing one). You should remove the old stack by doing `delete-stack` first.
`aws cloudformation delete-stack --stack-name temp-vpc --profile admin`
### <a name='Failure'></a>Failure
If something goes wrong, you can use `describe-stack-events` and pass the `stack-name` to find the events leading up to the failure: `aws cloudformation describe-stack-events --stack-name strapi-s3`.
If this is the first time you are creating a stack you will not be able to re-deploy the stack. You must first delete the stack entirely and then re-deploy with any fixes.
You can delete a stack by running: `aws --profile admin cloudformation delete-stack --stack-name strapi-s3`.
### <a name='Stacks'></a>Stacks
<https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacks.html>
A cloudformation stack is a collection of AWS resources that you can manage as a single unit. You can group different resources under one stack then create, update or destroy everything under this stack.
Using stacks means AWS will treat all resources as a single unit. They must all be created or destroyed successfully to be created or deleted. If a resource cannot be created, Cloudformation will roll the stack back to the previous configuration and delete any interim resources that were created.
### <a name='Snippets'></a>Snippets
#### <a name='Deployatemplatestack'></a>Deploy a template/stack
`aws --profile admin cloudformation deploy --template-file ./01-stack-storage.yaml --stack-name strapi-s3`
You can pass paramter values in with `--paramter-overrides KEY=VALUE`:
`--parameter-overrides TestParameter="some test string"`
#### <a name='Destroyastack'></a>Destroy a stack
`aws --profile admin cloudformation delete-stack --stack-name strapi-s3`
## <a name='Tags-1'></a>Tags
Suggested tags for all AWS resources are:
| Tag | Description | Example |
| ----------- | ---------------------------------- | ------------------------ |
| git | git repo that contains the code | `web-dev` |
| owner | who the resource is for/owned | `home`, `work`, `elliot` |
| project | what project it belongs to | `strapi-elb` |
| test | flag for a temporary test resource | `true` |
| environment | environment resource belongs to | `dev`, `prod` |
| deployment | AWS tool used for deployment | `cloudformation`, `elb` |
### <a name='Cloudformationdefaulttags'></a>Cloudformation default tags
For Cloudformation resources the following tags get applied automatically:
| Tag | Description | Example |
| ----------------------------- | ------------------------------- | -------------------------------------------------------------------------------------------------- |
| aws:cloudformation:stack-name | stack-name of the resource | `strapi-s3` |
| aws:cloudformation:logical-id | resource name given in template | `ELBExampleBucket` |
| aws:cloudformation:stack-id | ARN of the cloudformation stack | arn:aws:cloudformation:eu-west-1:745437999005:stack/strapi-s3/459ebbf0-88aa-11ea-beac-02f0c9b42810 |

2
documentation/steps.todo Normal file
View File

@@ -0,0 +1,2 @@
Connecting external DB:
☐ Create RDS using TF @important @today

346
documentation/tempnotes.md Normal file
View File

@@ -0,0 +1,346 @@
<!-- vscode-markdown-toc -->
- [Decoupling](#Decoupling)
- [Creating Database + VPC + Subnets in Cloudformation](#CreatingDatabaseVPCSubnetsinCloudformation)
- [Single instance (no load balancer)](#Singleinstancenoloadbalancer)
_ [EC2::VPC](#EC2::VPC)
_ [Enable DNS](#EnableDNS)
_ [EC2::Subnet](#EC2::Subnet)
_ [EC2::InternetGateway](#EC2::InternetGateway)
_ [EC2::VPCGatewayAttachment](#EC2::VPCGatewayAttachment)
_ [AWS::EC2::RouteTable](#AWS::EC2::RouteTable)
_ [AWS::EC2::Route](#AWS::EC2::Route)
_ [AWS::EC2::SubnetRouteTableAssociation](#AWS::EC2::SubnetRouteTableAssociation)
- [Running notes](#Runningnotes) \* [Database](#Database)
- [Work Commands](#WorkCommands)
_ [tags](#tags)
_ [deploy](#deploy)
_ [delete](#delete)
_ [describe-stack-resources](#describe-stack-resources)
- [Adding SSL to ELB](#AddingSSLtoELB) \* [With load balancer](#Withloadbalancer)
- [EB Templates/Resources](#EBTemplatesResources)
- [Configuring security groups](#Configuringsecuritygroups)
- [Elastic Load Balancer](#ElasticLoadBalancer)
_ [Elastic Scaler](#ElasticScaler)
_ [RDS](#RDS) \* [Security group to allow EC2 instances to talk to each other](#SecuritygrouptoallowEC2instancestotalktoeachother)
- [Custom VPC + Subnets in EB](#CustomVPCSubnetsinEB)
- [Using cloudformation functions in EB config files](#UsingcloudformationfunctionsinEBconfigfiles)
- [Creating a read replica RDS](#CreatingareadreplicaRDS)
- [Multiple security groups on the same resource](#Multiplesecuritygroupsonthesameresource)
- [Private subnets](#Privatesubnets)
<!-- vscode-markdown-toc-config
numbering=false
autoSave=true
/vscode-markdown-toc-config -->
<!-- /vscode-markdown-toc -->
# Temp Notes
## <a name='Decoupling'></a>Decoupling
When creating an EB instance with `--single` and `--database` the following is created as part of the EB deployment:
- security group
- EIP
- RDS database
Is the security group created without a databse? (probably yes...)
## <a name='CreatingDatabaseVPCSubnetsinCloudformation'></a>Creating Database + VPC + Subnets in Cloudformation
Template from AWS showing cross-stack referencing and creating and referencing a VPC: <https://s3.amazonaws.com/cloudformation-examples/user-guide/cross-stack/SampleNetworkCrossStack.template>.
Export these in the CF template with stackname (<https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-importvalue.html>)
A security group is a resource that defines what IPs/Ports are allowed on inbound/outbound for an AWS resource. You can have one for EC2 instance, or RDS among others.
EB will create a VPC for your EC2 instances.
You should use this VPC for you RDS instance.
Creating a VPC for EB (with RDS) <https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/vpc-rds.html>
## <a name='Singleinstancenoloadbalancer'></a>Single instance (no load balancer)
Example cloudformation template that EB uses: <https://raw.githubusercontent.com/awslabs/elastic-beanstalk-samples/master/cfn-templates/vpc-public.yaml>.
Create a VPC - this is an object that spans all availability zones in a region. You assign a VPC a CIDR block. This is a set of IP addresses that this VPC has access to.
You should create public subnets inside this VPC - these subnets should cover all availablility zones in your region. The CIDR block you specified in the VPC defines all the ips, you should create N subnets that equally contain these IP addresses for your region.
For example a VPC in `eu-west-1` has a CIDR block of `172.31.0.0/16`.
There are 3 availablity zones in `eu-west-1`: `eu-west-1a`, `eu-west-1b` and `eu-west-1c`.
To find other availablity zones you should go to the EC2 Dashboard for the region you want to work in, and scroll down to the Service health header. Here, a list of all availability zones will be shown.
You should create subnets with the following:
| Availability Zone | Subnet CIDR | Real IP Range |
| ----------------- | -------------- | --------------------------- |
| `eu-west-1a` | 172.31.0.0/20 | 172.31.0.0 - 172.31.15.255 |
| `eu-west-1b` | 172.31.16.0/20 | 172.31.16.0 - 172.31.31.255 |
| `eu-west-1c` | 172.31.32.0/20 | 172.31.32.0 - 172.31.47.255 |
This covers all IP addresses across all availability zones in the VPC.
To make these subnets actually public, you should associate them with an internet gateway.
An internet gateway is an object that allows communication to the internet. In Cloudformation you should create an internet gateway and a VPC Gateway attachment. This attachment should reference the VPC you have created and reference the internet gateway object you create as well. Then, in your subnets (which are public) you can use `MapPublicIpOnLaunch: true` in the `Properties` block for each subnet.
You should then create a public route table and associate it with the VPC you have created.
You should then create a public route. You can then attach the internet gateway attachment to this route and specify a list of IPs that will go out to the internet. To allow all trafic to the internet set a `DestinationCidrBlock` of `0.0.0.0/0`.
### <a name='EC2::VPC'></a>EC2::VPC
#### <a name='EnableDNS'></a>Enable DNS
Enable `EnableDnsHostnames` + `EnableDnsSupport` - this allows resources in the VPC to use DNS in AWS.
### <a name='EC2::Subnet'></a>EC2::Subnet
Go to the EC2 dashboard to find all availability zones. Create a subnet for each zone.
- `AvailabilityZone`
- `VpcId`
- `CidrBlock`
- `MapPublicIpOnLaunch`
### <a name='EC2::InternetGateway'></a>EC2::InternetGateway
### <a name='EC2::VPCGatewayAttachment'></a>EC2::VPCGatewayAttachment
- `VpcId`
- `InternetGatewayId`
### <a name='AWS::EC2::RouteTable'></a>AWS::EC2::RouteTable
- `VpcId`
### <a name='AWS::EC2::Route'></a>AWS::EC2::Route
- `RouteTableId`
- `DestinationCidrBlock`
- `GatewayId`
### <a name='AWS::EC2::SubnetRouteTableAssociation'></a>AWS::EC2::SubnetRouteTableAssociation
- `SubnetId`
- `RouteTableId`
## <a name='Runningnotes'></a>Running notes
If we specify the VPC + Subnets from Cloudformation in a config file, will it create the security groups automatically for the EC2 instances? - Yes
Database can use existing subnets.
Database needs a security group creating
EC2 security groups automatically created and associated with the VPC.
Use aws:ec2:vpc (https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-general.html#command-options-general-ec2vpc)
### <a name='Database'></a>Database
Needs:
- `AWS::RDS::DBSubnetGroup`
- `AWS::EC2::SecurityGroupIngress`
- `AWS::RDS::DBInstance`
Default ports:
| Database Engine | Default Port |
| -------------------- | ------------ |
| Aurora/MySQL/MariaDB | 3306 |
| PostgreSQL | 5432 |
| Oracle | 1521 |
| SQL Server | 1433 |
| DynamoDB | 8000 |
## <a name='WorkCommands'></a>Work Commands
### <a name='tags'></a>tags
`--tags git=web-dev owner=home project=strapi-eb test=true deployment=cloudformation`
### <a name='deploy'></a>deploy
`aws --profile admin cloudformation deploy --template-file ./02-stack-vpc.yaml --stack-name strapi-vpc --tags git=web-dev owner=home project=strapi-eb test=true deployment=cloudformation`
`aws --profile admin cloudformation deploy --template-file ./03-stack-rdsinstance.yaml --stack-name strapi-rds --parameter-overrides StackName=strapi-vpc --tags git=web-dev owner=home project=strapi-eb test=true deployment=cloudformation`
### <a name='delete'></a>delete
`aws --profile admin cloudformation delete-stack --stack-name strapi-vpc`
`aws --profile admin cloudformation delete-stack --stack-name strapi-rds`
`aws --profile admin cloudformation delete-stack --stack-name temp`
List of all RDS Engines available under "Engine" header: <https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBInstance.html>.
### <a name='describe-stack-resources'></a>describe-stack-resources
Will print a json list of all resources in the stack
`aws --profile admin cloudformation describe-stack-resources --stack-name strapi-vpc`
Using `jq` for formatting:
`aws --profile admin cloudformation describe-stack-resources --stack-name strapi-vpc | jq -r '.StackResources[] | .ResourceType + ": " + .PhysicalResourceId'`
## <a name='AddingSSLtoELB'></a>Adding SSL to ELB
You should generate an SSL Certificate in Certificate Manager for your domain. To do this you will need to create a CNAME record to verify you have access to the DNS settings.
At the same time you should create a CNAME record that maps your subdomain (<strapi.panaetius.co.uk>) to the DNS name AWS has given your load balancer (<awseb-AWSEB-68CXGV0UTROU-1492520139.eu-west-1.elb.amazonaws.com>).
### <a name='Withloadbalancer'></a>With load balancer
A load balancer is not free! It costs ~£15 a month.
- Configure the load balancer listener in a EB `.config` file:
```yaml
option_settings:
aws:elbv2:listener:443:
Protocol: HTTPS
SSLCertificateArns: arn:aws:acm:eu-west-1:745437999005:certificate/218876af-7f8d-4022-97af-ad982aa540bc
```
## <a name='EBTemplatesResources'></a>EB Templates/Resources
Good repo for examples: <https://github.com/awsdocs/elastic-beanstalk-samples>
Creating a VPC for RDS in EB: <https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/vpc-rds.html>
CF RDS EB template: <https://github.com/garystafford/aws-rds-postgres/blob/master/cfn-templates/rds.template>
Decouple an exisitng RDS instance from ELB to RDS: <https://aws.amazon.com/premiumsupport/knowledge-center/decouple-rds-from-beanstalk/>
## <a name='Configuringsecuritygroups'></a>Configuring security groups
## <a name='ElasticLoadBalancer'></a>Elastic Load Balancer
Should set: inbound/outbound 80/443 on 0.0.0.0/0
The option_settings: aws:elbv2:loadbalancer has two options for security groups.
| Option | Description |
| -------------------- | --------------------------------------------------------------------- |
| ManagedSecurityGroup | Defines the security group that is used for the load balancer itself. |
| SecurityGroups | Is a list of additional security groups you want to attach. |
If you define a ManagedSecurityGroup you should set SecurityGroups as well to the same one.
Load balancer needs a security group that allows incoming 80 + 443 from anywhere
It should also set the same for outbound as well
This security group should be set in `aws:elbv2:loadbalancer` under
`ManagedSecurityGroup` and `SecurityGroups`
### <a name='ElasticScaler'></a>Elastic Scaler
Should set inbound 80/443 from LBSG.
EB will create a security group for the EC2 instances. In addition to this, you can create a new security group that will be applied to EC2 instances the elastic scaler creates.
This is set under `aws:autoscaling:launchconfiguration`.
### <a name='RDS'></a>RDS
Should set: inbound 5432 from Scaling SG + home ip (change port and home ip).
The database should have a security group creating that allows incoming connections from the EC2 instances only.
### <a name='SecuritygrouptoallowEC2instancestotalktoeachother'></a>Security group to allow EC2 instances to talk to each other
Security group rule to allow instances in the same security group to talk to one another: <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/security-group-rules-reference.html#sg-rules-other-instances>.
## <a name='CustomVPCSubnetsinEB'></a>Custom VPC + Subnets in EB
In a `.config` file specify the subnets for each tier of your app:
```yaml
option_settings:
aws:ec2:vpc:
VPCId: "vpc-003597eb63a0a3efe"
Subnets: "subnet-02cd8f7981ddfe345,subnet-02d9e1338e8d92d09,subnet-0e07d4d35394db524"
DBSubnets: "subnet-02cd8f7981ddfe345,subnet-02d9e1338e8d92d09,subnet-0e07d4d35394db524"
```
## <a name='UsingcloudformationfunctionsinEBconfigfiles'></a>Using cloudformation functions in EB config files
Only certain CF functions can be used in EB config files. For anything more advanced you should use Terraform to deploy additional resources alongside an EB template.
Reddit discussion on the topic: <https://www.reddit.com/r/aws/comments/a2uoae/is_there_a_way_to_reference_an_elastic_beanstalk/>.
EB documentaion on what functions are supported: <https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions-functions.html#ebextensions-functions-getatt>.
You cannot use `FN::ImportValue` to reference a resource in another Cloudformation stack.
You can use join for resources that EB creates itself: `!Join [ ":", [ !Ref "AWS::StackName", AccountVPC ] ]`.
## <a name='CreatingareadreplicaRDS'></a>Creating a read replica RDS
To have a replica database you should create a new DB instance with same AllocatedStorage size and DBInstanceClass. You should set the SourceDBInstanceIdentifier to be a `!Ref` of your primary DB. You should also set the SourceRegion.
Read replica CF docs: <https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-database-instance.html#cfn-rds-dbinstance-sourcedbinstanceidentifier>
## <a name='Multiplesecuritygroupsonthesameresource'></a>Multiple security groups on the same resource
Multiple security groups get squashed to determine what is and isn't allowed: <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security-groups.html>.
## <a name='Privatesubnets'></a>Private subnets
You can create private subnets that do not have an internet gateway attached to them. An example of CF template is <https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/cfn-templates/vpc-privatepublic.yaml>.
You need a nat gateway to allow private subnets to go out to the internet.
If you use private subnets, the nat gateway is not cheap - £30 a month.
You dont need the nat gateway, you can achieve the same thing with security groups (block all incoming) (explained <https://www.reddit.com/r/aws/comments/75bjei/private_subnets_nats_vs_simply_only_allowing/>).
An advantage to NAT is all outgoing requests to the internet come from a single IP.
## Using certbot CLI to generate SSL
### Wildcard certificate
In a new virtualenv install certbot:
```bash
pip install certbot
```
Run the `certbot` command:
```bash
sudo certbot certonly --manual --preferred-challenges=dns --email dtomlinson@panaetius.co.uk --server https://acme-v02.api.letsencrypt.org/directory --agree-tos -d "*.panaetius.co.uk"
```
Follow the instructions to add a `TXT` record to your DNS server for validation.
When finished you should see:
```markdown
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/panaetius.co.uk/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/panaetius.co.uk/privkey.pem
Your cert will expire on 2020-08-01. To obtain a new or tweaked
version of this certificate in the future, simply run certbot
again. To non-interactively renew _all_ of your certificates, run
"certbot renew"
- Your account credentials have been saved in your Certbot
configuration directory at /etc/letsencrypt. You should make a
secure backup of this folder now. This configuration directory will
also contain certificates and private keys obtained by Certbot so
making regular backups of this folder is ideal.
```
## Terraform
### Elastic Beanstalk
Editing the EB default resources in Terraform: <https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/issues/98#issuecomment-620677233>.

72
documentation/todo.md Normal file
View File

@@ -0,0 +1,72 @@
# To Do
## Immediate
Merge the CF templates into one, make sure all the importing and other snippets are documented.
- Create single instance deployment + https (document)
- For https: use letsencrypt to generate ssl, configure the eb config to use this.
- Final git branch for each version of the app (load balanced https/http, single http/https).
- Terraform it all up (excluding single + https).
## Long term
Use codebuild to update strapi
Use circle CI instead
Cloudformation template to deploy an S3 bucket
## Documentation
Summarise the flow -> VPC, internet gateway, attachment + route tables, subnets etc. Mention the nat gateway but show how it can be replaced with security groups. Document each individual resource needed bullet point and link to the git repo for the TF/CF templates.
## Running Notes
Various deployments:
- Single instance with EBCLI
- Load balanced with EBCLI
- Single instance with terraform
- Load balanced with terraform
HTTP + HTTPS
Single instance with terraform isn't possible with HTTPS - this is because you can't edit `Resources` or `Files` (and the other advanced EB configs). A workaround would be to create a docker image.
Single instance with EBCLI isn't possible with HTTPS if you're using Certificate Manager to generate the certificates - this is because you need to edit the nginx proxy config locally on the instance to allow https. You don't have access to the private certificate with Cerficiate Manager.
One solution would be to generate your SSL using letsencrypt - then configure the instance with this.
Another solution would be to use Docker and build a custom image. In this image you could install and configure nginx, (using lets encrypt as multistage build to get your certificate).
HTTPS for load balanced environment just requires pointing a domain to the EB endpoint. You can tell the load balancer to forward 443 in the security group without using it.
For final deployment - use an EC2 instance (deploy with TF).
### Other
Work:
Can we use APIGateway + Fargate to run an API containerised?
Fargate documentation: <https://aws.amazon.com/fargate/>.
Fargate option in ECS terraform: <https://www.terraform.io/docs/providers/aws/r/ecs_service.html#launch_type>.
Lambda vs Fargate differences: <https://www.learnaws.org/2019/09/14/deep-dive-aws-fargate/>.
Fargate vs EC2 pricing: <https://www.reddit.com/r/aws/comments/8reem9/fargate_t2small_cost_comparison_dollar_to_dollar/>.
Reddit thread on using API Gateway + Fargate: <https://www.reddit.com/r/aws/comments/bgqz4g/can_api_gateway_route_to_a_container_in_fargate/>.
Using API Gateway + Private endpoints (in a VPC): <https://aws.amazon.com/blogs/compute/introducing-amazon-api-gateway-private-endpoints/>.
Fargate is just running containers serverless - but it isn't a direct replacement to lambda. The spin up times can be long, but if you need to run a task on a schedule and this doesn't matter, you can save money and time as you don't need to manage and run an EC2 instance for docker containers. It's not ideal for tasks that need to be running 24/7.
Have a seperate repos for Terraform + Ansible. Split them inside by project. One central place for all TF and Ansible will make things easier to reference from later.
Generate SSH keys for EC2.
Provision EC2 using TF - set SG to allow SSH from your IP.
Configure EC2 with an Ansible playbook.
## Single options
- Dockerise it + run on EC2/ECS/Fargate
- Use EBCLI + Config options for https. Generate SSL using lets encrypt.
Using certbot with docker: <https://certbot.eff.org/docs/install.html#running-with-docker>
Forcing http > https redirection: <https://github.com/awsdocs/elastic-beanstalk-samples/tree/master/configuration-files/aws-provided/security-configuration/https-redirect/nodejs>.

28
documentation/updated.md Normal file
View File

@@ -0,0 +1,28 @@
Follow this tutorial to do python with asgi
Try with native python deployment + docker
<https://towardsdatascience.com/building-web-app-for-computer-vision-model-deploying-to-production-in-10-minutes-a-detailed-ec6ac52ec7e4>
Try with single instance - does it use the DB settings in .ebextensions?
Have documented options for
- Single instance
- Single instance with DB
- Load balanced instance
Create an RDS instance, ensure the default SG is allowed on ingress to the DB.
Use this SG to define an ebextensions file
<https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/rds-external-defaultvpc.html>
<https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/security-configuration/securitygroup-addexisting.config>
Using a custom VPC created yourself (how it's done now): <https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/vpc.html>
Allows complete control over the security settings.
Q? If we use `--single` it will only create:
Instance subnets – One of the public subnets
Instance security groups – Add the default security group
Will it ignore the loadbalancer + autoscaling settings even if we define them in 07.config?