معرفی شرکت ها
ecsdep-0.2.9
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
AWS ECS Deployment Tool With Terraform
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | ecsdep-0.2.9 |
| نام | ecsdep |
| نسخه کتابخانه | 0.2.9 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | Hans Roh |
| ایمیل نویسنده | hansroh@gmail.com |
| آدرس صفحه اصلی | https://gitlab.com/skitai/ecsdep |
| آدرس اینترنتی | https://pypi.org/project/ecsdep/ |
| مجوز | MIT |
## Introduce
ECS deploy using `docker compose` and `terraform`.
You need to manage just `yml` file for `docker compose` and:
```shell
ecsdep cluster create
ecsdep service up
```
That's all.
## Prequisition
### Gitlab Repository Read Credential
Create Gitlab access token in `Project Settings > Access Tokens`,
then check `read/write registry` grants.
With this token, create secret in AWS
https://console.aws.amazon.com/secretsmanager/
```json
{
"username" : "<gitlab username>",
"password" : "<access token>"
}
```
And save `secret arn`.
### Issuing Domain Certification
It need AWS domain certification for linking load balancer
### S3 Bucket For Terraform State Backend
Create bucket like `terraform.my-company.com`
### SSH Key Pairs
SSH key file, for generating key file,
```shell
ssh-keygen -t rsa -b 2048 -C "email@example.com" -f ./mykeypair
```
It makes 2 files. `mykeypair` is private key and `mykeypair.pub` is public key.
Please keep private key file carefully for accessing your EC2 instances.
### AWS Access Key For ECS Deploy
Create policy.
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"acm:*",
"application-autoscaling:*",
"autoscaling:*",
"cloudformation:*",
"cognito-identity:*",
"ec2:*",
"ecs:*",
"elasticloadbalancing:*",
"iam:*",
"kms:DescribeKey",
"kms:ListAliases",
"kms:ListKeys",
"logs:*",
"route53:*",
"s3:*",
"secretsmanager:*",
"servicediscovery:*",
"tag:GetResources"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
},
{
"Action": "iam:PassRole",
"Effect": "Allow",
"Resource": [
"*"
],
"Condition": {
"StringLike": {
"iam:PassedToService": "ecs-tasks.amazonaws.com"
}
}
},
{
"Action": "iam:PassRole",
"Effect": "Allow",
"Resource": [
"arn:aws:iam::*:role/ecsInstanceRole*"
],
"Condition": {
"StringLike": {
"iam:PassedToService": [
"ec2.amazonaws.com",
"ec2.amazonaws.com.cn"
]
}
}
},
{
"Action": "iam:PassRole",
"Effect": "Allow",
"Resource": [
"arn:aws:iam::*:role/ecsAutoscaleRole*"
],
"Condition": {
"StringLike": {
"iam:PassedToService": [
"application-autoscaling.amazonaws.com",
"application-autoscaling.amazonaws.com.cn"
]
}
}
}
]
}
```
Create user and save access key.
## Starting Deploy Docker
### Local Deploy
Docker contains `terrform`, `awscli` and `ecsdep`.
```shell
docker run -d --privileged \
--name ecsdep \
-v /path/to/myproject:/app \
hansroh/dep:dind
docker exec -it ecsdep bash
```
Within docker,
```shell
pip3 install -U ecsdep
```
### Gitlab CI/CD Deploy
Add these lines into `.gitlab-ci.yml`:
```yml
image: hansroh/dep:latest
services:
- name: docker:dind
alias: dind-service
before_script:
- pip3 install -U ecsdep
```
## Setup Deploy Environment
### Docker Login
```shell
docker login -u <username> -p <personal access token> registry.gitlab.com
```
### Confoguring AWS Access Key
First of all, configure AWS access key,
```shell
aws configure set aws_access_key_id <value>
aws configure set aws_secret_access_key <value>
aws configure set region ap-northeast-2
```
## Creating ECS Cluster
Create file with `docker-compose.yml` or `docker.ecs.yml`.
### Terraform Setting
```yaml
x-terraform:
provider: aws
region: ap-northeast-2
template-version: 1.1
state-backend:
region: "ap-northeast-2"
bucket: "states-data"
key-prefix: "terraform/ecs-cluster"
```
Make sure you create s3 bucket.
### Cluster Settings
```yaml
x-ecs-cluster:
name: my-cluster
public-key-file: "mykeypair.pub"
instance-type: t3.medium
ami: amzn2-ami-ecs-hvm-*-x86_64-*
autoscaling:
desired: 2
min: 2
max: 10
cpu: 60
memory: 60
target-capacity: 0
loadbalancer:
cert-name: mydoamin.com
vpc:
cidr_block: 10.0.0.0/16
```
It creates resources like:
- Load Balancer
- VPC with subnets: 10.0.10.0/24, 10.0.20.0/24 and 10.0.30.0/24
- Cluster Instance Auto Scaling Launch Configuration
- Cluster Auto Scaling Group
- Cluster
#### Cluster-Level Auto Scaling
- `autoscaling.cpu` mean that your cpu reservations of your containers
reaches 60% of all cluster instances's CPU units, scale out ECS instances.
If 0, nothing will be happen.
- `autoscaling.memory` is same.
- `autoscaling.target-capacity` try to keep new instances by percent of `desired`.
- value is 100: auto scaling group doesn't need to scale in or scale out
- value under 100: at least one instance that's not running a non-daemon task
#### Instance Type
If you specified `x-ecs-gpus`, `instance-type` must have GPU.
#### VPC
If you consider VPC peering, choose carefully `cidr_block`.
If you would like to create default VPC, just remove `vpc` key.
For VPC peering,
```yml
x-ecs-cluster:
vpc:
peering_vpc_ids:
- default # mean default VPC
- vpc-e2ecb79ef6da46
```
Note that this will be work only if your own VPCs.
#### Container AWS Resouce Access Policies
```yml
x-ecs-cluster:
task-iam-policies:
- arn:aws:iam::aws:policy/AmazonS3FullAccess
```
#### Creating/Destroying ECS Cluster
Finally you can create cluster,
`ecsdep` find default yaml named as `docker-compose.yml` or `compose.ecs.yml`.
```shell
ecsdep cluster create
ecsdep cluster destroy
```
Specifying file is also possible.
```shell
ecsdep -f /path/to/docker-compose.yml cluster create
ecsdep -f /path/to/docker-compose.yml cluster destroy
```
#### Creating S3 Bucket and Cognito Pool
```yml
x-ecs-cluster:
s3-cors-hosts:
- mayservice.com
```
## Configuring Containsers
This example launch 1 container named `skitai-app`.
```yaml
version: '3.3'
services:
skitai-app:
image: registry.gitlab.com/skitai/ecsdep
container_name: skitai-app
build:
context: .
dockerfile: ./Dockerfile
ports:
- 5000:5000
```
Make sure service name (skitai-app) is same as `container_name`.
And test image build and container.
```shell
docker-compose build
docker-compose up -d
docker-compose down
```
### Adding ECS Related Settings
#### Specify Deploy Containers
Add `services.skitai-app.deploy` key. Otherwise this container will not be included in ECS service.
```yaml
services:
skitai-app:
image: registry.gitlab.com/skitai/ecsdep
deploy:
```
#### Docker Container Registry Pull Credential
```yaml
services:
skitai-app:
image: registry.gitlab.com/skitai/ecsdep
x-ecs-pull-credentials: arn:aws:secretsmanager:ap-northeast-2:000000000:secret:gitlab/registry/mysecret-PrENMF
```
See `Prequisition` section.
#### Logging (Optaional)
For integrating Cloud Watch log group.
```yaml
services:
skitai-app:
logging:
x-ecs-driver: awslogs
```
#### Container Health Checking (Optaional)
```yaml
healthcheck:
test:
- "CMD-SHELL"
- "wget -O/dev/null -q http://localhost:5000/ping || exit 1"
```
#### Container Level Reosurce Requirements
For minimal memory reservation - called soft memory limit,
```yaml
services:
skitai-app:
deploy:
resources:
reservations:
memory: "256M"
```
For hard memory limit,
```yaml
deploy:
resources:
reservations:
memory: "256M"
limits:
memory: "320M"
```
Make sure hard limits must be greater or equal than reservation value.
For miniaml CPU units reservation,
```yaml
deploy:
resources:
reservations:
cpus: "1024"
```
1024 means 1 vCPU. ECS will deploys container which fulfills these reservation requirements.
Value of `cpus` must be string.
Maybe you are using GPU,
```yaml
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
```
But `ecsdep` ignores above statement. Just add `x-ecs-gpus`.
```yaml
deploy:
resources:
reservations:
x-ecs-gpus: 1
devices: ...
```
Ok, test docker build and container.
```shell
docker-compose build
```
### Configuring ECS Task Definition
```yaml
x-ecs-service:
name: ecsdep
stages:
default:
env-service-stage: "qa"
hosts: ["qa.myservice.com"]
listener-priority: 100
production:
env-service-stage: "production"
hosts: ["myservice.com"]
listener-priority: 101
autoscaling:
min: 3
max: 7
loadbalancing:
pathes:
- /*
protocol: http
healthcheck:
path: "/ping"
matcher: "200,301,302,404"
deploy:
compatibilities:
- ec2
resources:
limits:
memory: 256M
cpus: "1024"
autoscaling:
desired: 1
min: 1
max: 4
cpu: 100
memory: 80
strategy:
minimum_healthy_percent: 50
maximum_percent: 150
```
#### Staging
`stages` can make deploy stages like `production`, `qa` or `staging`.
It find your current environment variable named `SERVICE_STAGE`.
```shell
export SERVICE_STAGE=qa
```
Current deploy stages is selected by current `SERVICE_STAGE` value matched with `env-service-stage`.
If `SERVICE_STAGE` is `qa`, your container routed `qa.myservice.com` by load balancer.
#### Deployment Strategy
- `strategy.maximum_percent`: The upper limit (as a percentage of the service's desiredCount) of the number of running tasks that can be running in a service during a deployment
- `strategy.minimum_healthy_percent`: The lower limit (as a percentage of the service's desiredCount) of the number of running tasks that must remain running and healthy in a service during a deployment
#### Auto Scaling
Default container-level auto scaling settings are placed in `x-ecs-service.autoscaling`.
But each stages can overwrite this values by `x-ecs-service.[workspace name].autoscaling`.
#### Load Balancing (Optional)
Your container routed to `loadbalancing.pathes` by your load balancer.
##### Using Fargate
If you wnat `fargate` deploy, set `x-ecs-service.deploy.compatibilities` to `- fargate`
#### Resource Limiting
It is required when 'fargate' laubching, but it is optional for 'ec2' launching.
`x-ecs-service.deploy.autoscaling.cpu` and `x-ecs-service.deploy.autoscaling.memory`
are both percentages of reserved cpu units or memory MBytes.
If `x-ecs-service.deploy.resources.limits.cpus` is defined, service cannot use units
over this value.
If `x-ecs-service.deploy.resources.limits.memory` is defined and your container over
this value, container will be terminated.
#### Deploying Service
you need env var `CI_COMMIT_SHA`. It is used as image tag with first 8chars from git commit hash string. it will be provided on gitlab runner, but at local testing, `lateset` is just OK.
```shell
export CI_COMMIT_SHA=latest
export SERVICE_STAGE=qa
ecsdep -f dep/compose.ecs.yml service up
```
Whenever commanding `ecsdep service up`, your containers will be deployed to ECS by rolling update way.
As a results, AWS resources will be created.
- Task Definition
- Update Service and Run
*Note*: Sometimes, service-level auto-scaling settings are not applied at initial deploy.
I don't know why but please deploy twice in this case.
#### Shutdown/Remove Service
```shell
ecsdep service down
```
### Deploying Other Services Into Cluster
It is recommended cluster settings are kept in your main app only.
In other service's yml, keep `x-ecs-cluster.name` only and remove other `x-ecs-cluster` settings.
You just care about `x-ecs-service` and `services` definition.
```yaml
services:
skitai-app-2:
...
x-terrform:
...
x-ecs-cluster:
name: my-cluster
```
### Deploying Service With Multiple Containers
```yaml
services:
skitai-app:
deploy:
ports:
- 5000
healthcheck:
test:
- "CMD-SHELL"
- "wget -O/dev/null -q http://localhost:5000 || exit 1"
skitai-nginx:
depends_on:
- skitai-app
x-ecs-wait-conditions:
- HEALTHY
ports:
- "80:80"
```
Make sure only single loadbalancable container can have host port
mapping like "80:80", other wise just docker internal port like "5000".
### Deploying Non Web Service
Get rid of ports and load balancing settings like:
- `services.your-app.ports`
- `x-ecs-service.loadbalancing`
- `x-ecs-service.stages.default.hosts`
- `x-ecs-service.stages.default.listener-priority`
### Using Secrets
```yaml
version: '3.3'
services:
skitai-app:
environment:
DB_PASSWORD=$DB_PASSWORD
secrets:
DB_PASSWORD:
name: "arn:aws:secretsmanager:ap-northeast-1:0000000000:secret:gitlab/registry/hansroh-PrENMF:DBPASSWORD::"
external: true
```
At ECS deploy time, environment `DB_PASSWORD` will be overwritten with
`secrets.DB_PASSWORD.name` value by AWS ECS Service.
نیازمندی
| مقدار | نام |
|---|---|
| - | colorama |
نحوه نصب
نصب پکیج whl ecsdep-0.2.9:
pip install ecsdep-0.2.9.whl
نصب پکیج tar.gz ecsdep-0.2.9:
pip install ecsdep-0.2.9.tar.gz