aws-is-how

• aws-is-how
  • 常见故障排除及支持手册
  • AWS Skill builder
  • freeCodeCamp
  • AI/ML
    • ML Study
    • SageMaker
    • Jupyter Notebooks
    • Compute vision
    • ChatGPT and AIGC
    • NLP
    • Translate and Audio
    • Forecasting
    • Fraud Detection
    • Recommandation
    • Labeling
    • Federated ML
    • Prediction Maintenance
    • ML Hardware
  • Cost
    • Cost Explorer
    • Network cost
    • Tagging
    • Sustainablity
  • Computing
    • EC2
    • Load Balancer
    • System Manager
    • HPC
  • Analytics
    • High Level Data Engineering and Data Analytics
    • Data integration service: Glue
    • Analysis: EMR
    • Stream - Flink and Spark Streaming
    • Stream - Kinesis
    • Stream - Kafka
    • Ad-hoc and Interactive query: Athena
    • Data Warehouse: Redshfit
    • Search and analytics: Elasticsearch Service
    • Governance
    • BI
    • Delta Lake
  • IOT
    • IoT Core
    • IoT Timeseries
    • OEE
    • IoT anaytics
    • Edge
    • OTA
    • AIOT
  • Security
    • Encryption - KMS
    • Credential - Secret Manager
    • Certificate - Certificate Manager
    • Asset Management and Compliance
    • AuthN and AuthZ
    • Sentitive Data
    • Threat detection - GuardDuty
    • WAF
    • Permission - IAM Policy, S3 Policy, RAM Policy
    • Multi accounts structure
    • SIEM and SOC
    • Vulnerability Assessment - Inspector and Alternative
  • Network
    • VPC
    • Keep private - VPC Endpoint and PrivateLink
    • NAT and proxy
    • Load balancers
    • Cross data center and cloud Leasing Line - Direct Connect and VPN
    • Cross board transfer
    • Cross accounts and Cross VPCs - TGW
    • Acceleration network
    • Edge
    • Network Secuirty
  • DNS
    • Route 53
    • HTTPDNS
  • Serverless
    • Serverless Workshop
    • Function as Service - Lambda
    • API Gateway
    • Step function
    • Build the serverless - SAM, Chalice, Serverless framwork, CDK
    • Serverless with AI/ML
  • Migration
    • Journey to Adopt Cloud-Native Architecture
    • Active Directory
    • Database
    • Data migration tool - DMS
    • Data migration tool - 3rd party tool
    • Cross Cloud Migration
    • File migration
  • Storage
    • S3 cross region or cross cloud OSS
    • S3
    • EBS
    • Storage Gatewway
    • EFS and FSx or other shared file system
  • Database
    • RDS
      • RDS usage
      • RDS Cross region, cross account, data replication and backup
      • RDS upgrade
      • RDS Security
      • RDS Performance
    • Graph Database
    • ElastiCache
    • Key-Value and Document
      • DynamoDB
      • MongoDB and DocumentDB
    • Time series
  • Container
    • EKS
      • EKS networking
      • EKS practice
      • Data On EKS
      • DevOps on EKS
    • ECS
    • Fargate
    • Istio, Envoy, App Mesh, Service discovery
    • ECR
  • DevOps
    • Management
    • CI/CD
      • Serverless CICD
      • Container CICD
    • Monitoring and Tracing
    • Logging
    • Change configuration
    • Developer
    • Infra as Code
  • Integration
    • Quque, notification
    • Call Center
    • MQ
    • Email
  • Media
    • Video on Demand
    • Video Streaming
  • Mobile
    • Moible app development
    • GraphQL - AppSync
  • Business continuity
    • Backup
    • DR
      • RDS HA/DR
    • Resilience
  • Game
    • GameLift
  • SAP
    • HA/DR
  • Office and business application
    • Workspaces - VDI
  • Metaverse
  • Automotive

常见故障排除及支持手册

AWS Skill builder

freeCodeCamp

AI/ML

ML Study

ML入门的知识，以及ML项目中的一些经验总结分享

SageMaker

• SageMaker-Workshop

• SageMaker Learning Series

• Install External Libraries and Kernels in SageMaker Notebook Instances

• CloudFormation to launch SageMaker Notebook on Glue Dev Endpoint

• Invoke SageMaker Notebook via Event

  • Lambda-Trigger-SageMaker-Notebook
  • Scheduling Jupyter notebooks on SageMaker ephemeral instances

• SageMaker input mode: pipe mode and file mode

• Save costs by automatically shutting down idle resources within Amazon SageMaker Studio

• SageMaker Neo supported devices edge devices

Jupyter Notebooks

• A gallery of interesting Jupyter Notebooks

• Set up a Jupyter Notebook Server on deep learning AMI

Compute vision

• Use SageMaker for Automotive Image Classification

• ML Bot Workshop

• IP Camera AI SaaS Solution

• image classification using resnet

• Open CV on Lambda

• Train and deploy OCR model on SageMaker

• Scale YOLOv5 inference with Amazon SageMaker endpoints and AWS Lambda

• PaddleOCR的轻量级车牌识别

• 新希望-构建云上智慧牧场

• Bedrock 大语言模型加速 OCR 场景精准提取

• How to improve user engagement with real-time AR effects using BytePlus Effects and Amazon IVS

• PaddleOCR

• PaddleOCR on SageMaker

ChatGPT and AIGC

• [GenAI Overview]

  • 一文读懂AIGC
    • 跨模态深度学习模型CLIP（Contrastive Language-Image Pre-Training）
    • “对抗生成网络”GAN（Generative Adverserial Network）
    • Diffusion模型
  • Token, Embeding, Self-Attention, Transformer, Vector, Encoding output 101
  • ChatGPT Overview
  • AIGC workshop
  • 三类场景赋能行业创新
  • A guide to making your AI vision a reality

• [Stable Diffsusion]

  • stable-diffusion-webui self hosted on g4dn.xlarge with Ubuntu 22.04 LTS
    • Remember run sudo apt-get update before sudo apt install wget git python3 python3-venv.
    • Run ssh -L 7862:localhost:7862 [email protected] or bash stable-diffusion-webui/webui.sh --share
    • Install Nvida Cuda
    • create-your-own-stable-diffusion-ui-on-aws
    • sagemaker-stablediffusion-quick-kit
    • Stabule Diffusion on EKS
    • Stable Diffusion Quick Kit 动手实践 – 基础篇
    • Stable Diffusion Extention hosting on AWS
    • SageMaker Notebook 机器学习服务轻松托管 Stable Diffusion WebUI
    • Stable Diffusion on Amazon SageMaker Workshop
    • inpaint-images-with-stable-diffusion-using-amazon-sagemaker-jumpstart

• [Industry focus]

  • 基于 Amazon SageMaker 使用 Grounded-SAM 加速电商广告素材生成
  • AIGC 助力电商虚拟试穿新体验
  • Towards General Purpose Virtual Try-on
  • SageMaker LMI+Streaming 构建 端到端GenAI Text2Image应用
  • IPC GenAI 应用场景与方案概述
  • 使用 Bedrock Agent 实现发票查询知识库和开发票

• [Vector database]

  • RDS for PostgreSQL now supports pgvector for simplified ML model integration

• [LLM]

  • LangChain for LLM Application Development
  • Baichuan on Sagemaker
  • ChatGLM on SageMaker
  • ChatYuan on SageMaker
  • ColossalAI for LLM quick training
  • 基于亚马逊云科技 AI 服务打造多模态智能化内容审核
  • Reduce container startup time on Amazon EKS with Bottlerocket data volume
  • Patterns for Building Generative AI Applications on Amazon Bedrock
  • 使用 Amazon SageMaker 和 Bedrock 构建营销场景端到端应用
  • 构建端到端生成式 AI 应用
  • Amazon Bedrock Workshop
  • Amazon Bedrock Claude3 Workshop

• [RAG - retrieval-augmented generation. It solves inherent knowledge limitations of FMs by integrating with data that are not part of the model’s training]

  • 基于智能搜索的大语言模型增强方案
  • 基于智能搜索的大语言模型增强方案2
  • 基于智能搜索和大模型打造企业下一代知识库
  • 基于智能搜索和大模型打造企业下一代知识库 之 制造/金融/教育/医疗行业实战场景
  • 基于 RDS 和 Confluence 数据源构建端到端的RAG
  • 基于大语言模型和推荐系统构建电商智能导购机器人
  • 基于大语言模型知识问答应用落地实践 – 知识召回调优
  • 基于LLM 和 Amazon Opensearch 或 Amazon Kendra 打造企业私有知识库
  • 基于Amazon Open Search+大语言模型的智能问答系统
  • AI Powered Chatbot
  • Building a serverless document chat with AWS Lambda and Amazon Bedrock
  • streaming response from Amazon Bedrock with FastAPI on AWS Lambda
  • GenAI Data Foundation Workshop - Healthcare RAG chatbot
  • 使用 Bedrock Agent 实现发票 Agent

• [Promote-Engineering]

  • FlagEmbedding - retrieval, classification, clustering, or semantic search. And it also can be used in vector databases for LLMs

• [Text2SQL]

  • DB-GPT与百川社区强强联手，无缝支持百川模型推理与Text2SQL微调
  • text2sql_gen_demo notebook

• [Video and Audio]

  • Video summarization
  • 构建文生音场景定制化人声解决方案

• [Code Generation]

  • Amazon Q & CodeWhisperer for VS Code
  • Claude3 code assistant

NLP

• NLP and Text Classification by using blazing text
• Use AWS SageMaker BlazingText to process un-balance data for text multiple classification The git repo
• Chinese-BERT

Translate and Audio

• 使用 Amazon Translate 自动翻译PPT

Forecasting

• Forecasting scalar (one-dimensional) time series data
• GluonTS for time series data

Fraud Detection

• SageMaker End-to-End Demo- Fraud Detection for Auto Claims and github repo

Recommandation

• Amazon Personalize workshop

• Amazon Personalize Get Start

Labeling

• 使用 Amazon SageMaker Ground Truth 标记 3D 点云 and guide
• [CV Labeling]
  • cvat-on-aws-china
  • CV Labeling: VOTT

Federated ML

• Amazon Redshift ML: Create, train, and deploy machine learning (ML) models using familiar SQL commands

Prediction Maintenance

Using AWS IoT and Amazon SageMaker to do IoT Devices Predictive Maintenance

IoT Time-series Forecasting for Predictive Maintenance

ML Hardware

• Hands-on Deep Learning Inference with Amazon EC2 Inf1 Instance

Cost

Cost Explorer

• Simple generate a report using the AWS Cost Explorer API

• Cost and Usage Report analysis

• Price Calculator

• Price List API

• Get the spot instance price

• Cloud Intelligence Dashboards (https://github.com/aws-samples/aws-cudos-framework-deployment)

• Cloud Intelligence Dashboards in China

Network cost

• AWS China region network cost details

• Networking calculator

• How to calculate data transfor cost on AWS

• Understanding AWS Direct Connect multi-account pricing

• Data transfer costs for common architectures

Tagging

• [Tagging when instance and object created]

  • Automatically Tag AWS EC2 Instances and Volumes Similar solution
  • S3 Object Tagging 以及怎么管理Tagging
  • 用S3 Batch Operation 批量对已有对象打标签
  • 写程序基于我们的SDK去打Tag
  • 基于S3 upload Event 触发lambda去对新的对象打标签

• Tagging AWS resources

Sustainablity

• Customer Carbon Footprint Tool

Computing

EC2

• [Linux Daily usage]

  • How to connect to Linux EC2 via NICE DCV Client
  • Amazon Linux how to support chinese
  • Upgrade-C4-CentOS-instance-to-C5-instance

• [Performance]

  • 4 vCPU 实例达成 100 万 JSON API 请求/秒的优化实践

• [Windows Daily usage]

  • How to connect to Windows EC2 via NICE DCV Client
  • Amazon EC2 針對 Windows Server 2012/R2 EOL

• [GPU Daily usage]

  • How to build Graphics Workstation on Amazon EC2 G4 Instances
  • Deploying Unreal Engine Pixel Streaming Server on EC2

• [Network of EC2]

  • Python code attach EC2 EIP
  • EC2 network performance
  • How to get the IP address under my account
  • How can I connect to my Amazon EC2 instance if I lost my SSH key pair after its initial launch
  • Keep EC2 primary private IP for a 'new' instance

• [Graviton]

  • Graviton 2 workshop (https://github.com/aws-samples/graviton2-workshop)
  • AWS Workshop - Graviton2 China
  • porting-advisor-for-graviton
    • sample blog
  • AWS Graviton2-based services
  • 3rd party
  • container
  • GRAVITON2 电商独立站
  • Porting Advisor for Graviton

• [Operation]

  • What does :-1 mean in python
  • Change EC2 Time-Zone
  • How can I set up a CloudWatch alarm to automatically recover my EC2 instance?
  • Move EC2 instance to other AZ
  • Best practices for handling EC2 Spot Instance interruptions
  • How to share the EC2 AMI
  • Copy AMI from global to China
  • How to handle EC2 detected degradation
  • Check if a reboot is required after installing Linux updates

• [Nitro] -在 AWS Nitro Enclaves 中运行传统 Web 应用迁移实践

  • Introduce the nitro-enclaves

Load Balancer

• ALB and NLB Route Traffic to Peering VPC

• Domain and Host based routing for ALB

• ALB Redirect Domain

• Hostname-as-Target for Network Load Balancers

• alb troubleshoot 502 errors

• Redirect HTTP requests to HTTPS using an Application Load Balancer

System Manager

• Query for AWS Regions, Endpoints, and More Using AWS Systems Manager Parameter Store

aws ssm get-parameters-by-path --path /aws/service/global-infrastructure/regions --output json --profile us-east-1 --region us-east-1 | jq '.Parameters[].Name'

aws ssm get-parameters-by-path --path /aws/service/global-infrastructure/regions --output json --profile us-east-1 --region us-east-1 | jq '.Parameters[].Name' | wc -l

• ssm-connection-lost-status

• Session-manager QuickStart

• Manage private EC2 instances without internet access

HPC

• Running CFD on AWS Parrell cluster

• AWS Batch Getting Start demo

• Orchestrating high performance computing with AWS Step Functions and AWS Batch

• [NICE DCV]

  • NICE DCV Guide
  • NICE DCV Connection Gateway - enables users to access a fleet of NICE DCV servers through a single access point to a LAN or VPC
  • NICE DCV Session Manager - the Agents, a Broker and API that makes it easy to build front-end applications that programmatically create and manage the lifecycle of NICE DCV sessions across a fleet of NICE DCV servers

• SOCA

  • SOCA 帮助半导体企业快速启动 EDA 云上部署

Analytics

High Level Data Engineering and Data Analytics

• AWS Data Engineering Day Workshop

  • Chinese version

• 数据分析的技术源流

• Analytics Reference Architecture

• Data Science on AWS - Quick Start Workshop

• Build a Lake House Architecture on AWS

• Harness the power of your data with AWS Analytics with Lake House

• Serverless Data Lake Workshop

• Serverless Data Lake Framework WORKSHOP

• 敦煌网集团大数据上云实践

• 解密Data Fabric的核心技术 – 数据虚拟化 - Data Virtualization

  • Data Fabric and Data Mesh 区别

Data integration service: Glue

• [ETL]

  • Quick demo for Glue ETL + Athena + Superset BI
  • Glue ETL for kinesis / Kafka and RDS MySQL
  • Update and Insert (upsert) Data from AWS Glue
  • Introducing PII data identification and handling using AWS Glue DataBrew
  • Best practices to scale Apache Spark jobs and partition data with AWS Glue

• [Glue Crawler]

  • Glue Crawler handle the CSV contains quote string

• Glue Workshop

  • Building Python modules for Spark ETL workloads using AWS Glue

• [Workflow]

  • Amazon Glue ETL 作业调度工具选型初探
  • Airflow and Glue workflow
  • Deploy an AWS Glue job with an AWS CodePipeline CI/CD pipeline
  • How to unit test and deploy AWS Glue jobs using AWS CodePipeline

• [Catalogs]

  • 如何提供对 AWS Glue 数据目录中资源的跨账户访问权限
  • Replication utility for AWS Glue Data Catalog
  • Open Source Data Catalog (https://github.com/bluishglc/serverless-datalake-example)

• [Delta Lake]

  • Glue and Hudi

Analysis: EMR

• AWS EMR Workshop

• aws-emr-best-practices

• [Develop Code]

  • EMR Notebooks and SageMaker Use EMR notebooks to prepare data for machine learning and call SageMaker from the notebook to train and deploy a machine learning model.
  • Tool to convert spark-submit to StartJobRun EMR on EKS API
  • Submit EMR Job remotely

• [Workflow]

  • Orchestrate an Amazon EMR on Amazon EKS Spark job with AWS Step Functions

• [Install and Delopyment]

  • How can I permanently install a Spark or Scala-based library on an Amazon EMR cluster
  • EMR_On_Graviton2
  • Why use the Glue Catalog v.s other external metastore for Hive

• [EMR on Kubernetes]

  • EMR on EKS Best Practice Guide
  • EMR on EKS workshop

• [Performance and HA]

  • Resolve s3 503 slowdown throttling
  • Hadoop high availability features of HDFS NameNode and YARN ResourceManager in an Amazon EMR cluster
  • Spark 小文件合并功能在 AWS S3 上的应用与实践
  • Amazon EMR实战心得浅谈

• [Security]

  • Introducing Amazon EMR integration with Apache Ranger
  • Enable federated governance using Trino and Apache Ranger on Amazon EMR

Stream - Flink and Spark Streaming

• [Flink on EMR]
  • 可持续性最佳架构实践—基于Spot的Flink作业集群部署与优化
• 基于 Hudi + Flink多流拼接（大宽表）最佳实践
• ClickStream workshop
• Clickstream Analytics on AWS
• 多库多表场景下使用Amazon EMR CDC实时入湖

Stream - Kinesis

• How to do analysis and virtulization DynamoDB
• AWS Kinesis Workshop
• Sending Data to an Amazon Kinesis Data Firehose Delivery Stream
• lambda as a consumer for kinesis

Stream - Kafka

• MSK Workshop

• AWS Streaming Data Solution for Amazon MSK

• Kafka UI

• [Connection]

  • Secure connectivity patterns to access Amazon MSK across AWS Regions
  • exposing kafka public
  • Connect to Amazon MSK Serverless from your on-premises network
  • Kafka/MSK Cluster connection issue
  • MSK now offers multi-VPC private connectivity and cross-account access

• MSK Connect

  • Run Kafka Connect as Fargate Docker containers and deploy MirrorMaker configuration files
  • MirrorMaker deployment
  • Create a low-latency source-to-data lake pipeline using Amazon MSK Connect, Apache Flink, and Apache Hudi

• [Reliability]

  • MSK 可靠性最佳实践 msk-reliability-best-practice

• [Performance & Cost]

  • THe tiered storage for Amazon MSK

Ad-hoc and Interactive query: Athena

• Automate athena query by lambda and step function

  • Automate run Athena_name_query and prepared_statement

• How to use the Athena to create the complex embeded table and query the table

• Split and search comma separated column in Athena

• Amazon Athena Workshop

  • Athena_access_control

• [Athena Perfomrance]

  • Top 10 Performance Tuning Tips for Amazon Athena
  • Using athena partition projection or glue partition indexes to improve athena query performance

• Use CTAS statements with Amazon Athena to reduce cost and improve performance

• Athena adds cost details to query execution plans

• How to get results from Athena for the past 7 days

Data Warehouse: Redshfit

• [Usage]

  • Cross-Account Data Sharing for Amazon Redshift
  • redshift one page
  • Automate Redshift ETL
  • Redshift ML
  • Create-Redshift-ReadOnly-User
  • Glue Studio supports Redshift Serverless
  • Redshift auto mounting Glue Catalog
  • Iceberg table support on Redshift

• [Redshift performance]

  • 使用 Amazon Glue 来调度 Amazon Redshift 跑 TPC-DS Benchmark
  • Cloud DataWarehouse Benchmark

• [CDC to Redshift]

  • CDC from On-Premises SQL Server to Amazon Redshift

• [ClickHouse and S3]

  • Integrating ClickHouse and S3 Compatible Storage
  • ClickHouse S3 table function
  • ClickHouse Cloud & Amazon S3 Express One Zone: Making a blazing fast analytical database even faster
  • 字节跳动 ByteHouse 云原生之路 – 计算存储分离与性能优化

• Scheduling SQL queries on your Amazon Redshift

• [Streaming datawarehouse]

  • Real-time analytics with Amazon Redshift streaming ingestion
  • Streaming ingestion official doc

Search and analytics: Elasticsearch Service

• Loading Streaming Data into Amazon Elasticsearch Service

• Amazon Elasticsearch Service Workshop

• Elasticsearch Service Snapshot Lifecycle

• Automating Index State Management for Amazon OpenSearch Service

• SAML Authentication for Kibana

• Search DynamoDB Data with Amazon Elasticsearch Service

• Log Hub workshop

• 使用Fluent Bit与Amazon OpenSearch Service构建日志系统

• Simple FAQ Bot

• OpenSearch cross cluster replication - DR or HA

Governance

• [Lake Formation]

  • Lake Formation Workshop
  • AWS Lake Formation Tag-based access control
  • Athena Support Lake Formation fine-grained-access-control

• [Data Quality with Deequ]

  • AWS Lab deequ
  • Scala: Test data quality at scale with Deequ
  • Python: Testing data quality at scale with PyDeequ
  • Monitor data quality in your data lake using PyDeequ and AWS Glue
  • Building a serverless data quality and analysis framework with Deequ and AWS Glue

• [Data Quality with Great Expectations]

  • Great Expectations Github
  • Great Expectations Home page
  • Expectations List
  • Provide data reliability in Amazon Redshift at scale using Great Expectations library
  • Monitoring Data Quality in a Data Lake Using Great Expectations and Allure-Built Serverless

• Data Lineage

BI

• Amazon QuickSight Workshop

• Athena integrated with PowerBI Desktop and PowerBI Service

• Integrate Power BI with Amazon Redshift for insights and analytics

Delta Lake

-[DataBricks]

• Migrating Transactional Data to a Delta Lake using AWS DMS

• [Hudi]

  • How EMR Hudi works

IOT

IoT Core

• IoT-Workshop

• AWS IoT Events Quick Start

• Ingest data to IoT core and using lambda write date to RDS PostgreSQL

• IoT DR solution

IoT Timeseries

• IoT Time-series Forecasting for Predictive Maintenance

OEE

• AWS IoT SiteWise Workshop

IoT anaytics

• AWS IoT Analytics Workshop

• AWS IoT Analytics Performance

• Using AWS IoT and Amazon SageMaker to do IoT Devices Predictive Maintenance

• IoT Time-series Forecasting for Predictive Maintenance

Edge

• AWS IoT Greengrass V2 Workshop

OTA

• Building a Scalable Standardized Pipeline for Automotive OTA on AWS

AIOT

• 制造业端到端(AIOT)动手训练营

Security

• AWS Security Hands on Lab - URL need whitelist
• AWS Security Hands on Lab2
• Public Access Consideration
• Curated list of links, references, books videos, tutorials, Exploit, CTFs, Hacking Practices etc. which are related to AWS Security
• An AWS Pentesting tool that lets you use one-liner commands to backdoor an AWS account's resources
• Top 2022 AWS data protection service and cryptography tool

Encryption - KMS

• Share-CMK-across-multiple-AWS-accounts
• Using-SM-Key-Algorithm-in-China
• Demystifying KMS keys operations, bring your own key (BYOK), custom key store, and ciphertext portability
• bring your own key to AWS KMS
• Multi-Region keys in AWS KMS

Credential - Secret Manager

• Secret Manager quick start demo

• Cross-Accounts-Secrets

Certificate - Certificate Manager

• Upload-SSL-Certificate

• Create certificate using openssl

• Free SSL certificate

• 3rd party PCA

• [Validate the ACM certificate]

  • Switch acm certificate validation
  • Troubleshooting acm certificate email validation

Asset Management and Compliance

• How to use the RDK for AWS Config Automation
• select * from cloud

AuthN and AuthZ

• Connect to Your Existing AD Infrastructure

• Cognito User Pool alternative solution - Authing demo

• Cognito with WeChat integration

• Summary the Single-Sign-On cases

  • Enabling Federation to AWS console using Windows Active Directory, ADFS, and SAML 2.0
  • Using IAM federation and Switch role to implement the Single Sign On multiple AWS Accounts
  • Okta-OpenID-AWS-in-the-Browser
  • Enabling custom identity broker access to the AWS console
  • Grant my Active Directory users access to the API or AWS CLI with AD FS
  • Using-temporary-credentials-with-AWS-resources
  • Okta - AWS China multi-account console integration
  • Keycloak on aws
  • Keycloak with Okta OpenID Connect Provider
  • Managing temporary elevated access just-in-time access to your AWS environment
  • Using global region SSO service to federate China region console

• Automatically rotate IAM user access keys at scale

• 利用IAM Roles Anywhere 授权云外设备访问AWS资源

Sentitive Data

• How to bootstrap sensitive data in EC2 User Data

Threat detection - GuardDuty

• GuardDuty Simulator

WAF

• aws-deployment-with-fortiweb-waf Source Code

• AWS WAF-Workshop

• WAF-Simulation-With-DVWA

• 使用 Amazon WAF 进行 Captcha人机验证

• WAF的托管规则说明

Permission - IAM Policy, S3 Policy, RAM Policy

• Policy evaluation logic

• How can I use permissions boundaries to limit the scope of IAM users and roles and prevent privilege escalation?

• Enforce MFA authentication for IAM users

• How can I use IAM roles to restrict API calls from specific IP addresses

Multi accounts structure

• Accessing and administering the member accounts in your organization

• Moving an memeber accounts to another organization-Part1

• Moving an memeber accounts to another organization-Part2

• Moving an memeber accounts to another organization-Part13

• Share a subnet with other account

SIEM and SOC

• Security Hub quick start
• Customer security findings for security hub

Vulnerability Assessment - Inspector and Alternative

• Inspector alternantive in China region - tenable nessus

Network

VPC

• How to solve private ip exhaustion with private nat solution

• How do I modify the IPv4 CIDR block of my Amazon VPC

  • How can I modify the CIDR block on my VPC to accommodate more hosts

Keep private - VPC Endpoint and PrivateLink

• How to verify EC2 access S3 via VPC S3 Endpoint?

• Why can’t I connect to an S3 bucket using a gateway VPC endpoint?

• The customer have a private subnet without NAT and want to use ssm vpc endpoint to connected to SSM service

• Using VPC PrivateLink to do cross VPC traffic

• Cross region VPC endpoint access

• How do I configure cross-Region Amazon VPC interface endpoints to access AWS PrivateLink resources?

NAT and proxy

• How I can setup transparent proxy - squid

• Nginx S3 Reverse Proxy

Load balancers

• NLB-TLS-Termination + Access log

• Current limits of AWS network load balancers

Cross data center and cloud Leasing Line - Direct Connect and VPN

• Direct Connect Monitoring

  • DX_Ping_check

• DX-Resillency

  • How to achieve active-active/active-passive Direct Connect connection

• Amazon Direct Connect inter-region routing for public access resources

• Connect alibaba cloud to aws via vpn

• AWS Direct Connect SiteLink: send data from one Direct Connect location to another, bypassing AWS Regions

• AWS Site-to-Site VPN Private IP VPNs

• Direct Connect and AWS Local Zones interoperability patterns

Cross board transfer

• Cross region EC2 to EC2 transfering speed testing

Cross accounts and Cross VPCs - TGW

• TGW cross account sharing and inter-connection testing

• VPC-Cross-Account-Connection

• Building a Solution for China Cross-Border VPC Connection

Acceleration network

• Using Amazon Global Accelerator to improve cross board request improvement

• Measuring AWS Global Accelerator performance and analyzing results

• Amazon CloudFront Extensions

• Enable the HTTPS access for CloudFront

  • create-ssl-with-cloudfront - China CloudFront SSL Plugin
  • External Server Authorization with Lambda@Edge

• Optimizing performance for users in China with Amazon Route 53 and Amazon CloudFront

Edge

• Protecting workloads on AWS from the Instance to the Edge

• CloudFront support HTTP/3

Network Secuirty

• GWLB Example

• Transit Gateway Connect 集成FortiGate安全服务

• How to check the Internet Traffic with VPC Flow?

• [Traffic Mirror]

  • Using VPC Traffic Mirroring to monitor and secure your VPC
  • 借助 VPC Traffic Mirroring 构建网络入侵检测系统
  • VPC traffic mirror 集成FortiGate安全分析
  • 使用 GWLB 和 FortiGate 作为流量镜像的替代方案

DNS

Route 53

• Route53 in China region

• How do I troubleshoot Route53 geolocation routing issues

• Route53 Routing Policy

• Route53 Resolver

• Route53 cross-account-dns

HTTPDNS

DNS hijacked using http dns bypass

Serverless

Serverless Workshop

• AWS Serverless Day

• Serverless Patterns Collection

• Serverless code repo Collection

• Serverless coffee workshop

• AWS Serverless SaaS Workshop

Function as Service - Lambda

• Lambda integration

  • Using AWS Lambda with Amazon Kinesis
  • How to put the S3 event to Kafka using lambda
  • Demo how to send the Lambda logs to S3 and ElasticSearch by using Kiensis Firehose
  • Run the serverless wordpress with AWS Lambda and AWS EFS
  • AWS 告警通知到微信
  • Lambda write PostgreSQL
  • Lambda sent email
    • Using Amazon SES
    • Using SendCloud
    • 利用 Lambda 调用 smtp
  • 使用 Lambda 函数URL + CloudFront 实现S3镜像回源

• Lambda usage

  • Schedule-Invoke-Lambda
  • AWS Lambda Custom Runtime for PHP
  • How to clean up the elastic network interface created by Lambda in VPC mode
  • How to get the lambda public IP address
  • How to retrieve the System Manager Parameter Store from lambda
  • Understanding the Different Ways to Invoke Lambda Functions
  • Run web applications on AWS Lambda without changing code
  • Disney use the open source and serverless
  • Liftshift web app to serverless - part1
  • Liftshift web app to serverless - part2
  • lambda extensions

• Lambda cost

  • 使用 Graviton 2优化Serverless车联网架构
  • Optimizing your AWS Lambda costs – Part 1
  • Optimizing your AWS Lambda costs – Part 2

• Lambda performance

  • Understanding AWS Lambda scaling and throughput
  • Lambda provisioned capacity autoscaling的实践

API Gateway

• Build Private API with API Gateway and integrate with VPC resource via API Gateway private integration

• API Gateway for API design patterns

• Understanding VPC links in Amazon API Gateway private integrations

• API Gateway private customer domain

Step function

• Configure Step Functions state machine as a target of Event

Build the serverless - SAM, Chalice, Serverless framwork, CDK

• hello-cdk

• Build and deploy a serverless application with the SAM CLI in China reigon

• SAM templates and lightweight web frameworks

• AWS Serverless Workshop

• Chalice - A framework for writing serverless applications

Serverless with AI/ML

• Create the pandas layer for lambda

• AWS Lambda – Container Image Support

• Lambda invoke AWS Rekgonition

• Lambda OpenCV

Migration

Journey to Adopt Cloud-Native Architecture

• #1 – Preparing your Applications for Hypergrowth
• #2 – Maximizing System Throughput
• #3 – Improved Resilience and Standardized Observability
• #4 – Governing Security at Scale and IAM Baselining
• #5 – Enhancing Threat Detection, Data Protection, and Incident Response

Active Directory

• How to migrate your on-premises domain to AWS Managed AD?

Database

• aws-database-migration-samples

• How to migrate MySQL to Amazon Aurora by Physical backup

• Migrating SQL Server to Amazon RDS using native backup and restore

• Microsoft SQL Server to Amazon S3

• 适用于Babelfish为目标的SQL Server数据迁移方法

• Best practices for migrating PostgreSQL databases to Amazon RDS and Amazon Aurora

• Migrating data to Amazon Aurora with PostgreSQL

• rds-for-postgresql v.s aurora-postgresql

• Bytebase 为 Amazon Aurora MySQL 设置数据库变更管理

• Aurora launches instances in at least 3 AZ even if less are specified

Data migration tool - DMS

• DMS Workshop
• AWS Database Migration Workshop scenario based

Data migration tool - 3rd party tool

• Migration-Data-From-AliCloud
• [XData])migration/DataMigration/XData.md
• Flink CDC Database Data

Cross Cloud Migration

Migrate from AliCoud workshop

File migration

• Getting Start Transfer Family

• Transfer Family to access EFS

• SFTP on AWS

Storage

S3 cross region or cross cloud OSS

• How to sync S3 bucket data between global region and China region

• Cross region S3 file download and upload

• S3 trasnfer tool

• s3-benchmark-testing

• Cross cloud OSS sync to S3

• RClone Quickstart

• Synchronize S3 bucket contents with Amazon S3 Batch Replication

S3

• S3 Web Explorer

• S3-Presign-URL

  • Embeded the image in html with S3-Presign-URL

• Uploading to Amazon S3 directly from a web or mobile application

• S3 disale TLS1.1 access or enforce TLS1.2 for in-transit encryption

• 使用 VPC Endpoint 从 VPC 或 IDC 内访问 S3

• Adding and removing object tags with S3 Batch Operations

• S3 inventory usage

• How Trend Micro uses Amazon S3 Object Lambda to help keep sensitive data secure

• Using S3 Intelligent-Tiering

• How to Check S3 object integrity

• 通过 STS Session Tags 来对 AWS 资源进行更灵活的权限控制 - 但是需要一个认证机制去确保userid可信的

• 将Amazon EC2到Amazon S3的数据传输推向100Gbps线速

• 如何在短时间里遍历 Amazon S3 亿级对象桶（原理篇）

• Hosting Internal HTTPS Static Websites with ALB, S3, and PrivateLink

EBS

• How do I create a snapshot of an EBS RAID array

• EBS benchmark testing

Storage Gatewway

• storage-gateway-demo and performance testing
• How can I troubleshoot an S3AccessDenied error from my file gateway
• How can I set up a private network connection between a file gateway and Amazon S3
• Resolve an internal error when activating my Storage Gateway

EFS and FSx or other shared file system

• EFS Workshop for GCR

• Amazon FSx for Lustre or Amazon FSx for Windows File Server Workshop

• Amazon FSx for Windows File Server file share on an Amazon EC2 Linux instance You can mount an Amazon FSx for Windows File Server file share on an Amazon EC2 Linux instance that is either joined to your Active Directory or not joined.

• goofys on AWS

• EFS Web Browser for list files and directory

• Global Region Simple File Manager for Amazon EFS

• 基于AWS DataSync 迁移 NetApp NAS上云

• Deploying IPFS Cluster using AWS Fargate and Amazon EFS One Zone

Database

RDS

RDS usage

• Amazon Aurora MySQL Database Quick Start Reference Deployment

• RDS common questions

• Use Proxysql for RDS for MySQL or Aurora databases connection pool and Read/Write Split

• Proxy for PostgreSQL

• AWS Bookstore Demo App - Purpose-built databases enable you to create scalable, high-performing, and functional backend infrastructures to power your applications

• [PostgreSQL Logging]

  • 如何使用 Amazon RDS for PostgreSQL 启用查询日志记录 query logging？
  • Working with RDS and Aurora PostgreSQL logs: Part 1
  • Working with RDS and Aurora PostgreSQL logs: Part 2

• RDS mysql max connections

• rds-postgresql ERROR: <module/extension> must be loaded via shared_preload_libraries

• [MySQL 手工分库分表]

  • Amazon Aurora的读写能力扩展之ShardingSphere-Proxy篇
  • 拓展 Aurora的读写能力之Gaea篇

RDS Cross region, cross account, data replication and backup

• MySQL Cross Region Replica

• 通过 Debezium and MSK Connect 一站式解决所有数据库 CDC 问题

• Cross vpc access RDS MySQL via VPC endpoint

• DB Snapshot cross region copy and backup cross region replication

• QuickStart RDS PostgreSQL and backup

• How-to-achive-postgreSQL-Table

• Amazon Aurora 故障恢复之降低DNS切换对应用影响篇

• Amazon Aurora 故障恢复不同JDBC Driver下的时延分析

RDS upgrade

• Achieving minimum downtime for major version upgrades in Amazon RDS PostgreSQL

• How to Migrate from Amazon RDS Aurora or MySQL to Amazon Aurora Serverless

• Aurora MySQL 5.6 upgrade blog

• Minimum downtime Aurora MySQL5.6 upgrade using clone

RDS Security

• Managing postgresql users and roles

• best-practices-for-working-with-amazon-aurora-serverless

• Securing sensitive data in Amazon RDS

• MySQL validate_password plugin

• Encrypt the Unencrypted RDS

• Disable_RDS_encryption

• RDS MySQL and Aurora MySQL Audit logs

• RDS PostgreSQL and Aurora PostgreSQL Audit

  • pgaudit extention

RDS Performance

• Amazon Aurora Performance Assessment

  • Managing performance and scaling for Amazon Aurora MySQL
  • Managing performance and scaling for Amazon Aurora PostgreSQL
  • "Too Many Connections" error when connecting to my Amazon Aurora MySQL instance

• Automate benchmark tests for Amazon Aurora PostgreSQL

• benchmarking-read-write-speed-on-amazon-aurora-classic-rds-and-local-disk

-Amazon Aurora 压力测试

• Aurora MySQL enhanced binlog to reduces the performance overhead of enabling binlog down to 13%

Graph Database

• Neo4j-On-AWS

• How to use the Neptune to Build Your First Graph Application

• 利用Neptune图数据库构建工厂知识图谱实践

• Diagram-as-code using generative AI to build a data model for Amazon Neptune

ElastiCache

• Building a fast session store for your online applications with Amazon ElastiCache for Redis

• Database Caching Strategies Using Redis

• 使用Redisson连接Amazon ElastiCache for redis 集群

• ElastiCache for Redis 慢日志可视化平台

• RedisInsight 官方可视化工具

• Amazon ElastiCache for Redis 分层存储

Key-Value and Document

DynamoDB

• DynamoDB labs

• [Migration and Replication]

  • 如何将我的 DynamoDB 表从一个 AWS 账户迁移到另一个账户
  • Streaming Amazon DynamoDB data into a centralized data lake
  • 中国区与 Global 区域 DynamoDB 表双向同步
  • aws-dynamodb-cross-region-replication
  • DynamoDB table initial migration from global to China
  • 使用 Lambda 订阅Amazon DynamoDB 变更数据，并传输到Amazon OpenSearch，实现全文检索
  • Understanding Amazon DynamoDB latency
  • 大规模 DynamoDB 表数据跨账号迁移指南

• [Security]

  • Securing sensitive data in Amazon DynamoDB

• [Performance]

  • DynamoDB_Pagenation
  • Understanding Amazon DynamoDB latency

MongoDB and DocumentDB

• Get-Start-DocumentDB

  • Program-with-DocumentDB

• DocumentDB performance benchmark

• 利用ChangeStream实现Amazon DocumentDB表级别容灾复制

• Amazon DocumentDB（兼容 MongoDB）增加了对跨区域快照复制的支持

• 带您玩转云原生文档数据库DocumentDB的地理空间

• Run full text search queries on Amazon DocumentDB

• AWS 云上 MongoDB/DocumentDB 数据定期归档

Time series

• Amazon TimeStream Performance Testing

Container

EKS

Serverless or Kubernetes on AWS

EKS networking

• 关于Amazon EKS中Service和Ingress深入分析和研究

• Exposing Kubernetes Applications via service and ingress resource

• How do I expose the Kubernetes services running on my Amazon EKS cluster

• Network load balancing on Amazon EKS instance and ip type

• 一文看懂 Amazon EKS 中的网络规划

• [How do I use multiple CIDR ranges with Amazon EKS]

  • Check your VPC setting
  • Add the additioal CIDR

• Cluster networking for Amazon EKS worker nodes

• mutual TLS authentication for EKS Application

EKS practice

• eks-workshop-greater-china

• EKS-Workshop-China

  • EKS in Beijing 3 AZ

• Advanced EKS workshop

  • EKS Multi-Tatent

• EKS Best Practices Guides

• Windows pod in EKS

• [EKS Managed Group]

  • Overview
  • Quotas
  • Official doc
  • Cluster autoscaler Managed node groups are managed using Amazon EC2 Auto Scaling groups, and are compatible with the Cluster Autoscaler. You can deploy the Cluster Autoscaler to your Amazon EKS cluster and configure it to modify your Amazon EC2 Auto Scaling groups.
  • Vertical Pod Autoscaler
  • Horizontal Pod Autoscaler

Data On EKS

• data-on-eks

DevOps on EKS

• Install SSM Agent on Amazon EKS worker nodes by using Kubernetes DaemonSet

• How can I check, scale, delete, or drain my worker nodes on EKS

• 关于Amazon EKS基于Gitlab的CICD实践

• DataDog for EKS control plane monitoring

• Exclusive Node from EKS ELB

• Securing Kubernetes with Private CA

• Public container images mirror to China ECR solution

• Global to China 跨国企业 Kubernetes 应用跨境复制和部署方案

• Application first delivery on Kubernetes with Open Application Model

• EKS Performance

• [Karpenter]

  • Karpenter 实践：一种多 EKS 集群下的 Spot 实例中断事件处理总线设计
  • Karpenter 实践：使用 Spot 实例进行成本优化
  • Karpenter 实践：部署GPU推理应用
  • EKS with KEDA HPA & Karpenter cluster autoscaler

ECS

• ECS workshop for china region

• ECS quick start demo workshop

• ECR Sync up from global from China and ECS Service Discovery

• How can I create an Application Load Balancer and then register Amazon ECS tasks automatically

• How can I a ECS service serve traffic from multiple port?

• How to launch tomcat server on ECS

• graceful-shutdowns-with-ecs

• [Amazon ECS firelens]

  • firelens examples
  • firelens demo

• amazon-ecr-cross-region-replication

Fargate

• Recursive Scaling Fargate with Amazon SQS

• aws-fargate-fast-autoscaler

• EKS on Fargate QuickStart

Istio, Envoy, App Mesh, Service discovery

• AWS App Mesh Workshop

• AWS App Mesh ingress and route enhancements

• Running microservices in Amazon EKS with AWS App Mesh and Kong

• [EKS and CloudMap]

  • Introducing AWS Cloud Map MCS Controller for K8s
  • Cross Amazon EKS cluster App Mesh using AWS Cloud Map
  • Cloud Map for serverless applications

ECR

• ecr-cross-region-replication

• ECR Sync up from global from China and ECS Service Discovery

DevOps

Management

• AWS Management Tool stack workshop

• How to make the Trust Advisor Check automatically

[AWS Well-Architected]

• AWS Well-Architected labs
• AWS Well-Architected Labs in Chinese

[Organizing Your AWS Environment Using Multiple Accounts]

• Organizing Your AWS Environment Using Multiple Accounts Whitepaper

• Landing Zone example

• Landing Zone plus - Cloud Foundations

• Deploy a multi-account cloud foundation to support highly-regulated workloads and complex compliance requirements

• AWS Services Autoscaling

CI/CD

• CodeCommit and - CodeCommit setup

• Codebuild Get Start

• CodePiple Workshop

Serverless CICD

• Serverless CI/CD based on Jenkins

• Lambda CICD with Jenkins and CodeBuild and CodeDeploy

• AWS Serverless CI/CD hands on lab

Container CICD

• 在 AWS 中国区 EKS 上以 GitOps 方式构建 CI/CD 流水线

Monitoring and Tracing

• X-Ray in AWS China region

• AWS DevOps Management Observability workshop

• Accessing the AWS Health API

• 一键部署在钉钉群里自动创建 AWS Support Case 无服务器解决方案

  • Event Bridge rule to capture EC2 status change & AWS health event in different AWS region

• SMS notification for AWS health event

• Add alarms-in-batches for ec2 on cloudwatch

  • Use tags to create and maintain Amazon CloudWatch alarms for Amazon EC2 instances
  • Automatically create a set of CloudWatch alarms with tagging
  • 监控告警一点通：在CloudWatch上为 EC2批量添加告警
  • 企业微信、钉钉接收 Amazon CloudWatch 告警
    • 一键部署企业微信，钉钉，飞书，Slack 告警
    • 一键部署Telegram 告警

• Monitor using Prometheus and Grafana Here is how to deploy Grafana on EKS

• Set up cross-region metrics collection for Amazon Managed Service for Prometheus workspaces

• Create cross-account, custom Amazon Managed Grafana dashboards for Amazon Redshift

• Datadog integration with AWS China

• [Grafana and CloudWatch integration]

  • Grafana AWS CloudWatch data source
  • Grafana cloudwatch plugin

Quota Monitor on AWS

Logging

• How to send CloudWatch logs to S3

• Central Logging on AWS

• How to stream logs from CloudWatch logs to Splunk

• [Log Hub]

  • Log Hub for EKS
  • Log Hub for WAF

Change configuration

• AWS AppConfig Workshop

• AWS Config for resource housekeeping and cost optimization

• Community Config Rules

  • AWS Config related blogs

• Create a Lambda Function for a Cross-Account Config Rule

• How-to-get-public-resources

Developer

• CSV tools

• Python GUI lib

Infra as Code

• How to migrate global cloudformation to China reigon?

• Terraform_Demo

• CloudFormation Stack Set

• AWS Cloud Control API QuickStart

Integration

Quque, notification

• How to build Amazon SNS HTTP Subscription?

• SNS Basic Example

• SQS quick start demo for Standard Queue and JMS

• Sent message to SQS queue using Lambda

Call Center

• Use the Amazon Connect to call out the mobile phone

• Automotive Call Center Services Solution Using Amazon Connect

MQ

• AmazonMQ-Workshop
  • Automate RabbitMQ configuration in Amazon MQ
  • consumer_timeout Amazon RabbitMQ 不支持修改
  • [Jenkins 与 RabbitMQ集成]
    • Jenkins 与 RabbitMQ集成 一
    • Jenkins 与 RabbitMQ集成 二
    • Jenkins plugin to connect RabbitMQ then consume messages in queue
    • RabbitMQ Jenkins build trigger

Email

• Amazon Simple Email Service SES Usaga

Media

Video on Demand

• Video on Demand on AWS

Video Streaming

• 无服务器直播解决方案

• Open source web conferencing solution - Jitsi

• Unreal Engine 像素流送在g4dn上实现容器化部署实践

Mobile

Moible app development

• Tutorial: Intro to React – React
• 在中国区 AWS 上使用 Amplify 开发离线应用的使用心得

GraphQL - AppSync

• AppSync-Workshop

Business continuity

Backup

• Backup FAQ

DR

• Understand resiliency patterns and trade-offs to architect efficiently in the cloud

• AWS Fault Isolation Boundaries

• [Building a disaster recovery site on AWS for workloads on Google Cloud]

  • Part 1
  • Part 2

• resiliency-analyser

• ec2-reachability

• Reducing the Scope of Impact with Cell-Based Architecture

• 依托亚马逊云科技构建韧性应用

RDS HA/DR

• Amazon RDS Under the Hood: Multi-AZ

• • 使用 Amazon RDS for Oracle 配合 Oracle Active Data Guard 建立托管的灾难恢复与只读副本

• RDS MySQL Automated frequency backup

Resilience

• Verify the resilience of your workloads using Chaos Engineering

Game

GameLift

• [unreal engine game server]
  • Add Amazon GameLift to an unreal engine game server project - Amazon GameLift
  • Host Unreal Engine UE Game on GameLift

SAP

HA/DR

• SAP云上自适应跨可用区高可用方案
• 基于CloudEndure的新一代云上一键灾备解决方案与最佳实践

Office and business application

Workspaces - VDI

• How to Enable Multi-Factor Authentication for AWS Services by Using AWS Microsoft AD and On-Premises Credentials

Metaverse

• possibilities-of-healthcare-in-the-metaverse
• life-sciences-in-the-metaverse

Automotive

• [Software Define Vechile - SDV]

  • soafee aws iotfleetwise demo
  • 在AWS上构建基于SOAFEE的云原生软件定义汽车实践
  • A Cloud-Native Environment for Distributed Automotive Software Development
  • Simulating Automotive E/E Architectures in AWS – Part 1: Accelerating the V-Model
  • Automotive Demo Lab

• [Autonomous Driving]

  • ADDF is a collection of modules, deployed using the SeedFarmer orchestration tool. ADDF modules enable users to quickly bootstrap environments
  • Develop and deploy a customized workflow using Autonomous Driving Data Framework (ADDF) on AWS
  • 详解智能驾驶的功能与场景体系
  • Deploy and Visualize ROS Bag Data on AWS
    • Scene Intelligence with Rosbag on AWS
    • Autonomous Driving Data Service with Rosbag on EKS

• [Vechile Data Plantform - VDP]

  • BMW Cloud Data Hub: A reference implementation of the modern data architecture on AWS