hongbomiao.com
hongbomiao.com copied to clipboard
π¦ A personal research and development (R&D) lab that facilitates the sharing of knowledge.
HONGBO MIAO
Making magic happen
Introduction
Personal cutting-edge technology lab. Happy learning and exchanging!
Architecture
This diagram shows the architecture of this repository.
It is way over-engineering. Please make sure to know the tradeoffs before onboarding any technology to your project.
Setup
make setup
Clean
make clean
Tech Stack
Web
- React - Web framework
- Redux - State container
- React Query - Hooks for fetching, caching and updating asynchronous data
- redux-observable - Side effects
- RxJS - Asynchronous programming with observable streams
- graphql-tag - GraphQL query parsing
- Bulma - CSS framework
- PurgeCSS - Unused CSS removing
- Jest - Unit testing, snapshot Testing
- React Testing Library - React component testing
- Storybook - Visual testing
- rxjs/testing - Marble testing
- Cypress - End-to-end testing
- Lighthouse CI - Performance, accessibility, SEO, progressive web app (PWA) analysis
Mobile
- React Native - Mobile application framework
- UI Kitten - UI library
- React Native Testing Library - React Native component testing
API Server - Go
- Gin - Web Framework
- gRPC - Remote procedure call (RPC) framework
- graphql-go - GraphQL
- jwt-go - JWT
- gin-contrib/cors - CORS
- opa - Open Policy Agent
- dgo - Dgraph client
- minio-go - MinIO client
- go-redis - Redis client
- pgx - PostgreSQL driver
- Resty - HTTP client
- Squirrel - SQL query builder
- apm-agent-go - APM agent
- OpenTelemetry Go - OpenTelemetry
- Prometheus Go - Prometheus
- Testify - Unit testing
- GoDotEnv - Environment variables loading
- jsonparser - JSON parser
- zerolog - Logging
API Server - Node.js
- Express - Node.js web application framework
-
GraphQL.js, express-graphql - GraphQL
- graphql-ws, graphql-subscriptions - GraphQL subscriptions
- graphql-upload - GraphQL upload
- graphql-shield - GraphQL permissions
- graphql-depth-limit - GraphQL depth limit
- graphql-query-complexity - GraphQL query complexity analysis
- DataLoader - Batching and caching
- Knex.js - SQL query builder
- node-postgres - PostgreSQL client
- ioredis - Redis client
- rate-limiter-flexible - Rate limiting
- expressjs/cors - CORS
- csurf - CSRF protection
- jsonwebtoken, express-jwt - JSON Web Tokens (JWT)
- bcrypt - Password hashing
- axios - HTTP client
-
Helmet - HTTP header
Content-Security-Policy
,Expect-CT
,Referrer-Policy
,Strict-Transport-Security
,X-Content-Type-Options
,X-DNS-Prefetch-Control
,X-Download-Options
,X-Frame-Options
,X-Permitted-Cross-Domain-Policies
,X-XSS-Protection
-
Report To - HTTP header
Report-To
-
Network Error Logging - HTTP header
NEL
-
express-request-id - HTTP header
X-Request-ID
-
response-time - HTTP header
X-Response-Time
- connect-timeout - Request timeout
- Terminus - Health check and graceful shutdown
- Opossum - Circuit breaker
- pino - Logging
- dotenv-flow - Environment variables loading
- Stryker - Mutation testing
- SuperTest - HTTP testing
- autocannon - HTTP benchmarking
- Clinic.js - Performance profiling
Data
- Trino - Distributed SQL query engine
- YugabyteDB - Distributed SQL database
- TimescaleDB - Time-series SQL database
- Cassandra - Distributed wide-column NoSQL database
- Dgraph - Distributed graph database
- Elasticsearch - Distributed document-oriented search engine
- PostgreSQL - Object-relational database
- KeyDB - High performance fork of Redis
- MinIO - High performance object storage
- Temporal - Microservice orchestration platform
- Airflow - Workflow management platform
- Spark - Data processing framework
-
Flink - Data processing framework
- flink-streaming-java - Flink
- flink-connector-twitter - Flink Twitter connector
- flink-connector-jdbc - Flink JDBC Connector
- flink-connector-redis - Flink Redis connector
-
Kafka - Distributed event streaming platform
- Debezium - Distributed change-data-capture platform
- debezium-connector-postgres - PostgreSQL connector
- kafka-connect-elasticsearchkafka-connect-elasticsearch - Elasticsearch sink connector
- http-connector-for-apache-kafka - HTTP sink connector
- Superset - Data exploration and data visualization platform
- golang-migrate/migrate - Database migrations
Cloud Native
-
Hasura - GraphQL Engine
- hasura-metric-adapter - Hasura GraphQL Engine metric adapter
- Ory Hydra - OAuth 2.0 and OpenID Connect server
- Terraform - Infrastructure as code
- TorchServe - PyTorch models serving
- Linkerd - Service mesh
- Traefik - Reverse proxy and load balancer
- nginx - Reverse proxy, load balancer
- Open Policy Agent (OPA) - Policy-based control
- OPAL - Open-policy administration layer
- Kibana - Data visualization dashboard for Elasticsearch
- Elastic APM - Application performance monitoring
- OpenTelemetry - Observability framework
- Jaeger - Distributed tracing system
- Grafana - Monitoring and observability platform
- Prometheus - Monitoring system
- Thanos - Highly available Prometheus setup with long term storage capabilities
- Fluent Bit - Log processor and forwarder
- Pixie - Observability tool for Kubernetes applications
- Docker - Container
- Skaffold - Continuous development for Kubernetes applications
- Multipass - VM manager
- Locust - Load testing
- Cloudflare Tunnel - Tunneling
- Kubernetes - Container-orchestration system
- K3s - Lightweight Kubernetes
- containerd - Container runtime
Ops
- Argo CD - Declarative GitOps CD for Kubernetes
- Rancher - Kubernetes container management platform
- Goldilocks - Kubernetes resource requests recommending
- Polaris - Kubernetes best practices validating
- Kubecost - Kubernetes cost monitoring and management
- Sloop - Kubernetes history visualization
- Ansible - IT automation system
- CodeQL - Variant analysis
Machine Learning
-
PyTorch - Machine learning framework
- PyTorch Geometric - PyTorch geometric deep learning extension
- Rasa - Machine learning framework for automated text and voice-based conversations
- CML - Continuous machine learning
- DVC - Data version control
- Feast - Feature store
- OGB - Open graph benchmark
- Weights & Biases - Machine learning experiment tracking
Embedded
- FreeRTOS - Real-time operating system
- Arduino Uno - Microcontroller board
Quantum Computing
- Qiskit - Quantum computing SDK
OPAL Client
- asyncpg - PostgreSQL client
- pydantic - Data validation
- Tenacity - General-purpose retrying library
Ethereum
- Solidity - Contract-oriented programming language
- solc-js - JavaScript bindings for the Solidity compiler
Code
- Prettier - Code formatter
- ClangFormat - C/C++ code formatter
- gofmt - Go code formatter
- Black - Python code formatter
- opa - Rego code formatter
- Mypy - Python static type checker
- tsc - TypeScript static type checker
- Stylelint - CSS linter
- hadolint - Dockerfile linter
- golangci-lint - Go linter
- ESLint - JavaScript linter
- Kubeval - Kubernetes configuration file linter
- markdownlint-cli2 - Markdown linter
- Buf - Protocol Buffers linter
- ShellCheck - Shell linter
- solhint - Solidity linter
- commitlint - Commit message linter
- Husky - Bad git commit and push preventing
Design
- Fusion 360 - Industrial design
- Figma - UX design
Services
- Sentry - Error tracking
- Report URI - Security reporting
- Google Tag Manager - Tag management
- Google Analytics - Web analytics
- FullStory - Experience analytics, session replay, heatmaps
- Namecheap - Domain
- Cloudflare - CDN, DNS, DDoS protection
- Discord - ChatOps
- Opsgenie - Incident management platform
- GitHub Actions - Continuous integration
- SonarCloud, Codacy, Code Climate, LGTM - Code reviews and analytics
- Codecov - Code coverage reports
- Chromatic - UI reviewing and feedback collecting
- HTTP/3 Check - HTTP/3 checking
- hstspreload.org - HSTS checking
- Mozilla Observatory, Snyk - Security monitoring
- Depfu, Requires.io - Dependency monitoring
- Uptime Robot - Uptime monitoring
- FOSSA - License compliance
Bots
- Renovate - Dependency update
- Mergify - Automatically merging
- Stale - Stale issues and pull requests closing
- ImgBot - Image compression
- semantic-release - Version management and package publishing
Highlights
Cloud Native
Pixie - Kubernetes Application Observing
Linkerd - Service Mesh
Hasura - GraphQL Engine
Traefik - Reverse Proxy and Load Balancer
Elastic APM - Application Performance Management
Jaeger - Distributed Tracing
Grafana - Monitoring and Observability Platform
Prometheus - Metrics
Kibana
Locust - Load Testing
Ops
Argo CD - GitOps
Discord - ChatOps
Rancher - Kubernetes Container Management
Kubecost - Kubernetes Cost Monitoring
Polaris - Kubernetes Best Practices Validating
Goldilocks - Kubernetes Resource Requests Recommending
Sloop - Kubernetes History Visualization
Data
Flink - Data Processing
Airflow - Workflow Management
Dgraph - Distributed Graph Database
Redis with RedisGraph Module
Machine Learning
Contextual AI assistant
Chatbot on Telegram powered by Rasa.
Weights & Biases - Machine Learning Experiment Tracking
Distributed hyperparameter optimization result by Weights & Biases.
HTTP/3
The website supports HTTP/3.
AVIF
Images on the website are using AVIF format.
βRoughly speaking, at an acceptable quality, the WebP is almost half the size of JPEG, and AVIF is under half the size of WebP.β β Jake Archibald, 2020
Security
Below is the website security report generated by Mozilla Observatory.
Profiling
Profiling result by Clinic.js and autocannon.
Automation
This pull request shows how these technologies work together from different aspects to achieve automation.