Learn Node.js by building a backend framework - Velocy

:warning: Live stream: The content of this book, as well as the backend framework we're building, is being live-streamed on YouTube.

You can access the current version of the book in the chapters directory or in PDF format (both Light and Dark modes are available) by clicking here. Note that this version includes the current release of the content, and is not the final version.

This book is still in a very early stage. It contains an insignificant portion of the total content that the book is supposed to cover. There’s going to be 0 dependencies for our backend framework, as well as our logging library. Everything will be done using vanilla Node.js, the hard-way (the best way to learn).

Note

If you're not familiar with javascript, you may also check out my other repository - Learn Javascript - The Easy Way that takes you on a deep and a fun journey into Javascript - from the very basics to the advanced concepts that you'd ever need, without diving into too much theory. Only practical code examples.

To master a new concept, it's often best to begin from the ground up. This isn't just another Node.js guide; it's a comprehensive, code-along experience aimed at building a real world product that may be used by thousands of developers. The product that we're going to build will be a backend framework, that too from scratch.

You won't just learn how Node.js works, but also why it operates in a particular way. The guide also includes discussions on relevant data structures and design patterns.

The book also includes a wide range of exercises specifically created to challenge you, that may require commitment and consistent effort on your part. The first exercises start from chapter 7

This guide goes beyond the basics. We're focused on delivering a modular, optimized backend framework that is close to being production-ready. Topics like performance optimization, security measures, and various testing approaches will be covered to ensure the framework is both reliable and extendable.

I highly recommend actively coding alongside this guide, rather than just reading through it, for a full understanding of Node.js and its more intricate aspects.

The repo for our backend framework- Velocy. (W.I.P)

Table of contents

• (optional) Node.js is way faster than you think
  • Contenders for the test
    • Elysia - Bun
    • Axum - Rust
    • Express - Node.js
    • Velocy - Node.js
  • The benchmark
• What the hell is a web server any way?
  • Parts of a Web Server
  • Navigating the World of Protocols: A Quick Overview
  • The Relationship Between HTTP and TCP
    • Data Integrity and Order
    • Acknowledgment Mechanism
    • Complex Interactions
    • Transmission Overhead
  • How Web Servers Respond to Your Requests
    • The Request
      • Your Request
      • Finding the Address
      • Resolving the Address
    • The Response
      • Return Address
      • Sending the Request
      • Preparing the Content
      • Sending the Response
      • Enjoying the Content
• Your first web server with node.js
  • What exactly is node or nodejs?
  • Your first node.js program
  • How does console.log() work in Node.js?
  • The process Object
  • The stdout property of the process object
• Working with files
  • What will the logging library do
  • How do you work with files anyway?
  • Let's get back to files
  • A little more about file descriptors
  • Creating our first file
    • path argument
    • flag argument
    • mode argument
  • Reading from a file
  • A small primer to for..of and for await..of in javascript
    • for..of
    • for await..of
  • Reading the json file
  • Buffers
  • Parsing the json file
• logtar - Our Own logging library
  • Initializing a new project
  • A little about SemVer
  • Creating a LogLevel class
  • The Logger class
  • Encapsulation with private fields
  • The LogConfig class
  • Design Patterns
    • The Builder pattern
    • Using the Builder pattern with the LogConfig class
  • jsdoc comments
  • The RollingConfig class
    • The RollingSizeOptions class
    • The RollingTimeOptions class
  • Finishing up the RollingConfig class
  • Let's recap
  • Adding more useful methods in the LogConfig class
  • Why readFileSync?
• Refactoring the code
  • The need for refactoring
  • Creating Separate Files
    • Explanation
    • index.js file
    • lib/logtar.js file
    • lib/logger.js file
    • lib/config/log-config.js file
    • lib/config/rolling-config.js file
    • lib/utils/log-level.js file
    • lib/utils/rolling-options.js class
• Writing Logs
  • Re-using the file handle
  • Log rotation
  • Asynchronous logging
  • Getting caller information
  • Testing our current API
  • Implementing logging methods
    • DRY (Don't Repeat Yourself)
    • The log method
    • Considering the log_level member variable
  • Writing to a file
    • A small primer on Regular Expressions
    • Testing the log file creation
    • Another gotcha
    • Logs directory configuration
    • Our first script
  • The require object
  • Adding a new helper to create log directory
  • Updating the init method
  • Completing the log method
• Capturing metadata
  • What is a stack
  • Examples of stacks
  • The call stack
  • Getting the stack info
  • Getting the callee name and the line number
  • A more ergonomic way
  • Using the get_caller_info function
• A small intro to async vs sync
  • The Balance between Opposites
  • Mixing Asynchronous and Synchronous Code
  • Faster I/O out of the box
  • Blocking Code
  • Concurrency
• Adding Rolling File Support
  • Rolling features
    • #time_threshold
    • #size_threshold
  • The rolling_check() method
  • file_handle.stat()
  • Calling the rolling_check method
  • A big gotcha!
  • Stack traces across await points
    • The culprit
  • Testing the new Log file creation
• HTTP Deep Dive
  • A small web server
    • Starting our web server
    • Testing our web server
    • Testing with cURL
• HTTP Verbs, Versioning and the benefits of HTTP/1.1
  • GET - Retrieve data
  • POST - Create something
  • PUT - Replace or create
  • HEAD - Retrieve metadata
  • DELETE - Remove from existence
  • PATCH - Partial updates
  • A small recap
  • The / path
  • HTTP/0.9
  • HTTP/1.0
    • Introduction to the HTTP Header
    • Versioning
    • Status Codes
    • Content-Type Header
    • New Methods
  • HTTP/1.1
    • Persistent Connections
    • Pipelining
    • Chunked Transfer Encoding
    • The Host header
    • Caching improvements
    • Range Requests
    • New Methods: PUT, DELETE, CONNECT, OPTIONS, TRACE
• User Agents
  • User-Agent can be weird
• MIME Type and Content-Type
  • Understanding the Accept Header
    • Breaking Down the Line
    • Why the Wildcard?
    • Server Response
  • Mime Type
  • Anatomy of a MIME type
  • But why the wildcard */*?
  • The Content-Type header
    • Content-Type on request header
    • Content-Type on response header
  • The charset=UTF-8: character encoding
    • Universal Character Encoding
• Headers
  • Header Name
  • Colon (:)
  • Header Value
  • Whitespace
  • Custom X- based headers
• Request headers
  • Accept
  • Referer
  • Authorization
  • Cookie
  • Host
  • Content-Type
• Response Headers
  • Content-Type (response)
  • Cache-Control
    • How Caches Work
    • Cache-Control Directives
      • Always Cache (infrequent updates)
      • Always Cache (private only)
      • Never Cache (realtime data)
  • Set-Cookie
• Response and Status Codes
  • Connection: close in action
  • Status Codes
• velocy our backend framework
  • Why velocy?
  • What is a backend framework/library anyway?
  • Core features of our backend framework
    • Routing and URL Handling
    • Middlewares
    • Building our own database
      • Data Storage and Retrieval
      • Indexing
      • Querying
    • Caching
    • Rate limiting
    • Other features
      • Shared state
      • File uploads
      • Static file serving
      • Multi-part data
      • Websockets
      • Logging
      • Monitoring
• A basic Router implementation
  • A Toy Router
  • Chunks, oh no!
  • Specifying Content-Length
  • Code reusability
  • Separation of Concerns
• The Router class
  • Using Router with an HTTP server
  • this is not good
    • Using .bind()
    • Using Arrow function
  • Lexical context
  • Arrow functions are not free
  • Why should we care about memory?
  • Testing the updated code
• Improving the Router API
• The need for a trie
  • What is a Trie anyway?
• Exercise 1 - Implementing a Trie
  • Root Node
  • End of the word
  • Challenge 1: Basic Trie with insert Method
    • Requirements
    • More details
    • Solution
  • Challenge 2: Implement search method
    • Requirements
    • More details
    • Hints
    • Solution
• Exercise 2 - Implementing our Trie based Router
  • Challenge 1: Implementing the addRoute method
    • Requirements
    • More details
    • Hints
    • Solution
    • Explanation
  • Challenge 2: Implementing the findRoute method
    • Requirements
    • More details
    • Starting Boilerplate
    • Hints
    • Solution
    • Explanation