# Mrsheerluck Blog

> Personal notes on engineering, systems, and software.

Index: https://blog.sheerluck.dev/llms.txt

# Mrsheerluck Blog

> Personal notes on engineering, systems, and software.

Source: https://blog.sheerluck.dev/ · Markdown: https://blog.sheerluck.dev/index.md

Hi,

Thank you so much for visiting my blog. I write mostly on whatever I am learning and want to share my knowledge on. The topic does not always have to be based on tech. It can be science, maths, economics, philosophy, game design, animation, and more.

My point is this is my digital garden, and I am thanking you for being a part of this journey.

Browse all posts via the sidebar, or subscribe via [RSS](/rss.xml).

If you have any feedback or want to request any topic, please feel free to reach out.

- X: [@sheerluck_io](https://x.com/sheerluck_io)
- Bluesky: [@mrsheerluck.bsky.social](https://bsky.app/profile/mrsheerluck.bsky.social)
- GitHub: [MrSheerluck](https://github.com/MrSheerluck)
- YouTube: [@sheerluck-dev](https://www.youtube.com/@sheerluck-dev)

# Learn Axum Basics and Routing by Building a URL Shortener

> In this post, we are going to learn baics of Axum and routing and build a URL shortener in Axum. This is the first part of our backend series in Rust

Source: https://blog.sheerluck.dev/posts/axum/learn-axum-basics-and-routing-by-building-a-url-shortener/ · Markdown: https://blog.sheerluck.dev/posts/axum/learn-axum-basics-and-routing-by-building-a-url-shortener/index.md

Welcome to the backend engineering series. If you followed the learning-rust series, you now understand ownership, borrowing, structs, enums, error handling, generics, lifetimes, closures, smart pointers, concurrency, and async/await. You built a thread pool and a raw HTTP/1.1 server from scratch over TCP. That was the foundation. Now we move up a layer.

![Axum Basics Cover Image](/images/axum-basics-cover.png)

In this series, we are going to learn backend engineering as practiced in Rust today, routing, persistence, authentication, caching, middleware, background jobs, WebSockets, API contracts, security hardening, testing, and deployment. Every article introduces a concept and then builds a standalone project around it. No shared codebase across articles. No capstone. Just a deliberate sequence where each concept is easiest to absorb once the previous one is in place.

The only prerequisite is that you have completed the learning-rust series or are comfortable with async Rust and Tokio. I will assume you know how `Arc`, `tokio::spawn`, and `.await` work.

In this post, we are going to **build a URL shortener with Axum**. But before that, we will learn what Axum actually is, how a request flows through it, from the TCP socket all the way to your handler function and why it's designed the way it is. We will learn about `Router`, `route()`, the `Handler` trait, path parameters, `State<T>`, and the `IntoResponse` trait. I promise you by the end of this article, Axum won't feel like a black box.

Let's start, I can't wait.

Get the source code from [here](https://github.com/MrSheerluck/url-shortener-in-axum)

## From Raw TCP to Axum

In the async/Tokio article, we built an HTTP server by hand. We called `TcpListener::bind`, accepted connections in a loop, read raw bytes from each socket, split the bytes on `\r\n` to find the request line, parsed headers manually, matched on the HTTP method, and wrote raw response bytes back with `socket.write_all`. It worked. It taught us what HTTP actually is under the hood.

But that approach has a problem: every new endpoint means more manual byte-parsing logic. Every new feature like path parameters, query strings, JSON bodies, middleware is something you have to build yourself. And the result, even for a simple server, is hundreds of lines of infrastructure code before you write a single line of actual business logic.

Axum solves exactly this. It sits on top of `hyper` (Rust's low-level HTTP library) and `tower` (a middleware and service abstraction), and gives you a clean set of abstractions: routes, extractors, responses, and state. You write a function. Axum calls it when a matching request arrives. You return something. Axum converts it into an HTTP response using the `IntoResponse` trait and writes it to the socket. Everything in between like parsing headers, routing, serialization, error conversion is handled by the framework.

Here is the mental model to hold onto: Axum is a layer cake.

```
Your handler function
   |
Axum (Router, extractors, IntoResponse)
   |
Tower (Service, Layer middleware)
   |
Hyper (HTTP/1.1 and HTTP/2 protocol handling)
   |
Tokio (async runtime, TCP sockets, I/O multiplexing)
```

Every request that arrives on a TCP socket bubbles up through Tokio (which handles the async I/O), through Hyper (which parses the raw bytes into an HTTP request representation), through Tower (which runs middleware), through Axum's routing layer (which matches the path and method to your handler), and finally into your function. The response travels back down the same stack in reverse.

If you built the raw HTTP/1.1 server from the async article, you have already built the bottom two layers by hand. Axum gives you the top three.
### What is Hyper?

Hyper is a low-level HTTP implementation. It parses the raw bytes received over a TCP connection into HTTP request values, manages connection lifecycles, implements HTTP/1.1 and HTTP/2, handles features like chunked transfer encoding, and encodes HTTP responses into bytes before writing them to the socket. Axum builds on top of Hyper rather than talking to TCP sockets directly.
### What is Tower?

Tower provides the `Service` trait:

```rust
pub trait Service<Request> {
type Response;
type Error;
type Future: Future<Output = Result<Self::Response, Self::Error>>;

fn poll_ready(&self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>>;
fn call(&self, req: Request) -> Self::Future;
}
```

A `Service` is anything that takes a request and returns a future that resolves to a response. That is it. An Axum `Router` implements `Service`. A rate-limiting middleware is a `Service` that wraps another `Service`. Everything in the request pipeline is eventually represented as a Tower `Service`: routers implement `Service`, middleware wraps `Service`s, and Axum adapts your handler functions into `Service`s behind the scenes.

The `poll_ready` method lets a service indicate whether it is currently ready to receive another request. Tower uses this for backpressure. Most application code never interacts with it directly, so we'll ignore it in this series.

A `Layer` produces a new `Service` by wrapping an existing `Service`. When you call `.layer(some_middleware)` on a router, you are wrapping the router's `Service` inside the middleware's `Service`. Every request passes through the outer layer first, then the inner layer, then eventually reaches your handler.

We will go deep on Tower and middleware in Part 5. For now, just know it's there.

## Project Setup

Create a new project:

```
cargo new url_shortener
cd url_shortener
```

Open `Cargo.toml` and add the dependencies:

```toml
[package]
name = "url_shortener"
version = "0.1.0"
edition = "2024"

[dependencies]
axum = "0.8"
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
uuid = { version = "1", features = ["v4"] }
```

We have four dependencies:

- `axum` is the web framework
- `tokio` is the async runtime (Axum runs on top of it)
- `serde` and `serde_json` handle JSON serialization and deserialization
- `uuid` generates unique short codes for our URLs. The `v4` feature gives us random UUIDs

## The Simplest Possible Axum Server

Open `src/main.rs`, delete everything, and write this:

```rust
use axum::{routing::get, Router};
use tokio::net::TcpListener;

#[tokio::main]
async fn main() {
let app = Router::new().route("/", get(root));

let listener = TcpListener::bind("127.0.0.1:3000").await.unwrap();
println!("Listening on http://127.0.0.1:3000");

axum::serve(listener, app).await.unwrap();
}

async fn root() -> &'static str {
"Hello, Axum!"
}
```

Now, let me explain what we just did.

`Router::new()` creates an empty router. `.route("/", get(root))` registers a route: when a `GET` request arrives at `/`, call the `root` function. `route()` takes a path and a method handler. `get(root)` is shorthand. Axum provides `get()`, `post()`, `put()`, `delete()`, `patch()`, `head()`, and `options()` for every HTTP method. Each of them wraps your function in the router's internal machinery.

`TcpListener::bind("127.0.0.1:3000").await` creates a TCP listener bound to port 3000. This is the same `TcpListener` from Tokio that we used in the async article. You create the listener yourself and pass ownership of it to `axum::serve`.

`axum::serve(listener, app)` is the entry point. It takes your listener and your router, and runs an event loop: accept connections from the listener, parse HTTP requests with Hyper, route them through your router, and write responses back. It runs until the process is killed or the listener is closed.

The `root` function is a handler. It returns `&'static str`. Axum knows how to convert `&'static str` into an HTTP response because `&str` implements the `IntoResponse` trait.

Run it:

```
cargo run
```

Visit `http://127.0.0.1:3000` in your browser or run `curl http://localhost:3000`. You should see `Hello, Axum!`.

## The Handler Trait and async fn

Why does `async fn root() -> &'static str` work as a handler? Because Axum implements the `Handler` trait for async functions that satisfy certain conditions. The implementation is generic over the function's arguments and return type.

You do not implement `Handler` yourself. The framework does it for you through a blanket impl. The rule is simple: any async function whose arguments all implement `FromRequestParts` (or `FromRequest` for the body) and whose return type implements `IntoResponse` is a valid handler.

Extractors implementing `FromRequest` consume the request body, so a handler can have only one body extractor. Extractors implementing `FromRequestParts` only inspect the request metadata (such as the path, headers, or state), so they can be freely combined.

This is what the `Handler` trait looks like, simplified:

```rust
pub trait Handler<T, S>: Clone + Send + Sized + 'static {
type Future: Future<Output = Response> + Send + 'static;

fn call(self, req: Request, state: S) -> Self::Future;
}
```

When Axum matches a route to your function, it calls `Handler::call`. The function's arguments are extracted from the request. The function runs. The return value is converted into an `http::Response` via `IntoResponse`. The response is written back to the socket.

You never see this machinery. You just write a function and Axum does the rest.

## IntoResponse - Why So Many Types Work

In the raw HTTP server, we built response strings by hand: `format!("HTTP/1.1 200 OK\r\nContent-Length: ...\r\n\r\n{}", body)`. That was tedious and error-prone.

Axum solves this with the `IntoResponse` trait:

```rust
pub trait IntoResponse {
fn into_response(self) -> Response;
}
```

Anything that implements this trait can be returned from a handler. Axum provides implementations for:

| Type | What it produces |
|------|-----------------|
| `&'static str` | 200 OK with `text/plain; charset=utf-8` |
| `String` | 200 OK with `text/plain; charset=utf-8` |
| `StatusCode` | A response with just that status code, no body |
| `(StatusCode, T)` | A response with that status code and body `T` (where `T: IntoResponse`) |
| `(HeaderMap, T)` | A response with custom headers and body `T` |
| `(StatusCode, HeaderMap, T)` | Status code, custom headers, and body |
| `Json<T>` | 200 OK with `application/json` body (where `T: Serialize`) |
| `Html<T>` | 200 OK with `text/html; charset=utf-8` body |

The tuple implementations are important. They let you compose status codes and bodies without a single wrapper type:

```rust
async fn handler() -> (StatusCode, &'static str) {
(StatusCode::NOT_FOUND, "nothing here")
}
```

This returns a 404 with a plain-text body. The tuple `(StatusCode, T)` implements `IntoResponse` by combining the status code from the first element with the body from the second. The body's `IntoResponse` implementation handles the `Content-Type` header and the actual bytes.

`Json<T>` is another key type. It wraps any `T: Serialize` and returns a 200 with `Content-Type: application/json`. If serialization fails, it returns a 500 Internal Server Error.

## State - Sharing Data Across Handlers

In the raw HTTP server, we used `Arc<PathBuf>` to share the serve directory across all connection tasks. Axum has the same need, your handlers often need access to shared data like a database pool, a configuration struct, or, in our case, the map of shortened URLs.

Axum provides `State<T>` for this:

```rust
use axum::extract::State;
use std::sync::Arc;

#[derive(Clone)]
struct AppState {
message: String,
}

async fn handler(State(state): State<Arc<AppState>>) -> String {
state.message.clone()
}

#[tokio::main]
async fn main() {
let state = Arc::new(AppState {
    message: "hello from state".to_string(),
});

let app = Router::new()
    .route("/", get(handler))
    .with_state(state);

let listener = TcpListener::bind("127.0.0.1:3000").await.unwrap();
axum::serve(listener, app).await.unwrap();
}
```

Now, let me explain what we just did.

`State<Arc<AppState>>` is an extractor. When Axum calls your handler, it looks at the function's arguments and tries to extract each one from the request (or from the application state). `State<T>` extracts the shared state that you provided to the router via `.with_state()`.

`Arc<AppState>` is the actual state type. It implements `Clone`, and cloning an `Arc` is inexpensive because it only increments an atomic reference count. This makes it an ideal way to share immutable ownership of application state across all request handlers.

`.with_state(state)` attaches the state to the router. Every handler in that router (and any sub-routers merged into it) can extract it with `State<T>`.

> **Important:** The state type must be the same for the entire router. If you need different pieces of state in different handlers, put them all in one struct and wrap it in `Arc`.

## Path Parameters

In the raw HTTP server, we parsed paths manually, strip the leading `/`, check for `..`, join with the serve directory. For a URL shortener, we need something much simpler: extract a short code from the URL path like `/abc123`.

Axum handles this with `Path<T>`:

```rust
use axum::extract::Path;

async fn redirect(Path(code): Path<String>) -> String {
format!("You requested code: {}", code)
}
```

When a request arrives at `GET /abc123`, Axum extracts `"abc123"` from the path, deserializes it into the type you specified (`String`), and passes it to your handler. `Path` implements `FromRequestParts`, just like `State`.

The path parameter pattern in the route definition uses `{name}` syntax:

```rust
let app = Router::new().route("/{code}", get(redirect));
```

A request to `/abc123` matches this route, and the `Path(code)` extractor gives you `"abc123"`.

> Axum uses the `{name}` syntax (not the deprecated `:name` syntax from older versions). If you see `:name` in old tutorials, replace it with `{name}`.

## The Project: URL Shortener

Now that you understand `Router`, handlers, `State`, `Path`, and `IntoResponse`, let's build a URL shortener.

Our program will:

- Accept a `POST /shorten` with a JSON body containing a long URL
- Generate a unique short code using UUID v4
- Store the mapping in memory
- Return the short code as JSON
- Accept a `GET /{code}` that looks up the code and issues a 302 redirect
- Return 404 if the code does not exist

To keep the focus on Axum itself, the application stores its data in memory using a `HashMap`. In later articles we will replace this with a real database using SQLx.

### What is a URL Shortener?

A URL shortener takes a long URL like `https://en.wikipedia.org/wiki/Uniform_Resource_Locator` and maps it to a short one like `https://sho.rt/abc123`. When someone visits the short URL, the server looks up the original long URL and redirects them.

The mapping is one-way: long URL → short code. There is no requirement to reverse-lookup a long URL to find its short code. This keeps things simple.

### The Full Code

Replace everything in `src/main.rs` with this:

```rust
use axum::{
extract::{Path, State},
http::StatusCode,
response::{IntoResponse, Redirect},
routing::{get, post},
Json, Router,
};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::{Arc, RwLock};
use tokio::net::TcpListener;
use uuid::Uuid;

#[derive(Clone)]
struct AppState {
urls: Arc<RwLock<HashMap<String, String>>>,
}

#[derive(Deserialize)]
struct ShortenRequest {
url: String,
}

#[derive(Serialize)]
struct ShortenResponse {
short_url: String,
}

#[tokio::main]
async fn main() {
let state = AppState {
    urls: Arc::new(RwLock::new(HashMap::new())),
};

let app = Router::new()
    .route("/shorten", post(shorten))
    .route("/{code}", get(redirect))
    .with_state(state);

let listener = TcpListener::bind("127.0.0.1:3000").await.unwrap();
println!("Listening on http://127.0.0.1:3000");

axum::serve(listener, app).await.unwrap();
}

async fn shorten(
State(state): State<AppState>,
Json(payload): Json<ShortenRequest>,
) -> impl IntoResponse {
let code = &Uuid::new_v4().to_string()[..8];

state
    .urls
    .write()
    .unwrap()
    .insert(code.to_string(), payload.url);

let short_url = format!("http://127.0.0.1:3000/{}", code);

(
    StatusCode::CREATED,
    Json(ShortenResponse { short_url }),
)
}

async fn redirect(
State(state): State<AppState>,
Path(code): Path<String>,
) -> impl IntoResponse {
let urls = state.urls.read().unwrap();

match urls.get(&code) {
    Some(long_url) => Redirect::to(long_url).into_response(),
    None => (
        StatusCode::NOT_FOUND,
        "404 Not Found: No URL for this code",
    )
        .into_response(),
}
}
```

Now, let me explain what we just did.

### The State

```rust
#[derive(Clone)]
struct AppState {
urls: Arc<RwLock<HashMap<String, String>>>,
}
```

`AppState` holds our in-memory store, a `HashMap` mapping short codes to long URLs. It is wrapped in `Arc<RwLock<...>>` because multiple requests will read and write to it concurrently. `RwLock` allows many concurrent readers or one exclusive writer, which matches our access pattern: reads are common (every redirect) and writes are rare (only on shortening).

`#[derive(Clone)]` is necessary because Axum clones the state for internal use. Since `Arc` is cheap to clone, it only increments an atomic reference count, the derived `Clone` implementation simply creates another `Arc` pointing to the same `RwLock<HashMap<...>>`.

This example uses `std::sync::RwLock` because each critical section is very small and we never `.await` while holding the lock. If you need to hold a lock across asynchronous work, prefer `tokio::sync::RwLock`.
### The Request and Response Types

```rust
#[derive(Deserialize)]
struct ShortenRequest {
url: String,
}

#[derive(Serialize)]
struct ShortenResponse {
short_url: String,
}
```

`ShortenRequest` is what the client sends in the POST body. `#[derive(Deserialize)]` from `serde` tells the compiler to generate code that can parse this struct from JSON. `ShortenResponse` is what we send back. `#[derive(Serialize)]` generates code that converts this struct into JSON.

These types are used with the `Json<T>` extractor and response type. When Axum sees `Json<ShortenRequest>` as a handler argument, it reads the request body, parses the JSON, and  if it matches the struct, passes it to the handler. If the body is missing, malformed, or has the wrong types, Axum returns a 422 Unprocessable Entity automatically, before your handler ever runs.

### The Shorten Handler

```rust
async fn shorten(
State(state): State<AppState>,
Json(payload): Json<ShortenRequest>,
) -> impl IntoResponse {
let code = &Uuid::new_v4().to_string()[..8];

state
    .urls
    .write()
    .unwrap()
    .insert(code.to_string(), payload.url);

let short_url = format!("http://127.0.0.1:3000/{}", code);

(
    StatusCode::CREATED,
    Json(ShortenResponse { short_url }),
)
}
```

The handler takes two extractors: `State<AppState>` (the shared state) and `Json<ShortenRequest>` (the parsed JSON body). Axum runs both extractors before calling the function. If either fails, the handler is never called and an error response is sent instead.

`Uuid::new_v4()` generates a random UUID like `a1b2c3d4-e5f6-7890-abcd-ef1234567890`. We take the first 8 characters (`a1b2c3d4`) as our short code. Eight hexadecimal characters provide roughly 4.3 billion possible codes. For this toy project that is more than sufficient, although a production URL shortener would still need to detect and handle collisions before storing a newly generated code.

`state.urls.write().unwrap()` acquires a write lock on the `RwLock`, giving us exclusive access to the `HashMap`. We insert the short code as the key and the long URL as the value. The lock is released when the `RwLockWriteGuard` goes out of scope at the end of the expression.

The return type is `impl IntoResponse`. This means "I return something that implements `IntoResponse`, but I am not naming the exact type." The actual type is `(StatusCode, Json<ShortenResponse>)`, which is a tuple. The tuple implementation sets the status code to 201 Created and the body to the JSON-serialized response.

### The Redirect Handler

```rust
async fn redirect(
State(state): State<AppState>,
Path(code): Path<String>,
) -> impl IntoResponse {
let urls = state.urls.read().unwrap();

match urls.get(&code) {
    Some(long_url) => Redirect::to(long_url).into_response(),
    None => (
        StatusCode::NOT_FOUND,
        "404 Not Found: No URL for this code",
    )
        .into_response(),
}
}
```

The `Path(code): Path<String>` extractor pulls the `{code}` segment from the URL path. When a user visits `/abc12345`, `code` is `"abc12345"`.

`state.urls.read().unwrap()` acquires a read lock. Multiple redirect requests can hold read locks simultaneously, they do not block each other. Only the `shorten` handler's write lock blocks readers.

`urls.get(&code)` returns an `Option<&String>`. If the code exists, we get the long URL and return a `Redirect::to(long_url)`. `Redirect` is an Axum response type that produces a 303 Found with a `Location` header. The browser follows the redirect automatically.

If the code does not exist, we return a 404 status with a plain-text body. Both branches call `.into_response()` explicitly. This is necessary because `match` arms must have the same type, and the two arms here produce different types (`Redirect` and `(StatusCode, &str)`). Calling `.into_response()` on each arm converts them both to `Response`, which is a single concrete type.

> **Why `impl IntoResponse` and not `Response`?** When the return type is `impl IntoResponse`, Axum calls `.into_response()` for you after your handler runs. But when different branches of a `match` return different types, you need to call `.into_response()` inside each branch to unify them yourself. The `impl IntoResponse` in the signature still works because `Response` implements `IntoResponse`.

## Running the Project

Start the server:

```
cargo run
```

You should see:

```
Listening on http://127.0.0.1:3000
```

### Shorten a URL

Open another terminal:

```
curl -X POST http://localhost:3000/shorten \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.rust-lang.org"}'
```

Response:

```json
{"short_url":"http://127.0.0.1:3000/a1b2c3d4"}
```

### Follow the Redirect

```
curl -v http://localhost:3000/a1b2c3d4
```

You should see:

```
* Request completely sent off
< HTTP/1.1 303 See Other
< location: https://www.rust-lang.org
< content-length: 0
```

Use `-L` to follow the redirect:

```
curl -L http://localhost:3000/a1b2c3d4
```

This fetches the Rust home page.

### Test a Missing Code

```
curl -v http://localhost:3000/nonexist
```

Response:

```
< HTTP/1.1 404 Not Found
404 Not Found: No URL for this code
```

### Test a Bad POST Body

```
curl -X POST http://localhost:3000/shorten \
  -H "Content-Type: application/json" \
  -d '{"not_url": 123}'
```

Response:

```
< HTTP/1.1 422 Unprocessable Entity
```

Axum rejected this before our handler ever ran because the JSON did not match `ShortenRequest`. The required field `url` was missing, so deserializing the request into `ShortenRequest` failed before the handler was called. This is the extractor at work, `Json<ShortenRequest>` validates the body and returns a 422 if it does not match.

## How a Request Flows Through - The Full Trace

Let's trace a single `GET /abc12345` request through every layer of the stack to understand what actually happens.

1. **Tokio**: A TCP packet arrives on port 3000. Tokio's I/O driver is notified that the listening socket is ready, the runtime schedules the appropriate async task, and that task reads the incoming bytes from the socket.

2. **Hyper**: Hyper parses the raw bytes into an HTTP request representation, extracting the method, URI, version, headers, and request body before passing it to the next layer.

3. **Tower**: The request enters the Tower service stack. If we had middleware (we do not in this project, but we will in Part 5), each `Layer` would get a chance to inspect or modify the request before passing it inward. The outermost layer runs first.

4. **Axum Router**: The router receives the request and efficiently matches the request path against the registered routes. The pattern `/{code}` matches `/abc12345`, captures the path segment, verifies the HTTP method is `GET`, and selects the `redirect` handler.

5. **Handler extraction**: Before calling `redirect`, Axum runs the extractors. `State<AppState>` clones the `Arc` (cheap, just increments a reference count). `Path<String>` deserializes `abc12345` into a `String`. Both succeed. The handler is called with these two arguments.

6. **Your handler**: `redirect` acquires a read lock, looks up the code in the `HashMap`, finds the long URL, and returns `Redirect::to(long_url).into_response()`. This produces an `http::Response<Body>` with status 302 and a `Location` header.

7. **Axum response**: The `Response` travels back through the router. Axum does nothing further, the response is complete.

8. **Tower**: The response travels back out through the middleware stack. Each layer gets a chance to inspect or modify the response.

9. **Hyper**: Hyper encodes HTTP responses into bytes before writing them to the socket: `HTTP/1.1 302 Found\r\nlocation: https://...\r\ncontent-length: 0\r\n\r\n`. It writes these bytes to the socket.

10. **Tokio**: The socket write is async. If the socket buffer is full, the task yields. When the buffer drains, the runtime wakes the task. The response finishes writing. The connection is closed (or reused, if keep-alive were enabled).

All of this happens in under a millisecond for a simple in-memory lookup. The async runtime can handle thousands of these concurrently on a handful of OS threads.

Compare this to the raw HTTP server from the async article. The flow is the same. The difference is that steps 3 through 7, routing, extraction, handler dispatch, response serialization  were all written by hand in that article, and now Axum does them for us. 

## What We Skipped

There are a few things I am intentionally skipping in this article:

- **Persistent storage**: Our URL mappings live in an in-memory `HashMap`. They disappear when the server restarts. In Part 3, we will add PostgreSQL persistence with SQLx.
- **Request validation**: We accept any string as a URL. A real service would validate the format, reject empty strings, and potentially check that the URL is reachable. We will cover request validation in Part 2.
- **Error handling**: We use `.unwrap()` on lock acquisition. In a production service, a poisoned lock (from a panic in another thread holding the lock) should be handled gracefully. We will build a proper `AppError` enum in Part 2.
- **Custom short codes**: We use random UUID prefixes. A real service might let users choose custom codes or use a different encoding scheme (like base62) for shorter URLs.
- **Tower middleware, logging, tracing, CORS**: These come in later parts, deliberately sequenced.
- **Graceful shutdown**: The server stops immediately on Ctrl+C without draining in-flight requests. Part 9 covers this.

> Everything we skipped in this article exists for a reason in real-world services. Rather than introducing that complexity all at once, we focused on the Axum request lifecycle, routing, state, path parameters, and response types. This is the foundation that every subsequent part builds on.

## Conclusion

In this post, you learned what Axum is and where it sits in the stack: on top of Hyper and Tower, running inside Tokio. You learned the `Handler` trait, why any async function that takes extractors and returns `IntoResponse` just works as a handler. You learned `IntoResponse`, how tuples, `Json<T>`, status codes, and `Redirect` compose into HTTP responses without any boilerplate. You learned `State<T>` for shared application state and `Path<T>` for path parameters.

You built a URL shortener from scratch. The server accepts `POST /shorten` with a JSON body, generates a random short code, stores the mapping in an `Arc<RwLock<HashMap>>`, and returns the short URL. `GET /{code}` looks up the code and issues a 302 redirect. Missing codes get a 404. Invalid JSON gets a 422.

This is the foundation of every Axum application you will ever write. Router, extractors, state, and responses. Everything else like middleware, custom extractors, WebSockets, streaming is built on top of these four concepts.

In the next article, we will learn about request bodies and error handling by building a Pastebin API. We will build a proper `AppError` enum, handle validation, and learn why Axum deliberately makes you decide every status code yourself instead of guessing. See you soon.

If you like reading this, please subscribe and share this with others. It will really help me and motivate me to keep publishing more such articles.

# Learn Axum Error handling by Building a Pastebin API

> In this post, we are going to build a Pastebin API with proper error handling

Source: https://blog.sheerluck.dev/posts/axum/learn-axum-error-handling-by-building-pastebin-api-in-rust/ · Markdown: https://blog.sheerluck.dev/posts/axum/learn-axum-error-handling-by-building-pastebin-api-in-rust/index.md

In Part 1, we built a URL shortener and learned the Axum request lifecycle: `Router`, handlers, `State<T>`, `Path<T>`, and `IntoResponse`. But we left two things unfinished. Our error handling was `.unwrap()` on every lock acquisition, and our response for a missing URL was an ad-hoc `(StatusCode, &str)` tuple dropped directly into the handler. 

![Axum Error Handling](/images/axum-error-handling.png)

In this post, we are going to build a **Pastebin API with proper error handling**. We will learn the `Json<T>` extractor in depth, the `Query` extractor for optional parameters, request validation, and most importantly how to build a single `AppError` enum that converts every possible failure into the right HTTP status code. Along the way, we will see why Axum deliberately has no hidden error-handling behavior, and why that is a feature, not a missing feature.

Let's start, I can't wait.

Get the source code from [here](https://github.com/MrSheerluck/pastebin-api-in-axum)

## The Problem with Ad-Hoc Error Handling

In Part 1, when a short code was not found, we wrote:

```rust
match urls.get(&code) {
Some(long_url) => Redirect::to(long_url).into_response(),
None => (
    StatusCode::NOT_FOUND,
    "404 Not Found: No URL for this code",
)
    .into_response(),
}
```

This works for one handler. Now imagine a real application with twenty handlers. Every handler that does a lookup has its own 404 logic. Every handler that deals with validation has its own 400 logic. The status codes and error messages are scattered across the codebase, and there is no single place to change how errors are formatted.

The alternative that some frameworks choose is "hidden behavior": the framework catches your errors, guesses which HTTP status code to return, and sends a response for you. A missing database row becomes a 404. A failed deserialization becomes a 400. This sounds convenient, until it guesses wrong and your API returns a 500 for something that should be a 400, or worse, a 200 for something that should be a 500.

In Axum, you define your own error type. You implement `IntoResponse` on it exactly once. You decide every status code. The framework does not make application-specific decisions for you. This is what we are going to build.

## The Json Extractor - Deeper Than You Think

In Part 1, we used `Json<ShortenRequest>` without much explanation. Now let's understand what it actually does.

`Json<T>` implements `FromRequest` for any `T: DeserializeOwned`. When a handler declares `Json(payload): Json<SomeType>`, Axum does three things:

1. **Buffers the entire request body.** The request body arrives as a stream of bytes from the client. `Json` first reads the entire stream into memory before attempting deserialization. This is why a handler can have at most one body extractor: once the body has been consumed, there is nothing left for another body extractor to read.

2. **Checks the Content-Type header.** If the request is not sent with a JSON media type (such as `application/json` or `application/*+json`), Axum rejects it with a `415 Unsupported Media Type`. This happens before deserialization, so the client gets a clear signal that it sent the wrong content type.

3. **Deserializes the JSON.** If the body contains valid JSON but it cannot be deserialized into your type (for example, because a required field is missing, a field has the wrong type, or `#[serde(deny_unknown_fields)]` rejects an unknown field), Axum rejects the request with an appropriate deserialization error, typically `422 Unprocessable Entity`. Invalid JSON syntax is rejected earlier with `400 Bad Request`.

All three steps happen before your handler runs. If any step fails, the handler is never called. The request is rejected with the appropriate status code automatically. Your handler only runs when the body is present, correctly typed as JSON, and valid against your struct.

This is the first half of Axum's explicit error-handling model. The framework handles malformed requests such as unsupported content types, invalid JSON syntax, and deserialization failures, so you do not have to. But semantic errors like empty content, invalid values, expired resources are yours to handle. The line is drawn at `serde` deserialization. Everything beyond that is your responsibility.

## The Query Extractor

Some parameters are not part of the request body. They are part of the URL itself. Axum provides `Query<T>` for this:

```rust
use axum::extract::Query;
use serde::Deserialize;

#[derive(Deserialize)]
struct ListParams {
language: Option<String>,
page: Option<u32>,
}

async fn list_pastes(Query(params): Query<ListParams>) -> impl IntoResponse {
// params.language is Some("rust") or None
// params.page is Some(1) or None
todo!()
}
```

A request to `GET /pastes?language=rust&page=2` deserializes into `ListParams { language: Some("rust".into()), page: Some(2) }`. A request to `GET /pastes` with no query string gives `ListParams { language: None, page: None }`.

Unlike `Json`, which deserializes a JSON request body, `Query` uses the `serde_urlencoded` crate together with Serde to deserialize URL query parameters into your type. The same `serde` infrastructure handles both. The difference is where the data comes from: `Json` reads the request body, `Query` reads the URL query string.

> **When to use Query vs Json:** `Query` is most commonly used with `GET` requests (and sometimes `DELETE`). `Json` is most commonly used with `POST`, `PUT`, and `PATCH` requests. This is a widely followed REST convention rather than a rule enforced by HTTP, and some APIs legitimately use query parameters on other methods as well.

## Request Validation - Semantic Checks

Deserialization checks structure. Validation checks meaning. `serde` can tell you that `content` is missing, but it cannot tell you that `content` is an empty string. It can tell you that `expires_in_seconds` is an integer, but it cannot tell you that `-5` is nonsensical for an expiry duration.

In this project, we perform validation in the handler immediately after deserialization and before any business logic. Here is the pattern:

```rust
fn validate_create_request(req: &CreatePasteRequest) -> Result<(), AppError> {
if req.content.trim().is_empty() {
    return Err(AppError::ValidationError("content must not be empty".into()));
}
if req.content.len() > MAX_CONTENT_LENGTH {
    return Err(AppError::ValidationError(
        format!("content exceeds maximum length of {} bytes", MAX_CONTENT_LENGTH)
    ));
}
if let Some(ref lang) = req.language {
    if !SUPPORTED_LANGUAGES.contains(&lang.as_str()) {
        return Err(AppError::ValidationError(
            format!("unsupported language: {}", lang)
        ));
    }
}
if let Some(seconds) = req.expires_in_seconds {
    if seconds <= 0 {
        return Err(AppError::ValidationError(
            "expires_in_seconds must be positive".into()
        ));
    }
}
Ok(())
}
```

Every validation failure returns the same error variant: `AppError::ValidationError`. Every failure produces a 400 Bad Request. The difference is the message, which tells the client specifically what was wrong like is it empty content, too long, bad language, negative expiry.

## AppError - The Single Error Enum

Now we arrive at the core pattern of this article. An `AppError` enum that:

- Has one variant per category of error
- Implements `IntoResponse` once, mapping each variant to the right status code
- Has `From` impls so domain-level errors (like a failed lock acquisition) convert automatically

```rust
enum AppError {
ValidationError(String),
NotFound(String),
InternalError(String),
}
```

Three variants and that's it. A real application might have more like `Unauthorized`, `Forbidden`, `Conflict` but three is enough to cover everything a simple API can encounter.

### ValidationError → 400

The client sent something wrong. The error is theirs to fix. Return a 400 with a message explaining what to fix.

### NotFound → 404

The requested resource does not exist. This covers both "never existed" and "existed but expired." From the client's perspective, an expired paste is the same as a paste that was never created: you cannot read it.

### InternalError → 500

Something went wrong on the server. The client cannot fix it. In a real application, you log the full error internally but send a generic message to the client. Leaking internal details like file paths, stack traces, database error strings is a security risk and a bad user experience.

Now the `IntoResponse` implementation:

```rust
impl IntoResponse for AppError {
fn into_response(self) -> Response {
    let (status, message) = match &self {
        AppError::ValidationError(msg) => (StatusCode::BAD_REQUEST, msg.clone()),
        AppError::NotFound(msg) => (StatusCode::NOT_FOUND, msg.clone()),
        AppError::InternalError(msg) => {
            eprintln!("internal error: {}", msg);
            (
                StatusCode::INTERNAL_SERVER_ERROR,
                "internal server error".to_string(),
            )
        }
    };

    let body = Json(serde_json::json!({
        "error": message,
    }));

    (status, body).into_response()
}
}
```

Every error response is a JSON object with an `"error"` field. Validation errors include the specific message. Not-found errors include the resource name. Internal errors log the actual error to stderr via `eprintln!` but send a generic message to the client, the client sees `{"error": "internal server error"}` regardless of whether the database is down, the filesystem is full, or the lock is poisoned. We will replace `eprintln!` with structured logging via `tracing` in Part 16, but the principle is the same: the internal details are for the server operator, not the client.

### From Impls - Automatic Error Conversion

The `?` operator is the backbone of Rust error handling. It converts one error type into another via `From`. If we want to use `?` in our handlers, we need `From` impls that convert standard library and framework errors into `AppError`:

```rust
impl From<std::sync::PoisonError<std::sync::RwLockWriteGuard<'_, HashMap<String, Paste>>>> for AppError {
fn from(_: std::sync::PoisonError<std::sync::RwLockWriteGuard<'_, HashMap<String, Paste>>>) -> Self {
    AppError::InternalError("lock poisoned".into())
}
}
```

That type signature is... a lot. It is for `RwLockWriteGuard` specifically. We would need similar impls for `RwLockReadGuard`. Fortunately, in practice you rarely write these by hand. Many projects use `thiserror` to derive conversions for their own error types, while specific errors like `PoisonError` are often handled explicitly with `.map_err()` or wrapped in another error type.

In practice, many projects use `thiserror` to reduce boilerplate for their own error types. Since our goal here is to understand what `From` is doing under the hood, we'll write the conversion manually instead.

## Project Setup

Create a new project:

```
cargo new pastebin_api
cd pastebin_api
```

Open `Cargo.toml` and add the dependencies:

```toml
[package]
name = "pastebin_api"
version = "0.1.0"
edition = "2024"

[dependencies]
axum = "0.8"
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
uuid = { version = "1", features = ["v4"] }
chrono = { version = "0.4", features = ["serde"] }
```

We have two new dependencies compared to Part 1:

- `chrono` handles timestamps and expiry calculations. The `serde` feature lets us serialize and deserialize `DateTime<Utc>` as ISO 8601 strings, which is the standard JSON representation for dates.
- `uuid` for generating unique paste IDs, same as Part 1.

## The Project: Pastebin API

Our program will:

- Accept `POST /paste` with a JSON body containing content, an optional language, and an optional expiry duration
- Validate the input like reject empty content, oversized content, unsupported languages, and negative expiry
- Generate a unique ID and store the paste in memory with a creation timestamp
- Accept `GET /paste/{id}` that returns the paste as JSON
- Return 404 if the paste does not exist or has expired
- Return 400 for any validation failure with a specific message
- Return 500 for any internal failure with a generic message

### Data Model

A paste is defined by six fields:

```rust
struct Paste {
id: String,
content: String,
language: Option<String>,
created_at: DateTime<Utc>,
expires_at: Option<DateTime<Utc>>,
}
```

`language` is optional, not all pastes are code. `expires_at` is optional as pastes can live forever. `created_at` is always set, using the server's clock at creation time.

### Supported Languages

We limit language hints to a known set. This prevents typos like `"javscript"` and gives the client a clear error message:

```rust
const SUPPORTED_LANGUAGES: &[&str] = &[
"rust", "python", "javascript", "typescript", "go", "java", "c", "cpp",
"ruby", "php", "swift", "kotlin", "scala", "elixir", "haskell",
"bash", "sql", "html", "css", "json", "yaml", "toml", "markdown",
"plaintext",
];
```

### Maximum Content Length

We cap content at 500 KiB (500 × 1024 bytes). Larger pastes should be stored differently as object storage, not an in-memory API:

```rust
const MAX_CONTENT_LENGTH: usize = 500 * 1024; // 500 KiB
```

### The Full Code

Replace everything in `src/main.rs` with this:

```rust
use axum::{
extract::{Path, State},
http::StatusCode,
response::{IntoResponse, Response},
routing::{get, post},
Json, Router,
};
use chrono::{DateTime, Duration, Utc};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::{Arc, RwLock, RwLockReadGuard, RwLockWriteGuard};
use tokio::net::TcpListener;
use uuid::Uuid;

const SUPPORTED_LANGUAGES: &[&str] = &[
"rust", "python", "javascript", "typescript", "go", "java", "c", "cpp",
"ruby", "php", "swift", "kotlin", "scala", "elixir", "haskell",
"bash", "sql", "html", "css", "json", "yaml", "toml", "markdown",
"plaintext",
];

const MAX_CONTENT_LENGTH: usize = 500 * 1024;

#[derive(Clone)]
struct AppState {
pastes: Arc<RwLock<HashMap<String, Paste>>>,
}

#[derive(Clone, Serialize)]
struct Paste {
id: String,
content: String,
language: Option<String>,
created_at: DateTime<Utc>,
expires_at: Option<DateTime<Utc>>,
}

#[derive(Deserialize)]
struct CreatePasteRequest {
content: String,
language: Option<String>,
expires_in_seconds: Option<i64>,
}

#[derive(Serialize)]
struct CreatePasteResponse {
id: String,
}

#[derive(Serialize)]
struct GetPasteResponse {
id: String,
content: String,
language: Option<String>,
created_at: DateTime<Utc>,
expires_at: Option<DateTime<Utc>>,
}

enum AppError {
ValidationError(String),
NotFound(String),
InternalError(String),
}

impl IntoResponse for AppError {
fn into_response(self) -> Response {
    let (status, message) = match &self {
        AppError::ValidationError(msg) => (StatusCode::BAD_REQUEST, msg.clone()),
        AppError::NotFound(msg) => (StatusCode::NOT_FOUND, msg.clone()),
        AppError::InternalError(msg) => {
            eprintln!("internal error: {}", msg);
            (
                StatusCode::INTERNAL_SERVER_ERROR,
                "internal server error".to_string(),
            )
        }
    };

    let body = Json(serde_json::json!({
        "error": message,
    }));

    (status, body).into_response()
}
}

impl<T> From<std::sync::PoisonError<RwLockReadGuard<'_, T>>> for AppError {
fn from(_: std::sync::PoisonError<RwLockReadGuard<'_, T>>) -> Self {
    AppError::InternalError("lock poisoned".into())
}
}

impl<T> From<std::sync::PoisonError<RwLockWriteGuard<'_, T>>> for AppError {
fn from(_: std::sync::PoisonError<RwLockWriteGuard<'_, T>>) -> Self {
    AppError::InternalError("lock poisoned".into())
}
}

#[tokio::main]
async fn main() {
let state = AppState {
    pastes: Arc::new(RwLock::new(HashMap::new())),
};

let app = Router::new()
    .route("/paste", post(create_paste))
    .route("/paste/{id}", get(get_paste))
    .with_state(state);

let listener = TcpListener::bind("127.0.0.1:3000").await.unwrap();
println!("Listening on http://127.0.0.1:3000");

axum::serve(listener, app).await.unwrap();
}

async fn create_paste(
State(state): State<AppState>,
Json(payload): Json<CreatePasteRequest>,
) -> Result<impl IntoResponse, AppError> {
validate_create_request(&payload)?;

let id = &Uuid::new_v4().to_string()[..8];

let expires_at = payload.expires_in_seconds.map(|seconds| {
    Utc::now() + Duration::seconds(seconds)
});

let paste = Paste {
    id: id.to_string(),
    content: payload.content,
    language: payload.language,
    created_at: Utc::now(),
    expires_at,
};

state.pastes.write()?.insert(id.to_string(), paste);

Ok((
    StatusCode::CREATED,
    Json(CreatePasteResponse { id: id.to_string() }),
))
}

async fn get_paste(
State(state): State<AppState>,
Path(id): Path<String>,
) -> Result<impl IntoResponse, AppError> {
let pastes = state.pastes.read()?;

let paste = pastes
    .get(&id)
    .ok_or_else(|| AppError::NotFound(format!("paste with id '{}' not found", id)))?;

if let Some(expires_at) = paste.expires_at {
    if Utc::now() > expires_at {
        return Err(AppError::NotFound(format!(
            "paste with id '{}' has expired",
            id
        )));
    }
}

let response = GetPasteResponse {
    id: paste.id.clone(),
    content: paste.content.clone(),
    language: paste.language.clone(),
    created_at: paste.created_at,
    expires_at: paste.expires_at,
};

Ok((StatusCode::OK, Json(response)))
}

fn validate_create_request(req: &CreatePasteRequest) -> Result<(), AppError> {
if req.content.trim().is_empty() {
    return Err(AppError::ValidationError(
        "content must not be empty".into(),
    ));
}

if req.content.len() > MAX_CONTENT_LENGTH {
    return Err(AppError::ValidationError(format!(
        "content exceeds maximum length of {} bytes (got {} bytes)",
        MAX_CONTENT_LENGTH,
        req.content.len()
    )));
}

if let Some(ref lang) = req.language {
    if !SUPPORTED_LANGUAGES.contains(&lang.as_str()) {
        return Err(AppError::ValidationError(format!(
            "unsupported language '{}'. supported languages: {}",
            lang,
            SUPPORTED_LANGUAGES.join(", ")
        )));
    }
}

if let Some(seconds) = req.expires_in_seconds {
    if seconds <= 0 {
        return Err(AppError::ValidationError(
            "expires_in_seconds must be greater than 0".into(),
        ));
    }
}

Ok(())
}
```

Now, let me explain what we just did.

### The State

```rust
#[derive(Clone)]
struct AppState {
pastes: Arc<RwLock<HashMap<String, Paste>>>,
}
```

Same pattern as Part 1. `HashMap` mapping paste IDs to `Paste` structs, wrapped in `Arc<RwLock<...>>` for shared concurrent access. The difference is the value type, `Paste` is a struct with six fields, not just a `String`.

`Paste` derives `Clone` and `Serialize`. `Clone` is necessary because `HashMap::get` returns a reference, and we need to clone some fields when building the response. `Serialize` is for the `DateTime<Utc>` fields to serialize as ISO 8601 strings.

### The Request and Response Types

Three request/response types, designed for three concerns:

`CreatePasteRequest` - what the client sends. `content` is required. `language` and `expires_in_seconds` are optional. All validation logic references these fields.

`CreatePasteResponse` - what the server returns after creating a paste. Just the ID. The client can use that ID to fetch the full paste.

`GetPasteResponse` - the full paste returned on read. Includes the ID, content, language, timestamps, and expiry. This is the external representation. Notice it is a separate type from `Paste`, the internal `Paste` struct stores data, the response type formats it for the client.

> **Why separate request/response types from domain types?** The Paste struct is our internal representation. It has an `id` field that the client never sends on creation (we generate it). The `CreatePasteRequest` does not have an `id` field. Separating these types prevents the client from setting `id` and prevents us from accidentally returning internal fields. This is a pattern you will see in every well-structured Rust API.

### The AppError Enum and IntoResponse

```rust
enum AppError {
ValidationError(String),
NotFound(String),
InternalError(String),
}
```

Three variants, one per HTTP error category. The `IntoResponse` implementation is the single place where errors become HTTP responses:

```rust
impl IntoResponse for AppError {
fn into_response(self) -> Response {
    let (status, message) = match &self {
        AppError::ValidationError(msg) => (StatusCode::BAD_REQUEST, msg.clone()),
        AppError::NotFound(msg) => (StatusCode::NOT_FOUND, msg.clone()),
        AppError::InternalError(msg) => {
            eprintln!("internal error: {}", msg);
            (
                StatusCode::INTERNAL_SERVER_ERROR,
                "internal server error".to_string(),
            )
        }
    };

    let body = Json(serde_json::json!({
        "error": message,
    }));

    (status, body).into_response()
}
}
```

Every error response is `{"error": "..."}` with the appropriate status code. The client always gets a consistent JSON structure, regardless of which handler produced the error. If you later decide to add a `request_id` field to every error response, you change one function, not twenty handlers.

The `InternalError` variant logs the actual error to stderr via `eprintln!` but sends a generic message to the client: `{"error": "internal server error"}`. We will replace `eprintln!` with structured logging via `tracing` in Part 16, but the principle is the same, internal details are for the server operator, never for the client.

### From Impls for Lock Poisoning

```rust
impl<T> From<std::sync::PoisonError<RwLockReadGuard<'_, T>>> for AppError {
fn from(_: std::sync::PoisonError<RwLockReadGuard<'_, T>>) -> Self {
    AppError::InternalError("lock poisoned".into())
}
}

impl<T> From<std::sync::PoisonError<RwLockWriteGuard<'_, T>>> for AppError {
fn from(_: std::sync::PoisonError<RwLockWriteGuard<'_, T>>) -> Self {
    AppError::InternalError("lock poisoned".into())
}
}
```

These impls let us use `?` with lock acquisition. Instead of `.unwrap()`, we write:

```rust
let pastes = state.pastes.read()?;
```

The `?` converts `PoisonError<RwLockReadGuard<HashMap<...>>>` into `AppError::InternalError`, which `IntoResponse` converts into a 500 response. The handler never panics on a poisoned lock.

> **What is a poisoned lock?** When a thread panics while holding a `Mutex` or `RwLock`, the lock is "poisoned", subsequent attempts to acquire it return an error. This prevents other threads from reading data that might be in an inconsistent state (the panicking thread might have been halfway through an update). In our case, the data is a `HashMap` of pastes. Rust does allow recovering from a poisoned lock, but this example intentionally treats it as an internal server error and returns a `500` instead.

### The Create Paste Handler

```rust
async fn create_paste(
State(state): State<AppState>,
Json(payload): Json<CreatePasteRequest>,
) -> Result<impl IntoResponse, AppError> {
validate_create_request(&payload)?;

let id = &Uuid::new_v4().to_string()[..8];

let expires_at = payload.expires_in_seconds.map(|seconds| {
    Utc::now() + Duration::seconds(seconds)
});

let paste = Paste {
    id: id.to_string(),
    content: payload.content,
    language: payload.language,
    created_at: Utc::now(),
    expires_at,
};

state.pastes.write()?.insert(id.to_string(), paste);

Ok((
    StatusCode::CREATED,
    Json(CreatePasteResponse { id: id.to_string() }),
))
}
```

Notice the return type: `Result<impl IntoResponse, AppError>`. This is the key pattern. The `Ok` branch returns a success response, a 201 with a JSON body. The `Err` branch is an `AppError`, which Axum converts via `IntoResponse`. The `?` operator on `validate_create_request` and `state.pastes.write()` converts failures into `AppError` automatically.

`validate_create_request(&payload)?` runs all four validation checks before any work is done. If any check fails, the error is returned immediately. This is "fail fast", do not start creating a paste if the input is invalid.

`payload.expires_in_seconds.map(|seconds| { ... })` converts the optional expiry duration into an absolute timestamp. If `expires_in_seconds` is `None`, `expires_at` is `None`, the paste lives forever. If it is `Some(3600)`, `expires_at` is `Utc::now() + 1 hour`.

> For simplicity, we shorten the UUID to eight hexadecimal characters. Production systems typically use the full UUID or another identifier with collision guarantees appropriate for their scale.

### The Get Paste Handler

```rust
async fn get_paste(
State(state): State<AppState>,
Path(id): Path<String>,
) -> Result<impl IntoResponse, AppError> {
let pastes = state.pastes.read()?;

let paste = pastes
    .get(&id)
    .ok_or_else(|| AppError::NotFound(format!("paste with id '{}' not found", id)))?;

if let Some(expires_at) = paste.expires_at {
    if Utc::now() > expires_at {
        return Err(AppError::NotFound(format!(
            "paste with id '{}' has expired",
            id
        )));
    }
}

let response = GetPasteResponse {
    id: paste.id.clone(),
    content: paste.content.clone(),
    language: paste.language.clone(),
    created_at: paste.created_at,
    expires_at: paste.expires_at,
};

Ok((StatusCode::OK, Json(response)))
}
```

`pastes.get(&id)` returns `Option<&Paste>`. `.ok_or_else(|| ...)` converts `None` into an `AppError::NotFound`. The `?` propagates that error if the paste does not exist.

The expiry check happens after the lookup. If the paste exists but has expired, we return the same `NotFound` error. From the client's perspective, an expired paste and a non-existent paste are identical, you cannot access either one. A real application might delete expired pastes from the map periodically (a background cleanup task), but for this in-memory implementation, we check at read time.

The response construction clones the fields that are `String` or `Option<String>`. `DateTime<Utc>` implements `Copy`, so we can pass it directly. Cloning strings is not free, and cloning the paste content is the most expensive part of this handler. Production systems often avoid unnecessary copies by using shared ownership types such as `Arc<str>` or other ownership strategies, depending on the application's requirements.

### The Validation Function

```rust
fn validate_create_request(req: &CreatePasteRequest) -> Result<(), AppError> {
if req.content.trim().is_empty() {
    return Err(AppError::ValidationError("content must not be empty".into()));
}

if req.content.len() > MAX_CONTENT_LENGTH {
    return Err(AppError::ValidationError(format!(
        "content exceeds maximum length of {} bytes (got {} bytes)",
        MAX_CONTENT_LENGTH,
        req.content.len()
    )));
}

if let Some(ref lang) = req.language {
    if !SUPPORTED_LANGUAGES.contains(&lang.as_str()) {
        return Err(AppError::ValidationError(format!(
            "unsupported language '{}'. supported languages: {}",
            lang,
            SUPPORTED_LANGUAGES.join(", ")
        )));
    }
}

if let Some(seconds) = req.expires_in_seconds {
    if seconds <= 0 {
        return Err(AppError::ValidationError(
            "expires_in_seconds must be greater than 0".into(),
        ));
    }
}

Ok(())
}
```

A standalone function, not a method on the handler. This keeps the handler clean and makes the validation logic testable independently, you can call `validate_create_request` in a unit test without setting up an entire Axum router.

Four checks, each returning a specific error message:

1. **Content must not be empty** - `"".trim()` is empty, `"   ".trim()` is empty, `"hi".trim()` is not empty. Trimming prevents pastes that look empty but contain only whitespace.

2. **Content must not exceed 500 KiB** - includes the actual byte count in the error message so the client knows how much to trim.

3. **Language must be supported** - lists all supported languages in the error message. This is a self-documenting API: the error response tells the client what valid values look like.

4. **Expiry must be positive** - rejects zero and negative values. Zero-second expiry is nonsensical. Negative expiry is almost certainly a bug.

## Running the Project

Start the server:

```
cargo run
```

You should see:

```
Listening on http://127.0.0.1:3000
```

### Create a Paste

```
curl -X POST http://localhost:3000/paste \
  -H "Content-Type: application/json" \
  -d '{"content": "fn main() {\n    println!(\"hello world\");\n}", "language": "rust"}'
```

Response:

```json
{"id":"a1b2c3d4"}
```

### Read the Paste

```
curl http://localhost:3000/paste/a1b2c3d4
```

Response:

```json
{
  "id": "a1b2c3d4",
  "content": "fn main() {\n    println!(\"hello world\");\n}",
  "language": "rust",
  "created_at": "2026-07-06T12:00:00Z",
  "expires_at": null
}
```

### Create a Paste with Expiry

```
curl -X POST http://localhost:3000/paste \
  -H "Content-Type: application/json" \
  -d '{"content": "this will expire soon", "expires_in_seconds": 5}'
```

Wait 6 seconds, then read it:

```
curl http://localhost:3000/paste/b2c3d4e5
```

Response:

```json
{"error":"paste with id 'b2c3d4e5' has expired"}
```

Status code: 404.

### Test Validation - Empty Content

```
curl -X POST http://localhost:3000/paste \
  -H "Content-Type: application/json" \
  -d '{"content": ""}'
```

Response:

```json
{"error":"content must not be empty"}
```

Status code: 400.

### Test Validation - Unsupported Language

```
curl -X POST http://localhost:3000/paste \
  -H "Content-Type: application/json" \
  -d '{"content": "some code", "language": "brainfuck"}'
```

Response:

```json
{"error":"unsupported language 'brainfuck'. supported languages: rust, python, ..."}
```

Status code: 400.

### Test Validation - Negative Expiry

```
curl -X POST http://localhost:3000/paste \
  -H "Content-Type: application/json" \
  -d '{"content": "some code", "expires_in_seconds": -10}'
```

Response:

```json
{"error":"expires_in_seconds must be greater than 0"}
```

Status code: 400.

### Test Structural Error - Missing Content

```
curl -X POST http://localhost:3000/paste \
  -H "Content-Type: application/json" \
  -d '{"language": "rust"}'
```

Response: `422 Unprocessable Entity`. Axum generates this response before our handler runs because deserializing the request into `CreatePasteRequest` fails when the required `content` field is missing.

### Test Structural Error - Wrong Content-Type

```
curl -X POST http://localhost:3000/paste \
  -H "Content-Type: text/plain" \
  -d '{"content": "hello"}'
```

Response: 415 Unsupported Media Type. Axum rejects this because the request is not using a JSON media type.

### Test Not Found

```
curl http://localhost:3000/paste/nonexistent
```

Response:

```json
{"error":"paste with id 'nonexistent' not found"}
```

Status code: 404.

## The Boundary Between Framework and Application

Look at the curl examples again. There are three categories of failure:

1. **Structural errors** (422, 415) - handled by Axum automatically. Wrong content types, malformed JSON, and deserialization failures. You never write code for these.

2. **Semantic errors** (400) - handled by your validation function. Empty content, unsupported language, negative expiry. You write explicit checks and return `AppError::ValidationError`.

3. **Resource errors** (404) - handled by your handler logic. Missing paste, expired paste. You check the lookup result and return `AppError::NotFound`.

This is the boundary Axum draws. The framework handles everything it can detect from the structure of the request. You handle everything that requires understanding the meaning of the data. There is no overlap, no ambiguity, no framework guessing what status code your error should be.

This is the opposite of "convention over configuration." Axum says: "I will handle the HTTP plumbing - parsing, routing, serialization. You handle every business decision, including what status code means what. There is no hidden application-level error handling."

## How an Error Flows Through

Let us trace a single `GET /paste/nonexistent` request to see how the `AppError` pattern works end-to-end:

1. The router matches `/paste/{id}` with `id = "nonexistent"` and selects `get_paste` as the handler.

2. `State<AppState>` extracts the state (clones the `Arc`, cheap).

3. `Path<String>` extracts `"nonexistent"` from the path.

4. The handler runs. `state.pastes.read()?` succeeds (the lock is not poisoned). `pastes.get(&id)` returns `None`. `.ok_or_else(|| AppError::NotFound(...))` converts `None` into `Err(AppError::NotFound("paste with id 'nonexistent' not found"))`. The `?` operator propagates the error out of the handler.

5. The handler returns `Err(AppError::NotFound(...))`. Because the return type is `Result<impl IntoResponse, AppError>`, Axum calls `AppError::into_response()` on the error.

6. `AppError::into_response()` matches `NotFound`, sets the status to 404, formats the error message as `{"error": "paste with id 'nonexistent' not found"}`, and returns an `http::Response`.

7. The response travels back through Hyper and Tokio, same as a success response.

Compare this to the Part 1 error handling: an ad-hoc `(StatusCode::NOT_FOUND, &str)` in the handler. The new pattern separates error construction (in the handler) from error formatting (in `IntoResponse`). The handler says *what* went wrong. The `IntoResponse` implementation decides *how* to represent that to the client.

## What We Skipped

- **Persistence**: Pastes live in memory and disappear on restart. Part 3 replaces the `HashMap` with PostgreSQL via SQLx.
- **Pagination**: We return a single paste by ID. A real API would have `GET /pastes?language=rust&page=1` for listing. We touched on `Query` parameters, but full pagination with SQL joins comes in Part 3.
- **Content type validation on read**: A paste with `language: "markdown"` would ideally return `Content-Type: text/markdown`. For simplicity, we always return JSON.
- **Expired paste cleanup**: Expired pastes stay in the `HashMap` forever, only rejected at read time. A background task that periodically removes expired entries would be the production pattern. Background tasks are Part 9.
- **The thiserror crate**: We wrote `From` impls manually to show the mechanism. In practice, `thiserror` derives these for you. The manual impls are educational; the derive macro is what you use at work.

## Conclusion

In this post, you built a Pastebin API while learning Axum's extractors, validation, and centralized error handling. Next, we'll replace the in-memory `HashMap` with PostgreSQL and SQLx.

If you like reading this, please subscribe and share this with others. It will really help me and motivate me to keep publishing more such articles.

# Learn Axum Persistence and Transactions with SQLx by Building a Bookmark Manager

> In this post, we are going to build a Bookmark manager API with PostgreSQL

Source: https://blog.sheerluck.dev/posts/axum/learn-axum-persistence-and-transaction-by-building-a-bookmark-manager/ · Markdown: https://blog.sheerluck.dev/posts/axum/learn-axum-persistence-and-transaction-by-building-a-bookmark-manager/index.md

In Part 1 and Part 2, we built a URL shortener and a Pastebin API. Both had the same limitation: data lived in a `HashMap` wrapped in `Arc<RwLock<...>>`. Restart the server, everything disappears. That was intentional. We were learning routing, extractors, and error handling, persistence would have been a distraction.

![Axum Bookmark Manager](/images/axum-bookmark-manager.png)

In this post, we are going to fix that. We will replace the `HashMap` with PostgreSQL and SQLx. Adding a real database introduces connection pooling, schema design, migrations, compile-time-checked queries, joins, pagination, the N+1 query trap, explicit transactions, and isolation levels. Every one of these concepts matters in production, and every one of them is easier to learn in a single-service project than under the pressure of a distributed system.

Let's start, I can't wait.

Get the source code from [here](https://github.com/MrSheerluck/rust-bookmark-manager-api)

## The Problem with In-Memory Storage

The `HashMap` we used in Parts 1 and 2 has three problems. First, data is lost on restart. Second, capacity is bounded by available RAM. Third, and most subtly, there is no transactional safety. In Part 2, we created a paste by inserting it into a map. That operation is atomic only because `HashMap::insert` is a single method call. The moment you have a multi-step operation like "create a bookmark and attach three tags" and the third tag insertion fails, you have a bookmark in the map with two orphaned tags and no way to undo the first two inserts.

Real persistence solves all three: durability (survives restarts), scalability (data lives on disk, not in RAM), and atomicity (all-or-nothing multi-step operations via transactions).

## What is SQLx?

SQLx is a Rust crate for talking to SQL databases. It is async-native (built on Tokio), supports PostgreSQL, MySQL, and SQLite, and provides connection pooling. But its defining feature, the one that matters to this article is **compile-time checking of SQL queries**.

In most ORMs and query builders, a typo in a column name or a type mismatch between a Rust struct and a SQL column is a runtime error. You write `SELECT tite FROM bookmarks`, deploy to production, and find out when the first request hits that endpoint and panics. SQLx validates those queries during compilation. It parses every SQL string in `query!` and `query_as!` and verifies it against either a live database schema or the cached metadata in the `.sqlx/` directory. If a column is missing, the types don't match, or the SQL is invalid, compilation fails before the application ever runs.

`cargo sqlx prepare` connects to your database and generates query metadata in the `.sqlx/` directory. During offline compilation, the macros read this metadata instead of connecting to a live database. This allows SQL queries to remain compile-time checked even in CI environments without database access. If a column is missing, `cargo build` fails with a clear message pointing to the exact query and the exact mismatch. For queries written with SQLx's checked macros, typos and schema mismatches are caught before the application is built.

> **Why not an ORM like Diesel?** SQLx is deliberately *not* an ORM. It does not generate SQL from Rust structs. You write raw SQL in macros, and SQLx checks it for you. This gives you full control over the SQL joins, window functions, CTEs while still catching mistakes at compile time. If you prefer ORMs, Diesel is the go-to in Rust. This series uses SQLx because the goal is to learn backend engineering, and I want you to understand the SQL that your application is actually running.

## Connection Pooling

A connection pool is a cache of open database connections. Creating a new TCP connection to Postgres for every request is slow, TLS handshakes, authentication, and session setup take milliseconds. A pool keeps a set of connections open and ready, handing one to each request that needs it and returning it when the request is done.

SQLx provides `PgPool` for this:

```rust
let pool = PgPoolOptions::new()
.max_connections(10)
.connect(&database_url)
.await?;
```

`max_connections(10)` limits the pool to 10 simultaneous connections. Every request handler can call `pool.acquire()` to get a connection from the pool. If all 10 are in use, the 11th request waits until one is returned. Acquiring a connection from the pool is much cheaper than establishing a new database connection because the TCP connection, authentication, and session setup have already been completed.

The pool itself is cheap to clone. `PgPool` wraps an `Arc<PoolInner>`, so cloning it just increments a reference count exactly like cloning an `Arc`. This means you can put the pool directly in your `AppState` and use `#[derive(Clone)]` on the state struct without wrapping the pool in an extra `Arc`:

```rust
#[derive(Clone)]
struct AppState {
pool: PgPool,  // no Arc needed, PgPool is already Arc inside
}
```

## Database Migrations

A schema changes over time. You add a `description` column to `bookmarks`, or an index on `created_at`. These changes are called migrations. SQLx provides a CLI for managing them:

```
cargo install sqlx-cli --no-default-features --features postgres
sqlx migrate add initial_schema
```

This creates a file in `migrations/` with a timestamp prefix. You write your SQL in that file, and then:

```
sqlx migrate run
```

This applies any unapplied migrations to your database. SQLx tracks which migrations have been run in a `_sqlx_migrations` table, so it never applies the same migration twice.

Our application also runs migrations at startup via `sqlx::migrate!("./migrations").run(&pool)`. This keeps the schema synchronized with the application during development. Some production deployments instead run migrations as a separate deployment step before starting the application.

## Project Setup

We need PostgreSQL. The simplest way is Docker:

```
docker compose up -d
```

Create a `docker-compose.yml`:

```yaml
services:
  postgres:
image: postgres:16-alpine
container_name: bookmark_db
restart: unless-stopped
ports:
  - "5432:5432"
environment:
  POSTGRES_USER: bookmark
  POSTGRES_PASSWORD: bookmark
  POSTGRES_DB: bookmark
volumes:
  - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:
```

This gives us Postgres 16 on port 5432, with a `bookmark` database, user, and password. The `pgdata` volume persists data across container restarts.

Create a `.env` file for SQLx:

```
DATABASE_URL=postgres://bookmark:bookmark@localhost:5432/bookmark
```

Create the project:

```
cargo new bookmark_manager
cd bookmark_manager
```

Open `Cargo.toml`:

```toml
[package]
name = "bookmark_manager"
version = "0.1.0"
edition = "2024"

[dependencies]
axum = "0.8"
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sqlx = { version = "0.8", features = ["runtime-tokio", "tls-rustls", "postgres", "migrate", "chrono", "uuid"] }
uuid = { version = "1", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
dotenvy = "0.15"
```

The new dependencies worth explaining:

- `sqlx` with five features: `runtime-tokio` (async runtime), `tls-rustls` (TLS for secure connections), `postgres` (database driver), `migrate` (run migrations from code), and `chrono` + `uuid` (type support for `DateTime<Utc>` and `Uuid` columns). Without the `chrono` feature, SQLx wouldn't know how to read a `TIMESTAMPTZ` column into Rust's `DateTime<Utc>`.
- `dotenvy` loads the `.env` file at startup so `DATABASE_URL` is available as an environment variable.
- `uuid` now has the `serde` feature in addition to `v4` for serializing/deserializing UUIDs in JSON.

Install `sqlx-cli` and prepare the offline query cache:

```
cargo install sqlx-cli --no-default-features --features postgres
cargo sqlx prepare
```

The `prepare` command connects to your database, extracts the current schema, and generates a `.sqlx/` directory with cached query metadata. This is what the compiler uses to check your SQL. You should commit the `.sqlx/` directory to version control. It contains the metadata SQLx uses to verify queries during offline compilation, allowing CI to build the project without requiring a running database.

## The Schema

Create `migrations/20260712000000_initial_schema.sql`:

```sql
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

CREATE TABLE bookmarks (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
url TEXT NOT NULL,
title TEXT NOT NULL,
description TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE tags (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
name TEXT NOT NULL UNIQUE
);

CREATE TABLE bookmark_tags (
bookmark_id UUID NOT NULL REFERENCES bookmarks(id) ON DELETE CASCADE,
tag_id UUID NOT NULL REFERENCES tags(id) ON DELETE CASCADE,
PRIMARY KEY (bookmark_id, tag_id)
);

CREATE INDEX idx_bookmarks_created_at ON bookmarks(created_at DESC);
CREATE INDEX idx_bookmark_tags_bookmark_id ON bookmark_tags(bookmark_id);
CREATE INDEX idx_bookmark_tags_tag_id ON bookmark_tags(tag_id);
```

Now, let me explain each table and the design decisions.

### bookmarks

The core entity. Each row has a UUID primary key, a URL (the thing we are bookmarking), a title, an optional description, and two timestamps. `UUID` primary keys have two advantages over auto-incrementing integers in an API: they are unguessable (no sequential IDs to enumerate), and they can be generated client-side or server-side without coordination.

`uuid_generate_v4()` generates a random (v4) UUID. This requires the `uuid-ossp` extension, enabled at the top of the migration. Random UUIDs are slightly slower to insert than sequential identifiers because they fragment B-tree indexes. For many web applications this overhead is acceptable, while write-heavy systems often prefer sequential identifiers or newer UUID variants such as UUIDv7.

### tags

Tags are shared across bookmarks. A tag like `rust` can be attached to many bookmarks. The `name` column has a `UNIQUE` constraint, you cannot insert the same tag name twice. This constraint is essential because without it, two concurrent requests could create duplicate `rust` rows in the `tags` table, and the `bookmark_tags` junction table would then reference two different rows for what should be the same tag.

### bookmark_tags

This is the **junction table** (also called a join table, association table, or linking table). It models the many-to-many relationship between bookmarks and tags. One bookmark can have many tags. One tag can be attached to many bookmarks. A relational database cannot represent this directly with a foreign key in either table, you need a third table that pairs them.

Each row in `bookmark_tags` is a pair of foreign keys: `bookmark_id` references `bookmarks(id)`, and `tag_id` references `tags(id)`. The composite primary key `(bookmark_id, tag_id)` ensures you can never attach the same tag to the same bookmark twice. `ON DELETE CASCADE` on both foreign keys means deleting a bookmark automatically removes all its tag associations, and deleting a tag removes all its associations.

### Indexes

Three indexes, designed for our access patterns:

- `idx_bookmarks_created_at` supports the `ORDER BY created_at DESC` used in the list endpoint. Without this index, Postgres would sort the entire `bookmarks` table on every request.
- `idx_bookmark_tags_bookmark_id` supports the `WHERE bt.bookmark_id = $1` join used to fetch tags for a bookmark.
- `idx_bookmark_tags_tag_id` supports the `WHERE t.name = $1` join used to filter bookmarks by tag.

Indexes are a trade-off: they speed up reads at the cost of slower writes (every insert must also update the index) and additional disk usage. For this application, reads are far more common than writes, users browse their bookmarks more often than they create new ones so the indexes are worth it.

## The Project: Bookmark Manager

Our program will:

- Accept `POST /bookmarks` with a JSON body containing a URL, title, optional description, and optional tags
- Create the bookmark and attach any tags in a single transaction
- Accept `GET /bookmarks` with optional `?page=`, `?per_page=`, and `?tag=` query parameters for paginated listing
- Accept `GET /bookmarks/{id}` that returns a single bookmark with its tags
- Accept `PUT /bookmarks/{id}` to update a bookmark's fields
- Accept `DELETE /bookmarks/{id}` to delete a bookmark (tags are cascade-deleted)
- Accept `POST /bookmarks/{id}/tags` to attach new tags to an existing bookmark
- Accept `DELETE /bookmarks/{id}/tags/{tag_id}` to detach a single tag

### Data Model

The internal `Bookmark` struct maps directly to the `bookmarks` table:

```rust
#[derive(Debug, Serialize, sqlx::FromRow)]
struct Bookmark {
id: Uuid,
url: String,
title: String,
description: Option<String>,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}
```

`#[derive(sqlx::FromRow)]` is what makes `query_as!(Bookmark, ...)` work. It generates code that reads each column from a database row and constructs a `Bookmark`. The field names in the struct must match the column names in the database, and the types must be compatible  `Uuid` ↔ `UUID`, `String` ↔ `TEXT`, `Option<String>` ↔ nullable `TEXT`, `DateTime<Utc>` ↔ `TIMESTAMPTZ`. If any field is missing or has the wrong type, `cargo build` catches it.

`Tag` follows the same pattern:

```rust
#[derive(Debug, Clone, Serialize, sqlx::FromRow)]
struct Tag {
id: Uuid,
name: String,
}
```

`BookmarkResponse` is the external representation. It includes tags alongside the bookmark fields. This is a separate type from `Bookmark` , the internal `Bookmark` struct is a direct row mapping, while `BookmarkResponse` includes computed data (the joined tags):

```rust
#[derive(Debug, Serialize)]
struct BookmarkResponse {
id: Uuid,
url: String,
title: String,
description: Option<String>,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
tags: Vec<Tag>,
}
```

Why separate `BookmarkResponse` from `Bookmark`? `Bookmark` represents a single row in the `bookmarks` table. `BookmarkResponse` represents the JSON the client sees, which includes tags fetched from a separate query. The response type is a view, it combines data from multiple sources into one shape. The internal type is a model, it maps directly to storage. Keeping them separate lets you change the database schema without changing the API response shape, and vice versa.

### The Full Code

Replace everything in `src/main.rs` with this:

```rust
use axum::{
extract::{Path, Query, State},
http::StatusCode,
response::{IntoResponse, Response},
routing::{delete, get, post, put},
Json, Router,
};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use sqlx::postgres::PgPoolOptions;
use sqlx::PgPool;
use uuid::Uuid;

#[derive(Clone)]
struct AppState {
pool: PgPool,
}

#[derive(Debug, Serialize, sqlx::FromRow)]
struct Bookmark {
id: Uuid,
url: String,
title: String,
description: Option<String>,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
}

#[derive(Debug, Clone, Serialize, sqlx::FromRow)]
struct Tag {
id: Uuid,
name: String,
}

#[derive(Debug, Deserialize)]
struct CreateBookmarkRequest {
url: String,
title: String,
description: Option<String>,
tags: Option<Vec<String>>,
}

#[derive(Debug, Deserialize)]
struct UpdateBookmarkRequest {
url: Option<String>,
title: Option<String>,
description: Option<String>,
}

#[derive(Debug, Serialize)]
struct BookmarkResponse {
id: Uuid,
url: String,
title: String,
description: Option<String>,
created_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
tags: Vec<Tag>,
}

#[derive(Debug, Deserialize)]
struct PaginationParams {
page: Option<i64>,
per_page: Option<i64>,
tag: Option<String>,
}

#[derive(Debug, Serialize)]
struct PaginatedBookmarks {
bookmarks: Vec<BookmarkResponse>,
page: i64,
per_page: i64,
total: i64,
}

enum AppError {
ValidationError(String),
NotFound(String),
InternalError(String),
}

impl IntoResponse for AppError {
fn into_response(self) -> Response {
    let (status, message) = match &self {
        AppError::ValidationError(msg) => (StatusCode::BAD_REQUEST, msg.clone()),
        AppError::NotFound(msg) => (StatusCode::NOT_FOUND, msg.clone()),
        AppError::InternalError(msg) => {
            eprintln!("internal error: {}", msg);
            (StatusCode::INTERNAL_SERVER_ERROR, "internal server error".to_string())
        }
    };

    let body = Json(serde_json::json!({ "error": message }));
    (status, body).into_response()
}
}

impl From<sqlx::Error> for AppError {
fn from(err: sqlx::Error) -> Self {
    match err {
        sqlx::Error::RowNotFound => {
            AppError::NotFound("resource not found".into())
        }
        _ => AppError::InternalError(format!("database error: {}", err)),
    }
}
}

#[tokio::main]
async fn main() {
dotenvy::dotenv().ok();

let database_url =
    std::env::var("DATABASE_URL").expect("DATABASE_URL must be set");

let pool = PgPoolOptions::new()
    .max_connections(10)
    .connect(&database_url)
    .await
    .expect("failed to connect to database");

sqlx::migrate!("./migrations")
    .run(&pool)
    .await
    .expect("failed to run migrations");

let state = AppState { pool };

let app = Router::new()
    .route("/bookmarks", post(create_bookmark))
    .route("/bookmarks", get(list_bookmarks))
    .route("/bookmarks/{id}", get(get_bookmark))
    .route("/bookmarks/{id}", put(update_bookmark))
    .route("/bookmarks/{id}", delete(delete_bookmark))
    .route("/bookmarks/{id}/tags", post(attach_tags))
    .route("/bookmarks/{id}/tags/{tag_id}", delete(detach_tag))
    .route("/healthz", get(healthz))
    .with_state(state);

let listener = tokio::net::TcpListener::bind("127.0.0.1:3000")
    .await
    .unwrap();
println!("Listening on http://127.0.0.1:3000");

axum::serve(listener, app).await.unwrap();
}

async fn healthz() -> impl IntoResponse {
StatusCode::OK
}

async fn create_bookmark(
State(state): State<AppState>,
Json(payload): Json<CreateBookmarkRequest>,
) -> Result<impl IntoResponse, AppError> {
if payload.url.trim().is_empty() || payload.title.trim().is_empty() {
    return Err(AppError::ValidationError(
        "url and title must not be empty".into(),
    ));
}

let mut tx = state.pool.begin().await?;

let bookmark = sqlx::query_as!(
    Bookmark,
    r#"INSERT INTO bookmarks (url, title, description)
       VALUES ($1, $2, $3)
       RETURNING id, url, title, description, created_at, updated_at"#,
    payload.url,
    payload.title,
    payload.description,
)
.fetch_one(&mut *tx)
.await?;

let mut tags = Vec::new();

if let Some(ref tag_names) = payload.tags {
    for name in tag_names {
        let tag = sqlx::query_as!(
            Tag,
            r#"INSERT INTO tags (name) VALUES ($1)
               ON CONFLICT (name) DO UPDATE SET name = EXCLUDED.name
               RETURNING id, name"#,
            name.trim().to_lowercase(),
        )
        .fetch_one(&mut *tx)
        .await?;

        sqlx::query!(
            "INSERT INTO bookmark_tags (bookmark_id, tag_id) VALUES ($1, $2)",
            bookmark.id,
            tag.id,
        )
        .execute(&mut *tx)
        .await?;

        tags.push(tag);
    }
}

tx.commit().await?;

Ok((
    StatusCode::CREATED,
    Json(BookmarkResponse {
        id: bookmark.id,
        url: bookmark.url,
        title: bookmark.title,
        description: bookmark.description,
        created_at: bookmark.created_at,
        updated_at: bookmark.updated_at,
        tags,
    }),
))
}

async fn get_bookmark(
State(state): State<AppState>,
Path(id): Path<Uuid>,
) -> Result<impl IntoResponse, AppError> {
let bookmark = sqlx::query_as!(
    Bookmark,
    "SELECT id, url, title, description, created_at, updated_at FROM bookmarks WHERE id = $1",
    id,
)
.fetch_optional(&state.pool)
.await?
.ok_or_else(|| AppError::NotFound(format!("bookmark with id '{}' not found", id)))?;

let tags = sqlx::query_as!(
    Tag,
    r#"SELECT t.id, t.name
       FROM tags t
       INNER JOIN bookmark_tags bt ON bt.tag_id = t.id
       WHERE bt.bookmark_id = $1"#,
    bookmark.id,
)
.fetch_all(&state.pool)
.await?;

Ok(Json(BookmarkResponse {
    id: bookmark.id,
    url: bookmark.url,
    title: bookmark.title,
    description: bookmark.description,
    created_at: bookmark.created_at,
    updated_at: bookmark.updated_at,
    tags,
}))
}

async fn list_bookmarks(
State(state): State<AppState>,
Query(params): Query<PaginationParams>,
) -> Result<impl IntoResponse, AppError> {
let page = params.page.unwrap_or(1).max(1);
let per_page = params.per_page.unwrap_or(20).clamp(1, 100);
let offset = (page - 1) * per_page;

let (bookmarks, total): (Vec<Bookmark>, i64) = if let Some(ref tag_name) = params.tag {
    let total = sqlx::query_scalar!(
        r#"SELECT COUNT(DISTINCT b.id)::bigint
           FROM bookmarks b
           INNER JOIN bookmark_tags bt ON bt.bookmark_id = b.id
           INNER JOIN tags t ON t.id = bt.tag_id
           WHERE t.name = $1"#,
        tag_name.trim().to_lowercase(),
    )
    .fetch_one(&state.pool)
    .await?
    .unwrap_or(0);

    let bookmarks = sqlx::query_as!(
        Bookmark,
        r#"SELECT DISTINCT b.id, b.url, b.title, b.description, b.created_at, b.updated_at
           FROM bookmarks b
           INNER JOIN bookmark_tags bt ON bt.bookmark_id = b.id
           INNER JOIN tags t ON t.id = bt.tag_id
           WHERE t.name = $1
           ORDER BY b.created_at DESC
           LIMIT $2 OFFSET $3"#,
        tag_name.trim().to_lowercase(),
        per_page,
        offset,
    )
    .fetch_all(&state.pool)
    .await?;

    (bookmarks, total)
} else {
    let total = sqlx::query_scalar!("SELECT COUNT(*)::bigint FROM bookmarks")
        .fetch_one(&state.pool)
        .await?
        .unwrap_or(0);

    let bookmarks = sqlx::query_as!(
        Bookmark,
        r#"SELECT id, url, title, description, created_at, updated_at
           FROM bookmarks
           ORDER BY created_at DESC
           LIMIT $1 OFFSET $2"#,
        per_page,
        offset,
    )
    .fetch_all(&state.pool)
    .await?;

    (bookmarks, total)
};

let bookmark_ids: Vec<Uuid> = bookmarks.iter().map(|b| b.id).collect();

#[derive(Debug, sqlx::FromRow)]
struct TagWithBookmark {
    id: Uuid,
    name: String,
    bookmark_id: Uuid,
}

let all_tags: Vec<TagWithBookmark> = if bookmark_ids.is_empty() {
    Vec::new()
} else {
    sqlx::query_as!(
        TagWithBookmark,
        r#"SELECT t.id, t.name, bt.bookmark_id
           FROM tags t
           INNER JOIN bookmark_tags bt ON bt.tag_id = t.id
           WHERE bt.bookmark_id = ANY($1)"#,
        &bookmark_ids as &[Uuid],
    )
    .fetch_all(&state.pool)
    .await?
};

use std::collections::HashMap;
let mut tags_by_bookmark: HashMap<Uuid, Vec<Tag>> = HashMap::new();
for row in all_tags {
    tags_by_bookmark
        .entry(row.bookmark_id)
        .or_default()
        .push(Tag {
            id: row.id,
            name: row.name,
        });
}

let bookmark_responses: Vec<BookmarkResponse> = bookmarks
    .into_iter()
    .map(|bookmark| {
        let tags = tags_by_bookmark
            .get(&bookmark.id)
            .cloned()
            .unwrap_or_default();
        BookmarkResponse {
            id: bookmark.id,
            url: bookmark.url,
            title: bookmark.title,
            description: bookmark.description,
            created_at: bookmark.created_at,
            updated_at: bookmark.updated_at,
            tags,
        }
    })
    .collect();

Ok(Json(PaginatedBookmarks {
    bookmarks: bookmark_responses,
    page,
    per_page,
    total,
}))
}

async fn update_bookmark(
State(state): State<AppState>,
Path(id): Path<Uuid>,
Json(payload): Json<UpdateBookmarkRequest>,
) -> Result<impl IntoResponse, AppError> {
let existing = sqlx::query_as!(
    Bookmark,
    "SELECT id, url, title, description, created_at, updated_at FROM bookmarks WHERE id = $1",
    id,
)
.fetch_optional(&state.pool)
.await?
.ok_or_else(|| AppError::NotFound(format!("bookmark with id '{}' not found", id)))?;

let url = payload.url.unwrap_or(existing.url);
let title = payload.title.unwrap_or(existing.title);
let description = payload.description.or(existing.description);

let bookmark = sqlx::query_as!(
    Bookmark,
    r#"UPDATE bookmarks
       SET url = $1, title = $2, description = $3, updated_at = NOW()
       WHERE id = $4
       RETURNING id, url, title, description, created_at, updated_at"#,
    url,
    title,
    description,
    id,
)
.fetch_one(&state.pool)
.await?;

let tags = sqlx::query_as!(
    Tag,
    r#"SELECT t.id, t.name
       FROM tags t
       INNER JOIN bookmark_tags bt ON bt.tag_id = t.id
       WHERE bt.bookmark_id = $1"#,
    bookmark.id,
)
.fetch_all(&state.pool)
.await?;

Ok(Json(BookmarkResponse {
    id: bookmark.id,
    url: bookmark.url,
    title: bookmark.title,
    description: bookmark.description,
    created_at: bookmark.created_at,
    updated_at: bookmark.updated_at,
    tags,
}))
}

async fn delete_bookmark(
State(state): State<AppState>,
Path(id): Path<Uuid>,
) -> Result<impl IntoResponse, AppError> {
let result = sqlx::query!("DELETE FROM bookmarks WHERE id = $1", id)
    .execute(&state.pool)
    .await?;

if result.rows_affected() == 0 {
    return Err(AppError::NotFound(format!(
        "bookmark with id '{}' not found",
        id
    )));
}

Ok(StatusCode::NO_CONTENT)
}

async fn attach_tags(
State(state): State<AppState>,
Path(id): Path<Uuid>,
Json(payload): Json<Vec<String>>,
) -> Result<impl IntoResponse, AppError> {
let exists = sqlx::query_scalar!("SELECT COUNT(*)::bigint FROM bookmarks WHERE id = $1", id)
    .fetch_one(&state.pool)
    .await?
    .unwrap_or(0)
    > 0;

if !exists {
    return Err(AppError::NotFound(format!(
        "bookmark with id '{}' not found",
        id
    )));
}

let mut tx = state.pool.begin().await?;
let mut tags = Vec::new();

for name in payload {
    let tag = sqlx::query_as!(
        Tag,
        r#"INSERT INTO tags (name) VALUES ($1)
           ON CONFLICT (name) DO UPDATE SET name = EXCLUDED.name
           RETURNING id, name"#,
        name.trim().to_lowercase(),
    )
    .fetch_one(&mut *tx)
    .await?;

    sqlx::query!(
        "INSERT INTO bookmark_tags (bookmark_id, tag_id) VALUES ($1, $2) ON CONFLICT DO NOTHING",
        id,
        tag.id,
    )
    .execute(&mut *tx)
    .await?;

    tags.push(tag);
}

tx.commit().await?;

Ok((StatusCode::CREATED, Json(tags)))
}

async fn detach_tag(
State(state): State<AppState>,
Path((bookmark_id, tag_id)): Path<(Uuid, Uuid)>,
) -> Result<impl IntoResponse, AppError> {
let result = sqlx::query!(
    "DELETE FROM bookmark_tags WHERE bookmark_id = $1 AND tag_id = $2",
    bookmark_id,
    tag_id,
)
.execute(&state.pool)
.await?;

if result.rows_affected() == 0 {
    return Err(AppError::NotFound("tag not attached to this bookmark".into()));
}

Ok(StatusCode::NO_CONTENT)
}
```

Now, let me explain each handler. Pay attention to the SQL, the queries are where the learning happens in this article.

### The State

```rust
#[derive(Clone)]
struct AppState {
pool: PgPool,
}
```

Notice what changed from Part 2. Previously, `AppState` held `Arc<RwLock<HashMap<String, Paste>>>`. Now it holds a `PgPool`. No `Arc`, no `RwLock`, no `HashMap`. The pool is inherently thread-safe (it uses `Arc` internally) and the database handles all concurrency.

### The From Impl for sqlx::Error

```rust
impl From<sqlx::Error> for AppError {
fn from(err: sqlx::Error) -> Self {
    match err {
        sqlx::Error::RowNotFound => {
            AppError::NotFound("resource not found".into())
        }
        _ => AppError::InternalError(format!("database error: {}", err)),
    }
}
}
```

This is new. In Parts 1 and 2, we had `From` impls for `PoisonError` on lock guards. Here we have one for `sqlx::Error`. This is what lets us use `?` after every database call, the error auto-converts into an `AppError`.

`sqlx::Error::RowNotFound` is the specific error returned when `fetch_one` finds zero rows. Notice we check for it explicitly and return a 404 instead of a 500. All other SQLx errors, connection failures, constraint violations, serialization failures map to `InternalError` with the full error string logged to stderr. In production, you would log this with `tracing` and discard the raw error string from the client response for security.

### The main Function

```rust
#[tokio::main]
async fn main() {
dotenvy::dotenv().ok();

let database_url =
    std::env::var("DATABASE_URL").expect("DATABASE_URL must be set");

let pool = PgPoolOptions::new()
    .max_connections(10)
    .connect(&database_url)
    .await
    .expect("failed to connect to database");

sqlx::migrate!("./migrations")
    .run(&pool)
    .await
    .expect("failed to run migrations");

let state = AppState { pool };

let app = Router::new()
    .route("/bookmarks", post(create_bookmark))
    .route("/bookmarks", get(list_bookmarks))
    .route("/bookmarks/{id}", get(get_bookmark))
    .route("/bookmarks/{id}", put(update_bookmark))
    .route("/bookmarks/{id}", delete(delete_bookmark))
    .route("/bookmarks/{id}/tags", post(attach_tags))
    .route("/bookmarks/{id}/tags/{tag_id}", delete(detach_tag))
    .route("/healthz", get(healthz))
    .with_state(state);

let listener = tokio::net::TcpListener::bind("127.0.0.1:3000")
    .await
    .unwrap();
println!("Listening on http://127.0.0.1:3000");

axum::serve(listener, app).await.unwrap();
}
```

`dotenvy::dotenv().ok()` loads the `.env` file into the process environment. The `.ok()` swallows the error if the file doesn't exist, which is fine for development.

`sqlx::migrate!("./migrations").run(&pool)` compiles the migration SQL into the binary at build time and runs any unapplied migrations at startup. This means you can deploy a new version of your application with a new migration, and the schema updates automatically. No separate migration step in your deployment pipeline.

The `Pool` is moved into `AppState`. After this, any handler can extract `State<AppState>` and call `state.pool.begin()` or use `&state.pool` directly for single queries.

### The Create Bookmark Handler - Transactions

```rust
async fn create_bookmark(
State(state): State<AppState>,
Json(payload): Json<CreateBookmarkRequest>,
) -> Result<impl IntoResponse, AppError> {
if payload.url.trim().is_empty() || payload.title.trim().is_empty() {
    return Err(AppError::ValidationError(
        "url and title must not be empty".into(),
    ));
}

let mut tx = state.pool.begin().await?;

let bookmark = sqlx::query_as!(
    Bookmark,
    r#"INSERT INTO bookmarks (url, title, description)
       VALUES ($1, $2, $3)
       RETURNING id, url, title, description, created_at, updated_at"#,
    payload.url,
    payload.title,
    payload.description,
)
.fetch_one(&mut *tx)
.await?;

let mut tags = Vec::new();

if let Some(ref tag_names) = payload.tags {
    for name in tag_names {
        let tag = sqlx::query_as!(
            Tag,
            r#"INSERT INTO tags (name) VALUES ($1)
               ON CONFLICT (name) DO UPDATE SET name = EXCLUDED.name
               RETURNING id, name"#,
            name.trim().to_lowercase(),
        )
        .fetch_one(&mut *tx)
        .await?;

        sqlx::query!(
            "INSERT INTO bookmark_tags (bookmark_id, tag_id) VALUES ($1, $2)",
            bookmark.id,
            tag.id,
        )
        .execute(&mut *tx)
        .await?;

        tags.push(tag);
    }
}

tx.commit().await?;

Ok((
    StatusCode::CREATED,
    Json(BookmarkResponse {
        id: bookmark.id,
        url: bookmark.url,
        title: bookmark.title,
        description: bookmark.description,
        created_at: bookmark.created_at,
        updated_at: bookmark.updated_at,
        tags,
    }),
))
}
```

This is the most important handler in the project. It demonstrates a **transaction**, a multi-step write that must either succeed entirely or leave the database unchanged.

`state.pool.begin().await?` starts a new transaction. This sends a `BEGIN` to Postgres. Everything executed on `tx` from this point until `tx.commit().await?` (which sends a `COMMIT`) is part of one atomic unit.

If any query inside the transaction fails for example, the tag insertion fails because of a constraint violation, the `?` operator returns an error from the handler. `tx` is dropped. When a `Transaction` is dropped without being committed, SQLx sends a `ROLLBACK` to Postgres, which undoes everything the transaction did. The bookmark that was inserted on line 164 is undone. The tags that were already inserted are undone. The database returns to the state it was in before `tx.begin()`.

This is the answer to the problem from the beginning of this article. Without transactions, a failure halfway through "create bookmark and attach three tags" leaves an orphaned bookmark in the database with some tags attached and others not. With transactions, it's all or nothing.

Let's trace the tag insertion logic step by step:

1. **`INSERT INTO tags ... ON CONFLICT (name) DO UPDATE SET name = EXCLUDED.name RETURNING id, name`** - This is an upsert. If a tag with this name already exists, Postgres returns the existing row. If it doesn't exist, Postgres inserts it and returns the new row. The `RETURNING` clause gives us the `id` and `name` of the row (whether it was newly inserted or already existed), so we know which `tag_id` to use in the junction table. The `ON CONFLICT DO UPDATE SET name = EXCLUDED.name` is a no-op update, it writes the existing value back to the row. We use it because we always need the tag's `id`. `ON CONFLICT DO NOTHING RETURNING ...` returns the inserted row only when a new row is created. If the conflict path is taken, it returns zero rows. Using `DO UPDATE ... RETURNING` guarantees that we receive the existing row in both cases.

2. **`INSERT INTO bookmark_tags ...`** - Creates the association between the bookmark and the tag in the junction table. No `ON CONFLICT` here, if the association already exists (because the client sent duplicate tags), the primary key constraint will reject it and the transaction rolls back. In production, you might handle this more gracefully with `ON CONFLICT DO NOTHING`.

3. **`tx.commit().await?`** - Commits the transaction. After this point, the bookmark and all its tags are durably stored. If the server crashes immediately after `commit()` returns, the data survives.

Every query in the transaction uses `&mut *tx` (a mutable reference to the transaction) instead of `&state.pool`. This is critical, queries on the pool use separate connections, which are not part of the transaction. If you accidentally used `&state.pool` for one of the tag insertions, that tag would be inserted outside the transaction and would survive a rollback.

> **What happens if the server crashes between two queries in the transaction?** The TCP connection to Postgres drops. Postgres detects the lost connection and automatically rolls back the transaction. The partial state is never persisted.

### The Get Bookmark Handler - Two Queries, Two Tables

```rust
async fn get_bookmark(
State(state): State<AppState>,
Path(id): Path<Uuid>,
) -> Result<impl IntoResponse, AppError> {
let bookmark = sqlx::query_as!(
    Bookmark,
    "SELECT id, url, title, description, created_at, updated_at FROM bookmarks WHERE id = $1",
    id,
)
.fetch_optional(&state.pool)
.await?
.ok_or_else(|| AppError::NotFound(format!("bookmark with id '{}' not found", id)))?;

let tags = sqlx::query_as!(
    Tag,
    r#"SELECT t.id, t.name
       FROM tags t
       INNER JOIN bookmark_tags bt ON bt.tag_id = t.id
       WHERE bt.bookmark_id = $1"#,
    bookmark.id,
)
.fetch_all(&state.pool)
.await?;

Ok(Json(BookmarkResponse {
    id: bookmark.id,
    url: bookmark.url,
    title: bookmark.title,
    description: bookmark.description,
    created_at: bookmark.created_at,
    updated_at: bookmark.updated_at,
    tags,
}))
}
```

Two queries on two separate connections from the pool. First, fetch the bookmark by its `Path(id)`, notice `Path<Uuid>` deserializes the path segment directly into a `Uuid`, which is more type-safe than `Path<String>`. Second, fetch all tags associated with that bookmark via the junction table.

`fetch_optional` returns `Option<Bookmark>`. If the row exists, we get `Some`. If it doesn't, we get `None`. The `.ok_or_else(|| ...)` converts `None` into an `AppError::NotFound`. If `fetch_optional` itself fails (network error, connection timeout), the `?` converts the `sqlx::Error` into `AppError::InternalError` via the `From` impl.

The tag query joins `tags` with `bookmark_tags` to find all tags attached to this bookmark. `INNER JOIN` means only tags that actually have an association are returned. If the bookmark has no tags, `fetch_all` returns an empty `Vec`, no error, just an empty tags list in the response.

### The List Bookmarks Handler - Pagination, Tag Filtering, and the N+1 Trap

This is the longest handler. Let's break it into three parts: pagination, the conditional query, and tag fetching.

**Pagination:**

```rust
let page = params.page.unwrap_or(1).max(1);
let per_page = params.per_page.unwrap_or(20).clamp(1, 100);
let offset = (page - 1) * per_page;
```

`.max(1)` prevents negative or zero page numbers. `.clamp(1, 100)` caps the page size between 1 and 100 - requesting 1000 items per page is either a bug or an abuse vector. `offset` is calculated from `(page - 1) * per_page`. Page 1, 20 per page gives offset 0. Page 2 gives offset 20. This is standard cursor-less pagination.

**The conditional query - tagged vs untagged:**

```rust
let (bookmarks, total): (Vec<Bookmark>, i64) = if let Some(ref tag_name) = params.tag {
// Tagged: count and fetch bookmarks with a specific tag
let total = sqlx::query_scalar!(
    r#"SELECT COUNT(DISTINCT b.id)::bigint
       FROM bookmarks b
       INNER JOIN bookmark_tags bt ON bt.bookmark_id = b.id
       INNER JOIN tags t ON t.id = bt.tag_id
       WHERE t.name = $1"#,
    tag_name.trim().to_lowercase(),
)
.fetch_one(&state.pool)
.await?
.unwrap_or(0);

let bookmarks = sqlx::query_as!(
    Bookmark,
    r#"SELECT DISTINCT b.id, b.url, b.title, b.description, b.created_at, b.updated_at
       FROM bookmarks b
       INNER JOIN bookmark_tags bt ON bt.bookmark_id = b.id
       INNER JOIN tags t ON t.id = bt.tag_id
       WHERE t.name = $1
       ORDER BY b.created_at DESC
       LIMIT $2 OFFSET $3"#,
    tag_name.trim().to_lowercase(),
    per_page,
    offset,
)
.fetch_all(&state.pool)
.await?;

(bookmarks, total)
} else {
// Untagged: count and fetch all bookmarks
let total = sqlx::query_scalar!("SELECT COUNT(*)::bigint FROM bookmarks")
    .fetch_one(&state.pool)
    .await?
    .unwrap_or(0);

let bookmarks = sqlx::query_as!(
    Bookmark,
    r#"SELECT id, url, title, description, created_at, updated_at
       FROM bookmarks
       ORDER BY created_at DESC
       LIMIT $1 OFFSET $2"#,
    per_page,
    offset,
)
.fetch_all(&state.pool)
.await?;

(bookmarks, total)
};
```

Two separate query blocks, one with tag filtering, one without. Conditionally branching on `params.tag` is simpler than building a dynamic query string, and it avoids SQL injection entirely.

In the tagged path, the query joins `bookmarks` → `bookmark_tags` → `tags` and filters by tag name. `DISTINCT` prevents duplicate bookmarks if a bookmark has multiple tags (the join would otherwise produce one row per tag per bookmark). The count query counts distinct bookmark IDs for the same reason, without `DISTINCT`, the count would include duplicates.

`query_scalar!` is a new macro. It's for queries that return a single column, like `COUNT(*)` or `SELECT name FROM ... WHERE id = $1`. It deserializes directly into a Rust scalar type (`i64`, `String`, etc.) instead of a struct.

**The tag fetching - avoiding the N+1 trap:**

Here is where this handler differs from a naive approach. A naive approach would be:

```rust
// DON'T DO THIS - the N+1 trap
for bookmark in &bookmarks {
let tags = sqlx::query_as!(Tag, "... WHERE bt.bookmark_id = $1", bookmark.id)
    .fetch_all(&state.pool)
    .await?;
// ...
}
```

This is the **N+1 query problem**. If the page has 20 bookmarks, you make 1 query for the bookmarks themselves, then 20 more queries for the tags (one per bookmark). That's 21 queries total. On the next page, another 21. At scale, this multiplies your database load by the page size and it's easy to write by accident because each lookup feels natural in a loop.

The fix: **fetch all tags for all bookmarks on the current page in a single query**, then group them in application code:

```rust
let bookmark_ids: Vec<Uuid> = bookmarks.iter().map(|b| b.id).collect();

#[derive(Debug, sqlx::FromRow)]
struct TagWithBookmark {
id: Uuid,
name: String,
bookmark_id: Uuid,
}

let all_tags: Vec<TagWithBookmark> = if bookmark_ids.is_empty() {
Vec::new()
} else {
sqlx::query_as!(
    TagWithBookmark,
    r#"SELECT t.id, t.name, bt.bookmark_id
       FROM tags t
       INNER JOIN bookmark_tags bt ON bt.tag_id = t.id
       WHERE bt.bookmark_id = ANY($1)"#,
    &bookmark_ids as &[Uuid],
)
.fetch_all(&state.pool)
.await?
};
```

`WHERE bt.bookmark_id = ANY($1)` with the slice `&bookmark_ids as &[Uuid]` tells Postgres "give me all tag associations for any of these bookmark IDs." Postgres uses the `idx_bookmark_tags_bookmark_id` index for each ID in the list and returns all matching rows in a single result set.

`TagWithBookmark` is a helper struct used only inside this handler. It includes `bookmark_id` so we know which bookmark each tag belongs to when grouping.

The `if bookmark_ids.is_empty()` check simply avoids making an unnecessary database query. `ANY()` works correctly with an empty array, but since we already know there are no bookmark IDs, we can return an empty vector immediately.

Then we group by bookmark ID:

```rust
let mut tags_by_bookmark: HashMap<Uuid, Vec<Tag>> = HashMap::new();
for row in all_tags {
tags_by_bookmark
    .entry(row.bookmark_id)
    .or_default()
    .push(Tag { id: row.id, name: row.name });
}
```

And finally map each bookmark to its response, looking up tags from the HashMap:

```rust
let bookmark_responses: Vec<BookmarkResponse> = bookmarks
.into_iter()
.map(|bookmark| {
    let tags = tags_by_bookmark
        .get(&bookmark.id)
        .cloned()
        .unwrap_or_default();
    BookmarkResponse { /* ... */ tags }
})
.collect();
```

The total number of queries is now 3 (bookmarks, count, tags), regardless of page size. This is the difference between an API that handles 1000 concurrent users and one that falls over at 10.

### The Update Bookmark Handler

```rust
async fn update_bookmark(
State(state): State<AppState>,
Path(id): Path<Uuid>,
Json(payload): Json<UpdateBookmarkRequest>,
) -> Result<impl IntoResponse, AppError> {
let existing = sqlx::query_as!(
    Bookmark,
    "SELECT id, url, title, description, created_at, updated_at FROM bookmarks WHERE id = $1",
    id,
)
.fetch_optional(&state.pool)
.await?
.ok_or_else(|| AppError::NotFound(format!("bookmark with id '{}' not found", id)))?;

let url = payload.url.unwrap_or(existing.url);
let title = payload.title.unwrap_or(existing.title);
let description = payload.description.or(existing.description);

let bookmark = sqlx::query_as!(
    Bookmark,
    r#"UPDATE bookmarks
       SET url = $1, title = $2, description = $3, updated_at = NOW()
       WHERE id = $4
       RETURNING id, url, title, description, created_at, updated_at"#,
    url,
    title,
    description,
    id,
)
.fetch_one(&state.pool)
.await?;

// ... fetch tags and return
}
```

A partial update pattern. The client sends only the fields it wants to change. Fields not present in the request default to their existing values from the database.

`payload.url.unwrap_or(existing.url)` takes the client-supplied value if present, otherwise falls back to the existing value. `payload.description.or(existing.description)` does the same for `Option<String>` , notice `or()` instead of `unwrap_or()`, since both are `Option` types.

`updated_at = NOW()` sets the timestamp to the current server time. This is done in SQL rather than Rust because the database clock is the single source of truth for when a row was last modified. If you set it in Rust, clock skew between application servers could cause inconsistent timestamps.

### The Delete Bookmark Handler - Cascade

```rust
async fn delete_bookmark(
State(state): State<AppState>,
Path(id): Path<Uuid>,
) -> Result<impl IntoResponse, AppError> {
let result = sqlx::query!("DELETE FROM bookmarks WHERE id = $1", id)
    .execute(&state.pool)
    .await?;

if result.rows_affected() == 0 {
    return Err(AppError::NotFound(format!(
        "bookmark with id '{}' not found",
        id
    )));
}

Ok(StatusCode::NO_CONTENT)
}
```

A single `DELETE` statement. The `ON DELETE CASCADE` foreign key on `bookmark_tags.bookmark_id` ensures that deleting a bookmark also deletes all rows in the junction table. Without `CASCADE`, you would need to delete from `bookmark_tags` first two queries, or a transaction. `CASCADE` handles it in one.

`rows_affected()` returns the number of rows that were modified. If it's 0, the bookmark didn't exist and we return 404.

### The Attach Tags Handler - Transactional Tag Creation

```rust
async fn attach_tags(
State(state): State<AppState>,
Path(id): Path<Uuid>,
Json(payload): Json<Vec<String>>,
) -> Result<impl IntoResponse, AppError> {
// Check bookmark exists
let exists = sqlx::query_scalar!("SELECT COUNT(*)::bigint FROM bookmarks WHERE id = $1", id)
    .fetch_one(&state.pool)
    .await?
    .unwrap_or(0) > 0;

if !exists {
    return Err(AppError::NotFound(...));
}

let mut tx = state.pool.begin().await?;
let mut tags = Vec::new();

for name in payload {
    // Upsert tag, insert into bookmark_tags
    // ...
}

tx.commit().await?;

Ok((StatusCode::CREATED, Json(tags)))
}
```

Similar tag creation logic to `create_bookmark`, but for an existing bookmark. The existence check happens before the transaction starts, no point opening a transaction if the bookmark doesn't exist.

The `ON CONFLICT DO NOTHING` on the `bookmark_tags` insert is important here: if the client sends a tag that's already attached, the insert is silently skipped instead of failing with a unique constraint violation. Without `DO NOTHING`, attaching `["rust", "rust"]` would cause the second insert to fail and roll back the entire transaction, losing the first tag.

### The Detach Tag Handler - Tuple Path Parameters

```rust
async fn detach_tag(
State(state): State<AppState>,
Path((bookmark_id, tag_id)): Path<(Uuid, Uuid)>,
) -> Result<impl IntoResponse, AppError> {
let result = sqlx::query!(
    "DELETE FROM bookmark_tags WHERE bookmark_id = $1 AND tag_id = $2",
    bookmark_id,
    tag_id,
)
.execute(&state.pool)
.await?;

if result.rows_affected() == 0 {
    return Err(AppError::NotFound("tag not attached to this bookmark".into()));
}

Ok(StatusCode::NO_CONTENT)
}
```

`Path<(Uuid, Uuid)>` is a new Axum trick. The route is `/bookmarks/{id}/tags/{tag_id}`. Axum deserializes the two path segments `{id}` and `{tag_id}` into a tuple `(Uuid, Uuid)`. This is cleaner than writing `Path(id): Path<Uuid>, Path(tag_id): Path<Uuid>` as separate extractor parameters, though both work. The tuple pattern works for up to 16 segments and saves a line of function signature.

> **Why not use `Path` for both?** You can `async fn detach_tag(Path(bookmark_id): Path<Uuid>, Path(tag_id): Path<Uuid>)`  but Axum's `Path` extractor can only appear once per handler since it's a `FromRequestParts` extractor limited to capturing a single path pattern. The tuple syntax `Path((a, b))` captures both in one extractor.

## Understanding Transactions

Transactions are the single most important database concept beyond basic CRUD. Here is the exact lifecycle of the `create_bookmark` transaction:

1. **`state.pool.begin().await?`** - SQLx acquires a connection from the pool and sends `BEGIN` to Postgres. Postgres marks this connection as "in a transaction." All subsequent queries on this connection are part of the transaction.

2. **`INSERT INTO bookmarks ...`** - The bookmark is inserted, but it is not visible to any other connection. Other users querying `bookmarks` will not see this row until the transaction commits. This is Postgres's default isolation level, `READ COMMITTED`, at work.

3. **`INSERT INTO tags ... RETURNING ...`** - Each tag is upserted. The `RETURNING` clause returns the tag's `id` and `name`, which we need for the junction table insert. The upsert uses the transaction's snapshot of the database — if another concurrent transaction creates a tag with the same name at the same moment, Postgres would block one of them (because of the `UNIQUE` constraint) and the second would see the first's row.

4. **`INSERT INTO bookmark_tags ...`** - The junction table row is inserted, linking the bookmark and the tag.

5. **`tx.commit().await?`** - SQLx sends `COMMIT` to Postgres. Postgres writes all the transaction's changes to the write-ahead log (WAL), making them durable. At this moment, the changes become visible to all other connections. The pooled connection is returned to the pool.

If any step fails, the `?` operator returns an `Err`, `tx` is dropped. SQLx's `Transaction` destructor sends `ROLLBACK` to Postgres. Postgres undoes all changes made during the transaction as if they never happened. No orphaned bookmarks. No partial tag attachments.

### What If Two Transactions Run at the Same Time?

The `name TEXT NOT NULL UNIQUE` constraint on `tags` is the key. If two concurrent requests both try to create a tag called `rust`:

1. Transaction A inserts `rust` into `tags`. Postgres acquires an exclusive lock on the new row.
2. Transaction B also tries to insert `rust`. Because of the `UNIQUE` constraint, Postgres blocks B until A commits or rolls back.
3. If A commits, B's insert fails with a unique violation but our query uses `ON CONFLICT (name) DO UPDATE`, so instead of failing, it returns the existing `rust` row that A inserted. B proceeds with A's tag ID.
4. If A rolls back, B's insert succeeds because the conflicting row no longer exists.

Without `ON CONFLICT`, B's insert would fail, the `?` would propagate the error, and B's entire transaction would roll back. The bookmark B was creating would be lost. The upsert pattern is a concurrency-safe way to "get or create" a shared resource like a tag.

## Isolation Levels - A Brief Look

Postgres's default isolation level is `READ COMMITTED`. You can see it if you run `SHOW default_transaction_isolation;` in `psql`. Here is what it means, and what the alternatives are:

### READ COMMITTED (the default)

Each statement in a transaction sees only data that was committed before the statement began, not before the transaction began. This means:

- **No dirty reads:** You never see uncommitted data from another transaction.
- **Non-repeatable reads are possible:** If transaction A reads a row, transaction B updates and commits that row, then A reads it again, A sees B's update. The row changed between reads within A's transaction.

This is sufficient for most applications. In our bookmark manager, the only write that involves multiple tables is `create_bookmark` (bookmark + tags), and that's wrapped in a transaction. Since the bookmark, tags, and their associations are all created inside a single transaction, other transactions do not observe any of those changes until the transaction commits.

### REPEATABLE READ

Each statement in a transaction sees only data that was committed before the **transaction** began. The snapshot is frozen at transaction start. Non-repeatable reads cannot happen. This is stronger isolation, but it can lead to serialization failures, if two transactions modify the same row, one must be retried.

### SERIALIZABLE

The strongest level. Transactions run as if they were executed one after another, even though they actually run concurrently. Postgres detects conflicts and forces one transaction to fail with a serialization failure, which the application must retry. This is the safest level, but it has the highest overhead and requires retry logic in your application code.

For a bookmark manager, `READ COMMITTED` (the default) is correct. For a banking application, you want `SERIALIZABLE` or at least `REPEATABLE READ`. The isolation level is set per transaction:

```rust
let mut tx = pool.begin().await?;
sqlx::query("SET TRANSACTION ISOLATION LEVEL SERIALIZABLE")
.execute(&mut *tx)
.await?;
```

You probably won't need this in practice, but knowing what the options mean and why Postgres defaults to `READ COMMITTED` is table stakes for backend engineering.

## Running the Project

Make sure Docker is running, then:

```
docker compose up -d
cargo sqlx prepare
cargo run
```

You should see:

```
Listening on http://127.0.0.1:3000
```

### Create a Bookmark

```
curl -X POST http://localhost:3000/bookmarks \
  -H "Content-Type: application/json" \
  -d '{
"url": "https://www.rust-lang.org",
"title": "Rust Programming Language",
"description": "The official Rust website",
"tags": ["rust", "programming"]
  }'
```

Response:

```json
{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "url": "https://www.rust-lang.org",
  "title": "Rust Programming Language",
  "description": "The official Rust website",
  "created_at": "2026-07-12T17:00:00.000000Z",
  "updated_at": "2026-07-12T17:00:00.000000Z",
  "tags": [
{"id": "b2c3d4e5-f6a7-8901-bcde-f12345678901", "name": "rust"},
{"id": "c3d4e5f6-a7b8-9012-cdef-123456789012", "name": "programming"}
  ]
}
```

The bookmark and both tags were created in a single transaction. If the server crashed after inserting the bookmark but before inserting the `programming` tag, the entire operation rolls back, no orphaned bookmark.

### Get a Bookmark

```
curl http://localhost:3000/bookmarks/a1b2c3d4-e5f6-7890-abcd-ef1234567890
```

### List All Bookmarks

```
curl http://localhost:3000/bookmarks
```

Response includes pagination metadata:

```json
{
  "bookmarks": [...],
  "page": 1,
  "per_page": 20,
  "total": 1
}
```

### List Bookmarks with Pagination

```
curl "http://localhost:3000/bookmarks?page=2&per_page=10"
```

### Filter by Tag

```
curl "http://localhost:3000/bookmarks?tag=rust"
```

This returns only bookmarks that have the `rust` tag.

### Update a Bookmark

```
curl -X PUT http://localhost:3000/bookmarks/a1b2c3d4-e5f6-7890-abcd-ef1234567890 \
  -H "Content-Type: application/json" \
  -d '{"title": "Rust Lang (Updated)"}'
```

Only the `title` field is updated. `url` and `description` retain their existing values. `updated_at` is set to `NOW()` by the database.

### Attach Tags to an Existing Bookmark

```
curl -X POST http://localhost:3000/bookmarks/a1b2c3d4-e5f6-7890-abcd-ef1234567890/tags \
  -H "Content-Type: application/json" \
  -d '["web", "systems"]'
```

Upserts the tags if they don't exist, inserts the associations, returns the full tag objects. If a tag is already attached, `ON CONFLICT DO NOTHING` skips it silently.

### Detach a Tag

```
curl -X DELETE http://localhost:3000/bookmarks/a1b2c3d4-e5f6-7890-abcd-ef1234567890/tags/b2c3d4e5-f6a7-8901-bcde-f12345678901
```

Returns 204 No Content on success, 404 if the tag wasn't attached.

### Delete a Bookmark

```
curl -X DELETE http://localhost:3000/bookmarks/a1b2c3d4-e5f6-7890-abcd-ef1234567890
```

Returns 204. The bookmark and all its tag associations are removed. `ON DELETE CASCADE` handles the junction table rows.

### Test Validation

```
curl -X POST http://localhost:3000/bookmarks \
  -H "Content-Type: application/json" \
  -d '{"url": "", "title": ""}'
```

Response:

```json
{"error": "url and title must not be empty"}
```

Status code: 400.

### Test Not Found

```
curl http://localhost:3000/bookmarks/00000000-0000-0000-0000-000000000000
```

Response:

```json
{"error": "bookmark with id '00000000-0000-0000-0000-000000000000' not found"}
```

Status code: 404.

## What We Skipped

- **The `thiserror` crate**: We wrote `From` impls manually to show the mechanism. In practice, `thiserror` derives these for you compactly. Use it in your own projects.
- **Compile-time query checking without a running database**: The `.sqlx/` directory generated by `cargo sqlx prepare` is the offline query cache. Commit it. CI builds won't need a database. We covered this in the setup.
- **Full-text search**: Searching bookmarks by URL or title substring would require a `tsvector` and GIN index. That's a different problem from tag-based filtering.
- **User accounts**: Bookmarks belong to everyone. Part 6 adds authentication and user-scoped data.
- **Orphaned tags**: If all bookmarks with a given tag are deleted, the tag row remains in the `tags` table with no associations. This is harmless (takes negligible space) but a production system might periodically clean up unused tags with a background job (Part 9).
- **Soft deletes**: `DELETE` is permanent. A real application might use a `deleted_at` column and a `WHERE deleted_at IS NULL` filter on all queries. This is an application-level decision, not a database one.

## Conclusion

You now have a solid foundation for building data-driven Axum applications. In the next article, we'll build on it by introducing caching and idempotency, two techniques that make APIs faster, more resilient, and safer to retry. See you soon

# Handbook

> My notes and thoughts while reading the handbook from Epictetus - The Complete Works book.

Source: https://blog.sheerluck.dev/posts/book-notes/epictetus-the-complete-works/epictetus-the-complete-works-handbook/ · Markdown: https://blog.sheerluck.dev/posts/book-notes/epictetus-the-complete-works/epictetus-the-complete-works-handbook/index.md

> **Reading this book?**
>
> These notes are based on **The Complete Works of Epictetus**.
>
> If you'd like to read the same edition, you can get it here:
>
> 👉 **[The Complete Works of Epictetus (Amazon)](https://www.amazon.in/Complete-Works-Handbook-Discourses-Fragments/dp/022676947X?&linkCode=ll2&tag=blog048bf-21&linkId=1da38e6b092c6a93a423d190d5702856&ref_=as_li_ss_tl)**
>
> *Disclosure: This is an Amazon affiliate link. If you purchase through it, I may earn a small commission at no extra cost to you.*

## 01 - What is Actually Mine?

> Some things are up to us and some are not. Up to us are judgement, inclination, desire, aversion - in short, whatever is our own doing. Not up to us are our bodies, possessions, reputations, public offices - in short, whatever isn’t our own doing

To me, it seems reasonable what Epictetus said. Things like judgments, our desire or things that we want to avoid are completely under our control. To give you some examples:

Let’s say someone insults you. Then it is up to your judgement to either ignore it, react to it, calmly handle it or whatever.

I think for the rest of the 3 things (inclination, desire and aversion), these come after we made a judgement

For things that are not up to us. Clearly, reputation depends on others, what other people think of you or you opinions and so on. For bodies, some other people or an external event can impact your body like an accident even if you were just sitting still or doing nothing.

> If you regard things that are naturally enslaved as free, if you regard things that are not yours are yours, you’ll be obstructed, dejected, and troubled, and you’ll blame both gods and men. 

For example, if we try to think that we have control over getting a job, clearing some exam, getting that promotion, getting certain viewers or followers count. These things are not up to us, hence we are moving away from reality and in frustration we will blame god or other people.

> If you regard as yours only what is yours, and as not yours what is not yours, which is the way things are in reality, no one will ever constrain you, no one will impede you, you’ll blame no one, you’; reproach no one, you’ll never act reluctantly, no one will harm you, and you’ll have no enemies, because you’ll never be harmed

Epictetus clearly says that this is actually difficult to do. So we need to practice this as much as we can. So every time, we face a situation, we should tell ourselves to examine if the situation or event is under our control or not. If not, then leave it and no need to do anything but if its up to us, then make judgement and take action towards it

# The Basic Laws of Human Stupidity

> My notes and thoughts while reading The Basic Laws of Human Stupidity.

Source: https://blog.sheerluck.dev/posts/book-notes/the-basic-laws-of-human-stupidity/ · Markdown: https://blog.sheerluck.dev/posts/book-notes/the-basic-laws-of-human-stupidity/index.md

This is a collection of my notes while reading **The Basic Laws of Human Stupidity** by Carlo M. Cipolla. It is not a review or a comprehensive summary. My goal is simply to document the ideas that stood out to me and my understanding of them.

## Foreword by Nassim Nicholas Taleb

This foreword made me chuckle quite a bit (though I admit I laugh easily). I recommend reading it before continuing.

> **Note:** Despite the name, these "laws" are not scientific laws. The book is a humorous essay rather than an empirical study, so it's best read as a thought-provoking framework rather than established scientific fact.

## Introduction

The author says that human life has always had problems. Struggles are a part of life for every living being. But humans have one extra problem: the harm caused by other humans. He believes that a lot of this harm comes from stupid people. They are not organized and have no leader, but their actions still create a big negative impact on society. The author also says this book is not meant to be negative or hopeless. Its purpose is to understand stupidity better so we can recognize it and reduce the damage it causes.

## The First Basic Law

> Always and inevitably everyone underestimates the number of stupid individuals in circulation

Author says that no matter how high our estimate is, the actual number is usually higher. We often think someone is intelligent, but later they do something surprisingly stupid. We also keep running into stupid people at the worst possible time and place. Because of this, the author says it is impossible to give an exact number of stupid people in society. Instead, he uses the symbol $\sigma$ (sigma) to represent their fraction in a population

## The Second Basic Law

> The probability that a certain person be stupid is independent of any other characteristic of that person

According to the author, stupidity is not linked to education, intelligence, wealth, social status, gender, race or profession. It is something people are born with and is spread evenly across every group in society.

The author claims that every group be it workers, students, professors and even Nobel Prize winners has roughly the same proportion of stupid people. He uses this to argue that no group is free from stupidity. So, no matter where you go or who you spend time with, you should expect to find the same percentage of stupid people. Combined with the first law, this also means that the number of stupid people will always be more than you expect

## Technical Interlude

Before introducing the third law, the author explains a simple way to understand human actions. Every action has two outcomes: one for ourselves and one for others. We may gain or lose from an action and at the same time other people may also gain or lose. Even choosing not to act can affect others because it can create a missed opportunity.

To explain this, the author uses a graph. The **X-axis** shows our gain or loss and the **Y-axis** shows the other person’s gain or loss.  This helps classify different kinds of human behaviour. The author also points out that we should judge another person’s gain or loss from **their perspective**, not our own. What feels like a benefit to us may actually be a loss for someone else.

## The Third (And Golden) Basic Law

> A stupid person is a person who causes losses to another person or to a group of persons while himself deriving no gain and even possibly incurring loss

Before this law, the author divides people into four types based on how their actions affect themselves and others:  

- Intelligent: Their actions benefit both themselves and others.
- Helpless: Their actions benefit others but cause a loss to themselves
- Bandit: Their actions benefit themselves by causing a loss to others
- Stupid: Their actions case a loss to others without bringing any benefit to themselves and they may even end up harming themselves

![types of people](/images/tblohs-first.png)

The third law defines a **stupid person** as someone who causes a loss to others while gaining nothing from it and sometimes even harming themselves in the process. This is what makes stupidity different from selfishness or crime. A bandit acts for personal gain, but a stupid person creates a damage without any benefit. According to the author, this makes stupid people the most unpredictable and dangerous type.

## Frequency Distribution of Human Types

The author says that most people are **not consistent**. A person may act intelligently in one situation and helplessly or selfishly in another. So, people are classified based on their **overall pattern of behaviour**, not on a single action. The only exception is **stupid people**, who tend to behave stupidly in almost every situation.

The author also explains that **bandits** are not all the same. A *perfect bandit* gains exactly as much as others lose. But most bandits either gain more than the damage they cause (making them closer to intelligent people) or more commonly, gain less than the damage they cause(making them closer to stupidity).

Unlike bandits, **stupid people are concentrated in one category**. Most of them repeatedly harm others without gaining anything themselves. Some even harm themselves while hurting others. The author calls these people **super-stupid**, as they create losses for everyone involved, including themselves.

## Stupidity and Power

The author says that not all stupid people are equally dangerous. Some cause only small problems, while others create huge damage that affects entire societies. Their ability to cause harm depends on two things: how naturally stupid they are and how much power they have.

![stupidity and power](/images/tblohs-second.png)

A stupid person in an important position such as a politician, military leader, bureaucrat or other authority can cause far greater damage than an ordinary person. The author also explains that throughout history, different systems like class, case, religion, political parties, bureaucracy and even democracy have allowed stupid people to reach positions of power. Since stupidity exists in every group of people, some of these individuals will always end up in a leadership role, increasing their ability to harm others.

## The Power of Stupidity

The author explains that the real danger of stupid people is their **unpredictability**. We can usually understand the motives of selfish or dishonest people because they act for personal gain. Their actions are rational, so we can often predict them and prepare for a response.

A stupid person is different. They may harm others without any reason, benefit or clear plan. Because their actions do not follow logic, they are almost impossible to predict. This makes it difficult to defend against them or respond effectively. According to the author, this unpredictability is what makes stupid people more dangerous than people who act out of self-interest.

## The Fourth Basic Law

> Non-stupid people always underestimate the damaging power of stupid individuals. In particular non-stupid people constantly forget that at all times and places and under any circumstances to deal and/or associate with stupid people infallibly turns out to be a costly mistake

What is more surprising is that even intelligent people and bandits make the same mistake. They often think they can predict, control or even use a stupid person for their own benefit.

According to the author, this never works because stupid people do not act logically. Their behaviour is unpredictable, making them impossible to control. Trying to work with or take advantage of a stupid person usually ends up causing losses.

## Macro Analysis and The Fifth Basic Law

> A stupid person is the most dangerous type of person. A stupid person is more dangerous than a bandit

A bandit harms others for personal gain, so wealth is simply transferred from one person to another. Society as a whole does not lose, it only changes who has the wealth.

A stupid person is different. They harm others without gaining anything themselves. This creates a net loss for everyone, making society poorer overall.

The author also explains that every society has about the same proportion of stupid people. The difference between a successful society and a failing one is how much influence these people have. Successful societies have enough intelligent people to limit damage caused by stupidity. Failing societies allow stupid people to become more active and influential, while the number of intelligent people decreases. As a result, the destructive effects of stupidity grow, leading to the decline of society.

## My Thoughts

I think the author's framework is useful because it forces us to think about outcomes rather than intentions. Whether the framework is literally true is less important than whether it helps explain situations we've observed.

I don't agree that stupidity is necessarily an innate trait distributed equally across all groups because the book doesn't provide evidence for that claim. However, I do think the third law offers an interesting way to distinguish irrational harmful behaviour from ordinary selfishness.

# Building Breakout in Bevy: Step by Step

> Build Breakout game step by step from scratch using the Bevy game engine in Rust

Source: https://blog.sheerluck.dev/posts/gamedev/bevy/build-breakout-in-bevy-step-by-step/ · Markdown: https://blog.sheerluck.dev/posts/gamedev/bevy/build-breakout-in-bevy-step-by-step/index.md

In this post, we are going to build **Breakout**, the classic arcade game where you control a paddle, bounce a ball, and destroy a grid of bricks. If you have been following the series from Pong and Snake, you already know the ECS basics, queries, resources, timers, and game states. Now we put it all together into a complete game with particles, lives, and a proper game over loop.

We will build the game incrementally. After every section, you can `cargo run` and see something new on screen. This article won't teach you anything new, the motive of this article is to reinforce what you learnt in the previous 2 articles of this series.

Get the full source code from [here](https://github.com/MrSheerluck/breakout-bevy).

![breakout-still-image](/images/breakout-still-image.png)
### What We Are Building

If you have not played Breakout before, here is how it works. A paddle sits near the bottom of the screen. A ball bounces around the play area. Bricks are arranged in rows at the top. Your goal is to destroy all the bricks by bouncing the ball into them. If the ball falls past the paddle, you lose a life. Lose all three lives and it is game over.

We will build this using colored rectangles for everything, no sprite sheets or image files needed. The bricks will have different colors per row. The ball will leave a particle burst every time it destroys a brick. Score and lives will be displayed as text.

### 1. A Window on Screen

Open your terminal and create a new Rust project:

```bash
cargo new bevy_breakout
cd bevy_breakout
```

Open `Cargo.toml` and replace its contents with:

```toml
[package]
name = "bevy_breakout"
version = "0.1.0"
edition = "2024"

[dependencies]
bevy = { version = "0.18", features = ["wav"] }
rand = "0.8"
```

We add `rand` for randomizing particle directions later. The `wav` feature enables Bevy audio support for the future.

Now open `src/main.rs` and write our first version, a window with a dark background, nothing else yet:

```rust
use bevy::prelude::*;
use bevy::window::WindowResolution;

const WINDOW_WIDTH: f32 = 800.0;
const WINDOW_HEIGHT: f32 = 600.0;

fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Breakout".into(),
            resolution: WindowResolution::new(WINDOW_WIDTH as u32, WINDOW_HEIGHT as u32),
            ..default()
        }),
        ..default()
    }))
    .insert_resource(ClearColor(Color::srgb(0.05, 0.05, 0.08)))
    .run();
}
```

Run it:

```bash
cargo run
```

You should see a dark 800×600 window titled "Breakout". Nothing moves yet but it compiles and runs, which means Bevy is set up correctly. Now we add the paddle.
### 2. The Paddle

We need a paddle that the player can move left and right. Add these constants and the `Paddle` marker component **above** `fn main()`:

```rust
const PADDLE_WIDTH: f32 = 100.0;
const PADDLE_HEIGHT: f32 = 16.0;
const PADDLE_SPEED: f32 = 600.0;
const PADDLE_Y: f32 = -250.0;

#[derive(Component)]
struct Paddle;
```

Now add the `setup` function that spawns the camera and the paddle, and the `move_paddle` system that handles keyboard input. Place both above `fn main()`:

```rust
fn setup(mut commands: Commands) {
commands.spawn(Camera2d);

commands.spawn((
    Paddle,
    Sprite::from_color(Color::srgb(0.6, 0.8, 1.0), Vec2::new(PADDLE_WIDTH, PADDLE_HEIGHT)),
    Transform::from_xyz(0.0, PADDLE_Y, 0.0),
));
}

fn move_paddle(
keyboard: Res<ButtonInput<KeyCode>>,
time: Res<Time>,
mut query: Query<&mut Transform, With<Paddle>>,
) {
for mut transform in &mut query {
    let mut direction = 0.0;

    if keyboard.pressed(KeyCode::ArrowLeft) || keyboard.pressed(KeyCode::KeyA) {
        direction -= 1.0;
    }
    if keyboard.pressed(KeyCode::ArrowRight) || keyboard.pressed(KeyCode::KeyD) {
        direction += 1.0;
    }

    transform.translation.x += direction * PADDLE_SPEED * time.delta_secs();

    let half_width = WINDOW_WIDTH / 2.0;
    let half_paddle = PADDLE_WIDTH / 2.0;
    transform.translation.x = transform
        .translation
        .x
        .clamp(-half_width + half_paddle, half_width - half_paddle);
}
}
```

The `clamp` call keeps the paddle from going off-screen. Now update `main()` to register these systems. Replace your current `main()` with:

```rust
fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Breakout".into(),
            resolution: WindowResolution::new(WINDOW_WIDTH as u32, WINDOW_HEIGHT as u32),
            ..default()
        }),
        ..default()
    }))
    .insert_resource(ClearColor(Color::srgb(0.05, 0.05, 0.08)))
    .add_systems(Startup, setup)
    .add_systems(Update, move_paddle)
    .run();
}
```

Run `cargo run`. You should see a light blue paddle near the bottom of the screen. Press Left/Right arrow keys or A/D to move it. The paddle stays within the window bounds.
### 3. Ball and Walls

Now we add a ball that moves on its own and bounces off walls. We will also introduce `GameState` even though we only have one state for now, it sets us up for the game over flow later.

Add these constants and the `Ball` component above `fn setup()`:

```rust
const BALL_SIZE: f32 = 12.0;
const BALL_SPEED: f32 = 350.0;

#[derive(Component)]
struct Ball {
velocity: Vec2,
}
```

Add the game state enum, for now it only has `Playing`:

```rust
#[derive(States, Default, Clone, Eq, PartialEq, Hash, Debug)]
enum GameState {
#[default]
Playing,
}
```

Now update `setup()` to spawn the walls and the ball. Replace your entire `setup()` function with:

```rust
fn setup(mut commands: Commands) {
commands.spawn(Camera2d);

// Walls
let wall_thickness = 10.0;
let wall_color = Color::srgb(0.3, 0.3, 0.4);

// Left wall
commands.spawn((
    Sprite::from_color(wall_color, Vec2::new(wall_thickness, WINDOW_HEIGHT + 200.0)),
    Transform::from_xyz(-WINDOW_WIDTH / 2.0 - wall_thickness / 2.0 + 1.0, 0.0, 0.0),
));
// Right wall
commands.spawn((
    Sprite::from_color(wall_color, Vec2::new(wall_thickness, WINDOW_HEIGHT + 200.0)),
    Transform::from_xyz(WINDOW_WIDTH / 2.0 + wall_thickness / 2.0 - 1.0, 0.0, 0.0),
));
// Top wall
commands.spawn((
    Sprite::from_color(wall_color, Vec2::new(WINDOW_WIDTH + 200.0, wall_thickness)),
    Transform::from_xyz(0.0, WINDOW_HEIGHT / 2.0 + wall_thickness / 2.0 - 1.0, 0.0),
));
// Bottom wall
commands.spawn((
    Sprite::from_color(wall_color, Vec2::new(WINDOW_WIDTH + 200.0, wall_thickness)),
    Transform::from_xyz(0.0, -WINDOW_HEIGHT / 2.0 - wall_thickness / 2.0 + 1.0, 0.0),
));

// Paddle
commands.spawn((
    Paddle,
    Sprite::from_color(Color::srgb(0.6, 0.8, 1.0), Vec2::new(PADDLE_WIDTH, PADDLE_HEIGHT)),
    Transform::from_xyz(0.0, PADDLE_Y, 0.0),
));

// Ball
commands.spawn((
    Ball {
        velocity: Vec2::new(BALL_SPEED, BALL_SPEED),
    },
    Sprite::from_color(Color::srgb(1.0, 1.0, 1.0), Vec2::splat(BALL_SIZE)),
    Transform::from_xyz(0.0, PADDLE_Y + PADDLE_HEIGHT / 2.0 + BALL_SIZE / 2.0, 0.0),
));
}
```

The walls are extended sprites placed just outside the visible area, they serve as invisible collision boundaries. The ball starts sitting on top of the paddle.

Now add the two new systems, ball movement and wall bouncing:

```rust
fn move_ball(time: Res<Time>, mut query: Query<(&mut Transform, &Ball)>) {
for (mut transform, ball) in &mut query {
    transform.translation += (ball.velocity * time.delta_secs()).extend(0.0);
}
}

fn bounce_ball_off_walls(mut query: Query<(&mut Transform, &mut Ball)>) {
for (mut transform, mut ball) in &mut query {
    let half_width = WINDOW_WIDTH / 2.0;
    let half_height = WINDOW_HEIGHT / 2.0;
    let half_ball = BALL_SIZE / 2.0;

    if transform.translation.x - half_ball < -half_width {
        transform.translation.x = -half_width + half_ball;
        ball.velocity.x = -ball.velocity.x;
    }
    if transform.translation.x + half_ball > half_width {
        transform.translation.x = half_width - half_ball;
        ball.velocity.x = -ball.velocity.x;
    }
    if transform.translation.y + half_ball > half_height {
        transform.translation.y = half_height - half_ball;
        ball.velocity.y = -ball.velocity.y;
    }
    if transform.translation.y - half_ball < -half_height {
        transform.translation.y = -half_height + half_ball;
        ball.velocity.y = -ball.velocity.y;
    }
}
}
```

The ball moves by adding velocity × delta time to its position. When it hits a wall, we reverse the appropriate velocity component and snap the ball back inside the play area to prevent it from getting stuck.

Update `main()` to include the state and the three gameplay systems:

```rust
fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Breakout".into(),
            resolution: WindowResolution::new(WINDOW_WIDTH as u32, WINDOW_HEIGHT as u32),
            ..default()
        }),
        ..default()
    }))
    .insert_resource(ClearColor(Color::srgb(0.05, 0.05, 0.08)))
    .init_state::<GameState>()
    .add_systems(Startup, setup)
    .add_systems(
        Update,
        (
            move_paddle.run_if(in_state(GameState::Playing)),
            move_ball.run_if(in_state(GameState::Playing)),
            bounce_ball_off_walls.run_if(in_state(GameState::Playing)),
        ),
    )
    .run();
}
```

Notice `run_if(in_state(GameState::Playing))`  these systems only run when the game is in the `Playing` state. For now we only have one state, but this pattern will matter when we add `GameOver` later.

Run `cargo run`. The ball launches upward and bounces off all four walls. The paddle still moves. The ball passes right through the paddle for now, we fix that next.

![breakout-start](/images/breakout-start.mp4)
### 4. Paddle Collision

Add the paddle collision system. This uses an AABB overlap check, the same one from Pong but with a smarter bounce. The angle depends on where the ball hits the paddle:

```rust
fn check_paddle_collision(
mut ball_query: Query<(&mut Transform, &mut Ball)>,
paddle_query: Query<&Transform, (With<Paddle>, Without<Ball>)>,
) {
for (mut ball_transform, mut ball) in &mut ball_query {
    for paddle_transform in &paddle_query {
        let ball_pos = ball_transform.translation.truncate();
        let paddle_pos = paddle_transform.translation.truncate();

        let half_ball = BALL_SIZE / 2.0;
        let half_paddle_w = PADDLE_WIDTH / 2.0;
        let half_paddle_h = PADDLE_HEIGHT / 2.0;

        let overlap = !(ball_pos.x + half_ball < paddle_pos.x - half_paddle_w
            || ball_pos.x - half_ball > paddle_pos.x + half_paddle_w
            || ball_pos.y + half_ball < paddle_pos.y - half_paddle_h
            || ball_pos.y - half_ball > paddle_pos.y + half_paddle_h);

        if overlap && ball.velocity.y < 0.0 {
            ball.velocity.y = -ball.velocity.y;

            let hit_offset = (ball_pos.x - paddle_pos.x) / half_paddle_w;
            ball.velocity.x = hit_offset * BALL_SPEED;

            ball_transform.translation.y = paddle_pos.y + half_paddle_h + half_ball;
        }
    }
}
}
```

`hit_offset` ranges from -1.0 (left edge) to 1.0 (right edge). The ball's horizontal velocity is set to `hit_offset * BALL_SPEED`, so hitting the edge sends the ball flying at an angle while hitting the center sends it straight up. This gives the player real control over ball direction.

The `ball.velocity.y < 0.0` check prevents the ball from bouncing upward if it approaches the paddle from below.

Add this system to the `Update` set in `main()`. Find the `Update` block and add the line:

```rust
check_paddle_collision.run_if(in_state(GameState::Playing)),
```

So the full `Update` block now looks like:

```rust
.add_systems(
	Update,
	(
		move_paddle.run_if(in_state(GameState::Playing)),
		move_ball.run_if(in_state(GameState::Playing)),
		bounce_ball_off_walls.run_if(in_state(GameState::Playing)),
		check_paddle_collision.run_if(in_state(GameState::Playing)),
	),
)
```

Run `cargo run`. The ball now bounces off the paddle. Try hitting it with different parts of the paddle, the angle changes.

![breakout-ball-bounce](/images/breakout-ball-bounce.mp4)
### 5. Bricks and Score

Time for the main event. We need a grid of colored bricks, collision detection that knows which side the ball hit, and a score counter.

Add the brick constants and new components:

```rust
const BRICK_WIDTH: f32 = 64.0;
const BRICK_HEIGHT: f32 = 24.0;
const BRICK_PADDING: f32 = 4.0;
const BRICK_TOP: f32 = 60.0;
const BRICK_COLS: i32 = 10;
const BRICK_ROWS: i32 = 5;

#[derive(Component)]
struct Brick;

#[derive(Resource, Default)]
struct Score(u32);

#[derive(Component)]
struct ScoreText;
```

Bricks are 64×24 pixels each, arranged in a 10×5 grid with 4 pixels of padding.

Update `setup()` to insert the score resource and spawn the score text. Add these lines **inside** the `setup()` function, anywhere after the camera spawn:

```rust
commands.insert_resource(Score(0));

commands.spawn((
    ScoreText,
    Text2d::new("Score: 0"),
    TextFont {
        font_size: 24.0,
        ..default()
    },
    TextColor(Color::WHITE),
    Transform::from_xyz(-WINDOW_WIDTH / 2.0 + 80.0, WINDOW_HEIGHT / 2.0 - 30.0, 0.0),
));
```

Now add the `spawn_bricks` system. We use `OnEnter(GameState::Playing)` so bricks reset every time the game restarts:

```rust
fn spawn_bricks(mut commands: Commands) {
let brick_colors = [
    Color::srgb(1.0, 0.2, 0.2),
    Color::srgb(1.0, 0.5, 0.1),
    Color::srgb(1.0, 0.9, 0.1),
    Color::srgb(0.4, 1.0, 0.2),
    Color::srgb(0.2, 0.6, 1.0),
];

let total_grid_width = BRICK_COLS as f32 * (BRICK_WIDTH + BRICK_PADDING) - BRICK_PADDING;
let start_x = -total_grid_width / 2.0 + BRICK_WIDTH / 2.0;

for row in 0..BRICK_ROWS {
    for col in 0..BRICK_COLS {
        let x = start_x + col as f32 * (BRICK_WIDTH + BRICK_PADDING);
        let y = BRICK_TOP + row as f32 * (BRICK_HEIGHT + BRICK_PADDING);
        commands.spawn((
            Brick,
            Sprite::from_color(brick_colors[row as usize], Vec2::new(BRICK_WIDTH, BRICK_HEIGHT)),
            Transform::from_xyz(x, y, 0.0),
        ));
    }
}
}
```

The grid is centered horizontally. Each row gets a different color: red, orange, yellow, green, blue.

Now the brick collision system. This is the most interesting system in the game, when the ball overlaps a brick, we figure out which side it hit so we can bounce in the correct direction:

```rust
fn check_brick_collision(
mut commands: Commands,
mut ball_query: Query<(&mut Transform, &mut Ball)>,
brick_query: Query<(Entity, &Transform), (With<Brick>, Without<Ball>)>,
mut score: ResMut<Score>,
) {
for (ball_transform, mut ball) in &mut ball_query {
    let ball_pos = ball_transform.translation.truncate();
    let half_ball = BALL_SIZE / 2.0;

    for (brick_entity, brick_transform) in &brick_query {
        let brick_pos = brick_transform.translation.truncate();
        let half_brick_w = BRICK_WIDTH / 2.0;
        let half_brick_h = BRICK_HEIGHT / 2.0;

        let overlap = !(ball_pos.x + half_ball < brick_pos.x - half_brick_w
            || ball_pos.x - half_ball > brick_pos.x + half_brick_w
            || ball_pos.y + half_ball < brick_pos.y - half_brick_h
            || ball_pos.y - half_ball > brick_pos.y + half_brick_h);

        if overlap {
            let overlap_left = (ball_pos.x + half_ball) - (brick_pos.x - half_brick_w);
            let overlap_right = (brick_pos.x + half_brick_w) - (ball_pos.x - half_ball);
            let overlap_top = (ball_pos.y + half_ball) - (brick_pos.y - half_brick_h);
            let overlap_bottom = (brick_pos.y + half_brick_h) - (ball_pos.y - half_ball);

            let min_overlap_x = overlap_left.min(overlap_right);
            let min_overlap_y = overlap_top.min(overlap_bottom);

            if min_overlap_x < min_overlap_y {
                ball.velocity.x = -ball.velocity.x;
            } else {
                ball.velocity.y = -ball.velocity.y;
            }

            commands.entity(brick_entity).despawn();
            score.0 += 10;

            break;
        }
    }
}
}
```

The technique: calculate how much the ball overlaps each edge of the brick. The axis with the smaller overlap is the one the ball hit. If `min_overlap_x` is smaller, the ball hit the left or right side. Otherwise, hit the top or bottom. The `break` ensures we only process one brick per frame.

Add the score text update system:

```rust
fn update_score_text(score: Res<Score>, mut query: Query<&mut Text2d, With<ScoreText>>) {
for mut text in &mut query {
    text.0 = format!("Score: {}", score.0);
}
}
```

Now update `main()`. Replace it with this version that adds `OnEnter` for bricks and the two new systems:

```rust
fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Breakout".into(),
            resolution: WindowResolution::new(WINDOW_WIDTH as u32, WINDOW_HEIGHT as u32),
            ..default()
        }),
        ..default()
    }))
    .insert_resource(ClearColor(Color::srgb(0.05, 0.05, 0.08)))
    .init_state::<GameState>()
    .add_systems(Startup, setup)
    .add_systems(OnEnter(GameState::Playing), spawn_bricks)
    .add_systems(
        Update,
        (
            move_paddle.run_if(in_state(GameState::Playing)),
            move_ball.run_if(in_state(GameState::Playing)),
            bounce_ball_off_walls.run_if(in_state(GameState::Playing)),
            check_paddle_collision.run_if(in_state(GameState::Playing)),
            check_brick_collision.run_if(in_state(GameState::Playing)),
            update_score_text,
        ),
    )
    .run();
}
```

Run `cargo run`. Five rows of colored bricks appear at the top. Bounce the ball into them, they disappear on contact and the score goes up. The ball bounces in the correct direction depending on which side of the brick it hits. "Score: 0" is displayed on the top left.
### 6. Particle Burst

Breaking bricks is more satisfying with a visual reward. We will spawn a burst of particles each time a brick is destroyed, with each particle flying outward and fading out over half a second.

Add the `Particle` component:

```rust
#[derive(Component)]
struct Particle {
velocity: Vec2,
lifetime: Timer,
}
```

Add the `spawn_brick_particles` helper function and the `update_particles` system:

```rust
fn spawn_brick_particles(commands: &mut Commands, position: Vec3) {
for _ in 0..8 {
    let angle = rand::random::<f32>() * std::f32::consts::TAU;
    let speed = 80.0 + rand::random::<f32>() * 120.0;
    commands.spawn((
        Particle {
            velocity: Vec2::new(angle.cos() * speed, angle.sin() * speed),
            lifetime: Timer::from_seconds(0.5, TimerMode::Once),
        },
        Sprite::from_color(Color::srgb(1.0, 0.8, 0.2), Vec2::splat(6.0)),
        Transform::from_translation(position),
    ));
}
}

fn update_particles(
mut commands: Commands,
time: Res<Time>,
mut particles: Query<(Entity, &mut Particle, &mut Transform, &mut Sprite)>,
) {
for (entity, mut particle, mut transform, mut sprite) in &mut particles {
    particle.lifetime.tick(time.delta());
    if particle.lifetime.just_finished() {
        commands.entity(entity).despawn();
        continue;
    }
    let t = particle.lifetime.fraction_remaining();
    transform.translation.x += particle.velocity.x * time.delta_secs();
    transform.translation.y += particle.velocity.y * time.delta_secs();
    sprite.color.set_alpha(t);
}
}
```

Eight yellow particles burst from the brick position. Each particle gets a random angle, `rand::random::<f32>() * TAU` gives us a full 360° spread and a speed between 80 and 200 pixels per second. The `Timer` gives each particle a 0.5 second lifetime. `fraction_remaining()` goes from 1.0 (just spawned) to 0.0 (about to despawn), which drives the alpha fade.

Now hook it up. In `check_brick_collision`, find the line `commands.entity(brick_entity).despawn();` and add `spawn_brick_particles` right before it:

```rust
            spawn_brick_particles(&mut commands, brick_transform.translation);
            commands.entity(brick_entity).despawn();
```

Add `update_particles` to the `Update` set in `main()`. Find the `Update` block and add:

```rust
            update_particles,
```

So the `Update` block now looks like:

```rust
    .add_systems(
        Update,
        (
            move_paddle.run_if(in_state(GameState::Playing)),
            move_ball.run_if(in_state(GameState::Playing)),
            bounce_ball_off_walls.run_if(in_state(GameState::Playing)),
            check_paddle_collision.run_if(in_state(GameState::Playing)),
            check_brick_collision.run_if(in_state(GameState::Playing)),
            update_particles,
            update_score_text,
        ),
    )
```

`update_particles` runs unconditionally, no `run_if`  so particles continue animating even when we add the game over screen later.

Run `cargo run`. Destroy a brick and watch eight yellow particles fly outward and fade. Feels satisfying.
### 7. Lives and Game Over

The final piece: if the ball falls past the paddle, the player loses a life. Lose all three and it is game over, with a prompt to restart.

Add the new components and resources:

```rust
#[derive(Resource)]
struct Lives(u32);

#[derive(Component)]
struct LivesText;

#[derive(Component)]
struct GameOverText;
```

Update the `GameState` enum to include `GameOver`:

```rust
#[derive(States, Default, Clone, Eq, PartialEq, Hash, Debug)]
enum GameState {
#[default]
Playing,
GameOver,
}
```

Update `setup()` to add the lives resource and lives text. Add these lines inside `setup()`:

```rust
commands.insert_resource(Lives(3));

commands.spawn((
    LivesText,
    Text2d::new("Lives: 3"),
    TextFont {
        font_size: 24.0,
        ..default()
    },
    TextColor(Color::WHITE),
    Transform::from_xyz(WINDOW_WIDTH / 2.0 - 80.0, WINDOW_HEIGHT / 2.0 - 30.0, 0.0),
));
```

Now replace the `bounce_ball_off_walls` function entirely. The bottom wall no longer bounces instead it decrements lives and either resets the ball or transitions to game over:

```rust
fn bounce_ball_off_walls(
mut ball_query: Query<(&mut Transform, &mut Ball)>,
mut lives: ResMut<Lives>,
mut next_state: ResMut<NextState<GameState>>,
) {
for (mut transform, mut ball) in &mut ball_query {
    let half_width = WINDOW_WIDTH / 2.0;
    let half_height = WINDOW_HEIGHT / 2.0;
    let half_ball = BALL_SIZE / 2.0;

    if transform.translation.x - half_ball < -half_width {
        transform.translation.x = -half_width + half_ball;
        ball.velocity.x = -ball.velocity.x;
    }
    if transform.translation.x + half_ball > half_width {
        transform.translation.x = half_width - half_ball;
        ball.velocity.x = -ball.velocity.x;
    }
    if transform.translation.y + half_ball > half_height {
        transform.translation.y = half_height - half_ball;
        ball.velocity.y = -ball.velocity.y;
    }
    if transform.translation.y - half_ball < -half_height {
        lives.0 = lives.0.saturating_sub(1);
        if lives.0 == 0 {
            next_state.set(GameState::GameOver);
        } else {
            transform.translation =
                Vec3::new(0.0, PADDLE_Y + PADDLE_HEIGHT / 2.0 + BALL_SIZE / 2.0, 0.0);
            ball.velocity = Vec2::new(BALL_SPEED, BALL_SPEED);
        }
    }
}
}
```

`saturating_sub(1)` ensures lives never go below zero even if multiple collisions happen in the same frame. When lives reach zero, the state transitions to `GameOver`. Otherwise the ball resets to its starting position on the paddle.

Add the brick cleanup, game over display, restart, and lives text systems:

```rust
fn cleanup_bricks(mut commands: Commands, bricks: Query<Entity, With<Brick>>) {
for entity in &bricks {
    commands.entity(entity).despawn();
}
}

fn show_game_over(mut commands: Commands) {
commands.spawn((
    GameOverText,
    Text2d::new("Game Over\nPress Enter to Restart"),
    TextFont {
        font_size: 40.0,
        ..default()
    },
    TextColor(Color::srgb(1.0, 0.2, 0.2)),
    Transform::from_xyz(0.0, 0.0, 1.0),
));
}

fn hide_game_over(mut commands: Commands, query: Query<Entity, With<GameOverText>>) {
for entity in &query {
    commands.entity(entity).despawn();
}
}

fn game_over_restart(
keyboard: Res<ButtonInput<KeyCode>>,
mut next_state: ResMut<NextState<GameState>>,
mut lives: ResMut<Lives>,
mut score: ResMut<Score>,
mut ball_query: Query<(&mut Transform, &mut Ball)>,
) {
if !keyboard.just_pressed(KeyCode::Enter) {
    return;
}

lives.0 = 3;
score.0 = 0;
for (mut transform, mut ball) in &mut ball_query {
    transform.translation =
        Vec3::new(0.0, PADDLE_Y + PADDLE_HEIGHT / 2.0 + BALL_SIZE / 2.0, 0.0);
    ball.velocity = Vec2::new(BALL_SPEED, BALL_SPEED);
}
next_state.set(GameState::Playing);
}

fn update_lives_text(lives: Res<Lives>, mut query: Query<&mut Text2d, With<LivesText>>) {
for mut text in &mut query {
    text.0 = format!("Lives: {}", lives.0);
}
}
```

The state transitions do the heavy lifting here:

- `OnExit(GameState::Playing)` → `cleanup_bricks` despawns all bricks.
- `OnEnter(GameState::Playing)` → `spawn_bricks` (already added in section 5) spawns fresh bricks.
- `OnEnter(GameState::GameOver)` → `show_game_over` spawns the message.
- `OnExit(GameState::GameOver)` → `hide_game_over` despawns the message.

This guarantees bricks and the game over message are always in sync with the state, whether we are starting fresh or restarting.

Finally, replace `main()` with the complete version:

```rust
fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Breakout".into(),
            resolution: WindowResolution::new(WINDOW_WIDTH as u32, WINDOW_HEIGHT as u32),
            ..default()
        }),
        ..default()
    }))
    .insert_resource(ClearColor(Color::srgb(0.05, 0.05, 0.08)))
    .init_state::<GameState>()
    .add_systems(Startup, setup)
    .add_systems(OnEnter(GameState::Playing), spawn_bricks)
    .add_systems(OnExit(GameState::Playing), cleanup_bricks)
    .add_systems(OnEnter(GameState::GameOver), show_game_over)
    .add_systems(OnExit(GameState::GameOver), hide_game_over)
    .add_systems(
        Update,
        (
            move_paddle.run_if(in_state(GameState::Playing)),
            move_ball.run_if(in_state(GameState::Playing)),
            bounce_ball_off_walls.run_if(in_state(GameState::Playing)),
            check_paddle_collision.run_if(in_state(GameState::Playing)),
            check_brick_collision.run_if(in_state(GameState::Playing)),
            update_particles,
            update_score_text,
            update_lives_text,
            game_over_restart.run_if(in_state(GameState::GameOver)),
        ),
    )
    .run();
}
```

Run `cargo run`. You now have the complete game:

![Breakout gameplay](/images/breakout-gameplay.mp4)
### Things We Could Improve

- Add sound effects for paddle hits, wall bounces, brick breaks, and life loss
- Add a high score that persists across sessions using file I/O
- Add a title screen with settings (volume sliders, fullscreen toggle)
- Add power-up drops from destroyed bricks (wider paddle, multi-ball)
- Remove the bottom wall sprite from `setup()` now that we handle the bottom with lives

### Conclusion

This was a fun one. We built a complete Breakout game in seven incremental steps, running the game after each one to see real progress. Along the way we covered collision side detection, particle bursts with lifetime timers, lives and game over with state transitions, and a clean restart loop using `OnEnter` and `OnExit` systems.

In the next post, we will start Phase 2 of our series and build Frogger, where we learn about physics engines, tilemaps, and moving platforms.

See you in the next one.

# Learn Bevy States, Timers, and Grid Movement by Building Snake

> Build Snake from scratch using the Bevy game engine in Rust. Learn states, timers, grid movement and particle effects. Then deploy a playable web version to Itch.io.

Source: https://blog.sheerluck.dev/posts/gamedev/bevy/learn-bevy-states-timers-by-building-snake/ · Markdown: https://blog.sheerluck.dev/posts/gamedev/bevy/learn-bevy-states-timers-by-building-snake/index.md

In this post, we are going to build **Snake**, the classic arcade game where you guide a growing snake to eat food while avoiding walls and yourself. If you read the Bevy Pong article, you already know the basics of ECS, queries, and resources. Now we level up.

We'll learn about **timers** for tick-based movement, **game states** to manage playing and game-over screens, **grid-based movement** that constrains objects to discrete positions, **movement queuing** so the snake doesn't accidentally reverse into itself, and **score tracking** with state transitions.

Get the full source code from [here](https://github.com/MrSheerluck/bevy-snake). Published game link is [here](https://mrsheerluck.itch.io/snake-with-bevy-engine).

### New Concepts

Before writing any code, let's understand the new ideas we haven't seen yet.

**Timers.** In Pong, the ball moved continuously every frame by adding `velocity * delta`. Snake is different. The snake moves one cell at a time at a fixed rate. It doesn't slide smoothly,  it teleports to the next cell every N milliseconds. This is called **tick-based** movement. Bevy provides a `Timer` type. You create it with a duration, call `tick(delta)` every frame, and when `just_finished()` returns true, it's time to move the snake. Reset and repeat.

**Game states.** Snake has two distinct modes: **playing** (the game is active) and **game over** (the player lost). In each mode, different systems should run. During game over, the snake shouldn't move and input should be different. Bevy's **state system** lets us define states and attach systems to specific ones. Only systems whose state matches the current state run.

**Grid coordinates.** Pong used continuous positions, the ball could be at x = 173.42 pixels. Snake uses a discrete grid. Every object sits on a `(col, row)` cell. We convert grid positions to world positions using a simple formula.

**Movement queuing.** Here is a classic Snake bug: the player presses Right then Down very quickly within one tick. If you change direction immediately, the snake goes right, then the same tick processes Down and the snake turns down. But what if the player pressed Left then Right? The snake reverses into itself and dies. We need to have a fix for this: a **queue** of directions. Each tick, pop one direction off the queue.

### Project Setup

Open your terminal and create a new Rust project:

```
cargo new bevy_snake
cd bevy_snake
```

Now open `Cargo.toml` and replace its contents with:

```toml
[package]
name = "bevy_snake"
version = "0.1.0"
edition = "2024"

[dependencies]
bevy = { version = "0.18", features = ["wav"] }
rand = "0.8"
```

We add `rand` because we need random positions for food. We enable the `wav` feature for Bevy audio support. We pin `rand` to `0.8` deliberately. Bevy 0.18's internal rand dependency was upgraded in 0.17 with breaking API changes, and `0.8` keeps our usage stable.

Now run:
```
cargo build
```

This will take a few minutes. Bevy is a large engine.
### The Skeleton: Constants and Window

Open `src/main.rs` and start with this:
```rust
use bevy::prelude::*;
use std::collections::VecDeque;

const GRID_WIDTH: i32 = 20;
const GRID_HEIGHT: i32 = 20;
const CELL_SIZE: f32 = 30.0;
const SNAKE_MOVE_INTERVAL: f32 = 0.15;

fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Snake".into(),
            resolution: bevy::window::WindowResolution::new(600, 680),
            resizable: false,
            ..default()
        }),
        ..default()
    }))
    .add_systems(Startup, setup)
    .run();
}

fn setup(mut commands: Commands, asset_server: Res<AssetServer>) {
commands.spawn(Camera2d);
}
```

Let me explain the constants.

`GRID_WIDTH` and `GRID_HEIGHT` are both 20, a 20x20 grid. `CELL_SIZE` is 30 pixels per cell. The play area is 20 * 30 = 600 pixels square. `SNAKE_MOVE_INTERVAL` is 0.15 seconds between each snake movement.

Note that `setup` takes `asset_server: Res<AssetServer>` from the start. We need it to load sound files, and Bevy injects it automatically as a system parameter.

Run `cargo run`. You should see a dark grey window.
### Grid Position Component

Every object in Snake lives on a discrete grid cell. Let's define a component for that:

```rust
#[derive(Component, Clone, Copy, PartialEq, Eq, Debug)]
struct GridPosition {
col: i32,
row: i32,
}
```

`#[derive(Component)]` registers it as a Bevy component. `Clone` and `Copy` let us pass it around easily. `PartialEq` lets us compare positions, we use that for collision detection.
### Direction and Helpers

Snake has four directions:

```rust
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum Direction {
Up,
Down,
Left,
Right,
}
```

We also need a function to convert grid positions to world positions (pixels on screen):

```rust
fn grid_to_world(pos: GridPosition) -> Vec3 {
let half_w = GRID_WIDTH as f32 * CELL_SIZE / 2.0;
let half_h = GRID_HEIGHT as f32 * CELL_SIZE / 2.0;
Vec3::new(
    pos.col as f32 * CELL_SIZE + CELL_SIZE / 2.0 - half_w,
    pos.row as f32 * CELL_SIZE + CELL_SIZE / 2.0 - half_h,
    0.0,
)
}
```

The math: `col * CELL_SIZE` gives the left edge of the cell. Adding `CELL_SIZE / 2` centers us in the cell. Subtracting `half_w` shifts the whole grid so the center of the screen is the center of the grid. Cell `(10, 10)` is right in the middle.
### Marker Components

Add these marker components to tag entities so we can find them in queries:

```rust
#[derive(Component)]
struct SnakeSegment;

#[derive(Component)]
struct FoodSprite;

#[derive(Component)]
struct ScoreText;

#[derive(Component)]
struct GameOverText;

#[derive(Component)]
struct ScorePop(Timer);

#[derive(Component)]
struct Particle {
velocity: Vec2,
lifetime: Timer,
}
```

These are the same pattern as `ScoreText` in Pong, empty structs (or small structs) that act as labels or lightweight state on entities. `ScorePop` and `Particle` carry per-entity timer and physics data rather than living in a global resource, because multiple of them can exist simultaneously.
### Resources

The game data lives in resources. In Pong we had a `Score` resource. Snake needs more:

```rust
#[derive(Resource)]
struct Snake {
segments: Vec<GridPosition>,
direction: Direction,
}

#[derive(Resource)]
struct Food(GridPosition);

#[derive(Resource)]
struct MoveTimer(Timer);

#[derive(Resource)]
struct DirectionQueue(VecDeque<Direction>);

#[derive(Resource)]
struct Score(u32);

#[derive(Resource)]
struct ScreenShake(Timer);

#[derive(Resource)]
struct EatSound(Handle<AudioSource>);

#[derive(Resource)]
struct DieSound(Handle<AudioSource>);
```

Let me explain each one.

**`Snake`** holds the logical state of the snake. `segments` is a `Vec<GridPosition>` where index 0 is the head. `direction` is which way the snake is currently moving.

**`Food`** is a newtype around a single `GridPosition`. Where the food sits on the grid.

**`MoveTimer`** wraps a Bevy `Timer`. We tick it every frame. When it finishes, the snake moves one cell.

**`DirectionQueue`** wraps a `VecDeque<Direction>`. When the player presses a key, we push onto this queue. Each tick, we pop one direction off.

**`Score`** is the player's score. Starts at 0.

**`ScreenShake`** wraps a `Timer`. When the player dies, we reset it. While it runs, `apply_screenshake` offsets the camera randomly each frame.

**`EatSound`** and **`DieSound`** hold handles to loaded audio assets. We load them once in `setup` and clone the handle whenever we want to play them.
### Game State

Add this to define our two game modes:

```rust
#[derive(States, Default, Clone, Eq, PartialEq, Hash, Debug)]
enum GameState {
#[default]
Playing,
GameOver,
}
```

`#[derive(States)]` tells Bevy this enum defines game states. `#[default]` on `Playing` means the game starts in Playing mode.

Register it in `main()`:

```rust
fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Snake".into(),
            resolution: bevy::window::WindowResolution::new(600, 680),
            resizable: false,
            ..default()
        }),
        ..default()
    }))
    .init_state::<GameState>()
    .add_systems(Startup, setup)
    .run();
}
```

### Random Position for Food

We need a way to generate random grid positions for food. We'll also need to avoid spawning food on top of the snake:

```rust
fn random_grid_position() -> GridPosition {
GridPosition {
    col: rand::random::<i32>().rem_euclid(GRID_WIDTH),
    row: rand::random::<i32>().rem_euclid(GRID_HEIGHT),
}
}

fn spawn_food_on_empty(snake: &Snake) -> GridPosition {
loop {
    let pos = random_grid_position();
    if !snake.segments.contains(&pos) {
        return pos;
    }
}
}
```

`rand::random::<i32>()` generates a random i32. `rem_euclid(GRID_WIDTH)` wraps it into (0, 20). The loop retries until it finds an empty cell that the snake does not occupy.
### The Setup System

Now the full `setup` function that spawns everything:

```rust
fn setup(mut commands: Commands, asset_server: Res<AssetServer>) {
// Load sounds
commands.insert_resource(EatSound(asset_server.load("audio/eat.wav")));
commands.insert_resource(DieSound(asset_server.load("audio/die.wav")));

// Initialize the screenshake timer as already finished so it doesn't
// fire on startup. We reset it manually when the player dies.
commands.insert_resource(ScreenShake({
    let mut t = Timer::from_seconds(0.3, TimerMode::Once);
    t.finish();
    t
}));

commands.spawn(Camera2d);

// Initial snake: 3 segments in the center
let start_col = GRID_WIDTH / 2;
let start_row = GRID_HEIGHT / 2;

let snake = Snake {
    segments: vec![
        GridPosition { col: start_col, row: start_row },
        GridPosition { col: start_col - 1, row: start_row },
        GridPosition { col: start_col - 2, row: start_row },
    ],
    direction: Direction::Right,
};

// Spawn one sprite per segment
for &pos in &snake.segments {
    commands.spawn((
        SnakeSegment,
        Sprite::from_color(Color::srgb(0.2, 0.8, 0.2), Vec2::splat(CELL_SIZE - 2.0)),
        Transform::from_translation(grid_to_world(pos)),
    ));
}
commands.insert_resource(snake);

// Food
let food_pos = GridPosition { col: 15, row: 10 };
commands.spawn((
    FoodSprite,
    Sprite::from_color(Color::srgb(0.8, 0.2, 0.2), Vec2::splat(CELL_SIZE - 2.0)),
    Transform::from_translation(grid_to_world(food_pos)),
));
commands.insert_resource(Food(food_pos));

// Timer, queue, score
commands.insert_resource(MoveTimer(
    Timer::from_seconds(SNAKE_MOVE_INTERVAL, TimerMode::Repeating),
));
commands.insert_resource(DirectionQueue(VecDeque::new()));
commands.insert_resource(Score(0));

// Score text at top of screen
commands.spawn((
    ScoreText,
    Text2d::new("Score: 0"),
    TextFont {
        font_size: 30.0,
        ..default()
    },
    TextColor(Color::WHITE),
    Transform::from_xyz(0.0, 290.0, 0.0),
));
}
```

Let me walk through the key parts.

The snake starts at the center with 3 segments: `(10, 10)`, `(9, 10)`, `(8, 10)` heading right. We spawn a green `Sprite::from_color` for each segment. `Vec2::splat(CELL_SIZE - 2.0)` makes each segment 28x28 pixels, with a 1 pixel gap on each side so segments don't bleed into each other.

Food is placed at `(15, 10)` on the right side of the grid. Red sprite.

The timer is `Timer::from_seconds(0.15, TimerMode::Repeating)`. It loops every 0.15 seconds. Each time it completes, the snake moves.

The `ScreenShake` timer is initialized with `t.finish()` called immediately after creation. This puts it in a completed state from the start. If we didn't do this, `apply_screenshake` would see a freshly-started timer on frame one and shake the camera for 0.3 seconds before any death has occurred. The timer only matters after we call `shake.0.reset()` on death.

The score text uses `Text2d`, `TextFont`, `TextColor`, same pattern as Pong. Positioned at `y = 320`, near the top of the grid.

Run `cargo run`. You should see a 3-segment green snake in the center, a red food square on the right, and "Score: 0" at the top. Nothing moves yet.

### Snake Input

Now we add movement. First, the input system. Arrow keys queue direction changes:

```rust
fn snake_input(
keyboard: Res<ButtonInput<KeyCode>>,
mut queue: ResMut<DirectionQueue>,
) {
if keyboard.just_pressed(KeyCode::ArrowUp) {
    queue.0.push_back(Direction::Up);
} else if keyboard.just_pressed(KeyCode::ArrowDown) {
    queue.0.push_back(Direction::Down);
} else if keyboard.just_pressed(KeyCode::ArrowLeft) {
    queue.0.push_back(Direction::Left);
} else if keyboard.just_pressed(KeyCode::ArrowRight) {
    queue.0.push_back(Direction::Right);
}
}
```

We use `just_pressed` (not `pressed`) so each key press adds exactly one entry to the queue. Holding a key down does not spam the queue.
### Opposite Direction Check

The snake should not reverse into itself:

```rust
fn is_opposite(a: Direction, b: Direction) -> bool {
matches!(
    (a, b),
    (Direction::Up, Direction::Down)
        | (Direction::Down, Direction::Up)
        | (Direction::Left, Direction::Right)
        | (Direction::Right, Direction::Left)
)
}
```

`matches!` is a Rust macro that checks if a value matches a pattern.
### Snake Movement

This is the core logic that runs on each timer tick:

```rust
fn snake_move(
mut commands: Commands,
time: Res<Time>,
mut timer: ResMut<MoveTimer>,
mut snake: ResMut<Snake>,
mut queue: ResMut<DirectionQueue>,
mut food: ResMut<Food>,
mut score: ResMut<Score>,
mut next_state: ResMut<NextState<GameState>>,
eat_sound: Res<EatSound>,
) {
// Advance the timer. Exit if it hasn't finished yet.
if !timer.0.tick(time.delta()).just_finished() {
    return;
}

// Pop the next direction from the queue, if any
while let Some(next_dir) = queue.0.pop_front() {
    if !is_opposite(next_dir, snake.direction) {
        snake.direction = next_dir;
        break;
    }
}

// Calculate the new head position
let head = snake.segments[0];
let new_head = match snake.direction {
    Direction::Up => GridPosition {
        col: head.col,
        row: head.row + 1,
    },
    Direction::Down => GridPosition {
        col: head.col,
        row: head.row - 1,
    },
    Direction::Left => GridPosition {
        col: head.col - 1,
        row: head.row,
    },
    Direction::Right => GridPosition {
        col: head.col + 1,
        row: head.row,
    },
};

// Wall collision
if new_head.col < 0
    || new_head.col >= GRID_WIDTH
    || new_head.row < 0
    || new_head.row >= GRID_HEIGHT
{
    next_state.set(GameState::GameOver);
    return;
}

// Self collision
if snake.segments.contains(&new_head) {
    next_state.set(GameState::GameOver);
    return;
}

// Move: insert new head at the front
snake.segments.insert(0, new_head);

// Check if the snake ate the food
if new_head == food.0 {
    score.0 += 1;
    commands.spawn(AudioPlayer(eat_sound.0.clone()));
    spawn_eat_particles(&mut commands, grid_to_world(new_head));
    food.0 = spawn_food_on_empty(&snake);
    // Don't remove the tail, snake grows
} else {
    snake.segments.pop(); // Remove tail
}
}
```

Let me explain the most important parts.

**System parameters.** `snake_move` takes `mut commands: Commands` and `eat_sound: Res<EatSound>` because eating food both plays a sound and spawns particles, both require `commands`. Bevy injects all system parameters automatically; you just declare what you need.

**Timer tick.** `timer.0.tick(time.delta())` advances the timer by the frame's delta time. `.just_finished()` returns true exactly once when the timer reaches its duration. On that frame, the snake moves. The timer is `Repeating`, so it resets automatically.

**Queue processing.** The `while` loop pops directions from the front of the queue until it finds one that is not opposite to the current direction. The `break` after a valid direction means only one turn per tick. The check is against `snake.direction`, the direction the snake is _currently moving_, not the last queued direction.

**New head position.** The head moves one cell in the current direction. `Up` increases `row`, `Down` decreases it. `Left` and `Right` change `col`.

**Wall collision.** If the new head is outside the 20x20 grid, game over.

**Self collision.** `.contains(&new_head)` checks if the new head overlaps any existing segment. If yes, game over.

**Food check.** If the head landed on the food cell, the score increases, a sound plays, particles burst, the food respawns, and the tail is NOT removed, the snake grows by one. Otherwise, the tail is popped off.
### Sync Sprites

The `snake_move` system updates the logical positions. Now we need to update the visual sprites to match. The simplest approach: despawn all old sprites and spawn new ones at the current positions:

```rust
fn sync_sprites(
mut commands: Commands,
snake: Res<Snake>,
food: Res<Food>,
segments: Query<Entity, With<SnakeSegment>>,
food_sprites: Query<Entity, With<FoodSprite>>,
) {
// Despawn old segment sprites
for entity in &segments {
    commands.entity(entity).despawn();
}
// Despawn old food sprite
for entity in &food_sprites {
    commands.entity(entity).despawn();
}

// Spawn new segment sprites
for (i, &pos) in snake.segments.iter().enumerate() {
    let color = if i == 0 {
        Color::srgb(0.0, 0.95, 0.0) // Head: bright green
    } else {
        Color::srgb(0.2, 0.7, 0.2) // Body: darker green
    };
    commands.spawn((
        SnakeSegment,
        Sprite::from_color(color, Vec2::splat(CELL_SIZE - 2.0)),
        Transform::from_translation(grid_to_world(pos)),
    ));
}

// Spawn new food sprite
commands.spawn((
    FoodSprite,
    Sprite::from_color(Color::srgb(0.9, 0.2, 0.2), Vec2::splat(CELL_SIZE - 2.0)),
    Transform::from_translation(grid_to_world(food.0)),
));
}
```

We despawn entities by querying for their marker components and calling `commands.entity(entity).despawn()`. We make the head bright green and the body darker green so the player can see which end is which. This despawn-and-respawn approach is simple and correct for a game this size.
### Score Text Update

```rust
fn update_score_text(
score: Res<Score>,
mut query: Query<&mut Text2d, With<ScoreText>>,
mut commands: Commands,
) {
for mut text in &mut query {
    let old = text.0.clone();
    text.0 = format!("Score: {}", score.0);
    if old != text.0 {
        commands.spawn((
            ScorePop(Timer::from_seconds(0.3, TimerMode::Once)),
            Text2d::new(text.0.clone()),
            TextFont {
                font_size: 44.0,
                ..default()
            },
            TextColor(Color::srgb(1.0, 1.0, 0.3)),
            Transform::from_xyz(0.0, 290.0, 2.0),
        ));
    }
}
}
```

`text.0` accesses the inner `String` of `Text2d`. We compare the old value to the new value. If they differ, the score just changed and we spawn a `ScorePop` entity, a larger yellow version of the score text that floats upward and fades. The `commands` parameter is needed here for that spawn.
### Score Pop Animation

```rust
fn update_score_pop(
mut commands: Commands,
time: Res<Time>,
mut pops: Query<(Entity, &mut ScorePop, &mut Transform, &mut TextColor)>,
) {
for (entity, mut pop, mut transform, mut color) in &mut pops {
    pop.0.tick(time.delta());
    if pop.0.is_finished() {
        commands.entity(entity).despawn();
        continue;
    }
    let t = pop.0.fraction_remaining();
    transform.translation.y = 290.0 + (1.0 - t) * 30.0;
    color.0.set_alpha(t);
}
}
```

`fraction_remaining()` goes from 1.0 at the start of the timer down to 0.0 at the end. We use it for both alpha (fades out) and to drive the upward float. When `t = 1.0` (just spawned), `y = 290.0 + 0.0 * 30.0 = 290.0`. When `t = 0.0` (about to despawn), `y = 290.0 + 1.0 * 30.0 = 320.0`. So the text floats 30 pixels upward while fading.

### Game Over and Restart

When the game is over, we show a message and wait for Enter:

```rust
fn game_over_restart(
keyboard: Res<ButtonInput<KeyCode>>,
mut next_state: ResMut<NextState<GameState>>,
mut snake: ResMut<Snake>,
mut food: ResMut<Food>,
mut score: ResMut<Score>,
mut timer: ResMut<MoveTimer>,
mut queue: ResMut<DirectionQueue>,
) {
if !keyboard.just_pressed(KeyCode::Enter) {
    return;
}

// Reset everything
let start_col = GRID_WIDTH / 2;
let start_row = GRID_HEIGHT / 2;
snake.segments = vec![
    GridPosition { col: start_col, row: start_row },
    GridPosition { col: start_col - 1, row: start_row },
    GridPosition { col: start_col - 2, row: start_row },
];
snake.direction = Direction::Right;
food.0 = GridPosition { col: 15, row: 10 };
score.0 = 0;
timer.0.reset();
queue.0.clear();

next_state.set(GameState::Playing);
}
```

The game over screen is shown and hidden via `OnEnter` and `OnExit`:

```rust
fn show_game_over(
mut commands: Commands,
die_sound: Res<DieSound>,
mut shake: ResMut<ScreenShake>,
) {
commands.spawn(AudioPlayer(die_sound.0.clone()));
shake.0.reset();
commands.spawn((
    GameOverText,
    Text2d::new("Game Over\nPress Enter to Restart"),
    TextFont {
        font_size: 40.0,
        ..default()
    },
    TextColor(Color::srgb(1.0, 0.2, 0.2)),
    Transform::from_xyz(0.0, 0.0, 1.0),
));
}

fn hide_game_over(mut commands: Commands, query: Query<Entity, With<GameOverText>>) {
for entity in &query {
    commands.entity(entity).despawn();
}
}
```

`show_game_over` does three things: plays the death sound, resets the screenshake timer (which triggers the camera shake), and spawns the game over text. `hide_game_over` despawns it when the player restarts.
### Polish: Particle Burst on Eat

When the snake eats, we spawn 10 small yellow particles that fly outward and fade:

```rust
fn spawn_eat_particles(commands: &mut Commands, position: Vec3) {
for _ in 0..10 {
    let angle = rand::random::<f32>() * std::f32::consts::TAU;
    let speed = 60.0 + rand::random::<f32>() * 80.0;
    commands.spawn((
        Particle {
            velocity: Vec2::new(angle.cos() * speed, angle.sin() * speed),
            lifetime: Timer::from_seconds(0.6, TimerMode::Once),
        },
        Sprite::from_color(Color::srgb(1.0, 0.9, 0.2), Vec2::splat(5.0)),
        Transform::from_translation(position),
    ));
}
}
```

`TAU` is 2π, a full circle in radians. Multiplying a random float by `TAU` gives a random angle. `angle.cos()` and `angle.sin()` give the x and y components of a unit vector in that direction, scaled by `speed`. Each particle gets its own `lifetime` timer. This is why `Particle` is a component rather than a resource, many particles can exist simultaneously, each with independent state.

A separate system updates particles each frame:

```rust
fn update_particles(
mut commands: Commands,
time: Res<Time>,
mut particles: Query<(Entity, &mut Particle, &mut Transform, &mut Sprite)>,
) {
for (entity, mut particle, mut transform, mut sprite) in &mut particles {
    particle.lifetime.tick(time.delta());
    if particle.lifetime.is_finished() {
        commands.entity(entity).despawn();
        continue;
    }
    let t = particle.lifetime.fraction_remaining();
    transform.translation.x += particle.velocity.x * time.delta_secs();
    transform.translation.y += particle.velocity.y * time.delta_secs();
    sprite.color.set_alpha(t);
    transform.scale = Vec3::splat(0.5 + t * 0.5);
}
}
```

`t` goes from 1.0 to 0.0 as the particle ages. Alpha fades from 1.0 to 0.0. Scale goes from `0.5 + 1.0 * 0.5 = 1.0` down to `0.5 + 0.0 * 0.5 = 0.5`, so particles shrink slightly as they die.
### Polish: Screen Shake on Death

When the player dies, the camera shakes briefly:

```rust
fn apply_screenshake(
mut shake: ResMut<ScreenShake>,
mut camera: Query<&mut Transform, With<Camera2d>>,
time: Res<Time>,
) {
shake.0.tick(time.delta());
for mut transform in &mut camera {
    if shake.0.is_finished() {
        if transform.translation != Vec3::ZERO {
            transform.translation = Vec3::ZERO;
        }
    } else {
        let intensity = 10.0 * (1.0 - shake.0.fraction());
        transform.translation = Vec3::new(
            rand::random::<f32>() * 2.0 * intensity - intensity,
            rand::random::<f32>() * 2.0 * intensity - intensity,
            0.0,
        );
    }
}
}
```

`shake.0.fraction()` goes from 0.0 to 1.0 as time passes, so `1.0 - fraction()` starts at 1.0 and decays to 0.0. Multiplying by 10.0 gives a starting intensity of 10 pixels that eases to zero. When finished, the camera is snapped back to `Vec3::ZERO`. Because the timer starts pre-finished (we called `t.finish()` in setup), this system runs every frame doing nothing until a death occurs and `shake.0.reset()` is called.
### Register Everything

The final `main` function ties it all together:

```rust
fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Snake".into(),
            resolution: bevy::window::WindowResolution::new(600, 680),
            resizable: false,
            ..default()
        }),
        ..default()
    }))
    .init_state::<GameState>()
    .add_systems(Startup, setup)
    .add_systems(OnEnter(GameState::GameOver), show_game_over)
    .add_systems(OnExit(GameState::GameOver), hide_game_over)
    .add_systems(
        Update,
        (
            snake_input,
            snake_move.run_if(in_state(GameState::Playing)),
            sync_sprites,
            update_score_text,
            update_score_pop,
            update_particles,
            apply_screenshake,
            game_over_restart.run_if(in_state(GameState::GameOver)),
        ),
    )
    .run();
}
```

Run `cargo run`. You should now have a fully polished Snake game:

- Steer with arrow keys. Eat food to grow and score points.
- Eat food -> hear a rising beep, yellow particles burst, score pops up bigger.
- Hit a wall or yourself -> hear a low tone, screen shakes, "Game Over" appears.
- Press Enter to restart.
### Deploying to Itch.io

Let's build a web version and publish it to itch.io.

Before building for the web, add this to your `Cargo.toml` after the `[dependencies]` block:

```toml
[target.'cfg(all(target_family = "wasm", any(target_os = "unknown", target_os = "none")))'.dependencies]
getrandom_02 = { version = "^0.2", features = ["js"], package = "getrandom" }
getrandom_03 = { version = "^0.3", features = ["wasm_js"], package = "getrandom" }
```

`rand` uses `getrandom` internally to access the system's entropy source. On native, this is straightforward. On `wasm32-unknown-unknown`, there is no OS, so `getrandom` needs to be told to use the browser's `crypto.getRandomValues()` API instead. Without this, `rand::random()` panics at runtime in the browser. The `cfg` target triple ensures these entries only apply to WASM builds and have no effect on native.

First, install the wasm target:

```
rustup target add wasm32-unknown-unknown
```

Install the Bevy CLI. It is not yet published to crates.io so install it from git:

```
cargo install --git https://github.com/TheBevyFlock/bevy_cli --locked bevy_cli
```

Build and bundle the web release:

```
bevy build --release web --bundle
```

The `--bundle` flag collects the wasm binary, JS bindings, and assets into a single folder at:

```
target/bevy_web/
```

That folder is self-contained. Create a zip from it:

```
cd target/bevy_web
zip -r ../../bevy_snake.zip .
```

Upload `bevy_snake.zip` to [itch.io](https://itch.io/) and enable "This file will be played in the browser".
### Things We Could Improve

- The food always spawns in random positions. It could prefer the area ahead of the snake for more interesting gameplay.
- We could add difficulty progression: the snake moves faster as the score increases.
- High score persistence with local storage.
- A proper title screen with settings (volume sliders, fullscreen toggle).
### Conclusion

You now understand how game states work in Bevy, how timers drive tick-based movement, how grid coordinates power tile-based games, and how to queue inputs to prevent unfair deaths. You also learned how to bake polish into every system like particles, screenshake, audio, and animations.

In the next post, we will build Breakout and learn about collision groups, sprite sheet animations, power-up drops, and tween-style animations. See you soon!

# Learn the Basics of Bevy 0.19 by Building and Deploying Pong to Itch.io

> Build Pong from scratch using the Bevy game engine in Rust. Learn ECS, the game loop, collision detection, player input, and scoring. Then deploy a playable web version to Itch.io.

Source: https://blog.sheerluck.dev/posts/gamedev/bevy/learn-the-basics-of-bevy-by-building-and-deploying-pong-to-itch-io/ · Markdown: https://blog.sheerluck.dev/posts/gamedev/bevy/learn-the-basics-of-bevy-by-building-and-deploying-pong-to-itch-io/index.md

I'm starting a new series on learning game dev and the Bevy game engine. In each article we build one game and learn different concepts along the way. All projects come from the [20 Games Challenge](https://20_games_challenge.gitlab.io/challenge/).

This is a learning series for me as well. I'm familiar with game dev but not with Bevy, this is why I'm writing these primarily for learning and sharing my knowledge with you. If there is any incorrect information, then please inform me, I'll rectify that asap.

The only prerequisite is that you know Rust (or can learn it on the go). I also have a [Rust learning series](https://blog.sheerluck.dev/series/learning-rust/) that follows the same project-based approach.

In this post, we build **Pong**, the classic arcade game where two players bounce a ball past each other's paddles and publish it on [Itch.io](https://itch.io). If you know Rust but have never built a game, this is the perfect place to start.

> **This is a rewrite of the original article** in this series, updated for **Bevy 0.19** and rewritten in an **incremental style**. Every section ends with `cargo run` , you'll see progress on screen after each step.

We'll learn: what a game loop is, how ECS (Entity-Component-System) works, how to detect collisions, how to handle keyboard input, and how to display text on screen.

> We might not follow standard patterns or best practices in our learning series as the goal is to build games incrementally. For example, in this article, we are not using any third party physics crate, instead we are implementing a manual AABB collision algorithm, which we won't do in future articles.

Get the full source code from [GitHub](https://github.com/MrSheerluck/bevy-pong). Published game is on [itch.io](https://mrsheerluck.itch.io/pong-with-bevy-engine).
## What Is a Game Engine?

Before writing code, let's understand what a game engine actually does.

At its core, a video game is just a loop. Every frame (usually 60 times per second), the game:

1. **Reads input** - what keys are the players pressing?
2. **Updates the world** - move paddles, move the ball, check for collisions.
3. **Renders** - draw everything on screen.

This is the **game loop**. Without an engine, you'd write this loop yourself, read input, update state, draw pixels directly. An engine like Bevy handles graphics, audio, windowing, and input so you focus on your game.

Bevy organizes game code using **ECS**: Entity-Component-System. Let's understand each part:

- **Entity**: A unique ID that represents something in your game. A paddle, the ball, the camera.
- **Component**: Data attached to an entity. Position, velocity, color. Plain Rust structs with no methods.
- **System**: A function that runs every frame and operates on entities that have specific components. A "movement" system finds all entities with a `Position` and `Velocity` and updates their positions.

**Data and logic are separate.** Components just sit there holding data. Systems read and modify components. This is different from object-oriented programming where objects contain both data and methods.
## Project Setup

Open your terminal and create a new Rust project:

```bash
cargo new bevy_pong
cd bevy_pong
```

Open `Cargo.toml` and replace its contents with:

```toml
[package]
name = "bevy_pong"
version = "0.1.0"
edition = "2024"

[dependencies]
bevy = "0.19"
```

The `[dependencies]` section tells Cargo what external libraries to download and compile. `bevy = "0.19"` pulls in the Bevy engine at version 0.19.

Run:

```bash
cargo build
```

This takes a few minutes. Bevy is a large engine, it compiles the renderer (OpenGL/Metal/Vulkan), audio system, asset loader, input handler, and more. Subsequent builds will be fast because only your code changes need recompilation.
## 1. A Window on Screen

Let's write the smallest possible Bevy program. Open `src/main.rs` and type this:

```rust
use bevy::prelude::*;
use bevy::window::WindowResolution;

const WINDOW_WIDTH: f32 = 800.0;
const WINDOW_HEIGHT: f32 = 600.0;

fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Pong".into(),
            resolution: WindowResolution::new(WINDOW_WIDTH as u32, WINDOW_HEIGHT as u32),
            ..default()
        }),
        ..default()
    }))
    .insert_resource(ClearColor(Color::srgb(0.0, 0.0, 0.0)))
    .add_systems(Startup, setup)
    .run();
}

fn setup(mut commands: Commands) {
commands.spawn(Camera2d);
}
```

### The `main` function - your app's headquarters

`App::new()` creates a new, empty Bevy application. Think of `App` as your game's headquarters, it holds everything: the world (all entities and their components), the schedule (which systems run and when), and the plugins (bundled features).

`.add_plugins(DefaultPlugins)` is the most important line in any Bevy project. `DefaultPlugins` is a bundle of everything Bevy needs to function:

- **WindowPlugin**: Creates a window on your screen.
- **RenderPlugin**: Renders graphics using your GPU.
- **AssetPlugin**: Loads files like images and sounds.
- **InputPlugin**: Handles keyboard, mouse, and gamepad input.
- **AudioPlugin**: Plays sounds.

We use `.set()` to replace the default `WindowPlugin` with our own. This lets us control the window resolution (800×600) and title ("Pong").

`WindowResolution::new(WINDOW_WIDTH, WINDOW_HEIGHT)` takes a width and height in logical pixels. On high-DPI displays, the actual physical pixels might be different, but Bevy handles that conversion for you.

`.insert_resource(ClearColor(Color::srgb(0.0, 0.0, 0.0)))` sets the background color. `ClearColor` is a **resource**, a singleton piece of data. Unlike components attached to entities, there's exactly one instance of it in the world. `Color::srgb(0.0, 0.0, 0.0)` is pure black (no red, no green, no blue).

`.add_systems(Startup, setup)` registers our `setup` function as a system that runs during the `Startup` schedule. A **schedule** determines *when* a system runs. Bevy has a few key schedules:

- **Startup**: Runs once when the app starts. Use this for creating entities, loading assets, setting up initial state.
- **Update**: Runs every frame. Use this for gameplay logic like movement, collision, input handling.

`.run()` starts the game loop. This function never returns, it runs until the player closes the window.

### The `setup` function - creating entities

`commands: Commands` is a queue of changes to make to the game world. When you call `commands.spawn(...)`, the entity is queued and created at a safe point during the frame. You never create entities directly, always go through `Commands`.

`commands.spawn(Camera2d)` creates a new entity with a `Camera2d` component. In Bevy, **components can require other components**. `Camera2d` automatically brings along `Camera` (the camera settings), `Projection` (the math that converts coordinates), and `Frustum` (for culling off-screen objects). Without a camera, there's nothing to render through.

### Understanding coordinates

Before we place objects, let's understand how positions work in a 2D game.

Your window is a flat rectangle. Every position on it is described by an **(x, y)** coordinate:

- **x**: Horizontal position. 0 is the center. Negative is left. Positive is right.
- **y**: Vertical position. 0 is the center. Negative is down. Positive is up.

Coordinates are measured in **pixels**. If your window is 800 pixels wide, the left edge is at x = -400, the right edge is at x = 400, and the center is at x = 0.

Each entity has a `Transform` component that stores its position, rotation, and scale. `Transform::from_xyz(x, y, z)` positions the entity at that coordinate. The z component controls which objects appear on top of others, higher z means closer to the camera. In a 2D game, we leave z at 0 for everything.

**Run it:**

```bash
cargo run
```

You should see a black 800×600 window titled "Pong". Nothing else yet, but Bevy is set up correctly and compiles without errors.

## 2. Two Paddles

Now we add the paddles that players control.
### Creating our game objects

In ECS, data lives in components. Let's define what a paddle is:

```rust
const PADDLE_WIDTH: f32 = 10.0;
const PADDLE_HEIGHT: f32 = 100.0;
const PADDLE_SPEED: f32 = 500.0;
const PADDLE_OFFSET: f32 = 50.0;

#[derive(Component)]
struct Paddle {
speed: f32,
side: Side,
}

#[derive(Component)]
enum Side {
Left,
Right,
}
```

`#[derive(Component)]` is a derive macro that tells Bevy "this struct can be used as a component on an entity." It implements the `Component` trait, which provides metadata Bevy needs to manage the data.

`Paddle { speed: f32, side: Side }`, each paddle has a movement speed (pixels per second) and a side identifier (which player controls it). We store this data in the component so each paddle knows which keys to listen to.

`Side` is an enum, in Bevy, enums can be components too. We use it to determine which keyboard keys move which paddle.

### Spawning the paddles

Now replace `setup()` with a version that spawns both paddles:

```rust
fn setup(mut commands: Commands) {
commands.spawn(Camera2d);

// Left paddle
commands.spawn((
    Paddle { speed: PADDLE_SPEED, side: Side::Left },
    Sprite::from_color(Color::WHITE, Vec2::new(PADDLE_WIDTH, PADDLE_HEIGHT)),
    Transform::from_xyz(-WINDOW_WIDTH / 2.0 + PADDLE_OFFSET, 0.0, 0.0),
));

// Right paddle
commands.spawn((
    Paddle { speed: PADDLE_SPEED, side: Side::Right },
    Sprite::from_color(Color::WHITE, Vec2::new(PADDLE_WIDTH, PADDLE_HEIGHT)),
    Transform::from_xyz(WINDOW_WIDTH / 2.0 - PADDLE_OFFSET, 0.0, 0.0),
));
}
```

Each `commands.spawn((...))` call creates an entity with multiple components at once. The components are passed as a tuple, Bevy treats each element as a separate component and attaches all of them to the same entity.

`Sprite::from_color(Color::WHITE, Vec2::new(10.0, 100.0))` creates a visual rectangle without loading any image files. The first argument is the fill color (white), the second is the size in pixels (10 wide, 100 tall).

`Transform::from_xyz(-WINDOW_WIDTH / 2.0 + PADDLE_OFFSET, 0.0, 0.0)` positions the left paddle. Our window is 800 pixels wide, so WINDOW_WIDTH / 2.0 = 400. The paddle sits at x = -400 + 50 = -350 (- 50 pixels from the left edge). The right paddle is at the mirror position.

### Making the paddles move

Now we write our first real gameplay system, a function that reads keyboard input and moves the paddle:

```rust
fn move_paddle(
keyboard: Res<ButtonInput<KeyCode>>,
mut paddle_query: Query<(&mut Transform, &Paddle)>,
time: Res<Time>,
) {
for (mut transform, paddle) in &mut paddle_query {
    let mut direction = 0.0;

    match paddle.side {
        Side::Left => {
            if keyboard.pressed(KeyCode::KeyW) { direction = 1.0; }
            if keyboard.pressed(KeyCode::KeyS) { direction = -1.0; }
        }
        Side::Right => {
            if keyboard.pressed(KeyCode::ArrowUp) { direction = 1.0; }
            if keyboard.pressed(KeyCode::ArrowDown) { direction = -1.0; }
        }
    }

    transform.translation.y += direction * paddle.speed * time.delta_secs();

    // Clamp to window bounds
    let half_paddle = PADDLE_HEIGHT / 2.0;
    let half_height = WINDOW_HEIGHT / 2.0;
    transform.translation.y = transform.translation.y.clamp(
        -half_height + half_paddle,
        half_height - half_paddle,
    );
}
}
```

### System parameters - how Bevy feeds your functions

A Bevy system is just a Rust function. What makes it special is the **parameters**, Bevy automatically injects the right data based on the types in your function signature.

`keyboard: Res<ButtonInput<KeyCode>>`, `Res<T>` gives read-only access to a resource. `ButtonInput<KeyCode>` is Bevy's keyboard input resource. It tracks which keys are currently held down, which were just pressed this frame, and which were just released. We use `.pressed(KeyCode::KeyW)` to check if the W key is being held down right now.

`mut paddle_query: Query<(&mut Transform, &Paddle)>`, `Query` is how you find entities in the world. The type parameter `(&mut Transform, &Paddle)` means "give me mutable access to Transform and read-only access to Paddle, but only for entities that have BOTH of these components." Since both paddles have Transform and Paddle, this system processes both of them. The `mut` keyword means we can modify the components we asked for mutably.

`time: Res<Time>` - the Time resource provides timing information. We use `delta_secs()`, the time in seconds since the last frame. This is crucial for smooth movement.

### Why delta time matters

Here's the problem every new game developer encounters. On a 60 Hz monitor, your game runs 60 times per second. On a 144 Hz monitor, it runs 144 times per second. If you moved the paddle 10 pixels every frame:

- At 60 FPS: 600 pixels per second
- At 144 FPS: 1440 pixels per second

The game runs at a completely different speed on different monitors. This is called a **frame rate dependency bug**.

The fix is to multiply your movement by `delta_secs()`:

- At 60 FPS: delta = 0.0167 seconds. 500 × 0.0167 = 8.35 pixels per frame.
- At 144 FPS: delta = 0.0069 seconds. 500 × 0.0069 = 3.47 pixels per frame.

The per-frame movement is different, but over one second: 500 pixels in both cases. The game runs at the same speed regardless of monitor refresh rate.

### Direction logic

We check which side this paddle is on and which keys are held. `direction` is 1.0 for up (positive y), -1.0 for down (negative y), and 0.0 if no keys are pressed.

We use two separate `if` statements (not `if-else`) because both keys could be pressed simultaneously. If W and S are both held, `direction` gets set to 1.0 then immediately to -1.0, the last key check wins. Not ideal (pressing both should cancel to zero), but functional enough for our first game.

### Clamping

The `clamp` call keeps the paddle from going off the top or bottom of the screen. We calculate how far the paddle can travel: from `-half_height + half_paddle` (bottom edge + half paddle height) to `half_height - half_paddle` (top edge - half paddle height). This ensures the paddle's edge never crosses the window edge.

Register the system in `main()`:

```rust
fn main() {
App::new()
    .add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            title: "Pong".into(),
            resolution: WindowResolution::new(WINDOW_WIDTH as u32, WINDOW_HEIGHT as u32),
            ..default()
        }),
        ..default()
    }))
    .insert_resource(ClearColor(Color::srgb(0.0, 0.0, 0.0)))
    .add_systems(Startup, setup)
    .add_systems(Update, move_paddle)
    .run();
}
```

**Run it:**

```
cargo run
```

Two white rectangles appear near the left and right edges. Move the left paddle with **W/S** and the right paddle with **Arrow Up/Down**. The paddles stop at the window edges.
## 3. The Ball

A Pong game needs a ball. Add the `Ball` component and the ball movement system:

```rust
const BALL_SIZE: f32 = 10.0;

#[derive(Component)]
struct Ball {
velocity: Vec3,
}
```

The ball's velocity is a 3D vector. Even in a 2D game, we use `Vec3` because `Transform` positions also use `Vec3`. The z component stays 0. The velocity determines how fast and in what direction the ball travels each frame.

Now add the ball to `setup()`, right after the paddle spawns:

```rust
// Ball
commands.spawn((
    Ball { velocity: Vec3::new(300.0, 150.0, 0.0) },
    Sprite::from_color(Color::WHITE, Vec2::splat(BALL_SIZE)),
    Transform::from_xyz(0.0, 0.0, 0.0),
));
```

`Vec2::splat(BALL_SIZE)` creates a `Vec2(10.0, 10.0)`. The ball starts at the exact center of the screen. Its velocity is 300 pixels/second to the right and 150 pixels/second upward. This diagonal path gives it an interesting angle.

Add the movement system:

```rust
fn move_ball(
mut ball_query: Query<(&mut Transform, &Ball)>,
time: Res<Time>,
) {
for (mut transform, ball) in &mut ball_query {
    transform.translation += ball.velocity * time.delta_secs();
}
}
```

The ball movement follows the exact same pattern as the paddle: add velocity multiplied by delta time to position. The difference is there's no input logic, the ball just moves according to its stored velocity.

Register `move_ball` alongside `move_paddle`:

```rust
    .add_systems(Update, (move_paddle, move_ball))
```

Using a tuple registers multiple systems in the same schedule. They run in the order listed.

**Run it:**

```
cargo run
```

The ball flies diagonally off-screen. That's expected, we haven't added walls yet.
## 4. Walls and Bouncing

The ball should bounce off the top and bottom of the window. Add this system:

```rust
fn bounce_ball(mut ball_query: Query<(&mut Transform, &mut Ball)>) {
let half_height = WINDOW_HEIGHT / 2.0;
let half_ball = BALL_SIZE / 2.0;

for (mut transform, mut ball) in &mut ball_query {
    // Top wall
    if transform.translation.y + half_ball >= half_height {
        transform.translation.y = half_height - half_ball;
        ball.velocity.y = -ball.velocity.y;
    }
    // Bottom wall
    if transform.translation.y - half_ball <= -half_height {
        transform.translation.y = -half_height + half_ball;
        ball.velocity.y = -ball.velocity.y;
    }
}
}
```

### The collision check

`half_height = WINDOW_HEIGHT / 2.0` - our window is 600 pixels tall, so the center is at y = 0, the top edge is at y = 300, and the bottom edge is at y = -300.

`half_ball = BALL_SIZE / 2.0` - our ball is 10×10 pixels, so its half-size (or "radius" for a square) is 5 pixels.

`transform.translation.y + half_ball >= half_height` checks if the ball's top edge (center y + half its height) has reached or passed the top of the window. If the ball center is at y = 297 and we add the radius of 5, we get 302, which is ≥ 300. Collision detected.

### The snap 

When a collision happens, we do two things:

```rust
transform.translation.y = half_height - half_ball;
ball.velocity.y = -ball.velocity.y;
```

First, we **snap the ball back inside** the play area. We set its y position to `half_height - half_ball` = 300 - 5 = 295. This prevents the ball from getting stuck outside the wall.

**Why would it get stuck?** Imagine the ball is fast (say 600 pixels per second) and the frame rate drops to 30 FPS. The ball moves 20 pixels in one frame. If it was at y = 290 and moved 20 pixels up, it'd be at y = 310. Next frame, it's still past the wall, so it bounces again, reversing direction again. But it's still past the wall because it bounced back only 10 pixels. It vibrates in place. Snapping ensures this never happens.

Second, we reverse the Y velocity. If the ball was moving upward (+150), it now moves downward (-150).

The bottom wall check is the mirror: `transform.translation.y - half_ball <= -half_height`.

Register it in `main()`:

```rust
    .add_systems(Update, (move_paddle, move_ball, bounce_ball))
```

**Run it:**

```
cargo run
```

The ball bounces off the top and bottom edges. It still passes through the paddles
## 5. Paddle Collision

Now we make the ball bounce off both paddles using **AABB collision detection** (Axis-Aligned Bounding Box). Two rectangles overlap if they are **not separated along any axis**.

Add this system:

```rust
fn check_paddle_collision(
mut ball_query: Query<(&mut Transform, &mut Ball), Without<Paddle>>,
paddle_query: Query<(&Transform, &Paddle), Without<Ball>>,
) {
for (mut ball_transform, mut ball) in &mut ball_query {
    let ball_pos = ball_transform.translation.truncate();
    let ball_size = Vec2::splat(BALL_SIZE);

    for (paddle_transform, _paddle) in &paddle_query {
        let paddle_pos = paddle_transform.translation.truncate();
        let paddle_size = Vec2::new(PADDLE_WIDTH, PADDLE_HEIGHT);

        // AABB overlap check
        let overlap = !(
            ball_pos.x + ball_size.x / 2.0 < paddle_pos.x - paddle_size.x / 2.0
            || ball_pos.x - ball_size.x / 2.0 > paddle_pos.x + paddle_size.x / 2.0
            || ball_pos.y + ball_size.y / 2.0 < paddle_pos.y - paddle_size.y / 2.0
            || ball_pos.y - ball_size.y / 2.0 > paddle_pos.y + paddle_size.y / 2.0
        );

        if overlap {
            ball.velocity.x = -ball.velocity.x;

            // Snap ball outside paddle to prevent sticking
            if ball.velocity.x > 0.0 {
                ball_transform.translation.x =
                    paddle_pos.x + paddle_size.x / 2.0 + ball_size.x / 2.0;
            } else {
                ball_transform.translation.x =
                    paddle_pos.x - paddle_size.x / 2.0 - ball_size.x / 2.0;
            }
        }
    }
}
}
```

### Disjoint queries - preventing data races

```rust
mut ball_query: Query<(&mut Transform, &mut Ball), Without<Paddle>>,
paddle_query: Query<(&Transform, &Paddle), Without<Ball>>,
```

Both queries use `Without<T>` filters. `Without<Paddle>` on the ball query means "only match entities that don't have a Paddle component." `Without<Ball>` on the paddle query means the opposite.

Why? Because Bevy needs to prove that these two queries never access the same entity. If they could, we'd have a data race, one query mutably accessing Transform while another query also accesses it. The `Without` filters tell Bevy "these are completely separate groups of entities," so it can run them safely without runtime checks.

### How AABB collision works

Imagine two rectangles on screen. They overlap if they are **not separated along any axis**. A rectangle is separated from another if:

1. Its right edge is to the left of the other's left edge.
2. Its left edge is to the right of the other's right edge.
3. Its top edge is below the other's bottom edge.
4. Its bottom edge is above the other's top edge.

If any one of these is true, they don't overlap. If all four are false, they overlap.

Our check:

```rust
let overlap = !(
ball_pos.x + ball_size.x / 2.0 < paddle_pos.x - paddle_size.x / 2.0
|| ball_pos.x - ball_size.x / 2.0 > paddle_pos.x + paddle_size.x / 2.0
|| ball_pos.y + ball_size.y / 2.0 < paddle_pos.y - paddle_size.y / 2.0
|| ball_pos.y - ball_size.y / 2.0 > paddle_pos.y + paddle_size.y / 2.0
);
```

`ball_pos.x + ball_size.x / 2.0` is the ball's right edge. `paddle_pos.x - paddle_size.x / 2.0` is the paddle's left edge. If the ball's right edge is to the left of the paddle's left edge, they're separated horizontally. We OR all four checks together and NOT the result, if none of the four separation conditions are true, the rectangles overlap.

### Collision response 

When a collision happens, we reverse the ball's X direction. Then we snap the ball to the outside of the paddle. If the ball is moving right (velocity.x > 0 after reversal is positive), it hit the left side of the paddle, so we place it just to the right of the paddle's right edge. If moving left, we place it just to the left of the paddle's left edge.

The snap prevents the same problem as wall bouncing, without it, the ball can get stuck inside the paddle, bouncing back and forth every frame until it wiggles through.

Register it:

```rust
    .add_systems(Update, (move_paddle, move_ball, bounce_ball, check_paddle_collision))
```

**Run it:**

```
cargo run
```

The ball bounces off both paddles and the top/bottom walls. Two-player Pong is playable.
## 6. Scoring

A game needs a score. We need a resource to store it and text to display it.
### Score resource and text marker

Add these **above** `setup()`:

```rust
#[derive(Resource, Default)]
struct Score {
left: u32,
right: u32,
}

#[derive(Component)]
struct ScoreText;
```

`Score` is a **resource** - a singleton piece of data. Unlike components attached to entities, resources are global. There's only one score in the entire game. `#[derive(Default)]` initializes both fields to 0.

We call it `insert_resource` instead of `spawn` because resources are not entities. They're singletons that any system can access. Entities are for things that exist in the game world (paddles, ball). Resources are for game-wide state (score, settings).

`ScoreText` is an empty **marker component**. We attach it to the text entity so we can find it later with a query. Without this, we'd have no way to distinguish the score text from other text entities.
### Spawning the score display

Now add the score display to `setup()`, right after the camera spawn:

```rust
use bevy::text::FontSize;

commands.insert_resource(Score::default());

commands.spawn((
    Text2d::new("0 - 0"),
    TextFont {
        font_size: FontSize::Px(40.0),
        ..default()
    },
    TextColor(Color::WHITE),
    Transform::from_xyz(0.0, WINDOW_HEIGHT / 2.0 - 50.0, 0.0),
    ScoreText,
));
```

`Text2d::new("0 - 0")` creates text positioned in the 2D world. It wraps a `String`. The initial text shows "0 - 0".

`TextFont { font_size: FontSize::Px(40.0), ..default() }` controls the font style. We set the size to 40 pixels and use the default font (Fira Mono, bundled with Bevy).

> **Bevy 0.19 note:** `TextFont::font_size` now uses `FontSize::Px(40.0)` instead of a bare `40.0`. This is one of the few breaking changes from 0.18.

`Transform::from_xyz(0.0, WINDOW_HEIGHT / 2.0 - 50.0, 0.0)` positions the text near the top of the screen. Our window is 600 pixels tall, so y = 300 is the top edge. We place it at y = 300 - 50 = 250, centered horizontally.

Add the `use bevy::text::FontSize;` import at the very top of the file alongside the other imports:

```rust
use bevy::prelude::*;
use bevy::text::FontSize;
use bevy::window::WindowResolution;
```
### The scoring system

Now add the system that detects when a player scores:

```rust
fn score_goal(
mut ball_query: Query<&mut Transform, With<Ball>>,
mut score: ResMut<Score>,
mut score_text: Query<&mut Text2d, With<ScoreText>>,
) {
let half_width = WINDOW_WIDTH / 2.0;

for mut transform in &mut ball_query {
    let scored = if transform.translation.x > half_width + 10.0 {
        score.left += 1;
        true
    } else if transform.translation.x < -half_width - 10.0 {
        score.right += 1;
        true
    } else {
        false
    };

    if !scored {
        continue;
    }

    transform.translation = Vec3::new(0.0, 0.0, 0.0);

    for mut text in &mut score_text {
        text.0 = format!("{} - {}", score.left, score.right);
    }
}
}
```

`ResMut<Score>` - `ResMut` gives mutable access to a resource. We need to modify the score counters.

`Query<&mut Transform, With<Ball>>` - we only want the ball entity, so we filter for entities that have a `Ball` component.

`Query<&mut Text2d, With<ScoreText>>` - finds the text entity by its marker component.

If the ball goes past the right edge (plus a 10-pixel buffer so it fully disappears), the **left** player scores, the right player failed to return it. If it goes past the left edge, the **right** player scores.

After scoring, we reset the ball to the center. The ball keeps its current velocity, it doesn't restart from zero speed.

`text.0` accesses the inner `String` of the `Text2d` tuple struct. We update it with the current score.

Register `score_goal` in `main()`:

```rust
    .add_systems(
        Update,
        (move_paddle, move_ball, bounce_ball, check_paddle_collision, score_goal),
    )
```

**Run it:**

```
cargo run
```

Play a game. Each time a player misses, the score at the top updates and the ball resets to center.

## Deploying to Itch.io

Let's build a web version and publish it so anyone can play in their browser.

Install the wasm target:

```
rustup target add wasm32-unknown-unknown
```

Install the Bevy CLI:

```
cargo install bevy_cli
```

Build the web release:

```
bevy build --release web
```

After it finishes, the generated files are in `target/wasm32-unknown-unknown/web-release/`. You should see:

```
bevy_pong.js
bevy_pong.wasm
bevy_pong_bg.wasm
```

Create an `index.html` file in that directory:

```html
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Bevy Pong</title>
  <style>
body { margin: 0; overflow: hidden; background: black; }
canvas { width: 100vw; height: 100vh; display: block; }
  </style>
</head>
<body>
  <script type="module">
import init from "./bevy_pong.js";
init();
  </script>
</body>
</html>
```

Create a clean deployment folder and copy only what itch.io needs:

```
mkdir itch_build
cp target/wasm32-unknown-unknown/web-release/bevy_pong.js itch_build/
cp target/wasm32-unknown-unknown/web-release/bevy_pong.wasm itch_build/
cp target/wasm32-unknown-unknown/web-release/bevy_pong_bg.wasm itch_build/
cp target/wasm32-unknown-unknown/web-release/index.html itch_build/
cp -r target/wasm32-unknown-unknown/web-release/assets itch_build/
```

Zip it:

```
cd itch_build
zip -r bevy_pong.zip .
```

Upload `bevy_pong.zip` to [itch.io](https://itch.io) as a new project. Under "Kind of project", select **HTML**. Enable **"This file will be played in the browser"**.
## Things We Could Improve

- Randomize the ball direction after each score (right now it keeps the same velocity).
- Add sound effects for paddle hits, wall bounces, and scoring.
- Add a particle effect on collision for game feel.
- Add a win condition (first to 5, for example) with a game over screen.

## What We Learned
This was a long one. I can go on and add other concepts as well but its already too long for our first article in this series. We have tons of chances to learn and reinforce our knowledge in future projects, so lets not worry about it. I hope you understood the overall basic concepts of Bevy and don't worry all these concepts will solidify in future with more and more projects.

In the next post, we'll build **Snake** and learn about game states, timers, and grid-based movement.

See you in the next one.

# Your First GPUI App - Building a Desktop UI in Rust

> Learn GPUI from scratch by building a Linear-inspired desktop UI in Rust. We'll cover GPUI's rendering model, elements, styling, layout, and component composition while coding the application step by step.

Source: https://blog.sheerluck.dev/posts/gpui/your-first-gpui-app-building-a-desktop-ui-in-rust/ · Markdown: https://blog.sheerluck.dev/posts/gpui/your-first-gpui-app-building-a-desktop-ui-in-rust/index.md

Hi, I'm starting this learning GPUI series completely out of curiosity to learn about GPUI and share whatever I learn with you. One thing that I can assure you is that we will follow the same pattern as the rest of my articles, we will build one interesting project per article and learn the concepts required to build that project. I'll try to post one article on GPUI every 2 weeks but can't promise on that, it might be 3 weeks per article as well.

By the end of this article, we will have this kind of UI:
![Final look of the GPUI issue tracker UI](/images/gpui/first/final-result.png)

<div class="aspect-video w-full overflow-hidden rounded-xl border border-border bg-black">
  <iframe
    class="h-full w-full"
    src="https://www.youtube.com/embed/NT2XPvtof-Y"
    title="Your First GPUI App - Building a Desktop UI in Rust"
    loading="lazy"
    allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
    referrerPolicy="strict-origin-when-cross-origin"
    allowFullScreen
  ></iframe>
</div>

Let's just start learning GPUI and in this article we are focusing on implementing layouts and styling in GPUI and we will build a linear like UI clone where we will have a static issue-tracking interface. It'll have a sidebar, navigation, projects, a header, tabs and issue cards.

You can get the source code from [here](https://github.com/MrSheerluck/gpui-linear-ui-clone)

GPUI is a Rust UI framework developed by the team behind [Zed](https://zed.dev). It is designed to build native desktop applications using Rust and the interface itself is described using Rust code.

## Pre-Requisites
- Rust programming language
- Have Rust installed on your machine ([installation guide](https://www.rust-lang.org/tools/install))
- HTML and CSS (just basics so that you'd clearly understand concepts like flex, margin, padding etc)
I would highly urge you to not move forward if you don't know the basics of Rust or basic Html and Css. Please get some basic knowledge on the pre-requisites and then come back. 

> As GPUI is evolving, you may encounter some breaking changes in future versions. If you do, then please inform me so that I can update the article accordingly. You can either go to the blog's Github repo's issues tab and create a new issue or inform me through any social media channels.

## Creating the Project
Let's start by creating a new Rust project:
```bash
cargo new gpui-issue-tracker-ui
```
Open the project in your preferred editor.

Lets start by adding dependencies to `Cargo.toml`:
```toml
[package]
name = "gpui-issue-tracker-ui"
version = "0.1.1"
edition = "2024"

[dependencies]
gpui-kit = "0.6"
rust-embed = "8"
anyhow = "1"
```
Here, we added the `gpui-kit` dependency, which brings GPUI, the platform-specific application entry point, GPUI Component, and its bundled assets together. `rust-embed` allows us to embed our own application assets into the binary, and `anyhow` is used for error handling.

## Creating the Smallest Possible GPUI Application
We are going to build this application incrementally, so that after every step you can run the application and check how its looking. Ok, lets start by opening `src/main.rs` and importing GPUI:
```rust
use gpui_kit::*;
```
Now, lets define an application view:
```rust
struct App;
```
At this point, `App` doesn't do anything, we need to tell GPUI how this view should be rendered.

### The `Render` trait
Lets implement `Render` for `App`:
```rust
impl Render for App {
	fn render(&mut self, _window: &mut Window, _cx: &mut Context<Self>) -> impl IntoElement {
		div().child("Hello, GPUI!")
	}
}
```
This `impl Render for App` tells GPUI that `App` is a view that can produce the UI. The `render` method returns `impl IntoElement` to construct an element tree. If you think in terms of HTML, you just created a `div` element and added a child that has the text `Hello, GPUI!`

You can visualise the element tree like this:
```tree
App
└── div
└── "Hello, GPUI!"
```
This is the kind of pattern we will follow for the rest of the article where a GPUI view renders an element and elements can contain other elements just like we do in HTML.

Let me explain a bit about the `Render` trait as well. `fn render` is the method i=required by the `Render` trait, the `&mut self` mean `render` method receives a mutable reference to our `App` instance. So, when we will start using things like app state in future, we can pass that state using this. The next thing is `_window: &mut Window`, this gives the render method mutable access to the current GPUI window. You can see its written like this `_window` because we are not using it right now, we are saying "we know this parameter exists, but I'm not using it". We will start using it later once we work on things like `focus`, `input` etc. Finally, we have the context `_cx: &mut Context<Self>`, this context provides access to GPUI functionality associated with this particular view. For example, in our future article we can use context to notify GPUI that the view's state has changed, its actually great when we are handling event. Here also we are using that underscore format just to meant that we are not using it right now.
## Starting the GPUI Application
Now, we need `Root` from GPUI Kit's component layer. Let's update our imports to:
```rust
use gpui_kit::*;
use gpui_kit::component::Root;
```
Now add a `main` function:
```rust
fn main() {
	let app = gpui_kit::application();

	app.run(|cx| {
		gpui_kit::init(cx);

		cx.open_window(
			WindowOptions::default(),
			|window, cx| {
let view = cx.new(|_| App);
cx.new(|cx| Root::new(view, window, cx))
			},
		)
		.expect("failed to open window");
		cx.activate(true);
	});
}
```
First, we are creating our application `let app = gpui_kit::application();`, then once our app is ready, `app.run(|cx| {...});` we start its event loop and this gives us an application context `cx`. After we get the context of the app, we initialise GPUI Kit and its enabled components with `gpui_kit::init(cx);`.
Once all this is done, we start by creating the window for our app `cx.open_window(...)`. I'm not going to explain all the details here, I'll explain them later in this article.

Now, if you'll run the app using `cargo run`, you should see a window with the text `Hello GPUI!`

## Giving the Window a Predictable Size
Right now, our app has a default window size due to this line`WindowOptions::default()`, lets change that and have a specific window size for our app. Instead of `WindowOptions::default()`, use this:
```rust
WindowOptions {
window_bounds: Some(WindowBounds::Windowed(Bounds {
    origin: point(px(0.0), px(0.0)),
    size: size(px(1200.0), px(760.0)),
})),
..Default::default()
}
```

Run it and you see something like this:
![Hello GPUI window with a fixed size](/images/gpui/first/hello-gpui-window.png)
You can see that with the default option and now with specific size, the window size actually changes, you can try different sizes to check for yourself

You need to note that, in GPUI, we need to represent dimensions using GPUI typed units like `px()`.

## Making our Element Fill the Window
Right now our `div` only contains the text, lets make it occupy the entire available space or window. Let's modify the `render` method to this:
```rust
impl Render for App {
fn render(&mut self, _window: &mut Window, _cx: &mut Context<Self>) -> impl IntoElement {
    div()
        .size_full()
        .child("Hello, GPUI!")
}
}
```
If you'll run it, you won't see any change because there's no background colour for our `div`, hence even though its now taking the entire window, we can't realise that, so lets add a background colour.

## Adding Colour
Add a background colour and text colour:
```rust
impl Render for App {
fn render(&mut self, _window: &mut Window, _cx: &mut Context<Self>) -> impl IntoElement {
    div()
        .size_full()
        .bg(rgb(0x0f1012))
        .text_color(rgb(0xe8e9ea))
        .child("Hello, GPUI!")
}
}
```
Now, run it and you'll see that the background is in black colour and text is in light colour. We are representing colour in RGB hexadecimal notation. 

One thing you can notice by now is that, in GPUI, we create an element and just configure it by chaining methods.

![Hello GPUI with dark background and light text](/images/gpui/first/hello-gpui-colours.png)

## Nested Elements
Of course, to build any complex layout, we need to know how to use or nest multiple elements, so lets' start learning about it.
Change the render method to this:
```rust
impl Render for App {
fn render(&mut self, _window: &mut Window, _cx: &mut Context<Self>) -> impl IntoElement {
    div()
        .size_full()
        .bg(rgb(0x0f1012))
        .text_color(rgb(0xe8e9ea))
        .child(
            div()
                .child("Hello, GPUI!"),
        )
}
}
```
Run it and you'll see that visually its still the same but the important thing to note here is, our element tree now looks like this:
```bash
App
└── div
└── div
    └── "Hello, GPUI!"
```
The outer `div` is our application container, the inner `div` is its child and the `.child()` method is one of the fundamental methods we'll use in GPUI. Its a way to have that parent-child structure for nested or sibling element structures

## Creating a Two-Column Layout
Our final app has a sidebar and a main content area. To achieve that, we need to know how to create column layouts and we can do that using flexbox. Let's change the inner `div` to this:
```rust
div()
.size_full()
.flex()
.flex_row()
.child(sidebar())
.child(main_content())
```

You can see we are using two functions `sidebar()` and `main_content()`, these are two helper functions that we are going to create, this will help us to maintain and reuse our code
```rust
fn sidebar() -> impl IntoElement {
div()
    .w(px(232.0))
    .h_full()
}
```
We are creating a sidebar component that has a width of `232px` and it takes complete height of the window and the main content layout takes the remaining width of the window and the full height of the window:
```rust
fn main_content() -> impl IntoElement {
div()
    .flex_1()
    .h_full()
}
```
Now, our `render` method should look something like this:
```rust
impl Render for App {
fn render(&mut self, _window: &mut Window, _cx: &mut Context<Self>) -> impl IntoElement {
    div()
        .size_full()
        .bg(rgb(0x0f1012))
        .text_color(rgb(0xe8e9ea))
        .child(
            div()
                .size_full()
                .flex()
                .flex_row()
                .child(sidebar())
                .child(main_content()),
        )
}
}
```
You can run and see that the text is gone and the entire app has a dark background colour, that's all, we can't visibly see the sidebar and main content area. Let's now work on the sidebar container

## Building the Sidebar Container
Replace the `sidebar()` with this:
```rust
fn sidebar() -> impl IntoElement {
div()
    .w(px(232.0))
    .h_full()
    .flex()
    .flex_col()
    .bg(rgb(0x131416))
    .border_r_1()
    .border_color(rgb(0x272a2f))
    .px(px(10.0))
    .py(px(10.0))
    .gap(px(14.0))
}
```
Now, if you'll run it, you can see that the sidebar layout is there. 
![Sidebar container with background and border](/images/gpui/first/sidebar-container.png)
We just have the same fixed width for sidebar and it takes height as the total height of the window. Then we turn this sidebar into a flex container with `flex()` and `flex_col()` is used so that its children will be arranged vertically. Then we set the background and border and finally we set the padding for horizontal and vertical axis and gap as well.

## Adding the Workspace Selector
Now, its time to add some content to the sidebar. The first thing we will add is the workspace selector. Add this child to the end of the sidebar:

> Please go ahead to the github repo and download the images from the assets folder and create a assets folder in your project's root as well. Otherwise the sections from now might not work or at least won't render the two custom assets that we are going to use. 

First, lets update our imports so that we can use icons and assets:
```rust
use gpui_kit::component::{
Icon,
IconName,
Root,
Sizable,
button::{Button, ButtonVariants},
};
use gpui_kit::gpui::prelude::FluentBuilder;
use gpui_kit::*;
```

```rust
.child(
div()
    .h(px(34.0))
    .flex()
    .items_center()
    .px(px(7.0))
    .rounded(px(7.0))
    .hover(|this| this.bg(rgb(0x1d2024)))
    .child(img("org.svg").size(px(22.0)).rounded(px(6.0)))
    .child(
        div()
            .ml(px(9.0))
            .flex_1()
            .text_size(px(13.0))
            .font_weight(FontWeight::MEDIUM)
            .child("SMBL"),
    )
    .child(
        Icon::new(IconName::ChevronDown)
            .size(px(14.0))
            .text_color(rgb(0x777b83)),
    ),
)
```

> From now on, I don't think I need to explain the styling methods that we are using as it should be easy to reason about what we are trying to do. If you still think, I should explain, please let me know, I'll edit the article and add more explanations if needed.

The two custom svgs that we use in this project are `org.svg`: its used by the workspace selector and `user.svg`: its used by the user section at the bottom of the sidebar.
We want these files embedded into the application rather than relying on them being present beside the executable at runtime.

That's why we added `rust-embed` to `Cargo.toml`. At the top of the `main.rs`, just add:
```rust
use std::borrow::Cow;

use anyhow::Result;
use rust_embed::RustEmbed;
```
Then define the embedded asset collection like this:
```rust
#[derive(RustEmbed)]
#[folder = "assets"]
struct AppAssets;
```
The `#[folder = "assets"]` attribute tells `rust-embed` to include everything inside our `assets` directory. Now we need to make these assets available to GPUI Kit. To do that, we need to write this:
```rust
impl AssetSource for AppAssets {
fn load(&self, path: &str) -> Result<Option<Cow<'static, [u8]>>> {
    if let Some(file) = AppAssets::get(path) {
        return Ok(Some(file.data));
    }

    gpui_kit::assets::Assets.load(path)
}

fn list(&self, path: &str) -> Result<Vec<SharedString>> {
    let mut files = AppAssets::iter()
        .filter(|file| file.starts_with(path))
        .map(SharedString::from)
        .collect::<Vec<_>>();

    files.extend(gpui_kit::assets::Assets.list(path)?);

    Ok(files)
}
}
```
So there are two asset sources here, the first one is our own application assets `AppAssets::get(path)`. If the requested file exists in our `assets` directory, we return it; otherwise, we fall back to GPUI Kit's bundled assets with `gpui_kit::assets::Assets.load(path)`. The same pattern is used by `list()` to expose both asset sources.

Now that this is done, we need to register this asset source with the application, so change this `let app = gpui_kit::application();` to this:
```rust
let app = gpui_kit::application().with_assets(AppAssets);
```

now, if you'll run this, you should see:
![Workspace selector in the sidebar](/images/gpui/first/workspace-selector.png)

## Building the Search Box
Now, lets add the search box right below the workspace selector. lets add a `child()` after the workspace element:
```rust
.child(
div()
    .h(px(32.0))
    .flex()
    .items_center()
    .px(px(8.0))
    .rounded(px(7.0))
    .bg(rgb(0x181a1d))
    .border_1()
    .border_color(rgb(0x272a2f))
    .child(
        Icon::new(IconName::Search)
            .size(px(14.0))
            .text_color(rgb(0x777b83)),
    )
    .child(
        div()
            .ml(px(8.0))
            .flex_1()
            .text_color(rgb(0x777b83))
            .text_size(px(12.0))
            .child("Search"),
    )
    .child(
        div()
            .px(px(5.0))
            .py(px(2.0))
            .rounded(px(4.0))
            .border_1()
            .border_color(rgb(0x303339))
            .text_color(rgb(0x696d75))
            .text_size(px(10.0))
            .child("⌘K"),
    ),
)
```

If you'll run this, you should see something like this:
![Search box added below the workspace selector](/images/gpui/first/search-box.png)

## Creating Reusable Navigation Items
Now, lets learn another pattern, sometimes we want to create several elements may be buttons, options or something else that share the same structure.

For example, right now we want to create several navigation entries and they all have the same structure, instead of duplicating the layout, lets create a helper function like we did for sidebar and main content:
```rust
fn sidebar_item(
icon: IconName,
label: &'static str,
count: Option<&'static str>,
active: bool,
) -> impl IntoElement {
div()
    .h(px(31.0))
    .flex()
    .items_center()
    .px(px(8.0))
    .rounded(px(6.0))
    .when(active, |this| this.bg(rgb(0x202227)))
    .hover(|this| this.bg(rgb(0x1d2024)))
    .child(Icon::new(icon).size(px(14.0)).text_color(if active {
        rgb(0xd9dbe0)
    } else {
        rgb(0x777b83)
    }))
    .child(
        div()
            .ml(px(9.0))
            .flex_1()
            .text_size(px(12.0))
            .text_color(if active { rgb(0xe0e1e4) } else { rgb(0x858990) })
            .child(label),
    )
    .when_some(count, |this, count| {
        this.child(
            div()
                .text_size(px(10.0))
                .text_color(rgb(0x656970))
                .child(count),
        )
    })
}
```
This function takes `icon`, `label`, `count`, `active`. These are all the things that we will pass to this function when we want to create a nav entry and depending on these values, the nav element will show up differently.

> We will use `'static` for all of our string literal inputs because we want their lifetime to be valid as long as the program exists. In later articles, we will see how we can use a store system to implement this in a better way

I want you to notice that `when()` is a conditional API that renders things depending on the condition. For now, we are checking if the element is active or not, if yes then apply a certain background colour, we are doing the same kind of thing for text colour as well.

There's one more called `when_some()`, this adds the count element only when a count was supplied. Of course, I can't cover all the methods or details about in-built GPUI methods, you can look it up later. But these things are interesting

Now, once this is done, we can add the navigation section to `sidebar()`:
```rust
.child(
div()
    .flex()
    .flex_col()
    .gap(px(1.0))
    .child(sidebar_item(IconName::Inbox, "Inbox", Some("3"), false))
    .child(sidebar_item(IconName::User, "My issues", Some("7"), false))
    .child(sidebar_item(IconName::CircleX, "Active issues", None, true))
    .child(sidebar_item(IconName::FolderOpen, "Projects", None, false))
    .child(sidebar_item(IconName::PanelLeft, "Views", None, false)),
)
```

If you run it, you'll see something like this, its look really good right now. We just added another child element to sidebar and added multiple child element by using our `sidebar_item` helper function with different values. Please experiment with different values to understand how its working
![Navigation items in the sidebar](/images/gpui/first/navigation-items.png)

## Adding the Projects Section
Lets add another section below the navigation. We will again write a helper function for this:
```rust
fn project_item(label: &'static str, color: u32) -> impl IntoElement {
div()
    .h(px(29.0))
    .flex()
    .items_center()
    .px(px(8.0))
    .rounded(px(6.0))
    .hover(|this| this.bg(rgb(0x1d2024)))
    .child(div().size(px(8.0)).rounded_full().bg(rgb(color)))
    .child(
        div()
            .ml(px(10.0))
            .text_size(px(12.0))
            .text_color(rgb(0x858990))
            .child(label),
    )
}
```
Now, lets add the projects section to the sidebar:
```rust
.child(
div()
    .flex()
    .flex_col()
    .gap(px(2.0))
    .child(
        div()
            .px(px(8.0))
            .pt(px(4.0))
            .pb(px(4.0))
            .text_size(px(10.0))
            .font_weight(FontWeight::SEMIBOLD)
            .text_color(rgb(0x656970))
            .child("PROJECTS"),
    )
    .child(project_item("Website", 0x7c6ff2))
    .child(project_item("Mobile", 0x4e9f7a))
    .child(project_item("API", 0xd9a441)),
)
```

If you'll run it, you should see something like this. We are again just follow the similar pattern that we followed for nav entries
![Projects section with coloured dots](/images/gpui/first/projects-section.png)

The small circles are actually themselves GPUI elements:
```rust
div()
.size(px(8.0))
.rounded_full()
.bg(rgb(color))
```

## Adding User Section to Sidebar
We want the user info to remain at the bottom of the sidebar just like many other apps do. Let's add this final child to `sidebar()`:
```rust
.child(
div()
    .mt_auto()
    .h(px(40.0))
    .flex()
    .items_center()
    .px(px(6.0))
    .rounded(px(7.0))
    .hover(|this| this.bg(rgb(0x1d2024)))
    .child(img("user.svg").size(px(26.0)).rounded_full())
    .child(
        div()
            .ml(px(8.0))
            .flex_1()
            .flex()
            .flex_col()
            .justify_center()
            .gap(px(1.0))
            .child(
                div()
                    .text_size(px(11.0))
                    .text_color(rgb(0xd0d2d6))
                    .child("Sheerluck"),
            )
            .child(
                div()
                    .text_size(px(9.0))
                    .text_color(rgb(0x656970))
                    .child("Personal"),
            ),
    )
    .child(
        Icon::new(IconName::EllipsisVertical)
            .size(px(14.0))
            .text_color(rgb(0x656970)),
    ),
)
```
If you'll run it, you can see something like this:
![User section at the bottom of the sidebar](/images/gpui/first/user-section.png)

## Building the Main Content Area
Now, lets work on the right side as we finished working on the sidebar. Replace the `main_content()` with this:
```rust
fn main_content() -> impl IntoElement {
div()
    .flex_1()
    .h_full()
    .flex()
    .flex_col()
}
```
We are making this main content layout as a flex container and a vertical flex container just like the sidebar
This will contain two major sections: `header` and `content`. Lets start with the header section.

## Creating the Header
Add this as the first child of `main_content()`:
```rust
.child(
div()
    .h(px(56.0))
    .flex()
    .items_center()
    .px(px(28.0))
    .border_b_1()
    .border_color(rgb(0x272a2f))
    .child(
        div()
            .text_color(rgb(0x686c74))
            .text_size(px(12.0))
            .child("Issues"),
    )
    .child(div().mx(px(8.0)).text_color(rgb(0x44474d)).child("/"))
    .child(div().text_size(px(12.0)).child("Active"))
    .child(div().flex_1())
    .child(
        div()
            .flex()
            .items_center()
            .gap(px(6.0))
            .child(Button::new("filter").ghost().label("Filter").small())
            .child(Button::new("sort").ghost().label("Sort").small())
            .child(Button::new("new").primary().label("New issue").small()),
    ),
)
```

Once you run it, you should see this:
![Main content header with breadcrumb and buttons](/images/gpui/first/main-header.png)
There's one thing that I want to explain: `ghost()` comes from GPUI Kit's component layer for its `Button` component. It's just a styling thing; we are not creating this style manually.

## Creating the Content Area
lets add the main content below the header:
```rust
.child(
div()
    .flex_1()
    .px(px(44.0))
    .py(px(30.0))
    .flex()
    .flex_col()
    .gap(px(24.0))
)
```
run it and you should see nothing different but this adds some spacing to our app for the main content.

## Adding the page Heading
inside the content container, add this:
```rust
.child(
div()
    .flex()
    .items_end()
    .child(
        div()
            .flex_1()
            .child(
                div()
                    .text_size(px(23.0))
                    .font_weight(FontWeight::SEMIBOLD)
                    .child("Active issues"),
            )
            .child(
                div()
                    .mt(px(5.0))
                    .text_size(px(12.0))
                    .text_color(rgb(0x777b83))
                    .child("Issues currently being worked on"),
            ),
    )
    .child(Button::new("more").ghost().label("•••").xsmall()),
)
```
Now, if you will run this, you can see that the heading for main content page is there
![Page heading for active issues](/images/gpui/first/page-heading.png)

## Adding Tabs
Lets add tabs to the main content. In these tabs, we will show options tabs like "All", "Assigned to me" and "Created by me". We will follow the same reusable helper function pattern:
```rust
fn tab(label: &'static str, active: bool) -> impl IntoElement {
div()
    .h_full()
    .flex()
    .items_center()
    .text_size(px(12.0))
    .text_color(if active { rgb(0xe2e3e6) } else { rgb(0x696d75) })
    .when(active, |this| this.border_b_2().border_color(rgb(0x7c6ff2)))
    .child(label)
}
```
Now add the tab container below the heading:
```rust
.child(
div()
    .h(px(36.0))
    .flex()
    .items_center()
    .gap(px(18.0))
    .border_b_1()
    .border_color(rgb(0x272a2f))
    .child(tab("All", true))
    .child(tab("Assigned to me", false))
    .child(tab("Created by me", false)),
)
```

You should see something like this:
![Tabs below the page heading](/images/gpui/first/tabs.png)

## Creating Issue Group Headers
Lets create a `group_header` helper function. These group headers will be used to set the title for a group and then we can have multiple issues per group
```rust
fn group_header(name: &'static str, count: u32) -> impl IntoElement {
div()
    .px(px(8.0))
    .py(px(8.0))
    .flex()
    .items_center()
    .gap(px(7.0))
    .child(
        div()
            .text_size(px(10.0))
            .font_weight(FontWeight::SEMIBOLD)
            .text_color(rgb(0x777b83))
            .child(name),
    )
    .child(
        div()
            .text_size(px(10.0))
            .text_color(rgb(0x50545b))
            .child(count.to_string()),
    )
}
```

## Building the Issue Component
The issue cards contain more information so lets create a reusable `issue()` function, these issue cards will be part of a certain group:
```rust
fn issue(
id: &'static str,
title: &'static str,
team: &'static str,
assignee: &'static str,
updated: &'static str,
priority: &'static str,
priority_color: u32,
) -> impl IntoElement {
div()
    .h(px(60.0))
    .flex()
    .items_center()
    .px(px(12.0))
    .rounded(px(7.0))
    .border_1()
    .border_color(rgb(0x202227))
    .bg(rgb(0x141619))
    .hover(|this| this.bg(rgb(0x1a1c20)).border_color(rgb(0x303339)))
    .child(
        div()
            .size(px(16.0))
            .rounded_full()
            .border_2()
            .border_color(rgb(priority_color))
            .mr(px(11.0)),
    )
    .child(
        div()
            .flex_1()
            .flex()
            .flex_col()
            .gap(px(3.0))
            .child(
                div()
                    .flex()
                    .items_center()
                    .gap(px(8.0))
                    .child(
                        div()
                            .text_size(px(13.0))
                            .font_weight(FontWeight::MEDIUM)
                            .child(title),
                    )
                    .child(
                        div()
                            .text_size(px(10.0))
                            .text_color(rgb(0x5f636b))
                            .child(id),
                    ),
            )
            .child(
                div()
                    .flex()
                    .items_center()
                    .gap(px(4.0))
                    .text_size(px(10.0))
                    .text_color(rgb(0x656970))
                    .child(team)
                    .child("·")
                    .child(format!("Updated {updated} ago")),
            ),
    )
    .child(
        div()
            .px(px(7.0))
            .py(px(3.0))
            .rounded(px(4.0))
            .bg(rgb(priority_color))
            .text_color(rgb(0x101113))
            .text_size(px(9.0))
            .font_weight(FontWeight::SEMIBOLD)
            .child(priority),
    )
    .child(
        div()
            .ml(px(16.0))
            .size(px(26.0))
            .rounded_full()
            .bg(rgb(0x30343b))
            .flex()
            .items_center()
            .justify_center()
            .text_size(px(10.0))
            .text_color(rgb(0xc4c6cb))
            .child(assignee.chars().next().unwrap_or('?').to_string()),
    )
}
```
I think this is a great example of why composition is so useful. Here, every issue has the same visual structure but the values changes like `title`, `team` and all the other parameters that the function is taking as input.

now that we have the reusable component with us, lets add this to the content container after the tabs:
```rust
// Engineering
.child(
div()
    .flex()
    .flex_col()
    .gap(px(1.0))
    .child(group_header("ENGINEERING", 3))
    .child(issue(
        "ENG-124",
        "Fix authentication redirect",
        "Backend",
        "John",
        "12m",
        "HIGH",
        0xd95c5c,
    ))
    .child(issue(
        "ENG-123",
        "Improve settings page",
        "Frontend",
        "Sarah",
        "1h",
        "MEDIUM",
        0xd9a441,
    ))
    .child(issue(
        "ENG-122",
        "Update API documentation",
        "Documentation",
        "Alex",
        "3h",
        "LOW",
        0x5b8fd8,
    )),
)
```

Now, you should see something like this, you can actually use some custom icons if you want for those priority labels.
![Issue cards with priority badges](/images/gpui/first/issue-cards.png)

## Adding the Product Issues
Lets add one more product group to see how it looks with multiple products on the page:
```rust
.child(
div()
    .mt(px(12.0))
    .flex()
    .flex_col()
    .gap(px(1.0))
    .child(group_header("PRODUCT", 2))
    .child(issue(
        "PRO-087",
        "Redesign onboarding flow",
        "Product",
        "Maya",
        "5h",
        "MEDIUM",
        0xd9a441,
    ))
    .child(issue(
        "PRO-086",
        "Add keyboard shortcuts",
        "Desktop",
        "David",
        "1d",
        "LOW",
        0x5b8fd8,
    )),
)
```
If you'll run it, you'll see something like this:
![Product issues group added](/images/gpui/first/product-issues.png)

## Adding the Issue Count
Lets add one last thing, this will be our small summary for the content area:
```rust
.child(
div()
    .pt(px(12.0))
    .text_size(px(11.0))
    .text_color(rgb(0x555960))
    .child("Showing 5 of 24 active issues"),
)
```
This is the final look that we will get
![Final look of the GPUI issue tracker UI](/images/gpui/first/final-result.png)

## Summarising What we just built
We started with a single element `div().child("Hello, GPUI!")`, then we slowly added elements and worked with parent-child and sibling element structure, customised each element with chained methods for styling.
We controlled the dimension with these:
```rust
.w(...)
.h(...)
.size_full()
```
we learned how to implement spacing using these:
```rust
.px(...)
.py(...)
.pt(...)
.mt(...)
.ml(...)
.gap(...)
```
colours by using these:
```rust
.bg(...)
.text_color(...)
```
borders from these:
```rust
.border_1()
.border_r_1()
.border_b_1()
.border_2()
.border_color(...)
```

We used `flex_1()` to let elements consume the remaining space, `mt_auto()` to push the user section to the bottom of the sidebar, and finally we also used some condition based methods:
```rust
.when(...)
.when_some(...)
.hover(...)
```

I hope you at least got the gist of how to design a descent UI with GPUI, we still need to learn how to make the interface respond to the user and that is what we will do in our next article and we will stick to this single project for our entire series. See you soon

# Scalars Explained

> In this post, we are going to learn about scalars

Source: https://blog.sheerluck.dev/posts/math/linear-algebra/scalars-explained/ · Markdown: https://blog.sheerluck.dev/posts/math/linear-algebra/scalars-explained/index.md

In this series we are going to learn all the maths that is required to learn machine learning or deep learning. All the post, will explain the concept, will have geometric intuition or understanding and then python code implementation. We will start from linear algebra.

## What is a Scalar?
Before going to vectors, lets first understand what is a scalar? 
A scalar is a single real number. That is the complete definition.
When we write:
$$s \in \\mathbb\{R\} $$

We are saying: $s$ is an element of the set of all real numbers.

> Scalar has only one piece of information: its **value**.

Scalars live in $\\mathbb\{R\}$ (the real number line). It includes every decimal, every fraction, every irrational number. Basically everything on a continuous number line.

## What Can You Do With Scalars
The standard arithmetic operations all apply:

| Operation      | Notation | Example     |
| -------------- | -------- | ----------- |
| Addition       | $a + b$  | $3 + 5 = 8$ |
| Subtraction    | $a-b$    | $3-5=-2$    |
| Multiplication | $a.b$    | $3.5=15$    |
| Division       | $a/b$    | $3/5=0.6$   |

These operations always produce another scalar. The output stays in $\\mathbb\{R\}$.

## Why Scalars Are Not Enough
Let's say you want to describe how an object is moving through 3D space. The object moves at 60 km/h. But 60 km/h where? Diagonally upward? A scalar cannot encode this. You need to know both how much (magnitude) and which way (direction)

In ML, you have a house with features: area = 1200 sq ft, bedrooms = 3, age = 10 years, price = $250,000. These are four separate scalars. But a model needs to process them together as one object, not as four disconnected numbers. A scalar cannot represent structured, multi-dimensional data.

This is the fundamental limitation of scalars:
> A scalar carries magnitude (size/value) only. It carries no directional or structural information.

## The Two Things We Need That Scalars Cannot Provide
1. Direction - When a quantity has both a size and direction like displacement, velocity, force. Here, a single number is insufficient. You need a mathematical object that encodes both simultaneously.
2. Structure across multiple dimensions - When data has multiple features that belong together as one unit like a data point in ML, you need an object that holds all dimensions at once and supports operations across them.

Both the things can be done with **vectors**, which is what we study from next article.

## Conclusion
This was a quick one. In the next one you'll learn about vectors and vector spaces. See you soon.

# Reading Lessons in Stoicism

> In this post, we are going to read the Lessons in Stoicism book

Source: https://blog.sheerluck.dev/posts/misc/lessons-in-stoicism/ · Markdown: https://blog.sheerluck.dev/posts/misc/lessons-in-stoicism/index.md

John Sellars is a philosophy lecturer at Royal Holloway and one of the people behind Modern Stoicism, the group that runs Stoic Week. This book is short, around 100 pages, and it does one thing. It walks you through the core ideas of the three major Roman Stoics, Seneca, Epictetus, and Marcus Aurelius, and explains what they were actually saying without making it boring.

The three could not have lived more different lives. Seneca was a tutor to Emperor Nero, a member of the Roman elite, and ended his life with a forced suicide. Epictetus was a slave from Asia Minor who was eventually freed, banished from Rome, and opened a philosophy school in Greece. Marcus Aurelius was the Emperor of Rome. And yet all three embraced the same philosophy.

By the time these three were writing, Stoicism was already hundreds of years old. It started in Athens around 300 BC with a man named Zeno from Cyprus, who lectured at the Painted Stoa, a covered colonnade, and gathered followers who became known as Stoics. None of the early Stoic writings survived. What we have comes almost entirely from these three Romans.

The book is structured around seven lessons. Lets go through each one.

## Chapter 1: The Philosopher as Doctor

Epictetus described the philosopher as a doctor and the philosophy school as a hospital for souls. This is not a metaphor he invented. It traces back to Socrates, who argued that the job of philosophy is to take care of your soul the same way a physician takes care of your body. By soul here they mean your mind, thoughts, and beliefs, not anything supernatural.

The core argument Socrates made, which the Stoics took up, is that material things are **value-neutral**. Money is neither good nor bad. Whether it benefits or harms depends entirely on the character of the person who has it. A good person uses money to do good things. A bad person uses it to cause harm. So the real source of value is not in the money itself but in the character of the person who holds it.

This led to the Stoic concept of **indifferents**. Only an excellent, virtuous character is genuinely good. Only a vicious character is genuinely bad. Everything else, wealth, health, reputation, success, is an indifferent. Not good or bad in itself.

But the Stoics were not like the Cynics. Diogenes of Sinope, the most famous Cynic, lived in a barrel and threw away his cup after seeing a child drink water with its hands. He praised poverty and rejected possessions entirely. Zeno, the first Stoic, found this too extreme. If money is truly neutral, why treat poverty as better than wealth?

Instead, Zeno introduced the idea of **preferred indifferents**. All else being equal, we naturally prefer to be healthy rather than sick, wealthy rather than broke, respected rather than despised. That is completely rational and nothing to feel bad about. But and this is the key point, you should never compromise your character to get these things. A virtuous character is the only truly good thing. Chasing money or fame at the cost of your integrity means trading the only real good for something that is not actually good at all.

So what does having good character actually mean? For the Stoics it means being wise, just, courageous and moderate. These are the four cardinal virtues. This is what it means to be a good human being, which the Stoics thought about the same way you might think about a good knife. A good knife cuts well. A good human, as a social animal, behaves well toward others.

## Chapter 2: What Do You Control?

This is probably the most well-known Stoic idea and it comes directly from Epictetus. The Handbook of Epictetus opens with this.

The things within our power: our judgements, impulses, and desires. Pretty much everything else is not within our power. Our bodies, our possessions, our reputation, our worldly success. All of these are ultimately outside our control.

Now, that sounds like Epictetus is saying we have almost no control over anything. But he is actually saying the opposite. Because **judgements are the source of everything**. We see something, judge it as good, which creates a desire, which drives us to pursue it. The whole chain of behavior starts with a judgement. If you master your judgements, you master your desires and your actions. Your happiness becomes entirely your own.

Marcus Aurelius put this into practice. He would pause before evaluating seemingly desirable things and think about them physically. A fine meal is just the dead body of a pig or a fish. An expensive car is just a lump of metal and plastic. Whatever value these things appear to have is something we project onto them with our judgements. It is not in the things themselves.

Epictetus makes another important point. If you tie your happiness to something outside your control, you have effectively handed your happiness over to forces that do not care about you. A romantic relationship, a career goal, a specific level of wealth. If any of these goes wrong and it might because you cannot control outcomes, only actions, then your wellbeing collapses with it.

The Stoics use the archer as an example here. Antipater, a Stoic writing before Epictetus, pointed out that even an expert archer sometimes misses because the wind might blow the arrow off course. The archer cannot control that. What the archer can control is the quality of the shot itself. So make your goal doing the best you can, not hitting the target. Then nothing can prevent you from succeeding at your actual goal.

Marcus Aurelius extends this to the whole of Nature. Everything changes. Nothing stays stable. He reminds himself of this constantly in the Meditations. All we can do is accept what happens that is not within our control and direct our efforts at what is.

Epictetus says this vigilance cannot be occasional. It has to be constant. He uses a sailing image. It is much easier for a mariner to wreck a ship than to keep it sailing safely. A momentary loss of attention produces disaster. Philosophy has to be a daily practice, not something you dip into when life gets hard.

Both Marcus and Seneca built reflection into their routines. Marcus would do morning reflection to prepare for the day. Seneca would do evening reflection to review what he did well, where he slipped, and what he could do better.

## Chapter 3: The Problem with Emotions

Someone came to Epictetus's school and complained about his brother, who had become angry with him. What could he do about his brother's anger? Epictetus said nothing. You cannot do anything about it. The only person who can change the brother's anger is the brother.

But Epictetus did not stop there. He shifted attention to what the man could actually control, his own reaction. The man was upset about his brother's anger and that upset was something the man himself could address. The real problem was not the brother. It was the judgement the man had made about the brother's anger.

The Stoic position on emotions is often misunderstood. The word stoic in everyday English means unfeeling and cold, which is not what the ancient Stoics meant. They did not want to eliminate all emotional responses. They wanted to eliminate the harmful ones. Anger, resentment, jealousy, possessiveness, obsession, perpetual fear.

The central claim is that **emotions are the product of judgements**. Because they come from judgements, and we control our judgements, we are responsible for our emotions. Not responsible for having involuntary physical reactions, but responsible for whether those reactions become full emotions.

Chrysippus compared having an emotion to running so fast you can no longer stop. Once an emotion has momentum, you are out of control and you cannot just switch it off. Seneca extended this. He likens being angry to having been thrown off a building and hurtling toward the ground. The emotion takes over the whole mind. This is what the Stoics warn against.

Seneca draws three stages. First, an **involuntary first movement**. A shock, a flinch, a sudden nervousness, even tears. These are physiological responses, not emotions in the Stoic sense. They are not within our control and that is fine. Second, a **judgement** in response to what happened. This is within our control and this is where the critical moment is. Third, an **emotion** that once formed is out of our control and must simply subside on its own.

The space between stage one and stage two is where the work happens. Epictetus says if someone insults you, pause before reacting. Ask whether what they said is true or false. If true, they have pointed out a fault you can now fix. They have helped you. If false, they are simply wrong and the only one being harmed is themselves. Either way, you suffer no harm from the insult itself. The only way you can be harmed is if you let it produce anger.

What about love? Are the Stoics saying to eliminate that too? No. A parent's love for a child is a natural instinct, not an irrational emotion. Healthy romantic relationships built on companionship and natural affection are fine. What they wanted to avoid were the destructive emotional attachments, possessiveness, jealousy, obsession. And even with grief, the Stoics acknowledge that first movements like crying are natural. What they want to prevent is grief becoming a debilitating, consuming habit of mind.

## Chapter 4: Dealing with Adversity

Seneca's own life was a test case. His son died. He was exiled to Corsica for nearly a decade. He was forced to become Nero's tutor on condition of being allowed back. He watched close friends die. He was eventually forced to commit suicide on Nero's orders. His wife insisted on dying with him. Neither died quickly.

His essay On Providence asks why bad things happen. His first answer is that nothing external is genuinely bad. Events are neither good nor bad in themselves. Someone who has internalized this will not rush to judge something terrible as terrible. They will simply accept what is.

But he goes further than that. He argues we should actually welcome adversity as a training exercise. He uses a wrestling image. A wrestler only improves by facing tough opponents. An easy opponent teaches nothing and allows skills to atrophy. Adversity in life works the same way. It gives you a chance to demonstrate your virtues and strengthen them.

He also makes the inverse argument. Excessive good fortune is bad for you. If everything always goes your way, you are never tested, never pushed to develop patience, courage, or resilience. Unending luxury breeds laziness, complacency, and greed. Real misfortune, Seneca says, is a life with no difficulty at all.

Then there is the **premeditation of future evils**, an idea advocated by Chrysippus and developed by Seneca. The idea is to regularly reflect on potentially bad things that could happen, so that you are better prepared to handle them if they do. Seneca wrote a letter of consolation to his friend Marcia, who had been consumed by grief for three years after the death of her son. Part of her problem, Seneca said, was that she had never reflected on the possibility of her son dying. But everyone dies. This is not something that might happen. It is something that must happen.

Grief hits hardest when it is unexpected. Seneca lists the uncomfortable truths. All our loved ones will die, and could do so at any moment. Whatever security or prosperity we have could be taken away at any moment by forces outside our control. When we think things are tough, they can always get worse. Not thinking about these things does not protect you from them. It just means you will be blindsided when they happen.

This is not pessimism. It is practical preparation. Seneca does not say you should go looking for adversity or pretend to welcome every hardship cheerfully. In one letter to Lucilius he is clear he does not agree with those who charge straight into difficult situations as a matter of principle. The goal is simply to be prepared so that when adversity comes, and it will come, you are not completely undone by it.

## Chapter 5: Our Place in Nature

Marcus Aurelius spent the later years of his reign on military campaigns near modern-day Vienna. He kept a private notebook during those campaigns, writing to himself to process his experiences and prepare for the next day. That notebook is the Meditations.

One of its central themes is how small we are.

Marcus imagines looking down at the earth from a great height, as astronauts have since actually done, and seeing how tiny each country is, how minuscule each city. From that perspective, the people living in those cities with their full lives of cares and concerns are practically nothing. He also reflects on time. Each human life is a tiny fraction of the boundless abyss of time.

This is not nihilism. The Stoic view is that Nature is not blind or chaotic. It is a single living organism, ordered and animated by a rational principle they called God or Zeus, though not in the personal sense. This rational principle within Nature is what accounts for the order and patterns we observe. The Stoics identified this with fate, which they meant in a physical sense, a chain of causes and effects. Physics, not superstition.

Sellars compares this to the Gaia hypothesis of James Lovelock, the idea that life on earth functions as a single self-regulating system, including organic and inorganic matter together. The Stoics had a remarkably similar view. Nature as a unified, intelligent organism, not a collection of isolated parts.

What does this mean practically? It means that the events of your life are part of a causal chain. When something happens, given all the causes at play at that moment, it could not have turned out otherwise. Wishing it had turned out differently is not just futile, it is a failure to understand how the world works.

Marcus puts it this way. Nature gives all and takes all back. The person educated into humility says to Nature: give what you will, take back what you will. Not in defiance, but as a loyal subject.

Marcus was honest enough to acknowledge uncertainty about whether Nature is truly providential, a rational order working for our benefit, or simply atoms colliding in a void. His view was that for practical purposes it does not matter. Whether the universe is guided by a providential deity, a cybernetic feedback system, or pure chance, the correct response is the same. Accept what happens and act as well as you can.

## Chapter 6: Life and Death

Seneca knew his life could end at any moment. A bad-tempered emperor, his own poor health. This made him think hard about time.

His essay On the Shortness of Life makes a counterintuitive argument. The problem is not that our lives are too short. The problem is that we waste most of the time we have. All of us have enough time. We just fritter it.

He lists the ways people waste their lives. Some chase wealth and success so they can buy luxury goods that end up in rubbish bins. Some go through daily routines on autopilot with no sense that time is running out. Some know what they want to do but postpone it, paralyzed by fear of failure, waiting for conditions to be perfect. All of these, Seneca says, are ways of failing to actually live.

He is also sharp about what comes with worldly success. People spend their whole lives chasing it. They get it and find they now lack the one thing they wanted most: time. Time for themselves, for peace, for quiet, for just being. Success brings its own demands, and many people who achieve it are more trapped than they were before.

Then there is distraction. Constant noise, interruptions, news, social media, all competing for attention to the point where it becomes difficult to complete anything or to simply be present in your own life. Seneca says living is the least important activity of the preoccupied man. They are effectively busy doing nothing.

His remedy is to hold the fact of death in your mind. Not morbidly, but practically. We do not know how much time we have left. Today could be your last day. Maybe you have weeks, maybe decades. The assumption that you will definitely make it to old age may simply be false. And even if it is true, assuming you have unlimited time encourages you to postpone everything into a future that may arrive too late or not at all.

Epictetus adds something important here. Life is a gift. It was given to you and it can be taken back. He says to think of it like being invited to a fair or a party. At some point the host says it is time to go. You can either leave feeling grateful for the time you had, or you can complain that it could not go on longer.

He applies this to the people you love too. Nothing and no one belongs to you permanently. Your wife, your child, your friends. They are on loan. One day they will be returned. He says plainly that it is foolish to want your loved ones to be immortal, since that is not within your power or theirs.

The goal for both Seneca and Epictetus is not to be gloomy about death but to make your appreciation of life more real and more urgent. Live each day as though it could be your last, not in the sense of giving up on planning, but in the sense of not deferring the actual experience of being alive to some imagined future.

## Chapter 7: How We Live Together

Most of what the Stoics teach looks self-focused. Control your judgements, manage your emotions, deal with your adversity. Marcus Aurelius talked about retreating to his inner citadel to escape external noise. Does any of this leave room for other people?

It does, and substantially so.

The Stoics agreed with Aristotle that humans are by nature social and political animals. We are born into communities, first family, then local community, then country, and ultimately all of humanity. The inner work is not a retreat from the world. It is preparation to engage with it better. By improving your character and eliminating destructive emotions like anger and resentment, you become a more effective member of the communities you belong to.

Epictetus stressed that each of us occupies multiple social **roles**. Some roles come from nature, like being a parent. Some come from social positions, like being a doctor or a magistrate. Each role comes with duties and responsibilities, and neglecting those duties is a failure.

He gives a concrete example. A magistrate visited his school. On being asked about his family, the man said his daughter had been so gravely ill that he could not bear to be with her and had run away. Epictetus challenged him on two counts. First, he was prioritizing his own feelings over his daughter's wellbeing. Second, he was neglecting his role as a father. He claimed to have left because of his love for her. But his love for her should have made him stay.

Beyond specific roles, the Stoics held a vision of a single human community. A slightly obscure Stoic called Hierocles developed this through the idea of expanding circles of concern. At the center is yourself. Then your immediate family. Then your local community. Then your country. Then all of humanity. You have duties at every level, and the outermost circle does not erase the inner ones. You are a member of both your local community and of humanity at large.

Seneca put it plainly. There are two commonwealths. One vast and common, embracing gods and men. One assigned to us by the accident of birth. We belong to both.

This was not just abstract theory. Stoics in Rome actually put their lives on the line for it. Helvidius Priscus, a senator, refused to back down when Emperor Vespasian abused his authority over the Senate. He was warned to stay away. He insisted on defending the rights of his fellow senators. He was executed. Marcus Aurelius later named him as one of the people who taught him the ideal of a community based on equality and freedom of speech.

Musonius Rufus, a Stoic lecturer who taught Epictetus, argued that women have the same capacity for reason and the same natural inclination toward virtue as men, and should therefore be able to study philosophy. This was roughly two thousand years before universal suffrage.

Epictetus also has a practical warning. If you are trying to develop new habits and improve your character, be very careful about who you spend time with. Spending time with people who embody the habits you are trying to escape is like brushing against someone covered in soot. You will get covered in soot. Find people whose values you share or admire.

# Welcome to My Blog

> A quick intro to this Zola + Terminimal setup.

Source: https://blog.sheerluck.dev/posts/misc/welcome-to-mrsheerluck-blog/ · Markdown: https://blog.sheerluck.dev/posts/misc/welcome-to-mrsheerluck-blog/index.md

Hi,

Thank you so much for subscribing to my newsletter. I write mostly on whatever I am learning and want to share my knowledge on. The topic does not always have to be based on tech. It can be science, maths, economics, philosophy, game design, animation, and more.

My point is this is my digital garden, and I am thanking you for being a part of this journey.

> I'm in the process of rewriting my Rust series and hence took down all the articles of Rust series. If you still want to read those articles, you can read from [here](https://github.com/MrSheerluck/blog/tree/main/content/posts)

If you have any feedback or want to request any topic, please feel free to reach out.

- X: [@sheerluck_io](https://x.com/sheerluck_io)
- Bluesky: [@mrsheerluck.bsky.social](https://bsky.app/profile/mrsheerluck.bsky.social)
- GitHub: [MrSheerluck](https://github.com/MrSheerluck)
- YouTube: [@sheerluck-dev](https://www.youtube.com/@sheerluck-dev)

# WTF is a supply chain attack

> A quick intro to supply chain attack and axios case study

Source: https://blog.sheerluck.dev/posts/misc/wtf-is-a-supply-chain-attack/ · Markdown: https://blog.sheerluck.dev/posts/misc/wtf-is-a-supply-chain-attack/index.md

![Axios supply chain issue screenshot](/images/axios-issue.webp)

Two major open source libraries LiteLLM and axios(still developing story) were compromised within a week of each other. In this article, you are going to learn what supply chain attacks are, why they keep working and what you need to know right now.

## What is a Supply Chain Attack?
When you write software, you pull packages written by other people like pulling packages from PyPI, npm or elsewhere. Those packages might pull in more packages of their own. That entire web of external code is your software supply chain.

A supply chain attack targets something your code actually trusts. If an attacker get malicious code into a library you install. Then, its game over. It'll have access to your files, secrets, infra.

## Axios (March 31, 2026) Issue (Still Developing)
Axios is one of the most used HTTP client library in JavaScript. This story just broke today (31st March 2026) and details are still coming, but lets just take this as a case study to understand supply chain attack clearly.

An attacker compromised the npm account of a lead axios maintainer, changed the account email id to an anon address and published two poisoned versions manually. 

The attacker added a dependency called `plain-crypto-js`. This package is never actually used by axios. It was not there previously. It's only purpose is to run a `postinstall` script that drops a Remote Access Trojan (RAT) targeting macOS, Windows and Linux. The RAT contacts a command and control server, downloads platform specific payloads, then deletes its own traces.

## What to Check Right Now
If you have installed `axios@1.14.1` or `axios@0.30.4` please remove them or revert back to a safer version. You can revert back these to `axios@1.14.0` and `axios@0.30.3`

> Update 1: The axios version has been rolled back to a safer version.
![axios-rolled-back-to-safer-version](/images/axios-rolled-back.png)

I'll update the story as new confirmed details come in.

# WTF is Claude Mythos

> Everything we know about Anthropic's Claude Mythos model

Source: https://blog.sheerluck.dev/posts/misc/wtf-is-claude-mythos/ · Markdown: https://blog.sheerluck.dev/posts/misc/wtf-is-claude-mythos/index.md

# WTF is Claude Mythos

Anthropic didn't announce Mythos. It leaked. A misconfigured CMS left a draft blog post in a publicly searchable data store. Security researchers found it and Anthropic confirmed it shortly after.

In this article you are going to learn what Mythos is, what it can actually do, and why Anthropic is scared of releasing it.

## What is Mythos?

Mythos is Anthropic's next model. It sits in a new tier above Opus, both in capability and cost. Internally they called this tier Capybara.

It is a general-purpose model. But it is strikingly capable at cybersecurity tasks specifically. Where prior Claude models respond to instructions one step at a time, Mythos plans and executes sequences of actions on its own, moving across systems and completing operations without waiting for human input at each stage.

Here are the benchmark numbers from Anthropic's own evaluation on agentic coding:

| Benchmark | Mythos Preview | Opus 4.6 |
|---|---|---|
| SWE-bench Pro | 77.8% | 53.4% |
| Terminal-Bench 2.0 | 82.0% | 65.4% |
| SWE-bench Multimodal | 59.0% | 27.1% |
| SWE-bench Multilingual | 87.3% | 77.8% |
| SWE-bench Verified | 93.9% | 80.8% |

These are not small improvements.

## What Can It Actually Do

This is where it gets serious.

During testing, Mythos Preview was capable of identifying and exploiting zero-day vulnerabilities in every major operating system and every major web browser. Many of the vulnerabilities it found are ten or twenty years old. The oldest being a 27-year-old bug in OpenBSD.

In one case it wrote a browser exploit that chained together four vulnerabilities, writing a complex JIT heap spray that escaped both the renderer and OS sandboxes. It autonomously obtained local privilege escalation on Linux by exploiting subtle race conditions and KASLR bypasses. It also wrote a remote code execution exploit on FreeBSD's NFS server that granted full root access to unauthenticated users by splitting a 20-gadget ROP chain over multiple packets.

The numbers make this concrete. Opus 4.6 turned Firefox 147 JavaScript engine vulnerabilities into working shell exploits only 2 times out of several hundred attempts. Mythos Preview developed working exploits 181 times on the same benchmark.

These capabilities were not explicitly trained into Mythos Preview. They emerged as a downstream consequence of general improvements in code, reasoning, and autonomy. The same improvements that make it better at patching vulnerabilities also make it better at exploiting them.

## Why It Is Not Public

Mythos Preview can find tens of thousands of vulnerabilities that even an advanced bug hunter would struggle to find, and it can write the exploits to go with them.

It is also extremely compute-intensive and expensive to run at scale. Anthropic has stated it is working on efficiency improvements before any general release.

So it is dangerous and expensive. Both problems they need to solve before shipping it broadly.

## What They Did Instead: Project Glasswing

On April 7, Anthropic launched Project Glasswing. More than 40 partner organizations get access to Mythos Preview strictly for defensive security work, scanning their own code and open-source software for vulnerabilities. Partners include Amazon, Apple, Microsoft, Cisco, CrowdStrike, Palo Alto Networks, and the Linux Foundation. Findings will be shared across the industry.

Anthropic is providing up to $100 million in usage credits to participating companies and $4 million to open-source security organizations including OpenSSF, Alpha-Omega, and the Apache Software Foundation.

Over 99% of the vulnerabilities found have not yet been patched, so details cannot be disclosed publicly yet. For findings they cannot discuss, Anthropic is publishing SHA-3 cryptographic commitments as proof they hold the findings. These will be revealed once responsible disclosure is complete.

There is no public release date.

## Conclusion

This was a big one. Mythos is real, it is the most capable model Anthropic has built, and it is deliberately locked away until they figure out how to release it safely. Project Glasswing is their attempt to use it for good in the meantime.

## Resources

- Anthropic Red Team: [Assessing Claude Mythos Preview's cybersecurity capabilities](https://red.anthropic.com/2026/mythos-preview/)
- Axios: [Anthropic withholds Mythos Preview over hacking concerns](https://www.axios.com/2026/04/07/anthropic-mythos-preview-cybersecurity-risks)
- TechCrunch: [Anthropic debuts Mythos preview in new cybersecurity initiative](https://techcrunch.com/2026/04/07/anthropic-mythos-ai-model-preview-security/)
- Fortune: [Anthropic confirms Mythos after data leak](https://fortune.com/2026/03/26/anthropic-says-testing-mythos-powerful-new-ai-model-after-data-leak-reveals-its-existence-step-change-in-capabilities/)
- Bloomberg: [Anthropic lets Apple, Amazon test Mythos](https://www.bloomberg.com/news/articles/2026-04-07/anthropic-lets-apple-amazon-test-more-powerful-mythos-ai-model)

# Reliability Over Ideology

> Thoughts on Codeberg's LLM policy, infrastructure reliability, and why predictable governance matters when choosing a Git hosting platform.

Source: https://blog.sheerluck.dev/posts/my-thoughts/reliability-over-ideology/ · Markdown: https://blog.sheerluck.dev/posts/my-thoughts/reliability-over-ideology/index.md

For a while, I’ve been thinking about moving away from Github to an Open Source Git Hosting Platform.

I don’t hate Github but I just don’t want to tied up with Microsoft’s ecosystem (Ironically, I still use Github Sponsors for donation). Naturally, Codeberg was one of the alternatives I started looking into.

But after reading Codeberg’s recent post on LLM-generated projects, I realized it probably isn’t the platform I’m looking for. Not because I disagree with its position on AI, but because it changed how I think about the platform’s long-term reliability.

When I choose infrastructure, I care about predictability.

A Git hosting platform should have stable governance and clear rules. I want to be confident that if my project follows the rules today, it’ll still follow them five years from now. That’s much more important to me than whether the platform agrees with or share my views on a particular technology or ideology.

What changed my mind was seeing a Git hosting platform make decisions about which categories of software it wants to encourage. Today its repositories that are “mostly” (again, this is debatable) generated using LLMs, previously it took a similar stance toward cryptocurrency related projects.

Maybe that’s where it stops. Maybe it doesn’t.

The point is that once a platform starts making decisions based on technology categories instead of project behaviour, its difficult to know where that line will be drawn in the future.

If I’m choosing a Git host, I’m making a long-term decision. I don’t want to wonder whether a project that’s acceptable today will become unwelcome because the platform’s view on a technology changes. That’s not the kind of uncertainty I want from infrastructure.

Personally, I’ve a fairly conservative workflow with LLMs. I use them regularly at work and when I need to prototype something quickly or for fun.

But when I’m learning something or building a product that I care about, I write almost all of the code myself because I want to understand the implementation and maintain complete control over it. You can say its a skill issue, but hey, that’s just me.

I don’t think everyone should work like this.

One of the most interesting things about LLMs is that they’ve lowered the barrier to contributing to open source. People who previously couldn’t navigate a large Rust or Go codebase can now understand it well enough to fix bugs, customize software or build tools for themselves.

Whether that’s good or bad depends on the quality of the result, not on the fact that an LLM was involved.

On the other side, I understand the concerns Codeberg raises. Repository spam, AI crawlers, hosting large number of low-quality repositories that are taking actual operational cost.

But to me, those are problems of behavior, not technology and those are the kinds of problems Git hosting providers should focus on resolving.

What I’m less convinced by is treating an entire category of software differently because of how it was produced.

I don’t mean to say Codeberg is wrong. It’s their platform and they are free to define the community they want to build. It does, however, make me reconsider whether its the kind of infrastructure I want to depend on.

For me, reliability is a more important property than alignment. I don’t expect a Git hosting provider to agree with every technology I use. But I expect it to host my projects consistently as long as I follow its rules

# WTF Is the Apology? Reading Plato's Apology of Socrates

> In this post, we are going to understand the Apology dialogue from the book Plato - The Complete Works

Source: https://blog.sheerluck.dev/posts/philosophy/plato/apology/ · Markdown: https://blog.sheerluck.dev/posts/philosophy/plato/apology/index.md

The Apology is not an apology. The Greek word *apologia* means defense speech, and that is exactly what this is. Socrates does not apologize for anything. He stands in front of a jury of 501 Athenians, faces a death sentence, and spends the entire time telling them they are wrong.

He is found guilty and executed.

This is the second dialogue in Plato's complete works, right after the Euthyphro. In the Euthyphro, Socrates was on his way into court. Now we are inside the court, watching the trial happen.

## The Setup

Athens, 399 BC. Socrates is 70 years old. Three men have brought charges against him: Meletus, Anytus, and Lycon. The formal charge is impiety, specifically that he does not believe in the gods the city believes in, that he introduces new spiritual beings, and that he corrupts the youth.

The jury is 501 ordinary Athenian citizens, randomly selected. A majority vote determines guilt. Then there is a second vote on the penalty.

Socrates opens by telling the jury that he will speak plainly, the way he talks in the marketplace, and asks them to judge him on what he says and not on how he says it. He warns them not to create a disturbance if his manner seems unusual. This is his first appearance in a lawcourt.

## Two Sets of Accusers

Before addressing the formal charges, Socrates says there are actually two sets of accusers he needs to deal with.

The recent accusers are Meletus and his associates. But there are older accusers, people who have been spreading a reputation about him for years. They say he is a wise man who studies things in the sky and below the earth, makes the worse argument the stronger, and teaches these things to others. Socrates says these old accusers are actually the more dangerous ones. They got to the jurors when they were young, when they would most readily believe, and there was no one there to refute them.

He also points to Aristophanes, the comedian, whose play *Clouds* portrayed a character called Socrates swinging in a basket and talking nonsense about the heavens. That image has stuck in people's minds for years.

So before he even gets to Meletus, Socrates has to fight a reputation built up over decades by people he cannot even identify or cross-examine.

## What Actually Caused All This

Socrates says his reputation for wisdom comes from something very specific. Here is the story.

His friend Chaerephon once went to the oracle at Delphi and asked whether anyone was wiser than Socrates. The oracle said no. When Socrates heard this, he was puzzled. He knew he had no great wisdom. So why would the god say this?

He decided to investigate by going to people who were considered wise and testing whether they actually were. First he went to politicians. He found that the ones with the greatest reputations were the most deficient, while they themselves thought they knew a great deal. Socrates concluded that he was slightly better off than them, because at least he did not think he knew things he did not know.

Then he went to the poets. He took their best work and asked them what they meant. He found that almost anyone could explain the poems better than the poets themselves. They did not compose from knowledge but from some kind of inspiration, like seers. And yet they thought themselves very wise in all matters because of their poetry.

Finally he went to the craftsmen. They did know things he did not know, and in that they were wiser. But they made the same mistake as the poets: because they were skilled at their craft, they assumed they were wise about everything else too.

After all this investigation, Socrates arrived at an interpretation of the oracle. The god is not saying Socrates is literally the wisest person. He is using Socrates as an example of someone who understands that his wisdom is worthless. **Human wisdom is worth little or nothing.** The one who is wisest is the one who, like Socrates, knows that he does not know.

This investigation is also why people hate him. The people he exposed in front of bystanders became angry, not with themselves but with Socrates. They started spreading the accusations that Meletus eventually formalized. Meletus speaks for the poets, Anytus for the craftsmen and politicians, Lycon for the orators.

## Dismantling Meletus

Socrates now cross-examines Meletus directly, which is where the dialogue gets sharp.

**On corrupting the youth.** Socrates asks Meletus who improves the youth. Meletus eventually says everyone does: the jurymen, the assembly members, all the Athenians. Socrates uses a horse-breeding example to expose the absurdity. With horses, one person or very few know how to improve them. The majority, if they have horses and use them without knowing what they are doing, corrupt them. Surely the same is true with people. The claim that only one man in all of Athens corrupts the youth while everyone else improves them is not just false, it is incoherent.

He then pushes further. Does Socrates corrupt them deliberately or unwillingly? Meletus says deliberately. Socrates says that cannot be right. If I deliberately make the people around me worse, I risk being harmed by them myself. No one makes others worse on purpose. So either I do not corrupt them, or I do so unwillingly. If unwillingly, the law does not bring people to court for that. It requires you to take them aside privately and instruct them. Meletus never did that.

**On not believing in gods.** Socrates shows that Meletus has contradicted himself. Meletus says Socrates does not believe in gods at all. But the formal charge also mentions that he introduces new spiritual beings. Does anyone believe in spiritual activities but not in spirits? And if he believes in spirits, which are either gods themselves or the children of gods, how can he simultaneously not believe in gods? Meletus is either saying Socrates is guilty of not believing in gods and of believing in gods, which is a contradiction. He has made this deposition either to test Socrates or because he genuinely could not find any real wrongdoing.

## Why He Will Not Stop

Socrates says something important here. Even if the jury offers to acquit him on the condition that he stop doing philosophy, he will refuse. He is not being stubborn. He genuinely believes he cannot stop, because stopping would mean disobeying the god who commanded him to examine himself and others.

And then he says the thing that probably sealed his fate:

> The unexamined life is not worth living for men.

He compares himself to a gadfly sent by the god to sting a great but sluggish horse into action. The city is the horse, and he is the gadfly. If they kill him, they will sleep undisturbed. Another gadfly will not easily come along.

He also explains why he never went into public life. His **divine sign**, a kind of inner voice, has always stopped him from doing things. It always says no, never yes. It stopped him from entering politics. He thinks that is fortunate, because a man who genuinely opposes the crowd on matters of justice does not survive long in public life.

He gives two examples from his own life to prove he has always acted on principle regardless of personal risk. During the trial of the ten generals after the battle of Arginusae, he was the only member of the presiding council to vote against an illegal mass trial. The crowd wanted blood and shouted him down, but he refused to participate. Under the Thirty Tyrants, when they ordered him and four others to go arrest Leon of Salamis for execution, Socrates simply went home. The others obeyed. He did not. He might have been killed for this but the regime fell shortly after.

## No Begging, No Dramatics

Socrates explicitly says he will not bring his family into court to cry in front of the jury. He has three sons, one a teenager and two younger children. He will not parade them before the jury to gain sympathy. He thinks this behavior is undignified and inappropriate, especially for someone who has spent his life claiming to care about virtue. If someone with a reputation for wisdom collapses into begging when facing death, it makes the city look bad.

He asks the jury to judge him by the facts of the case, not by pity.

## The Verdict and the Penalty

The jury votes guilty. He lost by a relatively narrow margin. He notes that if just 30 votes had gone the other way he would have been acquitted.

Meletus proposes the death penalty. Now Socrates has to propose a counter-penalty.

He starts by saying what he actually deserves: free meals in the Prytaneum, the magistrates' hall where Athenian olympic victors were honored. He genuinely believes he has been doing the city a great service by going around making people examine their lives. An Olympian victor makes you feel happy, he says. He makes you actually be happy.

He knows the jury will take this as arrogance. He is right that they do.

He considers the alternatives. A fine? He has no money. Imprisonment? Why would he want to live in prison? Exile? Wherever he goes, he will do the same thing and end up in the same situation. He eventually offers 30 minas, with his friends Plato, Crito, and others standing as guarantors.

The jury votes for death.

## After the Verdict

Socrates speaks to both groups separately. To those who voted against him, he says they have not escaped examination. After he is gone, more questioners will come, younger and harsher, and the jury will resent them even more. You cannot escape being held accountable by silencing the person holding you accountable.

Then he says something remarkable. His divine sign did not stop him at any point during this trial. Not when he left home, not when he entered the court, not at any moment during his speech. He takes this as a sign that what happened was actually good.

He lays out two possibilities for what death is. Either it is complete unconsciousness, like dreamless sleep, in which case it is nothing to fear and may even be pleasant. Or it is a relocation of the soul to another place, where all the famous dead are. In that case it would be an extraordinary opportunity to meet Homer, Hesiod, Odysseus, and the great heroes, and to question them and test whether they are actually wise. Socrates says he would happily die many times over for that chance.

He closes by saying a good man cannot be harmed in life or in death. He is not angry with those who convicted him. He only asks one thing: when his sons grow up, if they care more for money or reputation than for virtue, reproach them the way he reproached the Athenians.

The last line is this:

> Now the hour to part has come. I go to die, you go to live. Which of us goes to the better lot is known to no one, except the god.

# WTF Is the Crito? Reading Plato's Crito

> In this post, we are going to understand the Crito dialogue from the book Plato - The Complete Works

Source: https://blog.sheerluck.dev/posts/philosophy/plato/crito/ · Markdown: https://blog.sheerluck.dev/posts/philosophy/plato/crito/index.md

The trial is over. Socrates has been convicted and sentenced to death. But he is not dead yet.

In Athens at this time, there was a religious custom. Every year, a state galley sailed to the island of Delos for a sacred mission, and no executions were permitted while the ship was away. The ship had left the day before the trial, which means Socrates gets roughly a month in prison before his execution.

The Crito takes place in that prison, early one morning, very close to the end. His old friend Crito has come to tell him the ship is almost back and to make one final argument: Socrates should escape.

Socrates refuses. The question is why, and whether he is right.

## The Setup

Crito arrives before dawn. He has bribed the prison warder and slipped in. He watches Socrates sleeping peacefully and does not wake him. When Socrates wakes on his own, he is not surprised to see Crito. He tells him about a dream he had: a beautiful woman in white told him he would arrive at fertile Phthia on the third day. This is a quote from Homer, where Achilles says his ships will reach home in three days. Socrates takes it to mean his soul will reach its home in three days, which means the ship has not arrived yet and will not arrive today.

Crito confirms the ship is expected today. Socrates says it will be tomorrow. Either way, time is very short.

## Crito's Arguments for Escaping

Crito makes several arguments. Lets go through each one.

**His own reputation.** If Socrates dies when he could have been saved, people will think Crito valued money more than his friend. He will be blamed for not doing enough.

Socrates dismisses this immediately. Why should we care what the majority thinks? The majority cannot make a man wise or foolish. They inflict things haphazardly. What matters is what reasonable, informed people think, not the crowd.

**The money is there.** Crito says not to worry about cost. He has the money, and so do friends like Simmias and Cebes. There are people in other cities like Thessaly who would welcome Socrates and keep him safe.

Socrates does not directly respond to this but it becomes irrelevant once he establishes the core argument.

**Socrates is abandoning his children.** This is Crito's most emotionally charged point. By accepting death, Socrates is leaving his sons, one a teenager and two younger, effectively as orphans. He had a choice. He could have proposed exile as his penalty at the trial. He did not. Now he is choosing death when escape is possible. That is not courage, Crito says. It is the easy path. A good and courageous man would choose to stay alive for his children.

**It would be unjust to stay.** Crito also argues that it would actually be unjust for Socrates to accept the verdict. It hands a victory to his enemies. It abandons his friends. It means accepting a result that was itself unjust.

## Socrates' Framework

Before addressing any of this directly, Socrates lays down the principle he will reason from. He says he is not about to start listening to different things just because he is in danger of death. His principles are the same as they have always been.

The first principle: **you should only follow the opinions of those who know, not the opinions of the majority**. He walks Crito through an example. A man who is serious about physical training listens to his trainer, not to the crowd. The crowd's praise and blame do not determine what is actually good for his body. The expert's judgment does. The majority can sometimes put you to death, sure. But they cannot make something true or false, right or wrong. The one whose opinion matters is the one with genuine knowledge of the subject.

The second principle: **the most important thing is not life, but the good life**. And the good life, the beautiful life, and the just life are the same thing. Crito agrees with both of these.

This means the question is not "will Socrates survive?" but "will escaping be just or unjust?" Everything else, reputation, money, children, follows from that.

## The Voice of the Laws

Here is where the Crito gets genuinely original. Socrates says: imagine the Laws of Athens came to us right now and spoke. What would they say?

He then gives them a voice and lets them make the argument against escaping. This is one of the earliest examples of a social contract argument in philosophy.

The Laws say this: **You cannot destroy us just because a court made an error in your case.**

Their argument has several layers.

First, the Laws point out what they have done for Socrates. They governed the marriage of his parents, which is how he was born. They supervised his upbringing and education. They gave him everything he has. In that sense, he is their offspring. And you do not get to strike back at your parents just because they discipline you unfairly on one occasion.

Second, the Laws gave him a real choice. Every Athenian, when they reach adulthood, can look at how the city is run and decide to leave. No one is forced to stay. If you stay, you implicitly agree to live under the laws. Socrates stayed for 70 years. He barely left the city except for military service. He raised a family here. He chose this city more completely than almost any other Athenian. That choice is a binding agreement.

Third, the Laws point out that Socrates had a chance to propose exile as his penalty at the trial and did not. He chose to stay and fight. He even said in court he would rather die than stop doing philosophy. He cannot now, when the result has gone against him, decide exile is acceptable after all.

Fourth, the Laws ask what happens if he escapes. He goes to a nearby well-governed city like Thebes or Megara and arrives as a man who broke his agreement with Athens. Anyone who cares about the law will see him as a threat, someone who destroys the very thing that holds cities together. Or he goes to Thessaly, where Crito has friends, and lives out his days eating at banquets and telling people how he escaped in a disguise. What happens to his philosophical life then? And what happens to his children, who will either follow him as strangers in a foreign city or stay behind without him, with friends theoretically looking after them? The Laws point out that those same friends will look after the children whether Socrates goes to Thessaly or to the underworld, so his children are not actually a good reason to escape.

Fifth, and most sharply: if Socrates escapes, he has done a wrong in return for a wrong. He was wronged by men, not by the Laws. The Laws themselves did not fail him. A court of humans made an unjust decision using a just process. Responding to that by escaping means attacking the process itself, not the people who abused it.

## The Core Principle: No Retaliation

Throughout this whole argument, Socrates has established one underlying rule that he keeps returning to. **You should never do wrong in return for wrong done to you.**

This was not a commonly accepted view in ancient Athens. The traditional Greek idea of justice was helping your friends and harming your enemies. Crito would have been very comfortable with that framing. But Socrates rejects it completely. Doing harm is always wrong, regardless of what was done to you first. Retaliation does not become just because the original injury was real.

This is why the personal arguments about reputation, money, and children do not move him. All of those are arguments about consequences, about what will happen to various people. Socrates has already decided to reason from principle. If escaping is unjust, no consequence makes it right.

And then he adds something that shows he really means it. He says that the people who hold this view and the people who do not share no common ground. They inevitably despise each other's positions. He is not trying to win an argument about whether retaliation is generally a bad idea. He is identifying a genuine philosophical divide. If Crito agrees that you should never do wrong in return for wrong, then the rest of the argument follows. If he does not, they have nothing to talk about on this subject.

Crito agrees with the principle.

# What Is Piety? Reading Plato's Euthyphro

> In this post, we are going to understand the Euthyphro dialogue from the book Plato - The Complete Works

Source: https://blog.sheerluck.dev/posts/philosophy/plato/euthyphro/ · Markdown: https://blog.sheerluck.dev/posts/philosophy/plato/euthyphro/index.md

Plato wrote a bunch of dialogues. The Euthyphro is one of the shortest. In this post, we will understand the **Euthyphro** dialogue.

## The Setup

Athens, 399 BCE. Socrates is at the court of the king-archon because a man named Meletus has indicted him for impiety. Corrupting the youth and inventing new gods.

Outside the same court he runs into Euthyphro. Euthyphro is there to file murder charges against his own father. Here is the story. A laborer on the family estate killed a household slave in drunken anger. Euthyphro's father bound the laborer, threw him in a ditch, and sent someone to ask the priests what the correct ritual procedure was. The laborer died in the ditch before the messenger came back. Hunger, cold, and his bonds. Euthyphro believes his father is guilty of murder by negligence and that prosecuting him is the pious thing to do. His entire family thinks he has lost his mind.

In Athenian religion, murder caused **pollution** (miasma). If you knowingly kept company with a killer without ritual purification, the pollution spread to you. So Euthyphro is not just making a family decision. He is claiming he has expert, esoteric knowledge of piety, the kind that lets him navigate a case his whole family cannot agree on.

Socrates hears this and sees an opportunity. He is facing impiety charges himself. If Euthyphro genuinely knows what piety is, Socrates can learn from him and use that in court. So he asks. What is piety?

One thing to understand before we get into the answers. Socrates is not asking for examples of pious actions. He wants the **form**, the single thing present in every pious act and absent in every impious act. Something he can look at like a model and use to evaluate any action at all. This distinction between an instance and a definition is what every single answer in this dialogue misses.

## Definition 1: What I Am Doing Right Now

Euthyphro says piety is to do what he himself is doing right now. Prosecuting wrongdoers, be it about murder, temple robbery, or anything else, regardless of who they are. And he backs this up with a divine precedent. Zeus himself bound his own father Cronus because Cronus unjustly swallowed his sons. So prosecuting one's father is not crazy. It is what the gods do.

Socrates does not even argue about the Zeus example. He just points out the category error. That is one instance of a pious action, not a definition. There are many other pious actions. Giving one case does not tell us what makes any of them pious. What is the **form** that makes all pious actions pious?

Euthyphro agrees there are many pious actions and tries again.

## Definition 2: What Is Dear to the Gods

Now we get an actual definition. Piety is what is dear to the gods. Impiety is what is not.

Socrates says splendid, this is the kind of answer I wanted. Now let us examine whether it is true.

He reminds Euthyphro of something Euthyphro already agreed to earlier in the conversation. The gods are in a state of discord. They are at odds with each other. They have enmities and even wars, exactly like the stories the poets tell and the ones embroidered on the robe of Athena carried up to the Acropolis. Euthyphro confirmed all of this is true.

So here is the problem Socrates raises. What kinds of things cause disagreement that cannot be settled? Not numbers. If you and I disagree about which number is greater, we just count and resolve it. Not size. We measure. Not weight. We weigh. These disputes get resolved because there are procedures for settling them.

But what about the just and the unjust, the beautiful and the ugly, the good and the bad? Those are the things people genuinely fight over and cannot settle, because there is no neutral procedure. And that is exactly what the gods must be disagreeing about too, since they are at odds with each other.

So different gods consider different things just, beautiful, good. And each god loves what it considers good and hates the opposite. Which means the same action can be loved by some gods and hated by others. Which means under Definition 2, the same action would be both pious and impious at the same time.

Socrates puts it directly. Euthyphro's own action, prosecuting his father, might be pleasing to Zeus but displeasing to Cronus and Uranus, pleasing to Hephaestus but displeasing to Hera. The definition gives you contradictory answers for the same case.

Euthyphro tries to escape. He says surely on this particular subject, whether unjust killing must be punished, no gods would disagree with each other.

Socrates closes that exit. Think about how humans disagree. Nobody actually disputes the principle that wrongdoers should be punished. What they dispute is the specifics. Did this person actually do wrong? What exactly happened? When? The dispute is always about particular cases, not abstract principles. The gods would be the same. They might agree in the abstract that unjust killers should be punished, but they can still disagree about whether this specific killing was unjust.

And even if we grant that all gods agree on Euthyphro's case, that does not save the definition. Because the definition was "what the gods love is pious." Even if all gods agree here, we already showed that the same thing can be loved and hated by different gods in other cases. The definition still produces contradictions.

## Definition 2 Revised: What All the Gods Love

Socrates himself proposes the fix. What if we say: piety is what **all** the gods love, impiety is what all the gods hate, and whatever splits divine opinion is neither or both?

Euthyphro agrees. Yes, that is what he would say.

Socrates says good, lets examine it. And then he asks the most famous question in the dialogue.

> Is the pious being loved by the gods because it is pious, or is it pious because it is being loved by the gods?

This is the **Euthyphro Dilemma**. Two options.

**Option A.** Piety is prior. Things are pious independently, and the gods love them because they are pious. The gods are tracking something that already has value.

**Option B.** Divine love is prior. Things are pious because the gods love them. There is no independent standard. The gods' approval is what makes something pious.

Euthyphro does not understand the question at first, so Socrates works through it carefully using simpler examples. Something carried is a "carried thing" because it is being carried, not the other way around. Something seen is a "thing seen" because it is being seen. Something led is a "thing led" because it is being led. The state is always downstream of the act. A thing does not get carried because it is "a carried thing." It becomes "a carried thing" because something carries it.

The same goes for love. A thing is "something loved" because it is being loved by someone. That status comes from the love, not the other way around.

Now. Euthyphro already agreed that the pious is being loved because it is pious. That means piety is the prior thing, the reason the love happens. But "god-loved" works in the opposite direction, it gets its character from the loving. So the two are pointing in opposite directions and cannot be the same thing.

Conclusion: "god-loved" is a property pious things have as a result of being pious. It is not what piety is. Socrates says it plainly. You did not tell me what the pious is. You told me an **affect** of it, a quality it happens to have. The pious is god-loved, sure. But that tells me nothing about what piety actually is in itself.

At this point Euthyphro basically admits defeat on this one. He says whatever proposition they put forward goes around and refuses to stay put. Socrates jokes that his definitions behave like the statues of Daedalus, the legendary craftsman who made statues that could walk. They move and will not stay in place. Euthyphro says no, it is Socrates who is the Daedalus here, making everything move. Socrates says fair enough, but he would rather have stable answers and is clever without wanting to be. Lets keep going.

## Definition 3: Piety Is the Part of Justice Concerned with the Care of the Gods

Socrates pushes forward and gets Euthyphro to agree that everything pious is also just. But is everything just also pious? Not necessarily. Piety might be a **part** of justice, not all of it.

He explains this with an example about fear and shame. A poet wrote: "where there is fear there is also shame." Socrates disagrees. People fear disease and poverty but they are not ashamed of those things. So fear does not always come with shame. But the reverse is true. Where there is shame, there is always fear, because anyone who feels shame is afraid of having a reputation for wickedness. So shame is a **part of** fear, not equal to it. Fear covers more ground. Wherever you have shame you have fear, but not the other way around.

Same structure applies to piety and justice. Wherever you have piety you have justice, but not wherever you have justice do you have piety. Piety is a part of justice.

So now the question is: which part? If the pious is a part of the just, what specific part?

Euthyphro answers. The part of justice concerned with the **care of the gods** is piety. The part concerned with the care of men is the rest of justice.

Socrates likes this. But he immediately asks: what do you mean by care? In every other domain, care aims at the benefit of the thing being cared for. Horse breeders care for horses and horses become better. Hunters care for dogs. Farmers care for cattle. Care improves the object. So does piety improve the gods? Do our pious actions make the gods better?

Euthyphro says absolutely not. That is not what he means.

## Definition 3a: Piety Is the Service Slaves Give Their Masters

Euthyphro adjusts. The kind of care he means is not improvement-care. It is **service**. The kind a slave gives a master.

Socrates accepts this framing and asks the next question. Every service aims at producing something. Service to doctors produces health. Service to shipbuilders produces ships. Service to housebuilders produces houses. Every service has a goal it is working toward. So what excellent goal are the gods working toward that our service helps them achieve?

Euthyphro says the gods achieve many fine things.

Socrates says yes, so do generals. But you can easily say the main thing generals are working toward is victory in war. Farmers achieve many things too, but the main point is producing food. What is the main point of what the gods are doing, that our service contributes to?

Euthyphro cannot answer. He says it is a considerable task to acquire any precise knowledge of these things.

## Definition 3b: Piety Is Knowing How to Pray and Sacrifice

Pushed further, Euthyphro gives something more concrete. If a man knows how to say and do what is **pleasing to the gods** at prayer and sacrifice, those are pious actions, the kind that preserve private households and public affairs of state. Their opposites are impious and destroy everything.

Socrates compresses this. So piety is the knowledge of how to sacrifice and pray? To sacrifice is to make a gift to the gods. To pray is to beg from the gods. So piety is the knowledge of how to give to and beg from the gods correctly.

Euthyphro agrees.

## Definition 3c: Piety Is a Trading Skill Between Gods and Men

Socrates follows the logic one more step. If piety is knowing how to correctly give to and receive from the gods, it sounds like a kind of **trading skill**, an art of exchange between gods and men.

Euthyphro reluctantly agrees. Trading, yes, if you want to call it that.

Then Socrates asks the trap question. What do the gods actually get from us in this exchange? We receive everything good from the gods. What do they receive from us? Euthyphro says we give them honor, reverence, and what pleases them.

Socrates closes it. So the pious is what pleases the gods.

But that is the same as "what is dear to the gods."

We are right back at Definition 2. After the entire detour through care, service, prayer, sacrifice, and exchange, the argument has made a full circle and arrived exactly where it started. At a definition that was already shown to fail.

## The End

Euthyphro says he is in a hurry and has to go.

This is intentional. The dialogue is **aporetic**, from the Greek _aporia_, without a path forward. Euthyphro entered the conversation certain enough about piety to prosecute his own father on that certainty. He leaves unable to say what that certainty is actually based on.

Socrates is not satisfied. He says Euthyphro has cast him down from a great hope. He wanted to learn the nature of piety and use it to defend himself against Meletus. Now he has nothing. The dialogue failed to produce what it was looking for.

This is the first in a series reading through Plato's complete works. Next: the Apology

# Learn Go Basics By Building a Brainfuck Interpreter

> In this post, we are going to build a brainfuck language interpreter in Go

Source: https://blog.sheerluck.dev/posts/programming-languages/go/learn-go-basics-by-building-a-brainfuck-intepreter/ · Markdown: https://blog.sheerluck.dev/posts/programming-languages/go/learn-go-basics-by-building-a-brainfuck-intepreter/index.md

Coming soon

# Rust Control Flow in Practice - Build a Number Guessing Game

> In this article, we are going to learn about Rust control flow by building a number guessing game

Source: https://blog.sheerluck.dev/posts/programming-languages/rust/understanding-rust-control-flow-by-building-a-number-guessing-game/ · Markdown: https://blog.sheerluck.dev/posts/programming-languages/rust/understanding-rust-control-flow-by-building-a-number-guessing-game/index.md

Last time, we built a scientific calculator and used it to get comfortable with variables and types. Now we are going to build a number guessing game and learn how our program can repeat itself like the retry mechanism in games you have played in your life and a way to take decisions based on certain conditions.

<div class="aspect-video w-full overflow-hidden rounded-xl border border-border bg-black">
  <iframe
    class="h-full w-full"
    src="https://www.youtube.com/embed/ZietDOQYn5Y?si=qz4hozxxmADjfxek"
    title="Rust Control Flow in Practice - Build a Number Guessing Game"
    loading="lazy"
    allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
    referrerPolicy="strict-origin-when-cross-origin"
    allowFullScreen
  ></iframe>
</div>

In this game, our program will decide a number and then we have to guess that number in multiple retries. Once we guess a number, the program will reply if our guess is correct or our guessed number is higher or lower than the actual number and we will retry again.

We will use `if` to compare numbers, a loop to keep the game running, `match` to deal with input, and `break` or `continue` when we need to change what the loop does next.

## Control flow
Lets start with understanding about control flow. A program with no branches runs one statement after another. In our first article, our program had no branches, it was running or executing in a sequential manner. To give you an example

```rust
fn main() {
let a = 10;
let b = 20;
let result = a + b;

println!("{result}");
}
```

But for our guessing game, we need a way to write program so that it can take actions depending on certain conditions and also can have the retry loop. To get these things, we use the control flow concept not only in Rust but in any programming language. So, lets start with the concept of conditions and then we will learn about loops later in this article.

## `if`
In Rust, the `if` syntax looks very similar to other programming languages but there are still some differences
```rust
fn main() {
let number = 10;

if number > 5 {
    println!("Number is greater than 5");
}
}
```
In this program, we are first initialising the variable `number` with a value of `10` and then we are putting a condition using `if`. The condition is if the condition `number > 5` is true or satisfied, then go ahead and print the "Number is greater than 5" string. Like other languages, if this condition is not satsified, then it won't execute whatever is inside that `if` block, so our program won't print anything and then exit out of the program.

## Conditions Must be Boolean
Unlike some other languages, Rust does not have truthy and falsy values. Let me show you what I mean by that:
```rust
fn main() {
let number = 10;

if number {
    println!("Number exists");
}
}
```
if you'll run this, you'll see the compiler will complain saying expected bool but got integer. Remember, what I said in the first article, the `if` condition should produce a `bool` and right now `number` is actually an `i32` integer, we need to put a condition so that Rust can evaluate that condition and then the condition will return a bool value either true or false. So, to fix this, you can do this:
```rust
fn main() {
let number = 10;

if number != 0 {
    println!("Number is not zero");
}
}
```
You need to remember this because in some other languages, this is very common to use the concept of truthy and falsy values. I would say, this is a good thing from Rust because requiring a `bool` prevents rules such as `0` being false or a non-empty string being true or a null reference being false and so on

## `else` and `else if`
As you can guess from the name, we will use `else` for the remaining cases
```rust
fn main() {
let number = 3;

if number > 5 {
    println!("Number is greater than 5");
} else {
    println!("Number is 5 or less");
}
}
```
Just like other languages, `else` block is optional but you gotta use `if` block if you ever want to use `else` block. In the `else` block, we don't need to give any conditions because we are basically saying for all the remaining cases, execute whatever is in the `else` block just like other languages.

Now, for the `else if` (yeah, you guessed it right), this is used when there are more cases
```rust
fn main() {
let number = 10;

if number > 10 {
    println!("Number is greater than 10");
} else if number == 10 {
    println!("Number is exactly 10");
} else {
    println!("Number is less than 10");
}
}
```
This is again very similar to other languages, we are having multiple condition checks, if one of the condition satisfies, then the rest of the conditions won't execute at all. Our game will use this same structure as well.

## `if` is an expression
One important thing to note that in Rust, `if` can produce a value
```rust
fn main() {
let number = 10;

let message = if number > 5 {
    "greater than 5"
} else {
    "5 or less"
};

println!("{message}");
}
```

## Loops
Now, lets understand loops in Rust. in Rust, there are three common loop forms: `loop`, `while` and `for`. Like other languages, loop is used to repeat a block for a certain amount of times. 

## `loop`
Let's start with `loop`. `loop` has not condition in its syntax, it repeats until execution leaves it through `break`, `return`, a panic or any diverging operation. Let me show you what I mean by that
```rust
fn main() {
loop {
    println!("Hello");
}
}
```
So, you can see this loop is an infinite loop, you have to stop it manually with `Ctrl + C`. A loop that can never finish has the never type `!` in Rust's type model. The never type represents an expression that does not produce a normal value because execution does not continue past it.

So, to stop this infinite loop system we can use `break`, `break` exits the nearest enclosing loop and execution resumes after that loop. Its different than `return`, `return` exits the entire function but `break` just gets out of the current block. 
```rust
fn main() {
let mut number = 1;

loop {
    println!("{number}");

    if number == 5 {
        break;
    }

    number += 1;
}
}
```
We are printing the number inside the loop but you can see there is a `if` condition that says if `number == 5` then break out of this loop

There's also `continue`, we use this when the current iteration should end but the loop itself should keep running
```rust
fn main() {
let mut number = 0;

loop {
    number += 1;

    if number == 3 {
        continue;
    }

    println!("{number}");

    if number == 5 {
        break;
    }
}
}
```
Again, we are incrementing the `number` variable inside the loop, then there is a condition that says if the `number == 3`, then `continue` that means skip the rest of this specific iteration and transfer the control back to the loop boundary. 

So, this will print 1, 2, 4 and 5. It'll skip 3 because if you'll dry run this, first the number is 0, then we get into the loop and increment the number value to 1, then the if condition fails, so we skip that, then we print the number, so it prints 1, then there's another if check that fails, now it goes back to the loop again, in the same way it'll print 2, now it goes back to the top of the loop, and increments the value to 3, after this the if check satisfies and it says `continue`, this means skip the remaining lines for this iteration and go back to the top of the loop, then the value gets incremented to 4 and prints 4, then it prints 5 but in the end of this iteration, the final if check satisfies and `break` executes and we exit out of that loop.

One thing to note that both `break` and `continue` act on the nearest loop unless a loop label is supplied (i think we have something like this in python but not sure). Labels are useful when loops are nested:
```rust
fn main() {
'outer: loop {
    loop {
        break 'outer;
    }
}
}
```
The apostrophe begins the label; it is not a string. `break 'outer` exits the outer loop directly. The guessing game has only one loop, so it does not need a label. I would say this is a great addition because I have faced situations where inside the inner loop I want to have a condition and if that condition satisfies I want exit out of both the inner and outer loop, in that case this kind of feature really helps.

## `while`
Now lets look at another form of loop called `while`. `while` checks its condition before every iteration its pretty similar to other languages:
```rust
fn main() {
let mut number = 1;

while number <= 5 {
    println!("{number}");
    number += 1;
}
}
```
We have a `number` variable with value 1, then we are entering the loop and the condition check happens, if the condition satisfies then the code block will execute and if not we will get out of that loop. So this program will print numbers 1,2,3,4,5 then it'll break out as the condition won't be satisfied anymore.

In a way, `while` loop acts like a `loop` containing an `if` that calls `break` when the condition is `false`.

## `for`
The `for` loop takes values from an iterator
```rust
fn main() {
for number in 1..=5 {
    println!("{number}");
}
}
```
The range `1..=5` includes both endpoints. On every iteration, the next item is bound to `number` and the body runs once for that item. If you want to run the iteration from 1 to 5 (not inclusive), then you can just use `1..5` without the equals. More precisely, `for` accepts any value that implements `IntoIterator`, converts it into an iterator, repeatedly calls `next`, and stops when `next` returns `None`. The variable after `for` is technically a pattern, so it can destructure items as they are produced. Rust's standard `for` syntax also avoids the manual indexing and boundary checks commonly used to traverse collections in languages with C-style loops. We will cover iterators and patterns in depth later.

## `match`
`match` evaluates one value, called the scrutinee and compares it against a set of patterns
```rust
fn main() {
let number = 2;

match number {
    1 => println!("One"),
    2 => println!("Two"),
    3 => println!("Three"),
    _ => println!("Something else"),
}
}
```
Each of these lines or sections inside the `match` block is called an arm, each arm has a pattern on the left of `=>` and the expression for that pattern on the right. Rust tests the arms from top to bottom and runs the first matching arm. Since `number` is `2`, the second arm runs

The `_` pattern is a wildcard that handles every value which did not match an earlier arm. if you'll remove it, the compiler will complain because Rust requires every possible case to be covered before the program can compile. This is one of the things you should remember that when you're using `match`, you should cover all the possible values otherwise it won't compile.

Patterns can do more than compare literal values. They can bind names, destructure tuples and structs, select enum variants, ignore parts of a value, and include ranges. In this article we only need literal patterns, the wildcard, and the `Ok` and `Err` enum patterns.

Just like `if`, `match` is an expression and can produce a value
```rust
fn main() {
let number = 2;

let message = match number {
    1 => "One",
    2 => "Two",
    3 => "Three",
    _ => "Something else",
};

println!("{message}");
}
```
You can see that the value of `number` is 2, so the respective arm would match and the `message` will point to `"Two"`, so when we will print `message`, its gonna print `"Two"`

One thing you need to note that when using `match` as an expression make sure all arms must produce compatible type

`match` may remind you of `switch`, but exhaustiveness, expression values, and destructuring make it closer to algebraic pattern matching in languages that support tagged unions. We will use it here to handle the two possible results of parsing input.

## `Result`
In the previous article, we saw that when we get an input from user though command line, we get that as a string and sometimes we need to convert their type accordingly. For our guessing project, we need to convert the input to a number
```rust
fn main() {
let number = "42".parse::<u32>();

println!("{number:?}");
}
```
If you'll print this, the output will be `Ok(42)`. If the string is "hello", the output will look like `Err(ParseIntError {...})`. What is happening here is that parsing returns `Result<T, E>` because the operation may succeed or fail (we just saw both the cases from our example). `T` is the success type and `E` is the error type. A simplified version of its definition looks like this
```rust
enum Result<T, E> {
Ok(T),
Err(E),
}
```
Each `Result` value contains exactly one of those variants `Ok(value)` or `Err(error)`
`T` and `E` are generic parameters, so different operations can use the same `Result` structure with different success and error types. For `"42".parse::<u32>()`, the success type is `u32` and the error type is `ParseIntError`. The `::<u32>` syntax is commonly called the turbofish; it tells the generic `parse` method which target type we want when the surrounding code does not provide enough information.

We will cover enums and generics separately, but we can already handle the variants with `match`:

```rust
fn main() {
let result = "42".parse::<u32>();

match result {
    Ok(number) => println!("Parsed number: {number}"),
    Err(error) => println!("Failed to parse: {error}"),
}
}
```
The pattern `Ok(number)` both checks the variant and binds its contained value to `number`, while `Err(error)` does the same for the error. This is destructuring rather than a separate status check followed by a second call to obtain the value.

You may have used `unwrap()` in earlier Rust code
```rust
let number = "42".parse::<u32>().unwrap();
```
`unwrap()` returns the value inside `Ok`. Calling it on an `Err` causes a panic
```rust
fn main() {
let number = "hello".parse::<u32>().unwrap();

println!("{number}");
}
```
`unwrap()` is convenient when failure is impossible by construction, during quick experiments, or when a panic is the intended response. User input is expected to contain mistakes, so a panic would turn normal input into a program crash. Our game will handle the parsing result with `match` and continue running.

## Building the Number Guessing Game
Ok then, enough learning, now lets start building our project

### Create the project
Lets first create a cargo project and open this in your preferred IDE
```bash
cargo new number-guessing-game
```

Now, you know the drill, open `src/main.rs` and remove the existing hello world code

### Add Random Number Generation
The standard library does not include the random number function needed here, so add the `rand` crate to `Cargo.toml`. Cargo reads this manifest, resolves a compatible crate version, downloads its source, and records the exact resolved dependency graph in `Cargo.lock`
```toml
[package]
name = "number-guessing-game"
version = "0.1.0"
edition = "2024"

[dependencies]
rand = "0.10"
```

Ok now that we have the `rand` crate, lets start by generating a number from the inclusive range `1..=100`
```rust
fn main() {
let secret_number = rand::random_range(1..=100);

println!("Secret number: {secret_number}");
}
```
`random_range` is generic over the range's value type. In this isolated example, the unsuffixed integer literals default to `i32`, so `secret_number` is inferred as an `i32`. In the final program, comparing it with the parsed `u32` gives the compiler enough context to infer `u32` instead. We could make the type explicit at any stage with `let secret_number: u32 = ...`.

The `..=` syntax constructs an inclusive range, so both `1` and `100` are possible results. By comparison, `1..100` would exclude `100`. Run the program several times and confirm that the value changes. The `println!` is temporary and will be removed after this check.

### Read input
Now that we know how to generate a random number that the user needs to guess, lets now take the guessed number from the user
```rust
use std::io;

fn main() {
let secret_number = rand::random_range(1..=100);

println!("I'm thinking of a number between 1 and 100.");
println!("Enter your guess:");

let mut input = String::new();
io::stdin().read_line(&mut input).unwrap();

println!("You entered: {input}");
println!("Secret number: {secret_number}");
}
```
`String::new()` creates an owned, growable UTF-8 string. `read_line` appends the bytes read from standard input to that string, which is why it needs `&mut input` rather than an immutable reference. Passing a mutable reference allows `read_line` to change the string without taking ownership of it, so `main` can still use `input` afterward.

The method returns `io::Result<usize>`. Its success value is the number of bytes read, while its error value describes an input/output failure. This draft uses `unwrap()` for that lower-level failure so the article can stay focused on control flow and ordinary invalid guesses. A production program could handle this `Result` as well.

### Parse The Guess
Ok so now that we have received the value that the user has guessed, its time to parse it and convert it to a number

Currently, `input` is a `String` and `secret_number` is a number, so we can't compare these two values directly, we first need to convert the guessed number as a `u32` (we could go for i32 as well but as our secret number will always be positive, `u32` is good to use)
```rust
let guess = input.trim().parse::<u32>();
```
`read_line` normally keeps the newline created when the user presses Enter. `trim()` returns a string slice with leading and trailing whitespace removed, then `parse::<u32>()` attempts to build a number from that slice. Use `match` to handle the returned `Result`:
```rust
```rust
use std::io;

fn main() {
    let secret_number = rand::random_range(1..=100);

    println!("I'm thinking of a number between 1 and 100.");
    println!("Enter your guess:");

    let mut input = String::new();
    io::stdin().read_line(&mut input).unwrap();

    let guess = match input.trim().parse::<u32>() {
        Ok(number) => number,
        Err(error) => {
            println!("Invalid input: {error}");
            return;
        }
    };

    println!("Your guess: {guess}");
    println!("Secret number: {secret_number}");
}
```
The complete `match` expression becomes the value assigned to `guess`. The `Ok` arm produces a `u32`, while the `Err` arm prints the error and executes `return`. Although these arms appear to have different types, `return` has the never type `!` because it does not produce a value in the current function. A diverging expression can fit wherever another type is required, so the type of the complete match is `u32`.

This first version uses `return`, which exits `main` and therefore ends the program. The next version needs a different control-flow target because invalid text should end only the current attempt.

### Add Retries
Now that we have parsed the guessed number from the user correctly, its time to add the retry mechanism so that user can keep trying until they get it right
```rust
use std::io;

fn main() {
    let secret_number = rand::random_range(1..=100);

    println!("I'm thinking of a number between 1 and 100.");

    loop {
        println!("Enter your guess:");

        let mut input = String::new();
        io::stdin().read_line(&mut input).unwrap();

        let guess = match input.trim().parse::<u32>() {
            Ok(number) => number,
            Err(error) => {
                println!("Invalid input: {error}");
                continue;
            }
        };

        println!("Your guess: {guess}");
    }
}
```
Entering `hello` now prints an error and returns to the prompt. Like `return`, `continue` is a diverging expression with type `!`, so the `Err` arm does not need to produce the `u32` expected from the `Ok` arm. It transfers control to the next iteration before `guess` is created. A valid number reaches the final `println!`.

### Compare the Guess
Now, lets compare user's guessed number and the secret number, and we can tell them whether they are correct or their guess is too low or too high. Replace the last `println!` with the comparison
```rust
if guess < secret_number {
    println!("Too low!");
} else if guess > secret_number {
    println!("Too high!");
} else {
    println!("You got it!");
}
```
The conditions are mutually exclusive. For any two ordered integers, the guess must be less than, greater than, or equal to the secret number. The final `else` therefore represents equality without repeating the comparison.

The program can now give the correct hint, but the loop continues after a correct guess. Add `break` to the final branch. This plain `break` makes the complete loop evaluate to `()`, and execution resumes after the loop. Since the loop is the final operation in `main`, the function then ends

### Complete Program
Here's the complete code for your reference
```rust
use std::io;

fn main() {
    let secret_number = rand::random_range(1..=100);

    println!("I'm thinking of a number between 1 and 100.");

    loop {
        println!("Enter your guess:");

        let mut input = String::new();
        io::stdin().read_line(&mut input).unwrap();

        let guess = match input.trim().parse::<u32>() {
            Ok(number) => number,
            Err(error) => {
                println!("Invalid input: {error}");
                continue;
            }
        };

        if guess < secret_number {
            println!("Too low!");
        } else if guess > secret_number {
            println!("Too high!");
        } else {
            println!("You got it!");
            break;
        }
    }
}
```

If you'll run it now and play the game, you can have something like this:
```text
I'm thinking of a number between 1 and 100.
Enter your guess:
50
Too low!
Enter your guess:
75
Too high!
Enter your guess:
68
You got it!
```

## Exercises
From this article, I thought of giving you some exercise challenges that you can add and improve the program, this way you can gain some practice as well:

#### Count valid attempts

Add a mutable counter and increase it after each valid guess. Print the count when the player wins:

```text
You got it in 7 attempts!
```

Placing the increment after parsing means invalid text will not count as an attempt. Test the program with two invalid inputs followed by three valid guesses; the reported count should be `3`.

#### Validate the range

Parsing and validation are separate steps. `500` is a valid `u32`, so parsing succeeds even though the number is outside the game's range. Add an `if` condition after parsing and use `continue` when the guess is outside `1..=100`. Verify that `0`, `101`, and `500` are rejected while `1` and `100` reach the comparison.

#### Rewrite `while` using `loop`

Take the earlier `while number <= 5` example and express the same behavior with `loop`, `if`, and `break`. Both programs should print the numbers from `1` through `5`. This exercise checks the relationship between conditional loops and unconditional loops with explicit exits.

#### Return a value from a loop

Write a loop that counts upward and uses `break number;` when `number * number` is greater than `100`. Store the loop's value and print it. The result should be `11`.

Ok then, I think you learned some good stuff in this article, I admit that the project was simple but worry not, from 5th or 6th article, the projects are going to be increasingly challenging and interesting, till then we gotta learn the basics of Rust.

In the next article, we will cover functions and expressions and build a temperature converter (I know this is simple but simple projects are really good to learn the concepts faster without much cognitive load). I'll see you soon, till then have a great life!

# Build a Scientific Calculator in Rust - Understanding Variables and Types

> In this article, we are going to learn about Rust variables and types by building a simple scientific calculator.

Source: https://blog.sheerluck.dev/posts/programming-languages/rust/understanding-rust-variables-and-types-by-building-a-scientific-calculator/ · Markdown: https://blog.sheerluck.dev/posts/programming-languages/rust/understanding-rust-variables-and-types-by-building-a-scientific-calculator/index.md

Welcome to this learning Rust series where we will learn Rust by building one project per article and learn each and every concept in quite some depth. This is my new Rust series and it'll much more comprehensive than my previous Rust series. For the first 5-6 articles, the projects will be relatively simple but after that I assure you, we are going to build a ton of interesting projects and learn a lot of stuff. So, lets start with the first article

In this article, we are going to learn about Rust variables and types by building a simple scientific calculator.

<div class="aspect-video w-full overflow-hidden rounded-xl border border-border bg-black">
  <iframe
    class="h-full w-full"
    src="https://www.youtube.com/embed/g46VpaqoFa0"
    title="Build a Scientific Calculator in Rust - Understanding Variables and Types"
    loading="lazy"
    allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
    referrerPolicy="strict-origin-when-cross-origin"
    allowFullScreen
  ></iframe>
</div>

## Prerequisites
You should know any one programming language at least the basics of it like conditions, variables, loops, functions. *The most important thing, you should have will to learn and put effort*. This is going to be a long but tough journey so buckle up.

## Part 1 - Installing Rust
Just like any other language, we first need to install it before we can use it, so just go ahead and follow the Rust installation instruction and come back [Install Rust with rustup](https://www.rust-lang.org/tools/install?utm_source=chatgpt.com)

After installing Rust, open a new terminal and check that the compiler is available:
```bash
rustc --version
```

You should see a version similar to:
```text
rustc 1.97.0
```

The exact version will depend on when you install Rust.

Now check Cargo:
```bash
cargo --version
```

You may be thinking **what is Cargo?** Cargo is Rust's build system and package manager. We will use Cargo throughout this series.

You can also check rustup:
```bash
rustup --version
```

If all three commands work, your Rust installation is ready.

## Getting Started with Cargo

Before we start learning Rust, let's quickly understand how we will create, build, and run our Rust projects throughout this series.

### What is Cargo?

Cargo is Rust's build system and package manager. It handles tasks such as creating projects, compiling code, running programs, checking code, and managing dependencies.

You will be using Cargo throughout this series, so let's get familiar with the basic commands.

### Creating a Rust Project

Let's create our first Rust project.

Open your terminal, navigate to the folder where you want to keep your project, and run:

```bash
cargo new hello-rust
```

Cargo will create a new directory called `hello-rust` with a basic Rust project inside it.

Move into the project:

```bash
cd hello-rust
```

If you open the project in your editor, you will see a structure similar to this:

```text
hello-rust/
├── Cargo.toml
└── src/
└── main.rs
```

Let's understand what these files are.

`Cargo.toml` is the project's manifest file. It contains information about the project, such as its name, version, Rust edition, and dependencies.

`src/main.rs` is the entry point of a binary Rust application. This is where our `main` function will live.

Cargo also creates a `Cargo.lock` file after resolving dependencies during a build. You don't need to worry about it yet; we'll discuss dependency management later in the series.

### Running the Project

Open `src/main.rs`. Cargo generates a small Hello World program for us:

```rust
fn main() {
println!("Hello, world!");
}
```

We can run the program with:

```bash
cargo run
```

You should see:

```text
Hello, world!
```

`cargo run` first compiles the project if necessary and then runs the resulting executable.

### Checking Your Code

Another useful command is:

```bash
cargo check
```

`cargo check` checks whether your project compiles without producing the final executable.

It is useful while developing because it lets you quickly check whether your code is valid without doing a complete build.

You will see me use `cargo check` frequently throughout this series.

### Building Your Project

To compile the project, you can use:

```bash
cargo build
```

By default, this creates a debug build. When you eventually want an optimized release build, you can use:

```bash
cargo build --release
```

For now, you don't need to worry about the difference between debug and release builds. We'll come back to that when it becomes relevant.

### The Cargo Workflow

At this point, the basic workflow you need to remember is:

```text
cargo new       → create a new project
cargo check     → check that the code compiles
cargo run       → compile and run the program
cargo build     → compile the project
```

That's enough Cargo knowledge for now.

We will learn more about Cargo, dependencies, workspaces, builds, and project configuration later in the series when we actually need them.

Now that we know how to create and run a Rust project, let's start learning Rust itself.
## Variables
The very first thing that we will learn is **variables in Rust**. The syntax is very similar to other programming languages but still there are some differences that you should know and understand from the very beginning.
### Creating a Variable
Let's start by learning how to create a variable:
```rust
fn main() {
	let number = 10;

	println!("{number}")
}
```
You can run this by typing the following in your terminal:
```bash
cargo run
```
You should see the output `10`. The line `let number = 10;` creates a variable binding. Let me show you with the following image:

![image 1](/images/rust-02.png)

In Rust terminology, it is useful to think of `let` as creating a **binding** between a name and a value. Here, we are basically telling Rust that the name `number` refers to the value `10`. Don't worry about the types right now, we will learn about them shortly

Now you might be thinking that it seems very very similar to languages like JavaScript and Python, then what is the difference that I was talking about. Let me show you by explaining what are the pieces of information associated with binding:
![image 2](/images/rust-03.png)
For now, you just need to note that the binding has a name called `number` and it refers to a value `10`, the value has a type, the binding is immutable by default and the binding exists within a particular scope. We will comeback to these properties later in this section and the next one as well.
## Variables are immutable by default
Lets try and run this program:
```rust
fn main() {
let number = 10;

number = 20;

println!("{number}");
}
```

You will see an error saying we cannot assign to an immutable variable.
> You need to remember this: bindings created with `let` are immutable by default

If we want to change the value associated with the binding, we have to explicitly say that it is mutable and we do that by using the keyword `mut`:

```rust
fn main() {
let mut number = 10;

number = 20;

println!("{number}");
}
```

Now run the program using `cargo run` and you'll see the output `20`. 
> For now, that distinction is enough. Ownership and borrowing will make this distinction much more important later in the series.
### Reassignment
Let's try and run this program:
```rust
fn main() {
	let mut number = 10l;

	number = 20;
	number = 30;

	println!("{number}")
}
```
You'll get the output `30`. We first create the binding `number` with value `10`, then reassigned to `20` and then again reassigned to `30` and finally print `number`

### Shadowing
In Rust, we can declare another binding using the same name, like this:
```rust
fn main() {
	let number = 10;
	let number = 20;

	println!("{number}")
}
```
Now, run and check the output, you'll get the output `20`. At first, this may seem like reassignment but its not. This is what we call `shadowing`. There is a subtle difference between reassignment and shadowing, in shadowing you can see that the second `let` creates a new binding that has the same name as the previous binding. In reassignment, we don't create multiple binding with the same name.

Shadowing can also change the type as well:
```rust
fn main() {
	let value = 10;
	let value = "ten";

	println!("{value}")
}
```
If you'll run this, you can see the output will be `ten`. If you'll do this using reassignment, then it would fail and will show you an error:
```rust
fn main() {
	let mut value = 10;
	value = "ten";

	println!("{value}")
}
```
This shadowing concept will be used later in our series, so for now I'm leaving this concept here and we will pick it up in a later article to learn even more about it.

### Scope
These bindings also have a scope. Scope is the region of the program in which the binding can be used. Let me show you with an example:
```rust
fn main() {
	let outer = 10;

	{
		let inner = 20;
		println!("{outer}")
		println!("{inner}")
	}

	println!("{outer}")
}
```
I think you must have seen this kind of scope in other languages as well. The inner binding scope only exists inside the nested block and outer binding scope exists till the end of the program. So when you run the program, once our program gets out of the nested block, the `inner` binding is no longer available. That's why if you'll try to print `inner` binding value outside of that nested block, you'll get an error. Let me show you the scope of our above code with this visual:
![image 3](/images/rust-04.png)
We will later understand and connect the idea of scoping with ownership and lifetime, but for now, just understand that scope is a region in which a binding is available

### Where does the value actually live?
I know you may be wondering what actually happens when we write `let number = 10;` in a Rust program. I can give you a brief explanation now but we will dive deeper in a later article regarding this. 

So, for simple values such as integers, the compiler can generally represent the value directly in machine storage often in a register or stack memory depending on how the generated code is optimised and used. 
> Variables are not always live on the stack. Please take the above brief statement with a pinch of salt as I'm simplifying this a lot. As I said, we will learn about this in detail in a separate article, so it would be great if you'll be patient and for now just focus on learning the language level behaviour and not focus on machine level details.

### One Last Experiment
Let's just combine what we have learned with this program:
```rust
fn main() {
	let number = 10;

	{
		let mut number = number;

		number = 20;

		println!("{number}");
	}

	println!("{number}")
}
```
I would say try to guess the output first before running the program and checking the output. First, we created a outer binding `number` and the value is `10`, then inside the nested block, we created an inner binding with the same name, this will shadow the outer `number` binding and the value of this inner binding is `10`, then we are reassigning the value to be `20`(we can do this because of `mut` in inner binding), then printing `number` would give us `20`. Finally we exist out of nested block and prints `number` again but this time it'll point to the outer `number` binding, hence printing the value `10`.

I think, by now you should be comfortable with the basic variable model. Now, we can get into types

## Types
Like some other languages, we have types in Rust as well. Let's use the same example that we used first and look at the value itself:
```rust
let number = 10;
```
We know that the `number` refers to `10` but the question is *What kind of value if 10?* This is where we will focus for the rest of the section where we will understand all the common types that Rust provides.

In general, a type tells us what kind of value it is working with and what operations are valid for that value.

Let's start with a very basic example:
```rust
fn main() {
	let number = 10;

	println!("{number}")
}
```
Once you run it, you'll see the output `10`. We already know this, nothing surprising. Now, lets make its type explicit:
```rust
fn main() {
	let number: i32 = 10;

	println!("{number}")
}
```
This will again output `10` but in this program, we explicitly mentioned that type the value should have.

The syntax that you'll follow to explicitly mention the type of a value is this:
```rust
let name: Type = value;
```
So in our basic example, `number` is the binding name, `i32` is the type and `10` is the value

### What Does a Type Actually Determine?
A type determines things like what values can be represented, how operations on those values behave, which operations are valid, how values can interact with other values, how the compiler represents the value at runtime and may be something else but I think these are the most common things you should know
For example, integers support operations arithmetic operations:
```rust
fn main() {
	let a: i32 = 10;
	let b: i32 = 20;

	let result = a + b;

	println!("{result}");
}
```
This works because `i32` supports addition. But Rust does not allow arbitrary values of unrelated types to be combined. For example:
```rust
fn main() {
	let a: i32 = 10;
	let b: f64 = 20.5;

	let result = a + b;

	println!("result")
}
```
If you'll run this, this will fail because Rust will see two different types `i32` and `f64` and the compiler will complain saying both are not the same type so can't perform operation. We will later see how to convert a type to make this operation happen but for now just remember that performing operations on different types will fail in Rust.

### Rust's Scalar Types
In Rust, we have some primitive scalar types. The four main categories are:

| Scalar Types           |
| ---------------------- |
| Integers               |
| Floating-point numbers |
| Booleans               |
| Characters             |

The integer and floating-point categories contain multiple concrete types. Let's go over each of these types and understand a bit more about each of these types one by one

### Signed Integers
A signed integer can represent both positive and negative whole numbers. In Rust, we have several variants of signed integers, these variants represents the size of the integer in bits.

| Signed Integers |
| --------------- |
| `i8`            |
| `i16`           |
| `i32`           |
| `i64`           |
| `i128`          |
| `isize`         |

We have different variants of signed integers because programs sometimes need different numeric ranges. For example, `i8` has 8 bits and can represent `-128 through 127` while `i32` has 32 bits and can represent `-2,147,483,648 through 2,147,483,647` and `i64` can represent even more. I hope you get the idea. 

### Unsigned Integers
A unsigned integer (yeah, you guessed it right) can't represent negative values. For example:
```rust
fn main() {
	let count: u32 = 100;

	println!("{count}");
}
```
This will work perfectly fine but if you'll replace that `100` with any negative number like this:
```rust
fn main() {
	let count: u32 = -100;

	println!("{count}")
}
```
Now, if you'll try to run it, compiler will complain because the range of `u8` is `0 through 255` and that's why negative numbers are not valid.

### `usize` and `isize`
There are two integer types that I want you to give your special attention: `usize` and `isize`. Unlike `i32` or `u64`, their size depends on the target architecture. For example, on a 64-bit system: `usize` is 64 bits and `isize` is 64 bits but on a 32-bit system: `usize` is 32 bits and `isize` is 32 bits. `usize` is very commonly used when working with sizes and indexes. You'll frequently see ti later when we work with collections. We will see the usage of `usize` in a later article, for now I just want you to understand the gist of it and I hope you do.

### Floating Point Types
As we are building a mini scientific calculator, we obviously need decimal value support and to support that Rust provides two floating-point types: `f32` and `f64`
Let me show you a basic example for floating point as well:
```rust
fn main() {
	let a: f32 = 10.5;
	let b: f64 = 20.5;

	println!("{a}");
	println!("{b}");
}
```

### Boolean
Rust supports boolean type as well and its represented with `bool`. It has exactly two possible values: `true` and `false`. Let me give you a basic example:
```rust
fn main() {
	let is_running: bool = true;
	let is_finished: bool = false;

	println!("{is_running}");
	println!("{is_finished}");
}
```

### Character
Rust also supports character type and its represented by `char` keyword. A `char` represents a single Unicode scalar value. For example:
```rust
fn main() {
	let letter: char = 'R';
	let symbol: char = '😀';

	println!("{letter}");
	println!("{symbol}");
}
```
Remember that, we are using single quotes when we are defining a character. If you'll use double quotes, then it'll become a string literal. These two are different types.

So, from the above example, you can see that a Rust `char` can represent Unicode scalar values and this is the reason you should not think of `char` as "one byte". Rust's `char` is a four-byte type representing a Unicode scalar value.

### Type Inference
In the beginning, we were not writing types for our variables but recently we started explicitly mentioning the types of our variables. That means, Rust does not require us to annotate every variable, Rust will infer the type and this is called **type inference**. 
Type inference doesn't mean that a variable has not type. that variable still has a concrete compile-time type. Its just that Rust has simply determined that type for us. We can also verify this theory with an example:
```rust
fn main() {
	let number = 10;

	println!("{}", std::any::type_name_of_val(&number));
}
```
You should see the output as `i32`.

### Numeric Literal Suffixes
Rust allows us to attach a type suffix to a numeric literal. This is just another way of making type explicit.
```rust
fn main() {
	let a = 10_i32;
	let b = 10_i64;
	let c = 10.5_f64;
}
```
The suffix tells Rust what type the literal should have. So, `10_i64` means the integer literal `10` should have type `i64`. The first way that we learned uses type annotation on the binding and this one uses a type suffix on the literal.

### Rust Does Not Implicitly Convert Numeric Type
Let's go back to the example that we used earlier:
```rust
fn main() {
	let a: i32 = 10;
	let b: f64 = 20.5;

	let result = a + b;

	println!("{result}");
}
```
If you'll run this, this will fail because you are trying to do operation with two different types. To make it run successfully, we need to convert one of these two types and make both the types same:
```rust
fn main() {
	let a: i32 = 10;
	let b: f64 = 20.5;

	let result = a as f64 + b;

	println!("{result}");
}
```
With that `as f64`, we converted the integer `a` to `f64` and now the operation can happen. We will learn more about type conversion later in the series. 

## Building The Scientific Calculator
From now on, we are going to build our calculator incrementally. I hope you have installed Rust already, if not please go to the top of this article and get the Rust installation link and follow the instructions. Once done, come back and start working on this project.

### Create the project
Just open your terminal, go to your preferred folder where you want to create this project and then lets create a new Cargo project:
```bash
cargo new scientific-calculator
```
Then open this project in your preferred IDE. I'll be using [Zed IDE](https://zed.dev/)
Once you open the project, you'll see this folder structure:
```
scientific-calculator/
├── Cargo.toml
└── src/
└── main.rs
```
For this project, we don't need to modify `Cargo.toml` as we will only use Rust's standard library

Now, open `src/main.rs` and you'll see there is a Cargo generated hello world program, let's replace that and print `Scientific Calculator`:
```rust
fn main() {
	println!("Scientific Calculator");
}
```
Once you run it with `cargo run`, you should see `Scientific Calculator` on your terminal.

### Add the Four Basic Operations
Let's add all the four basic arithmetic operations:
```rust
fn main() {
let a: f64 = 20.0;
let b: f64 = 6.0;

let addition = a + b;
let subtraction = a - b;
let multiplication = a * b;
let division = a / b;

println!("Addition: {addition}");
println!("Subtraction: {subtraction}");
println!("Multiplication: {multiplication}");
println!("Division: {division}");
}
```
If you'll run this, you'll see that all the operations are working fine but formatting for division is missing. We will deal with that later.

### Add our First Scientific Operation
Lets add square root, power, trigonometric operations:
```rust
fn main() {
let a: f64 = 20.0;
let b: f64 = 6.0;

let addition = a + b;
let subtraction = a - b;
let multiplication = a * b;
let division = a / b;

let square_root = 25.0_f64.sqrt();
let power = 2.0_f64.powi(8);
let sine = 0.0_f64.sin();
let cosine = 0.0_f64.cos();

println!("Addition: {addition}");
println!("Subtraction: {subtraction}");
println!("Multiplication: {multiplication}");
println!("Division: {division}");
println!("Square root: {square_root}");
println!("Power: {power}");
println!("Sine: {sine}");
println!("Cosine: {cosine}");
}
```
Now, its looking a bit better but there is still one problem, the inputs are hardcoded, we need a way to take input from user. Let's work on that next.

### Reading Command Line Arguments
To read user inputs, we can read command line arguments and Rust provides `std::env::args` for accessing the arguments passed to a program. For now, lets not worry about parsing numbers or performing calculations. Lets first look at what are we receiving from the arguments through command line:
Comment out the complete `main.rs` code and then write this code:
```rust
fn main() {
	let args = std::env::args();

println!("{:?}", args);
}
```
Now run:
```bash
cargo run -- add 10 20
```
You should see output representing the arguments passed to the program. However, `args` is an iterator, and printing it directly isn't particularly useful for our purposes. Let's collect the arguments into a vector:
> Don't worry if you don't understand things like iterator or vector, we will learn about these later. Here, we are just using them to get user input. The main goal of this article is for you to understand about variables and types and that's already done at this point. 

```rust
fn main() {
let args: Vec<String> = std::env::args().collect();

println!("{args:?}");
}
```

Run:
```bash
cargo run -- add 10 20
```

You should see something similar to:
```bash
["target/debug/scientific-calculator", "add", "10", "20"]
```
The exact first value will depend on your environment.

The important part is:
```bash
"add"
"10"
"20"
```
These are the arguments that we supplied.

### Why There's a `--`?
You may have noticed that we're running the program like this:
```bash
cargo run -- add 10 20
```
The `--` separates Cargo's arguments from our program's arguments. Everything before `--` is interpreted by Cargo. Everything after `--` is passed to our program.

So:
```bash
cargo run -- add 10 20
```

means:
```bash
Cargo
  |
  └── run the program
      |
      ├── "add"
      ├── "10"
      └── "20"
```

### Accessing Individual Arguments
We can access individual elements of the collected arguments using an index.

Change the program to:
```rust
fn main() {
let args: Vec<String> = std::env::args().collect();

println!("Operation: {}", args[1]);
println!("First value: {}", args[2]);
println!("Second value: {}", args[3]);
}
```

Run:
```bash
cargo run -- add 10 20
```

You should get:
```bash
Operation: add
First value: 10
Second value: 20
```

> Why do we start at `args[1]` instead of `args[0]`? Because the first argument, `args[0]`, is the program's executable name or path.

![image 5](/images/rust-05.png)

### Parsing the numbers
We are able to get the operation that user wants to perform and the values but the problem is the values are coming as string from command line, we need to convert them to numbers to perform any mathematical operations. To parse, we can do something like this:
```rust
fn main() {
let args: Vec<String> = std::env::args().collect();

let a = args[2].parse::<f64>().unwrap();
let b = args[3].parse::<f64>().unwrap();

let result = a + b;

println!("{result}");
}
```
Run it and you should see `30` as output. Now the remaining work we need to do is bring back those operations that we commented out previously and use conditions to choose which operation to perform.

### Choosing the Operation
Till now, we can get the operation that user wants to execute and the values in floating-point type. We are going to use conditionals in Rust, I'm not going to dive deep into conditions right now but we will use basic if-else block in this project so no need to worry.
```rust
fn main() {
let args: Vec<String> = std::env::args().collect();

let operation = &args[1];

if operation == "add" {
    let a = args[2].parse::<f64>().unwrap();
    let b = args[3].parse::<f64>().unwrap();

    println!("{}", a + b);
} else if operation == "subtract" {
    let a = args[2].parse::<f64>().unwrap();
    let b = args[3].parse::<f64>().unwrap();

    println!("{}", a - b);
} else if operation == "multiply" {
    let a = args[2].parse::<f64>().unwrap();
    let b = args[3].parse::<f64>().unwrap();

    println!("{}", a * b);
} else if operation == "divide" {
    let a = args[2].parse::<f64>().unwrap();
    let b = args[3].parse::<f64>().unwrap();

    println!("{}", a / b);
} else if operation == "sqrt" {
    let value = args[2].parse::<f64>().unwrap();

    println!("{}", value.sqrt());
} else if operation == "pow" {
 let base = args[2].parse::<f64>().unwrap();
 let exponent = args[3].parse::<i32>().unwrap();

 println!("{}", base.powi(exponent));
} else if operation == "sin" {
    let value = args[2].parse::<f64>().unwrap();

    println!("{}", value.sin());
} else if operation == "cos" {
    let value = args[2].parse::<f64>().unwrap();

    println!("{}", value.cos());
} else {
    println!("Unknown operation: {operation}");
}
}
```
Ok, this seems a lot of lines of code but we are only using a lot of if-else blocks, that's all. Its very similar to other languages, so I'm not explaining anything. But to summarise, we are receiving the operation that user wants to execute, then we use a bunch of if-else blocks to check which block satisfies the operation and then in that block, we are reading the values from command line and performing the operation and printing the result. 

> We are reading values inside every block so that we can simply decide which operation requires how many values for example square root requires only one value but addition requires two

Now test them:
```bash
cargo run -- sqrt 25
```

```
5
```

```bash
cargo run -- sin 0
```

```
0
```

```bash
cargo run -- cos 0
```

```
1
```

There you go, we just finished building our mini scientific calculator in Rust. If you want to experiment further, try adding a few more operations yourself. You could add `tan`, `log`, or anything else you think would be useful.

And if you're following along and build the calculator yourself or make any interesting modifications, feel free to tag me on X. I'd love to see what you build and will repost most of them as a small gesture of appreciation for following along with the series.

In the next article, we'll learn about **control flow in Rust** by building a **number guessing game**. See you in the next one.

# Learn Error Handling in Rust By Building a TOML Config Parser

> In this post, we are going to build a toml config parser in Rust and learn error handling in Rust using Result

Source: https://blog.sheerluck.dev/posts/rust/learn-error-hanlding-in-rust/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-error-hanlding-in-rust/index.md

In this post, we are going to learn about error handling in Rust. Once we cover all the concepts, we will build a **TOML config parser with schema validation**. Every error must be handled properly using `Result`. Let's start.

You can get the source code from **[here](https://github.com/MrSheerluck/toml-config-parser-in-rust)**

## Why `panic!` and `unwrap` Are Not Enough

In the previous articles, we used `panic!` and `.expect()` and `.unwrap()` all over the place. When something went wrong, we just crashed the program. Let me remind you:

In the JSON parser, when the lexer hit an unexpected character, we did this:

```
panic!("Unexpected character: {}", c as char);
```

And when the parser encountered a token it didn't expect:

```
panic!("Expected ':' in object");
```

This is fine for a learning exercise. But real software can't just crash when something goes wrong. A web server can't crash because one request had bad input, it needs to return an error to that client and keep serving others. A config parser can't crash because one field has the wrong type, it needs to tell the user exactly what's wrong and move on.

Here's the rule you should internalise:

> `panic!` is for **programmer bugs**: things that should never happen if your code is correct. Like indexing past the end of an array, or dividing by zero. These are "I messed up" situations.
>
> `Result` is for **expected failures**: things that can happen even when your code is perfect. Like bad user input, missing files, network timeouts. These are "something went wrong and I need to handle it" situations.

In this article, we are going to replace every `panic!` and `.unwrap()` with proper `Result` based error handling. By the end, our program will never crash unexpectedly. It will always produce a clear, specific error message telling you exactly what went wrong and where.

## `Option<T>` - Dealing with Absence

Before we get to `Result`, let's talk about its simpler cousin: `Option`.

How many times have you written code that returns `null` or `-1` or some magic value to signal "nothing found"? In Rust, we don't do that. Instead, we use `Option<T>`.

`Option<T>` is an enum with two variants:

```rust
enum Option<T> {
Some(T),  // there is a value, and it's of type T
None,     // there is no value
}
```

Instead of returning `null` or `-1`, a function returns `Option<T>`. If it has a value, it returns `Some(value)`. If not, it returns `None`.

### Creating Options

```rust
let some_number = Some(42);
let some_string = Some("hello");
let nothing: Option<i32> = None;
```

When you create a `Some` value, Rust can infer the type `T`. But when you create a `None`, Rust needs to know what type `T` would be if it were `Some`, so you often need a type annotation.

### Pattern Matching on Option

You can `match` on an `Option` just like any other enum:

```rust
let maybe_number = Some(42);

match maybe_number {
Some(n) => println!("got {}", n),
None => println!("got nothing"),
}
```

Or use `if let` when you only care about one variant:

```rust
if let Some(n) = maybe_number {
println!("got {}", n);
}
```

We actually used `Option` in previous articles. Remember the brainfuck interpreter? `Lexer::current()` returned `Option<u8>`. `Some(byte)` when there was a byte to read, `None` when we reached the end of input. The JSON parser's `Parser::current()` also returned `Option<&Token>`. These are natural uses of `Option`, sometimes there's a value, sometimes there isn't, and the type system forces you to handle both cases.

### Useful Methods on Option

**`.unwrap()`**: extract the value inside `Some`, or panic if `None`:

```rust
let x = Some(5);
println!("{}", x.unwrap()); // 5

let y: Option<i32> = None;
println!("{}", y.unwrap()); // PANICS!
```

We're trying to get away from `unwrap`, but you should still know what it does.

**`.unwrap_or(default)`**:  extract the value, or return a default if `None`:

```rust
let x: Option<i32> = None;
println!("{}", x.unwrap_or(0)); // 0

let y = Some(42);
println!("{}", y.unwrap_or(0)); // 42
```

**`.is_some()` and `.is_none()`**: check which variant you have:

```rust
let x = Some(5);
println!("{}", x.is_some()); // true
println!("{}", x.is_none()); // false
```

## `Result<T, E>` - Dealing with Failure

`Option<T>` handles absence. `Result<T, E>` handles failure. They look similar, but serve different purposes:

- `Option<T>` - "there might be a value, or there might be nothing" (like searching for something in a collection)
- `Result<T, E>` - "this operation might succeed with a value of type T, or it might fail with an error of type E" (like parsing a file, reading from disk, making a network request)

`Result<T, E>` is an enum with two variants:

```rust
enum Result<T, E> {
Ok(T),   // success, contains a value of type T
Err(E),  // failure, contains an error of type E
}
```

We actually already used `Result` in the grep article. `Regex::new(pattern)` returns `Result<Regex, Error>`. If the pattern is valid regex, you get `Ok(regex)`. If it's malformed, you get `Err(error)`. And we matched on it:

```rust
let regex = match Regex::new(pattern) {
Ok(r) => r,
Err(e) => {
    eprintln!("Invalid pattern: {}", e);
    std::process::exit(1);
}
};
```

Now we're going to go much deeper.

### Pattern Matching on Result

Just like `Option`, you can match on `Result`:

```rust
match result {
Ok(value) => println!("success: {}", value),
Err(error) => println!("failure: {}", error),
}
```

### Useful Methods on Result

**`.unwrap()`** - extract the `Ok` value, or panic on `Err`. Same warning as above: we're moving away from this.

**`.expect("message")`** - like `unwrap`, but panics with your custom message:

```rust
let bad: Result<i32, &str> = Err("error");
bad.expect("the number should be valid"); // PANICS with "the number should be valid: error"
```

**`.unwrap_or(default)`** - extract the `Ok` value, or return a default on `Err`:

```rust
let bad: Result<i32, &str> = Err("oops");
println!("{}", bad.unwrap_or(0)); // 0
```

**`.is_ok()` and `.is_err()`** - check which variant you have:

```rust
let result: Result<i32, &str> = Ok(42);
println!("{}", result.is_ok());  // true
println!("{}", result.is_err()); // false
```

## The `?` Operator

Here's a common pattern in Rust. You call a function that returns a `Result`. If it's `Ok`, you continue. If it's `Err`, you return the error to the caller. Written out with `match`, it looks like this:

```rust
fn do_something() -> Result<i32, MyError> {
let a = match step_one() {
    Ok(value) => value,
    Err(e) => return Err(e),
};

let b = match step_two(a) {
    Ok(value) => value,
    Err(e) => return Err(e),
};

let c = match step_three(b) {
    Ok(value) => value,
    Err(e) => return Err(e),
};

Ok(c)
}
```

This is called the "pyramid of doom" and it's exhausting to write and read. Rust has the `?` operator to fix this:

```rust
fn do_something() -> Result<i32, MyError> {
let a = step_one()?;
let b = step_two(a)?;
let c = step_three(b)?;
Ok(c)
}
```

The `?` operator does exactly what the `match` version does:
- If the `Result` is `Ok(value)`, it extracts the value and continues
- If the `Result` is `Err(error)`, it returns the error from the current function immediately

It also works on `Option`:
- If `Some(value)`, it extracts the value and continues
- If `None`, it returns `None` from the current function

> There's one catch: the function you're using `?` in must return a `Result` (or `Option`). You can't use `?` in `main` unless `main` returns a `Result`. We'll handle this by keeping our main logic in a separate `run` function that returns `Result`.

### `?` Also Converts Error Types

If your function returns `Result<T, ErrorA>` and a function you call returns `Result<T, ErrorB>`, you can't just use `?` directly because the error types don't match. But if `ErrorA` implements `From<ErrorB>`, then `?` will automatically convert `ErrorB` into `ErrorA` for you.

We'll use this in our project our custom `ConfigError` type will implement `From<std::io::Error>` (via `thiserror`) so that we can use `?` on file-reading operations that return `io::Error`.

## Defining Custom Error Types

Using `String` as your error type works, but it's not great. You can't programmatically inspect what went wrong. You can't match on different kinds of errors. You can't add structured data like line numbers and column numbers.

The solution is to define your own error type as an enum. Let me show you how.

### Creating the Error Enum

First, define the error enum with a variant for each kind of error:

```rust
enum ConfigError {
UnexpectedCharacter {
    line: usize,
    col: usize,
    expected: String,
    found: String,
},
UnterminatedString {
    line: usize,
    col: usize,
},
InvalidNumber {
    line: usize,
    col: usize,
    detail: String,
},
}
```

Each variant carries specific data about what went wrong and where. `UnexpectedCharacter` tells you the line, column, what was expected, and what was found. `UnterminatedString` tells you where the string started. `InvalidNumber` gives you the position and what was wrong with the number.

This is the structural approach. When you need to display an error message, you match on the variant and format it. When you need to handle specific errors differently, you match on the variant and take different actions.

### Using `thiserror` to Automate the Boilerplate

Writing error messages by hand is repetitive. Every time you add a variant, you have to update the match. The `thiserror` crate automates this with derive macros:

```rust
use thiserror::Error;

#[derive(Debug, thiserror::Error)]
enum ConfigError {
#[error("unexpected character at line {line}, column {col}: expected {expected}, found '{found}'")]
UnexpectedCharacter {
    line: usize,
    col: usize,
    expected: String,
    found: String,
},
#[error("unterminated string starting at line {line}, column {col}")]
UnterminatedString { line: usize, col: usize },
#[error("invalid number at line {line}, column {col}: {detail}")]
InvalidNumber {
    line: usize,
    col: usize,
    detail: String,
},
}
```

The `#[error("...")]` attribute on each variant replaces the manual message formatting. The `{field}` placeholders inside the string are replaced with the field values, just like `println!`. And the `#[derive(thiserror::Error)]` macro generates all the necessary trait implementations for us.

Under the hood, `thiserror` implements two traits for our `ConfigError`:

- **The Display trait** - this controls how the error appears when you format it with `{}` (like `format!("{}", err)` or `println!("{}", err)`). The `#[error("...")]` attribute becomes the Display implementation.
- **The Error trait** - this marks our type as an official error in Rust's error system. It's required for interoperability. We'll learn more about traits in a future article, but for now, just know that `#[derive(thiserror::Error)]` handles it for us.

The `Io(#[from] std::io::Error)` variant is special. The `#[from]` attribute tells thiserror to automatically implement `From<std::io::Error> for ConfigError`. This means when we call a function that returns `Result<_, std::io::Error>` (like `fs::read_to_string`), we can use `?` and the `io::Error` will be automatically converted into `ConfigError::Io`.

Notice how every error variant carries a `line` and (often) a `col`. This is deliberate. When you tell a user "invalid character", the first question they'll ask is "where?". By embedding position information in the error type itself, every error message can point to the exact location in the file.

Compare this with the JSON parser from the last article, where errors just said `panic!("Unexpected character")` with no location information. The difference is night and day.

Now, let's start working on our project.

## What is TOML?

TOML stands for **Tom's Obvious Minimal Language**. It's a configuration file format that's designed to be easy for humans to read and write. If you've ever used Rust, you've already seen TOML, every `Cargo.toml` file is a TOML file.

Here's what a TOML file looks like:

```toml
# This is a comment
host = "localhost"
port = 8080
debug = true

[server]
host = "localhost"
port = 8080
debug = true
```

The subset we'll support:
- **Key-value pairs**: `key = value`
- **Value types**: strings (`"hello"`), integers (`8080`), floats (`3.14`), booleans (`true`/`false`)
- **Tables**: `[section_name]` groups key-value pairs into sections
- **Comments**: `#` lines
- **Dotted keys in table headers**: `[server.host]` creates a nested section

We won't support arrays, inline tables, or array-of-tables. This keeps the scope focused on error handling, which is the point of this article.

## Project Setup

```
cargo new config_parser
cd config_parser
```

Open `Cargo.toml` and add `thiserror`:

```toml
[package]
name = "config_parser"
version = "0.1.0"
edition = "2024"

[dependencies]
thiserror = "2"
```

Open `src/main.rs` and delete everything. We'll write it from scratch.

## Step 1: The Error Types

Type this into `src/main.rs`:

```rust
use std::fs;

#[derive(Debug, thiserror::Error)]
enum ConfigError {
#[error("unexpected character at line {line}, column {col}: expected {expected}, found '{found}'")]
UnexpectedCharacter {
    line: usize,
    col: usize,
    expected: String,
    found: String,
},
#[error("unterminated string starting at line {line}, column {col}")]
UnterminatedString { line: usize, col: usize },
#[error("invalid number at line {line}, column {col}: {detail}")]
InvalidNumber {
    line: usize,
    col: usize,
    detail: String,
},
#[error("expected {expected} at line {line}, column {col}, found {found}")]
ExpectedToken {
    line: usize,
    col: usize,
    expected: String,
    found: String,
},
#[error("missing value for key '{key}' at line {line}, column {col}")]
MissingValue {
    line: usize,
    col: usize,
    key: String,
},
#[error("schema violation at line {line}: key '{key}' expected type {expected}, found {found}")]
SchemaViolation {
    line: usize,
    key: String,
    expected: String,
    found: String,
},
#[error("missing required key '{key}'")]
MissingRequiredKey { key: String },
#[error("key '{key}' has value {value} which is outside range {min} to {max}")]
ValueOutOfRange {
    key: String,
    value: String,
    min: i64,
    max: i64,
},
#[error("unknown key '{key}' at line {line}")]
UnknownKey { key: String, line: usize },
#[error("io error: {0}")]
Io(#[from] std::io::Error),
}
```

Let me explain the key parts:

- `#[derive(Debug, thiserror::Error)]` - this tells Rust to automatically implement the `Debug` trait (for `{:?}` formatting) and the `Error` trait. The `thiserror::Error` derive macro generates everything based on the `#[error("...")]` attributes.
- Each `#[error("...")]` attribute is the error message for that variant. The `{field}` placeholders are replaced with the field values, just like in `println!`.
- The `Io(#[from] std::io::Error)` variant is special. The `#[from]` attribute tells thiserror to automatically implement `From<std::io::Error> for ConfigError`. This means when we call a function that returns `Result<_, std::io::Error>` (like `fs::read_to_string`), we can use `?` and the `io::Error` will be automatically converted into `ConfigError::Io`.

Notice how every error variant carries a `line` and (often) a `col`. When you tell a user "invalid character", the first question they'll ask is "where?". By embedding position information in the error type itself, every error message can point to the exact location in the file.

## Step 2: The Token Type and Lexer

### SpannedToken

In the JSON parser, the `Token` enum only carried the token's kind. In this project, we need each token to know where it came from. So we split the token into two parts: the kind and the location.

```rust
#[derive(Debug, Clone, PartialEq)]
enum TokenKind {
Equal,
StringLit(String),
Integer(i64),
Float(f64),
Boolean(bool),
Comment(String),
Identifier(String),
TableHeader(String),
Newline,
}

#[derive(Debug, Clone)]
struct SpannedToken {
kind: TokenKind,
line: usize,
col: usize,
}
```

- `TokenKind` is the "what kind of token is this" enum. `Equal` is `=`, `StringLit("hello")` is a quoted string, `Integer(42)` is a number, `Boolean(true)` is a boolean, `TableHeader("server")` is `[server]`, `Newline` is a line break.
- `SpannedToken` bundles the kind with its position in the source file (`line` and `col`). When we encounter an error, we can pull the position from the token that caused it.

Compare this with the JSON parser's `Token` enum. The key difference: our `SpannedToken` knows where it came from. The JSON parser's tokens had no position information, which is why its error messages were just "unexpected character" with no location.

### The Lexer

```rust
struct Lexer {
input: Vec<u8>,
pos: usize,
line: usize,
col: usize,
}
```

This is very similar to the JSON parser's lexer, but with one crucial addition: `line` and `col` tracking. Every time `advance()` is called, we check if the current byte is a newline. If it is, we increment `line` and reset `col` to 1. Otherwise, we just increment `col`. This gives us exact position information for every byte in the input.

### The tokenize Method

```rust
fn tokenize(&mut self) -> Result<Vec<SpannedToken>, ConfigError> {
```

Notice the return type: `Result<Vec<SpannedToken>, ConfigError>`. In the JSON parser, `tokenize` returned `Vec<Token>` and just paniced on errors. Here, it returns a `Result`. If tokenization succeeds, you get `Ok(tokens)`. If it fails, you get `Err(ConfigError)` with a specific variant telling you exactly what went wrong and where.

The body of `tokenize` is similar to the JSON parser, but every error case returns `Err(ConfigError::...)` instead of `panic!()`. Here's the key difference: in the JSON parser, an unexpected character caused `panic!("Unexpected character: {}", c as char)`. In our project, it returns `Err(ConfigError::UnexpectedCharacter { line, col, expected, found })` with the exact position built in.

### Reading Strings

```rust
fn read_string(&mut self) -> Result<(String, usize, usize), ConfigError> {
let start_line = self.line;
let start_col = self.col;
self.advance();

let mut result = String::new();
loop {
    match self.current() {
        None => {
            return Err(ConfigError::UnterminatedString {
                line: start_line,
                col: start_col,
            });
        }
        Some(b'"') => {
            self.advance();
            return Ok((result, start_line, start_col));
        }
        Some(b'\\') => {
            self.advance();
            match self.current() {
                Some(b'"') => { result.push('"'); self.advance(); }
                Some(b'\\') => { result.push('\\'); self.advance(); }
                Some(b'n') => { result.push('\n'); self.advance(); }
                Some(b't') => { result.push('\t'); self.advance(); }
                Some(b'r') => { result.push('\r'); self.advance(); }
                _ => {
                    let found = match self.current() {
                        Some(b) => (b as char).to_string(),
                        None => String::from("end of input"),
                    };
                    return Err(ConfigError::UnexpectedCharacter {
                        line: self.line,
                        col: self.col,
                        expected: "valid escape sequence".to_string(),
                        found,
                    });
                }
            }
        }
        Some(c) => {
            result.push(c as char);
            self.advance();
        }
    }
}
}
```

In the JSON parser, unterminated strings caused `panic!("Unterminated string")`. Now they return `Err(ConfigError::UnterminatedString { line, col })` with the exact position where the string started. The error message will say "unterminated string starting at line 3, column 12" instead of just crashing.

### Reading Numbers

```rust
fn read_number(&mut self) -> Result<(TokenKind, usize, usize), ConfigError> {
let start_line = self.line;
let start_col = self.col;
let mut s = String::new();
let mut has_dot = false;

if let Some(b'-') = self.current() {
    s.push('-');
    self.advance();
}

loop {
    match self.current() {
        Some(c @ b'0'..=b'9') => {
            s.push(c as char);
            self.advance();
        }
        Some(b'.') => {
            if has_dot {
                return Err(ConfigError::InvalidNumber {
                    line: start_line,
                    col: start_col,
                    detail: "multiple decimal points".to_string(),
                });
            }
            has_dot = true;
            s.push('.');
            self.advance();
        }
        _ => break,
    }
}

if has_dot {
    let value: f64 = match s.parse() {
        Ok(v) => v,
        Err(_) => {
            return Err(ConfigError::InvalidNumber {
                line: start_line,
                col: start_col,
                detail: format!("cannot parse '{}' as float", s),
            });
        }
    };
    Ok((TokenKind::Float(value), start_line, start_col))
} else {
    let value: i64 = match s.parse() {
        Ok(v) => v,
        Err(_) => {
            return Err(ConfigError::InvalidNumber {
                line: start_line,
                col: start_col,
                detail: format!("cannot parse '{}' as integer", s),
            });
        }
    };
    Ok((TokenKind::Integer(value), start_line, start_col))
}
}
```

Two important things here. First, multiple decimal points like `3.14.159` now return `Err(ConfigError::InvalidNumber { ... detail: "multiple decimal points" })` instead of panicking.

Second, the `.parse()` calls use `match` instead of `.unwrap()`. When you call `"abc".parse::<i64>()`, it returns `Result<i64, ParseIntError>`. We can't use `?` here because `ParseIntError` is not our `ConfigError` type. Instead, we `match` on the result and construct our own error with the position information. This is a common pattern: when a library returns an error type that doesn't match yours, you match on their result and create your own error.

## Step 3: The TomlValue Type

```rust
#[derive(Debug, Clone, PartialEq)]
enum TomlValue {
String(String),
Integer(i64),
Float(f64),
Boolean(bool),
Table(Vec<(String, TomlValue)>),
}
```

This is similar to the `JsonValue` enum from the JSON parser, but simpler because TOML doesn't have arrays or null. The `Table` variant uses a `Vec<(String, TomlValue)>` just like the JSON parser's `Object`, each key-value pair is a tuple.

To display a `TomlValue`, we write a standalone function, just like we did for `JsonValue` in the previous article:

```rust
fn display(value: &TomlValue) -> String {
match value {
    TomlValue::String(s) => format!("\"{}\"", s),
    TomlValue::Integer(n) => format!("{}", n),
    TomlValue::Float(n) => format!("{}", n),
    TomlValue::Boolean(b) => format!("{}", b),
    TomlValue::Table(pairs) => {
        let mut result = String::from("{");
        let mut first = true;
        for (k, v) in pairs {
            if !first {
                result.push_str(", ");
            }
            result.push_str(&format!("{}: {}", k, display(v)));
            first = false;
        }
        result.push('}');
        result
    }
}
}
```

This is exactly the same pattern we used for the JSON parser, a recursive function that pattern matches on each variant and builds a string.

## Step 4: The Parser

The parser takes a `Vec<SpannedToken>` and turns it into a `TomlValue`. It uses the same cursor pattern as the lexer, a current position that advances as tokens are consumed.

```rust
struct Parser {
tokens: Vec<SpannedToken>,
pos: usize,
}
```

### The Main Parse Method

The parser handles three kinds of top-level items:

1. **Comments and newlines** - skip them
2. **Table headers** like `[server]` - collect all following key-value pairs into a sub-table
3. **Key-value pairs** - parse key, equal sign, value

Every place the JSON parser had `panic!()`, we have `Err(ConfigError::...)`. Even better, every error includes the line and column from the token. When a key is followed by something other than `=`, the error says "expected = at line 5, column 12, found Integer" instead of crashing with no context.

The `?` operator is used throughout: `self.parse_value()?` propagates errors automatically. If `parse_value` fails, the error bubbles up through `parse` and up to the caller effortlessly.

### Parse Value

```rust
fn parse_value(&mut self) -> Result<TomlValue, ConfigError> {
let token = self.current();
match token {
    Some(t) => match &t.kind {
        TokenKind::StringLit(s) => {
            let val = TomlValue::String(s.clone());
            self.advance();
            Ok(val)
        }
        TokenKind::Integer(n) => {
            let val = TomlValue::Integer(*n);
            self.advance();
            Ok(val)
        }
        TokenKind::Float(n) => {
            let val = TomlValue::Float(*n);
            self.advance();
            Ok(val)
        }
        TokenKind::Boolean(b) => {
            let val = TomlValue::Boolean(*b);
            self.advance();
            Ok(val)
        }
        _ => Err(ConfigError::ExpectedToken {
            line: t.line,
            col: t.col,
            expected: "value (string, integer, float, or boolean)".to_string(),
            found: format!("{:?}", t.kind),
        }),
    },
    None => Err(ConfigError::ExpectedToken {
        line: 0,
        col: 0,
        expected: "value".to_string(),
        found: "end of input".to_string(),
    }),
}
}
```

Clean and straightforward. If we have a token that's a valid value type, we return it. If not, we return a specific error saying what we expected and what we found instead.

## Step 5: The Schema

The schema defines what keys are expected in the config file, what types they should be, and whether they're required. The schema is itself a TOML file, parsed by our own parser.

```rust
enum FieldType {
String,
Integer,
Float,
Boolean,
}

struct FieldSchema {
field_type: FieldType,
required: bool,
default: Option<TomlValue>,
min: Option<i64>,
max: Option<i64>,
}

impl FieldType {
fn from_str(s: &str) -> Option<FieldType> {
    match s {
        "string" => Some(FieldType::String),
        "integer" => Some(FieldType::Integer),
        "float" => Some(FieldType::Float),
        "boolean" => Some(FieldType::Boolean),
        _ => None,
    }
}

fn name(&self) -> &'static str {
    match self {
        FieldType::String => "string",
        FieldType::Integer => "integer",
        FieldType::Float => "float",
        FieldType::Boolean => "boolean",
    }
}
}

#[derive(Debug, Clone)]
struct FieldSchema {
field_type: FieldType,
required: bool,
default: Option<TomlValue>,
min: Option<i64>,
max: Option<i64>,
}

type Schema = Vec<(String, FieldSchema)>;
```

`FieldType` - the expected type for a config key. 
`FieldSchema` - the full validation spec for a single key. 
`required` - if true, the key must be present (unless it has a default). 
`default` - a value to use if the key is missing. 
`min` and `max` - for integers, the valid range. 
`Schema` - a list of key names and their validation rules. 

`FieldType::from_str` converts a string like `"integer"` into a `FieldType` variant (or `None` if the string isn't a valid type name). `FieldType::name` converts a variant back into a string for error messages.

The schema TOML file looks like this:

```toml
[server.host]
type = "string"
required = true

[server.port]
type = "integer"
min = 1
max = 65535
required = true
```

When our parser processes this, `[server.host]` becomes a `TableHeader("server.host")`, and the key-value pairs inside it become entries in a sub-table. The `parse_schema` function walks this sub-table and extracts the validation rules:

```rust
fn parse_schema(value: &TomlValue) -> Result<Schema, ConfigError> {
let mut schema = Vec::new();
let table = match value {
    TomlValue::Table(pairs) => pairs,
    _ => {
        return Err(ConfigError::UnexpectedCharacter {
            line: 0,
            col: 0,
            expected: "table".to_string(),
            found: "non-table value".to_string(),
        });
    }
};

for (key, val) in table {
    let inner = match val {
        TomlValue::Table(pairs) => pairs,
        _ => continue,
    };

    let mut field_type = FieldType::String;
    let mut required = false;
    let mut default_val: Option<TomlValue> = None;
    let mut min_val: Option<i64> = None;
    let mut max_val: Option<i64> = None;

    for (field_name, field_val) in inner {
        match field_name.as_str() {
            "type" => {
                let type_str = match field_val {
                    TomlValue::String(s) => s.clone(),
                    _ => String::new(),
                };
                let ft = FieldType::from_str(&type_str);
                match ft {
                    Some(t) => field_type = t,
                    None => {
                        return Err(ConfigError::SchemaViolation {
                            line: 0,
                            key: key.clone(),
                            expected: "valid type (string, integer, float, boolean)".to_string(),
                            found: type_str,
                        });
                    }
                }
            }
            "required" => {
                required = match field_val {
                    TomlValue::Boolean(b) => *b,
                    _ => false,
                };
            }
            "default" => {
                default_val = Some(field_val.clone());
            }
            "min" => {
                min_val = match field_val {
                    TomlValue::Integer(n) => Some(*n),
                    _ => None,
                };
            }
            "max" => {
                max_val = match field_val {
                    TomlValue::Integer(n) => Some(*n),
                    _ => None,
                };
            }
            _ => {}
        }
    }

    schema.push((
        key.clone(),
        FieldSchema {
            field_type,
            required,
            default: default_val,
            min: min_val,
            max: max_val,
        },
    ));
}

Ok(schema)
}
```

This function takes the parsed schema `TomlValue` (which is a `Table` of sub-tables) and walks each entry. For each key like `server.port`, it reads the `type`, `required`, `default`, `min`, and `max` fields from the sub-table. If the `type` string isn't one of our known types, it returns a `SchemaViolation` error.

The `flatten_table` helper is also needed, it converts the nested `TomlValue::Table` into a flat list of dotted keys like `server.host`, `server.port`, etc.:

```rust
fn flatten_table(table: &TomlValue, prefix: &str) -> Vec<(String, TomlValue)> {
let pairs = match table {
    TomlValue::Table(pairs) => pairs,
    _ => return vec![],
};

let mut result = Vec::new();
for (key, value) in pairs {
    let full_key = if prefix.is_empty() {
        key.clone()
    } else {
        format!("{}.{}", prefix, key)
    };

    match value {
        TomlValue::Table(_) => {
            let nested = flatten_table(value, &full_key);
            for item in nested {
                result.push(item);
            }
        }
        _ => {
            result.push((full_key, value.clone()));
        }
    }
}
result
}
```

And a small `toml_type_name` helper that maps a `TomlValue` variant to a human-readable type name, used in schema violation error messages:

```rust
fn toml_type_name(value: &TomlValue) -> &'static str {
match value {
    TomlValue::String(_) => "string",
    TomlValue::Integer(_) => "integer",
    TomlValue::Float(_) => "float",
    TomlValue::Boolean(_) => "boolean",
    TomlValue::Table(_) => "table",
}
}
```

## Step 6: Schema Validation

The `validate` function takes a parsed config, a schema, and the original source text. It checks every key in the config against the schema and collects all errors.

```rust
fn validate(config: &TomlValue, schema: &Schema, source: &str) -> Result<(), Vec<ConfigError>> {
let mut errors = Vec::new();
let flat_config = flatten_table(config, "");

for (key, field) in schema {
    let mut found_value: Option<&TomlValue> = None;
    for (k, v) in &flat_config {
        if k == key {
            found_value = Some(v);
            break;
        }
    }

    match found_value {
        None => {
            if field.required {
                match &field.default {
                    Some(_) => {}
                    None => {
                        errors.push(ConfigError::MissingRequiredKey { key: key.clone() });
                    }
                }
            }
        }
        Some(value) => {
            let type_matches = match (&field.field_type, value) {
                (FieldType::String, TomlValue::String(_)) => true,
                (FieldType::Integer, TomlValue::Integer(_)) => true,
                (FieldType::Float, TomlValue::Float(_)) => true,
                (FieldType::Float, TomlValue::Integer(_)) => true,
                (FieldType::Boolean, TomlValue::Boolean(_)) => true,
                _ => false,
            };

            if !type_matches {
                let line = find_line_for_key(source, key);
                errors.push(ConfigError::SchemaViolation {
                    line,
                    key: key.clone(),
                    expected: field.field_type.name().to_string(),
                    found: toml_type_name(value).to_string(),
                });
            }

            if let (TomlValue::Integer(n), Some(min)) = (value, field.min) {
                if *n < min {
                    errors.push(ConfigError::ValueOutOfRange {
                        key: key.clone(),
                        value: n.to_string(),
                        min,
                        max: field.max.unwrap_or(0),
                    });
                }
            }

            if let (TomlValue::Integer(n), Some(max)) = (value, field.max) {
                if *n > max {
                    errors.push(ConfigError::ValueOutOfRange {
                        key: key.clone(),
                        value: n.to_string(),
                        min: field.min.unwrap_or(0),
                        max,
                    });
                }
            }
        }
    }
}

for (key, _) in &flat_config {
    let mut is_known = false;
    for (k, _) in schema {
        if k == key {
            is_known = true;
            break;
        }
    }
    if !is_known {
        let line = find_line_for_key(source, key);
        errors.push(ConfigError::UnknownKey {
            key: key.clone(),
            line,
        });
    }
}

if errors.is_empty() {
    Ok(())
} else {
    Err(errors)
}
}
```

`validate` returns `Result<(), Vec<ConfigError>>`, not `Result<(), ConfigError>`. It collects **all** errors, not just the first one. This is important for a config validator — you want to see everything that's wrong, not fix one error, re-run, fix another, re-run, etc.

The `flatten_table` helper converts the nested `TomlValue::Table` into a flat list of dotted keys like `server.host`, `server.port`, etc. This makes it easy to match against the schema keys.

The `find_line_for_key` helper searches the original TOML source text for a key and returns its line number. It strips the section prefix from dotted keys (turning `server.host` into just `host`) and scans for lines matching `key =`:

```rust
fn find_line_for_key(source: &str, target_key: &str) -> usize {
let short_key = if target_key.contains('.') {
    let mut parts = target_key.rsplit('.');
    match parts.next() {
        Some(k) => k,
        None => target_key,
    }
} else {
    target_key
};

for (i, line) in source.lines().enumerate() {
    let trimmed = line.trim();
    if trimmed.starts_with('#') || trimmed.starts_with('[') || trimmed.is_empty() {
        continue;
    }
    if let Some(eq_pos) = trimmed.find('=') {
        let key_part = trimmed[..eq_pos].trim();
        if key_part == short_key {
            return i + 1;
        }
    }
}
0
}
```

It skips comments, section headers, and empty lines, then looks for lines where the part before `=` matches the key. If it can't find the key, it returns `0` (which `format_error` treats as "no line information available").

## Step 7: Displaying Errors

```rust
fn format_error(err: &ConfigError, source: &str) -> String {
let line_num = match err {
    ConfigError::UnexpectedCharacter { line, .. } => *line,
    ConfigError::UnterminatedString { line, .. } => *line,
    ConfigError::InvalidNumber { line, .. } => *line,
    ConfigError::ExpectedToken { line, .. } => *line,
    ConfigError::MissingValue { line, .. } => *line,
    ConfigError::SchemaViolation { line, .. } => *line,
    ConfigError::UnknownKey { line, .. } => *line,
    _ => 0,
};

if line_num == 0 {
    return format!("Error: {}", err);
}

let lines: Vec<&str> = source.lines().collect();
let line_content = match lines.get(line_num - 1) {
    Some(l) => *l,
    None => "",
};

format!(
    "Error: {}\n   |\n{:>2} | {}\n   |",
    err,
    line_num,
    line_content
)
}
```

This function takes an error and the original source text, finds the offending line, and formats it like:

```
Error: schema violation at line 3: key 'server.host' expected type string, found integer
   |
 3 | host = 1234
   |
```

The `match err { ... }` extracts the line number from whichever error variant it is. All the variants that carry a `line` field get their line number extracted. Variants like `MissingRequiredKey` that don't have a specific line get `0`, in which case we just print the error message without a source line.

## Step 8: Main Function and Demo

```rust
fn run(config_path: &str, schema_path: &str) -> Result<(), Vec<ConfigError>> {
let config_source = match fs::read_to_string(config_path) {
    Ok(s) => s,
    Err(e) => return Err(vec![ConfigError::from(e)]),
};
let schema_source = match fs::read_to_string(schema_path) {
    Ok(s) => s,
    Err(e) => return Err(vec![ConfigError::from(e)]),
};

let mut config_lexer = Lexer::new(&config_source);
let config_tokens = match config_lexer.tokenize() {
    Ok(tokens) => tokens,
    Err(e) => return Err(vec![e]),
};
let mut config_parser = Parser::new(config_tokens);
let config_value = match config_parser.parse() {
    Ok(v) => v,
    Err(e) => return Err(vec![e]),
};

let mut schema_lexer = Lexer::new(&schema_source);
let schema_tokens = match schema_lexer.tokenize() {
    Ok(tokens) => tokens,
    Err(e) => return Err(vec![e]),
};
let mut schema_parser = Parser::new(schema_tokens);
let schema_value = match schema_parser.parse() {
    Ok(v) => v,
    Err(e) => return Err(vec![e]),
};

let schema = match parse_schema(&schema_value) {
    Ok(s) => s,
    Err(e) => return Err(vec![e]),
};

validate(&config_value, &schema, &config_source)?;

println!("Config is valid!");
println!();
println!("Parsed config:");
for (key, value) in flatten_table(&config_value, "") {
    println!("  {} = {}", key, display(&value));
}

Ok(())
}
```

This is the beauty of proper error handling. Look at `run()`: it's a pipeline of operations, each of which can fail. Each `match` on a `Result` handles the error case explicitly. For cases where we want to convert a single error into a `Vec` of errors, we wrap it with `vec![e]`.

Notice we use `match` here instead of `?`. That's because `run` returns `Result<(), Vec<ConfigError>>`, but most of the inner calls return `Result<_, ConfigError>` (a single error, not a Vec). The `?` operator only works when the error types match, so we explicitly match and wrap single errors into a Vec where needed.

The `ConfigError::from(e)` on the `fs::read_to_string` error lines works because we defined `Io(#[from] std::io::Error)` in our error enum. The `#[from]` attribute generated a `From<std::io::Error> for ConfigError` implementation, so `ConfigError::from(e)` converts an `io::Error` into our `ConfigError::Io` variant.

The `main` function reads the file paths from command-line arguments (or defaults to the example files), calls `run`, and handles any errors:

```rust
fn main() {
let args: Vec<String> = std::env::args().collect();

let (config_path, schema_path) = if args.len() >= 3 {
    (args[1].clone(), args[2].clone())
} else {
    (String::from("examples/valid_config.toml"), String::from("examples/schema.toml"))
};

match run(&config_path, &schema_path) {
    Ok(()) => {}
    Err(errors) => {
        let source = match fs::read_to_string(&config_path) {
            Ok(s) => s,
            Err(_) => String::new(),
        };
        for err in &errors {
            eprintln!("{}", format_error(err, &source));
            eprintln!();
        }
        std::process::exit(1);
    }
}
}
```

If `run` returns `Ok(())`, we're done. If it returns `Err(errors)`, we load the source text (so we can point to specific lines), format each error with `format_error`, print them to stderr, and exit with code 1.

### Running with a Valid Config

Create `examples/valid_config.toml`:

```toml
# Server configuration
[server]
host = "localhost"
port = 8080
debug = true

[database]
url = "postgres://localhost/mydb"
max_connections = 10
```

Run it:

```
cargo run -- examples/valid_config.toml examples/schema.toml
```

Output:

```bash
Config is valid!

Parsed config:
  server.host = "localhost"
  server.port = 8080
  server.debug = true
  database.url = "postgres://localhost/mydb"
  database.max_connections = 10
```

### Running with an Invalid Config

Create `examples/invalid_config.toml`:

```toml
# Config with multiple errors
[server]
host = 1234
port = 99999
debug = "yes"

[database]
url = "postgres://localhost/mydb"
max_connections = 200
```

Run it:

```
cargo run -- examples/invalid_config.toml examples/schema.toml
```

Output:

```
Error: schema violation at line 3: key 'server.host' expected type string, found integer
   |
 3 | host = 1234
   |

Error: key 'server.port' has value 99999 which is outside range 1 to 65535

Error: schema violation at line 5: key 'server.debug' expected type boolean, found string
   |
 5 | debug = "yes"
   |

Error: key 'database.max_connections' has value 200 which is outside range 1 to 100
```

All four errors are reported at once. No panicking, no crashing, no vague messages. Every error tells you exactly what's wrong and where.

## Conclusion

In this post, we replaced every `panic!` and `unwrap` with proper `Result` based error handling. We defined a custom `ConfigError` enum with rich error variants carrying line numbers and context. We used `thiserror` to avoid writing boilerplate code. We used the `?` operator to propagate errors effortlessly. And we built a validator that collects **all** errors and displays them with source lines, not just the first one.

In the next article, we'll learn about generics, traits, and lifetimes and build a **Generic LRU Cache**. See you soon.

# Learn Rust Generics and Traits By Building a Mini Blackjack Game

> In this post, we are going to build a mini blackjack game in Rust and learn generics and traits in Rust

Source: https://blog.sheerluck.dev/posts/rust/learn-generics-traits-in-rust-by-building-blackjack-card-game-engine/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-generics-traits-in-rust-by-building-blackjack-card-game-engine/index.md

In this post, we are going to learn about generics and traits by building a blackjack card engine in Rust. I'm really excited for this project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles.

The only prerequisite is that you have read the previous articles in this series, as I will assume you know ownership, borrowing, structs, enums, pattern matching and error handling.

In this post, we will first learn what generics are, what traits are, how they work together, and then we will build a generic card engine that can work with any card type. We will also use proper error handling throughout, since we already learned about `Result`, the `?` operator, and custom error types in the previous article.

You can get the source code from [here](https://github.com/MrSheerluck/blackjack-card-engine)

Let's start. I can't wait.
## Generics
Let's say you want to write a function that returns the larger of two values. Without generics, you would need to write a separate function for every type you want to support:
```rust
fn max_i32(a: i32, b: i32) -> i32 {
if a > b { a } else { b }
}
fn max_f64(a: f64, b: f64) -> f64 {
if a > b { a } else { b }
}
```
These two functions have identical logic. The only difference is the type. If you also wanted to support `i64`, `u32`, `strings`, or your own custom types, you would need to duplicate this code forever. This is not maintainable at all.
To solve this issue, we have **generics**. Generics let you write logic once and use it with any type. You introduce a type parameter, which acts as a placeholder for a concrete type.
```rust
fn identity<T>(value: T) -> T {
value
}
```
The `<T>` is the type parameter. It says "this function works for any type T". When you call `identity(5)`, Rust replaces `T` with `i32`. When you call `identity("hello")`, Rust replaces `T` with `&str`. The compiler figures this out automatically from the argument you pass.
Now let me explain what we just did. The `T` is just a name. You can use any identifier, but single capital letters like `T`, `U`, `K`, `V` are conventional. `T` stands for "type". The compiler generates separate machine code for each concrete type you use. This is called **monomorphization**. It means there is zero runtime cost compared to writing separate functions. The generated code is exactly as fast as if you have written `identity_i32` and `identity_str` by hand.
### Generics Structs
Generics are not limited to functions. Structs can be generic too.
```rust
struct Point<T> {
x: T,
y: T,
}
```
`Point<i32>` and `Point<f64>` are completely different types. The compiler treats them as separate. You cannot mix them.
```rust
let p1 = Point { x: 5, y: 10 }; // Point<i32>
let p2 = Point { x: 1.5, y: 2.5 }; // Point<f64>
```
If you try `let p = Point { x: 4, y: 3.14 };`, you will get a compile error because `x` and `y` must be the same type. If you want to be different, you need two type parameters, like this:
```rust
struct Point<T, U> {
x: T,
y: U,
}
```
Now `Point<i32, f64>` is valid. The convention is to use `T` and `U` when they are unrelated and `K` and `V` when they represent keys and values.
### The Limitation
Here is where generics hit a wall. If you try to write that `max` function using generics:
```rust
fn max<T>(a: T, b: T) -> T {
if a > b {
    a
} else {
    b
}
}
```
You will get a compile error. The compiler will tell you that `T` does not support `>` operator. The problem is that `T` could be anything. It could be a struct you defined that has no concept of ordering. It could be a vector. It could be a file handle. The compiler cannot assume `>` is valid.
This is where traits come in. Traits let you constrain generics. They let you say "this generic function only works with types that can do X".
## Traits
A trait defines a set of methods that a type must implement. It is a contract. If a type implements a trait, it is promising that is can do certain things.
Think of it like this. In real world, a "vehicle" is a concept that promises certain behaviour. Any vehicle can start, stop and turn. A car implements vehicle. A motorcycle implements vehicle. They do it differently but they both fulfill the contract.
In Rust, a trait looks like this:
```rust
trait Card {
fn value(&self) -> u8;
fn display(&self) -> String;
}
```
Any type that wants to be a `Card` must provide two methods: `value` and `display`. The trait only defines signatures, not implementations. The actual code lives in the `impl` block for each type.
### Implementing a Trait
```rust
struct PokerCard {
rank: Rank,
suit: Suit,
}
impl Card for PokerCard {
fn value(&self) -> u8 {
    // implementation
}
fn display(&self) -> String {
    // implementation
}
}
```
The syntax is `impl TRAIT for TYPE`. This is different from inherent methods which are just `impl Type`. The trait implementation attaches the behavior to the type.
Now let me explain what we just did. `impl Card for PokerCard` says "PokerCard fulfills the Card contract." Inside the block, we provide the actual code for `value` and `display`. If we forget either method, the compiler will reject our code because the trait contract is incomplete.
### Default Implementations
Traits can provide default behaviour. The implementor can use it or override it.
```rust
trait Card {
fn value(&self) -> u8;
fn is_face(&self) -> bool {
    self.value() >= 10
}
}
```
Here `is_face` has a default body. If you implement `Card` for `PokerCard` and only provide `value`, you get `is_face` for free. But you can override it if you want custom logic.
```rust
impl Card for PokerCard {
fn value(&self) -> u8 {
    match self.rank {
        Rank::Two => 2,
        Rank::Three => 3,
        // ...
        Rank::Ace => 11,
    }
}
fn is_face(&self) -> bool {
    // custom override
    matches!(self.rank, Rank::Jack | Rank::Queen | Rank::King)
}
}
```
### Trait Bounds
Remember the max function that would not compile? Traits fix it.
```rust
fn max<T: PartialOrd>(a: T, b: T) -> T {
if a > b { a } else { b }
}
```
`T: PartialOrd` is a trait bound. It means "T must implement the PartialOrd trait." `PartialOrd` is a standard library trait that provides ordering operations like `<`, `>`, `<=`, `>=`. The compiler checks this at the call site. If you try to pass a type that does not implement PartialOrd, you get a clean compile error.
### Multiple Trait Bounds
Sometimes one bound is not enough.
```rust
fn process<T: Card + Clone + Debug>(card: T) {
println!("{:?}", card.clone());
println!("Value: {}", card.value());
}
```
This means `T` must implement `Card` and `Clone` and `Debug`. When you have many bounds, the where clause is cleaner:
```rust
fn process<T>(card: T)
where
T: Card + Clone + Debug,
{
println!("{:?}", card.clone());
println!("Value: {}", card.value());
}
```
Both mean the same thing. Pick whichever reads better. I prefer `where` when there are more than two bounds because it keeps the function signature readable.
### Derive
Rust can auto implement some traits for you.
```rust
#[derive(Debug, Clone, Copy, PartialEq)]
struct PokerCard {
rank: Rank,
suit: Suit,
}
```
This generates `Debug`, `Clone`, `Copy`, and `PartialEq` implementations automatically. You cannot derive everything. `Display` and custom traits like `Card` must be written by hand because the compiler does not know how you want them formatted.
Here is a table of commonly derived traits:
| Trait        | What It Gives You                          |
| ------------ | ------------------------------------------ |
| `Debug`      | `{:?}` formatting for programmer output    |
| `Clone`      | `.clone()` method to duplicate             |
| `Copy`       | Bitwise copy on assignment instead of move |
| `PartialEq`  | `==` and `!=` operators                    |
| `Eq`         | Reflexive equality, used by HashMap        |
| `PartialOrd` | `<`, `>`, `<=`, `>=` operators             |
| `Ord`        | Total ordering, used by sorting            |
| `Hash`       | Used by HashMap                            |
| `Default`    | `Default::default()` constructor           |
### The Orphan Rule
You can implement a trait for a type if either the trait or the type is defined in your crate. You cannot implement a foreign trait for a foreign type.
For example, you cannot do this:
```rust
impl Display for Vec<u8> {
// ERROR: both Display and Vec are from std
}
```
This prevents conflicting implementations from different crates. If crate A and crate B both implemented `Display` for `Vec<u8>`, and you used both crates, which implementation would win? Rust avoids this ambiguity entirely.
If you need to add behavior to a foreign type, wrap it in your own struct. This is called the newtype pattern.
```rust
struct ByteVec(Vec<u8>);
impl Display for ByteVec {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
    write!(f, "{:?}", self.0)
}
}
```
`ByteVec` is your type, so you can implement any trait for it.
### Traits on Generic Structs
This is where everything comes together. You can write impl blocks for generic structs with trait bounds.
```rust
struct Deck<T> {
cards: Vec<T>,
}
impl<T: Card + Clone + Debug> Deck<T> {
fn shuffle(&mut self) { ... }
fn deal(&mut self) -> Result<T, GameError> { ... }
fn len(&self) -> usize { ... }
}
```
`impl<T: Card + Clone + Debug> Deck<T>` means these methods only exist for `Deck<T>` when T implements all three traits. Inside the block, you can call `card.value()` because `T: Card`. You can `card.clone()` because `T: Clone`. Without the bounds, `T` is an unknown blob and the compiler will reject any method call.
But here is something important. A generic `Deck<T>` should not know how to create `T`. Creating 52 poker cards is specific to poker cards. A `Deck<UnoCard>` would need different cards entirely. So we keep the generic methods (shuffle, deal, len) on `impl<T> Deck<T>`, but we put the constructor on a separate, non-generic impl block:
```rust
impl Deck<PokerCard> {
fn new() -> Self {
    let mut cards = Vec::new();
    for suit in [Suit::Hearts, Suit::Diamonds, Suit::Clubs, Suit::Spades] {
        for rank in [
            Rank::Two, Rank::Three, Rank::Four, Rank::Five,
            Rank::Six, Rank::Seven, Rank::Eight, Rank::Nine,
            Rank::Ten, Rank::Jack, Rank::Queen, Rank::King, Rank::Ace,
        ] {
            cards.push(PokerCard { rank, suit });
        }
    }
    Deck { cards }
}
}
```
`impl Deck<PokerCard>` means these methods only exist for `Deck<PokerCard>`, not for any other `Deck<T>`. This is how we keep the engine generic while still having a concrete constructor for our specific card type.
## What We Are Skipping
We are not covering lifetimes in this article. We will cover them properly in a later article. We are also not covering trait objects. That comes later when we talk about smart pointers. We are also not covering associated types or generic associated types. Those are advanced topics for future articles.
## The Project
Now that you understand generics and traits, let's build the blackjack engine. We will use proper error handling throughout, since we already learned about `Result`, custom error types, and the `?` operator in the previous article.
We are going to build a command line blackjack game. We are going to build a generic card engine. The deck and the hand will work with any type of card that implements the `Card` trait.
### Project Setup
Open your terminal and run:
```bash
cargo new blackjack-card-engine
cd blackjack-card-engine
```
Now open the `Cargo.toml` file and add the dependencies:
```toml
[package]
name = "blackjack-card-engine"
version = "0.1.0"
edition = "2024"
[dependencies]
rand = "0.8"
thiserror = "2"
```
We added `thiserror` because we will define a custom error type for our game. We learned about `thiserror` in the previous article.
Now open `src/main.rs` and delete the generated code. We will start fresh.
### Defining Error Types
Since we are using proper error handling, let's define our error type first.
```rust
use thiserror::Error;
#[derive(Error, Debug)]
enum GameError {
#[error("Deck is empty")]
EmptyDeck,
#[error("Invalid input: {0}")]
InvalidInput(String),
#[error("IO error: {0}")]
Io(#[from] std::io::Error),
}
```
Now, let me explain what we just did. `#[derive(Error)]` from `thiserror` automatically implements the `Error` and `Display` traits for us. `#[error("...")]` defines the display message for each variant. `#[from] std::io::Error` auto-implements `From<std::io::Error>` for our `GameError`, which means we can use the ? operator on `std::io` operations and they will automatically convert.
### Rank and Suit
```rust
#[derive(Debug, Clone, Copy, PartialEq)]
enum Rank {
Two, Three, Four, Five, Six, Seven,
Eight, Nine, Ten, Jack, Queen, King, Ace,
}
#[derive(Debug, Clone, Copy, PartialEq)]
enum Suit {
Hearts, Diamonds, Clubs, Spades,
}
```
We derive `Debug`, `Clone`, `Copy`, and `PartialEq`. These are simple value types, so deriving all four makes sense. `Copy` means they are bitwise copied when assigned, not moved. This is important because we will be passing them around a lot.
### The Card Trait
```rust
trait Card {
fn value(&self) -> u8;
fn display(&self) -> String;
}
```
Any type that implements `Card` must provide a numeric value and a display string.
### The PokerCard Struct
```rust
#[derive(Debug, Clone, Copy, PartialEq)]
struct PokerCard {
rank: Rank,
suit: Suit,
}
```
### Display Implementation
We cannot derive `Display`, so we implement it manually.
```rust
use std::fmt;
impl fmt::Display for Rank {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
    let s = match self {
        Rank::Two => "2",
        Rank::Three => "3",
        Rank::Four => "4",
        Rank::Five => "5",
        Rank::Six => "6",
        Rank::Seven => "7",
        Rank::Eight => "8",
        Rank::Nine => "9",
        Rank::Ten => "10",
        Rank::Jack => "J",
        Rank::Queen => "Q",
        Rank::King => "K",
        Rank::Ace => "A",
    };
    write!(f, "{}", s)
}
}
impl fmt::Display for Suit {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
    let symbol = match self {
        Suit::Hearts => "♥",
        Suit::Diamonds => "♦",
        Suit::Clubs => "♣",
        Suit::Spades => "♠",
    };
    write!(f, "{}", symbol)
}
}
impl fmt::Display for PokerCard {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
    write!(f, "{} of {}", self.rank, self.suit)
}
}
```
Now, let me explain what we just did. `fmt::Display` is the trait for user-facing string formatting. It is what `println!("{}", value)` uses. The `Formatter<'_>` is the output target. 

> You might notice the `'_` inside `Formatter<'_>`. This is a lifetime annotation. Every reference in Rust has a lifetime, which is how long the reference stays valid. The Formatter struct borrows an output buffer, and Rust needs to track how long that borrow lasts. Writing `'_` tells the compiler 'I don't want to name this lifetime, just figure it out automatically.`'` This is called lifetime elision. The compiler looks at the function signature and infers the correct lifetime for you. We will cover lifetimes properly in a later article, but for now just understand that `'_` means 'the compiler will handle this borrow check for us.'

### Implement Card for PokerCard
```rust
impl Card for PokerCard {
fn value(&self) -> u8 {
    match self.rank {
        Rank::Two => 2,
        Rank::Three => 3,
        Rank::Four => 4,
        Rank::Five => 5,
        Rank::Six => 6,
        Rank::Seven => 7,
        Rank::Eight => 8,
        Rank::Nine => 9,
        Rank::Ten | Rank::Jack | Rank::Queen | Rank::King => 10,
        Rank::Ace => 11,
    }
}
fn display(&self) -> String {
    format!("{}", self)
}
}
```
The Ace returns 11. The hand scoring logic will handle the soft Ace calculation.
### The Generic Deck
We need the `Debug` trait in scope for our generic bounds, so we import it:
```rust
use std::fmt::Debug;
use rand::seq::SliceRandom;
struct Deck<T> {
cards: Vec<T>,
}
```
Now for the generic methods. These work with any `T` that implements `Card`, `Clone`, and `Debug`:
```rust
impl<T: Card + Clone + Debug> Deck<T> {
fn shuffle(&mut self) {
    self.cards.shuffle(&mut rand::thread_rng());
}
fn deal(&mut self) -> Result<T, GameError> {
    self.cards.pop().ok_or(GameError::EmptyDeck)
}
fn len(&self) -> usize {
    self.cards.len()
}
}
```
Now, let me explain what we just did. `impl<T: Card + Clone + Debug> Deck<T>` means these methods only exist when `T` implements all three traits. Inside this block, we can call `card.value()` because `T: Card`. We can `card.clone()` because `T: Clone`.
Notice that `deal` now returns `Result<T, GameError>` instead of `Option<T>`. We use `ok_or` to convert `None` into our custom error. This is proper error handling. In the previous article, we might have used `expect` or `unwrap`. Now we use `Result`.
But wait. Where is `new`? We cannot put `new` in the generic impl because a generic `Deck<T>` does not know how to create `T`. Creating 52 poker cards is specific to `PokerCard`. So we put it on a separate impl block that is only for `Deck<PokerCard>`:
```rust
impl Deck<PokerCard> {
fn new() -> Self {
    let mut cards = Vec::new();
    for suit in [Suit::Hearts, Suit::Diamonds, Suit::Clubs, Suit::Spades] {
        for rank in [
            Rank::Two, Rank::Three, Rank::Four, Rank::Five,
            Rank::Six, Rank::Seven, Rank::Eight, Rank::Nine,
            Rank::Ten, Rank::Jack, Rank::Queen, Rank::King, Rank::Ace,
        ] {
            cards.push(PokerCard { rank, suit });
        }
    }
    Deck { cards }
}
}
```
`impl Deck<PokerCard>` means `new()` only exists for `Deck<PokerCard>`. If you tried to write `Deck::<UnoCard>::new()`, you would get a compile error because there is no `new` for that type. This is the correct way to handle it: keep the engine generic, but put concrete constructors on concrete types.
### The Generic Hand
```rust
struct Hand<T> {
cards: Vec<T>,
}
impl<T: Card> Hand<T> {
fn new() -> Self {
    Hand { cards: Vec::new() }
}
fn add(&mut self, card: T) {
    self.cards.push(card);
}
fn score(&self) -> u8 {
    let mut total = 0;
    let mut aces = 0;
    for card in &self.cards {
        let value = card.value();
        total += value;
        if value == 11 {
            aces += 1;
        }
    }
    while total > 21 && aces > 0 {
        total -= 10;
        aces -= 1;
    }
    total
}
fn display(&self) -> String {
    self.cards.iter()
        .map(|c| c.display())
        .collect::<Vec<_>>()
        .join(", ")
}
}
```
Now, let me explain the scoring. We iterate through every card and add up the values. We also count how many Aces we saw. After the loop, if the total is over 21 and we have Aces, we convert each Ace from 11 to 1 by subtracting 10. We keep doing this until the total is 21 or below, or we run out of Aces. This is the standard blackjack soft hand rule.
### The Run Pattern
Just like in the previous article, we keep `main` simple and put our game logic in a `run` function that returns `Result`.
```rust
fn run() -> Result<(), GameError> {
let mut deck = Deck::new();
deck.shuffle();
let mut player = Hand::new();
let mut dealer = Hand::new();
// Deal initial cards
player.add(deck.deal()?);
dealer.add(deck.deal()?);
player.add(deck.deal()?);
dealer.add(deck.deal()?);
println!("Your hand: {} (score: {})", player.display(), player.score());
println!("Dealer shows: {}", dealer.cards[0].display());
// Player turn
loop {
    println!("Hit or stand? (h/s)");
    let mut input = String::new();
    std::io::stdin().read_line(&mut input)?;
    match input.trim() {
        "h" | "hit" => {
            let card = deck.deal()?;
            println!("You drew: {}", card.display());
            player.add(card);
            println!("Your hand: {} (score: {})", player.display(), player.score());
            if player.score() > 21 {
                println!("Bust! You lose.");
                return Ok(());
            }
        }
        "s" | "stand" => break,
        _ => {
            return Err(GameError::InvalidInput(input.trim().to_string()));
        }
    }
}
// Dealer turn
println!("Dealer hand: {} (score: {})", dealer.display(), dealer.score());
while dealer.score() < 17 {
    let card = deck.deal()?;
    println!("Dealer drew: {}", card.display());
    dealer.add(card);
    println!("Dealer hand: {} (score: {})", dealer.display(), dealer.score());
}
if dealer.score() > 21 {
    println!("Dealer busts! You win.");
    return Ok(());
}
// Determine winner
let player_score = player.score();
let dealer_score = dealer.score();
println!("Final - You: {}, Dealer: {}", player_score, dealer_score);
if player_score > dealer_score {
    println!("You win!");
} else if dealer_score > player_score {
    println!("Dealer wins!");
} else {
    println!("Push! It's a tie.");
}
Ok(())
}
fn main() {
if let Err(e) = run() {
    eprintln!("Error: {}", e);
    std::process::exit(1);
}
}
```
Now, let me explain what we just did. `run` returns `Result<(), GameError>`. We use `?` everywhere an operation can fail. `deck.deal()?` propagates the `EmptyDeck` error. `std::io::stdin().read_line(&mut input)?` propagates IO errors, and thanks to `#[from] std::io::Error` on our error type, the `?` operator automatically converts it.
`main` calls `run` and handles the error. If there is an error, it prints to stderr and exits with code 1. This is the exact pattern we used in the TOML parser article.
### Running the Project
Type this in your terminal:
```bash
cargo run
```
You should see output like this:
```
Your hand: A of ♠, 10 of ♥ (score: 21)
Dealer shows: 7 of ♦
Hit or stand? (h/s)
```
Try playing a few hands. The generic engine is working.
## Conclusion
In this post, you learned about generics and traits. You learned that generics let you write code once and use it with any type, and that traits define contracts that constrain those generics. 

You learned about trait bounds, default implementations, the orphan rule, and the derive macro. You also learned how to combine generics and traits to build truly reusable code.

You built a generic blackjack card engine with proper error handling using `Result`, custom error types with `thiserror`, and the `?` operator. The engine works with any card type that implements the `Card` trait.

In the next article, we will learn about HashMap and build an inverted index search engine. I hope to see you soon. Till then, goodbye.

# Learn Rust HashMap and Iterators by Building a Git Object Store Reader

> In this post, we are going to build a git object store reader in Rust and learn hashmaps and iterators in Rust

Source: https://blog.sheerluck.dev/posts/rust/learn-hashmap-iterators-by-building-a-git-object-store-reader/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-hashmap-iterators-by-building-a-git-object-store-reader/index.md

In this post, we are going to learn about `HashMap` and the `Iterator` trait in Rust. Once we cover all the concepts, we will build a **Git Object Store Reader** that reads a real `.git` folder, decompresses loose objects, decodes commits/trees/blobs, and renders `git log --oneline` output. I'm really excited for this project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles.

The only prerequisite is that you have read the previous articles in this series, as I will assume you know ownership, borrowing, structs, enums, pattern matching, error handling, generics, traits, and lifetimes.

Get the source code [here](https://github.com/MrSheerluck/git-log-in-rust)

## HashMap
We have been using `Vec<(String, JsonValue)>` as a stand-in for objects in the JSON parser because we had not covered HashMap yet. Now it's time to learn this.

`HashMap<K, V>` is a hash table that maps keys of type `K` to values of type `V`. It gives you O(1) average lookups, insertions, and deletions.

### HashMap Insertion Operation

```rust
use std::collections::HashMap;

let mut scores = HashMap::new();

scores.insert(String::from("Alice"), 100);
scores.insert(String::from("Bob"), 80);
```
### HashMap Retrieving Values

You can look up values by key:

```rust
let alice_score = scores.get("Alice"); // Option<&i32>
match alice_score {
Some(score) => println!("Alice scored {}", score),
None => println!("Alice not found"),
}
```

`get` returns `Option<&V>`. If the key exists, you get `Some(&value)`. If not, `None`.

You can iterate over entries:

```rust
for (name, score) in &scores {
println!("{}: {}", name, score);
}
```

### Entry API

The entry API is one of the most useful features of `HashMap`. It lets you check if a key exists and act on it in one step:

```rust
scores.entry(String::from("Alice")).or_insert(50);
```

`entry` returns an `Entry` enum. `or_insert` inserts the value if the key does not exist, or returns a mutable reference to the existing value.

Without the entry API, you'd have to write:

```rust
if !scores.contains_key("Alice") {
scores.insert(String::from("Alice"), 50);
}
```

That's two lookups instead of one. The entry API does it in a single operation.

You can also update based on the existing value:

```rust
let mut map = HashMap::new();
for word in "hello world hello".split_whitespace() {
let count = map.entry(word).or_insert(0);
*count += 1;
}
```

`map.entry(word)` looks up the word in the map. If the word doesn't exist, `or_insert(0)` inserts it with a starting count of 0 and returns a `&mut i32` pointing to that new entry. If the word already exists, `or_insert` returns a `&mut i32` pointing to the existing count. Either way, we get a mutable reference to the counter. Then `*count += 1` dereferences it and increments the value. After this loop, `map` contains `{"hello": 2, "world": 1}`.

### Ownership
When you insert a `String` into a `HashMap`, the map takes ownership of the string. If you insert a reference, you must ensure the reference outlives the map.

```rust
let mut map = HashMap::new();
let key = String::from("name");
let value = String::from("Alice");
map.insert(key, value); // ownership moves into map
// println!("{}", key); // ERROR: key was moved
```

Both `key` and `value` are `String`s. When we call `map.insert(key, value)`, ownership of both strings moves into the map. After that line, we can no longer use `key` or `value`. If we tried to print `key`, the compiler would say "value used here after move." This is the same ownership rule that applies everywhere in Rust: when you give something to a collection, the collection owns it now.

## The Iterator Trait

You have been using iterators since the mini grep project. Every time you call `.iter()`, `.into_iter()`, `.lines()`, or write a `for` loop, you are using the Iterator trait. Now let's understand what it actually is.

```rust
pub trait Iterator {
type Item;
fn next(&mut self) -> Option<Self::Item>;
// many provided methods
}
```

That's it. An iterator is just something that has a `next` method. Each call to `next` returns either `Some(item)` or `None` when the iteration is finished.

### `for` Loops Desugar to Iterators
When you write:

```rust
let v = vec![1, 2, 3];
for x in v {
println!("{}", x);
}
```

Rust desugars it into:
```rust
let mut iter = v.into_iter();
while let Some(x) = iter.next() {
println!("{}", x);
}
```

`for` calls `.into_iter()` on the collection, then repeatedly calls `.next()` on the resulting iterator.

### Three Forms of Iteration
Rust provides three ways to get an iterator from a collection:

| Method | Produces | Meaning |
|--------|----------|---------|
| `.iter()` | `&T` | Immutable references |
| `.iter_mut()` | `&mut T` | Mutable references |
| `into_iter()` | `T` | Owned values (consumes the collection) |

Here is what each looks like:
```rust
let v = vec![1, 2, 3];
for x in v.iter() {
// x: &i32 - we borrow each element
}
for x in v.iter_mut() {
// x: &mut i32 - we can modify each element
}
for x in v.into_iter() {
// x: i32 - we own each element, v is consumed
}
// println!("{:?}", v); // ERROR: v was moved in into_iter()
```

### Consuming Adapters
Some iterator methods consume the iterator and produce a value. `collect` and `count` are common examples.

```rust
let v = vec![1, 2, 3, 4, 5];
let count = v.iter().count(); // 5
let doubled: Vec<i32> = v.iter().map(|x| x * 2).collect();
```

`count` exhausts the iterator and returns the number of elements. `collect` exhausts the iterator and gathers elements into a collection. You need to specify the collection type (like `Vec<i32>`) because `collect` can produce many types.

### Iterator Adapters

Iterator adapters transform one iterator into another iterator. They are lazy, meaning they do not execute immediately. Instead, they create a new iterator that describes how values should be processed. The actual work only happens when a consuming operation like `collect`, `count`, or `for_each` starts pulling values out with `next()`.

`map` transforms each element:

```rust
let v = vec![1, 2, 3];

let squares: Vec<i32> = v.iter()
.map(|x| x * x)
.collect();

// squares = [1, 4, 9]
```

`v.iter()` produces `&i32` values. The closure receives each element one by one and returns a transformed value. Here, every number is squared. `map` does not modify the original vector. It creates a new iterator that yields transformed values lazily.

`filter` keeps only elements that match a condition:

```rust
let v = vec![1, 2, 3, 4, 5];

let evens: Vec<&i32> = v.iter()
.filter(|x| *x % 2 == 0)
.collect();

// evens = [2, 4]
```

`filter` takes a closure that returns `true` or `false`. If the closure returns `true`, the element stays in the iterator. If it returns `false`, the element is skipped.

Notice the resulting type is `Vec<&i32>`, not `Vec<i32>`. `v.iter()` yields references, so `filter` also yields references.

`take` limits how many elements are produced:

```rust
let v = vec![1, 2, 3, 4, 5];

let first_three: Vec<&i32> = v.iter()
.take(3)
.collect();

// first_three = [1, 2, 3]
```

`take(3)` stops the iterator after producing three elements. Even if the original iterator has more values, iteration ends early.

`skip` ignores the first N elements:

```rust
let v = vec![1, 2, 3, 4, 5];

let after_two: Vec<&i32> = v.iter()
.skip(2)
.collect();

// after_two = [3, 4, 5]
```

`skip(2)` discards the first two values before yielding anything.

`chain` combines two iterators into one continuous iterator:

```rust
let a = vec![1, 2];
let b = vec![3, 4];

let chained: Vec<&i32> = a.iter()
.chain(b.iter())
.collect();

// chained = [1, 2, 3, 4]
```

`chain` first yields all values from the first iterator, then continues with the second iterator.

`zip` pairs elements from two iterators together:

```rust
let names = vec!["Alice", "Bob"];
let scores = vec![100, 80];

let paired: Vec<(&str, &i32)> = names.into_iter()
.zip(scores.iter())
.collect();

// paired = [("Alice", 100), ("Bob", 80)]
```

`zip` walks both iterators at the same time. The first element from one iterator is paired with the first element from the other, the second with the second, and so on.

If one iterator ends early, `zip` stops immediately.

`filter_map` combines filtering and transformation into a single operation:

```rust
let v = vec!["1", "two", "3", "four"];

let numbers: Vec<i32> = v.iter()
.filter_map(|s| s.parse::<i32>().ok())
.collect();

// numbers = [1, 3]
```

`s.parse::<i32>()` returns `Result<i32, ParseIntError>`. Calling `.ok()` converts that `Result` into an `Option<i32>`.

- `Ok(value)` becomes `Some(value)`
- `Err(_)` becomes `None`

`filter_map` only keeps `Some` values and automatically unwraps them. Values that become `None` are skipped entirely.
So:
- `"1"` becomes `Some(1)`
- `"3"` becomes `Some(3)`
- `"two"` becomes `None`
- `"four"` becomes `None`

The final result is:
```rust
[1, 3]
```
### Chaining Adapters
The real power comes from chaining:

```rust
let v = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
let result: Vec<i32> = v.iter()
.filter(|x| *x % 2 == 0)    // keep evens
.map(|x| x * 10)             // multiply by 10
.take(3)                     // take first 3
.collect();
// result = [20, 40, 60]
```

Each adapter creates a new iterator struct that wraps the previous one. `v.iter()` creates an `Iter<i32>`. `.filter(...)` wraps it in a `Filter<Iter<i32>, _>`. `.map(...)` wraps it in a `Map<Filter<...>, _>`. `.take(3)` wraps it in a `Take<Map<...>>`. Nothing executes until a consuming operation like `collect`, `count`, or `for_each` starts calling `next` repeatedly. When it does, values flow through the pipeline one at a time: `1` comes out of the iterator, `filter` rejects it (odd), `2` comes out, `filter` accepts it, `map` turns it into `20`, `take` accepts it, `collect` stores it. Then `3` is rejected, `4` → `40` → accepted, `5` rejected, `6` → `60` → accepted. After three items, `take` returns `None` and iteration stops.

## Git Object Storage Model
Before we start the project, let's understand how Git stores data internally. Git is a content-addressable filesystem. In traditional Git repositories, every object is identified by the SHA-1 hash of its contents.
### Object Types

Git has four types of objects:

| Type | What It Stores |
|------|----------------|
| blob | File contents |
| tree | Directory listing (filenames, modes and hashes) |
| commit | Snapshot metadata (tree hash, parent hashes, author, message) |
| tag | Signed references (we will skip this) |

### Loose Object Format
Loose objects are stored in `.git/objects/XX/YYYYYY...` where `XX` is the first two hex characters of the SHA-1 hash and `YYYYYY...` is the remaining 38 characters.

The file contains the object data compressed with zlib (deflate). When decompressed, the format is:

```
<type> <size>\0<content>
```

For example, a blob containing "hello world\n" decompresses to:

```
blob 12\0hello world\n
```

The header is the type name, a space, the content size in bytes as a decimal string, a null byte, and then the raw content.

### The SHA-1 Hash
Traditional Git computes the SHA-1 hash of `<type> <size>\0<content>`. That hash becomes the object's filename. This means the same content always produces the same hash, and the hash verifies the content.

### Commit Object
```
commit <size>\0tree <tree_hash>
parent <parent_hash>
author Mrsheerluck <email> <timestamp>
committer Mrsheerluck <email> <timestamp>

commit message
```

The parent line is optional (the first commit has no parent). The tree line points to the tree object representing the directory structure at that commit.

### Tree Object
```
tree <size>\0<mode> <name>\0<hash (20 bytes)>
<mode> <name>\0<hash (20 bytes)>
```

Each entry is a file or subdirectory. The mode is a string like `100644` (regular file) or `040000` (directory). The name is the filename. The hash is 20 raw bytes of the SHA-1 hash (not hex).

## The Project: Git Object Store Reader
Now that you understand HashMap, iterators, and how Git stores objects, let's build our Git Object Store Reader.

Our program will:
- Accept a path to a `.git` directory
- Read loose objects from `.git/objects/XX/YYYYYY`
- Decompress them using `flate2`
- Parse the header and content into typed structs
- Build a commit graph using `HashMap<CommitHash, Commit>`
- Render `git log --oneline` output starting from HEAD

### Project Setup
Open your terminal and run:
```bash
cargo new git_log
cd git_log
```

Now open `Cargo.toml` and add the dependency:
```toml
[package]
name = "git_log"
version = "0.1.0"
edition = "2024"
[dependencies]
flate2 = "1"
```

The only external dependency we need is `flate2` for zlib decompression. Everything else is standard library: `std::collections::HashMap`, `std::fs`, `std::io::Read`.

### Imports

Open `src/main.rs` and delete everything. Write this:

```rust
use flate2::read::ZlibDecoder;
use std::collections::HashMap;
use std::fs;
use std::io::Read;
use std::path::Path;
```

- `flate2::read::ZlibDecoder` is used to decompress Git's zlib-compressed object files
- `std::collections::HashMap` is for storing the commit graph, mapping hashes to commit structs
- `std::fs` is for reading files from the .git directory
- `std::io::Read` gives us the `read_to_end` method we need for decompression
- `std::path::Path` is for manipulating filesystem paths

### Reading HEAD

Git always knows which branch you are on by reading the `.git/HEAD` file.

```rust
fn read_head(git_dir: &Path) -> String {
let head_path = git_dir.join("HEAD");
let content = fs::read_to_string(&head_path)
    .expect("Failed to read HEAD");
content.trim().to_string()
}
```

Now, let me explain what we just did. `git_dir.join("HEAD")` constructs the path to the HEAD file. If `git_dir` is the path to a `.git` directory, this gives us `.git/HEAD`. `fs::read_to_string` reads the entire file into a `String`. We `trim()` whitespace (HEAD files often have a trailing newline) and return the content.

The HEAD file typically contains something like `ref: refs/heads/main`. This means "main" is the current branch. To get the actual commit hash, we need to read `.git/refs/heads/main`.

```rust
fn resolve_ref(git_dir: &Path, reference: &str) -> Option<String> {
if reference.starts_with("ref: ") {
    let ref_path = &reference[5..];
    let full_path = git_dir.join(ref_path);
    fs::read_to_string(&full_path).ok()
        .map(|s| s.trim().to_string())
} else {
    Some(reference.to_string())
}
}
```

Now, let me explain what we just did. If the reference starts with `"ref: "`, we strip off those 5 characters to get something like `refs/heads/main`. Why 5? Because `"ref: "` is exactly 5 characters: `r`, `e`, `f`, `:`, ` `. We join that path with the git directory and read the file it points to.

`fs::read_to_string` returns `Result<String, io::Error>`. We call `.ok()` to convert it to `Option<String>`. If the file exists, we get `Some(content)`. If not (the branch doesn't exist), we get `None`. Then `.map(|s| s.trim().to_string())` strips whitespace from the content. The result is the actual commit hash, like `"a1b2c3d4e5f6..."`.

If the reference doesn't start with `"ref: "`, it means HEAD is already pointing directly at a commit hash — this is "detached HEAD" state. In that case, we just return it as-is wrapped in `Some`.

### Structs for Git Objects

We need a struct for the data we care about:

```rust
struct Commit {
tree_hash: String,
parent_hashes: Vec<String>,
message: String,
}
```

Each commit has a tree hash (pointing to the directory snapshot), zero or more parent hashes (the first commit has none, merge commits have two), and a commit message.

### Reading a Loose Object

Git stores objects in `.git/objects/XX/YYYYYY` where the path is derived from the SHA-1 hash. The first two hex characters become the directory name, and the remaining 38 characters become the filename.

A loose object file is compressed with zlib (deflate). Let's write a function to read and decompress one:

```rust
fn read_object(git_dir: &Path, hash: &str) -> Vec<u8> {
let object_path = git_dir
    .join("objects")
    .join(&hash[..2])
    .join(&hash[2..]);
let compressed = fs::read(&object_path)
    .unwrap_or_else(|_| panic!("Failed to read object: {}", hash));
let mut decoder = ZlibDecoder::new(&compressed[..]);
let mut decompressed = Vec::new();
decoder.read_to_end(&mut decompressed)
    .unwrap_or_else(|_| panic!("Failed to decompress object: {}", hash));
decompressed
}
```

Now, let me explain what we just did. We construct the path by splitting the hash. For example, if the hash is `a1b2c3d4e5f6...`, `&hash[..2]` gives us `"a1"` (the first two hex characters), and `&hash[2..]` gives us `"b2c3d4e5f6..."` (the remaining 38 characters). The resulting path becomes `.git/objects/a1/b2c3d4e5f6...`.

We read the compressed file into a byte vector using `fs::read`. Then `&compressed[..]` creates a byte slice of the entire compressed data. We wrap it in a `ZlibDecoder` which implements `Read`. When we call `decoder.read_to_end(&mut decompressed)`, it decompresses the zlib data and writes all decompressed bytes into our output vector.

When decompressed, a Git object looks like this:

```
blob 12\0hello world\n
```

or:

```
commit 234\0tree a1b2...\nparent c3d4...\nauthor ...
```

The format is always: type name, space, size in bytes as decimal, null byte, then the raw content.

### Parsing the Object Header

Let's write a function to split the decompressed data into the header and the content:

```rust
fn parse_object(data: &[u8]) -> (&str, &[u8]) {
let header_end = data.iter().position(|&b| b == 0)
    .expect("Invalid object: no null byte");
let header = std::str::from_utf8(&data[..header_end])
    .expect("Invalid object: header not valid UTF-8");
let content = &data[header_end + 1..];
let mut parts = header.splitn(2, ' ');
let obj_type = parts.next().unwrap();
let _size = parts.next().unwrap();
(obj_type, content)
}
```

Now, let me explain what we just did. `data.iter().position(|&b| b == 0)` walks through the byte array and returns the index of the first null byte. This is our separator between the header and the content. If there's no null byte, the data is corrupted and we panic.

`&data[..header_end]` gives us the header bytes (everything before the null). We convert those bytes to a string with `from_utf8`. Then `header.splitn(2, ' ')` splits the header on the first space into at most 2 parts. Part 1 is the object type (like `"commit"` or `"tree"` or `"blob"`). Part 2 is the size (like `"234"`). We don't actually need the size because the content is everything after the null byte, but we still parse it because Git object headers always include it.

The function returns a tuple: the type string and a byte slice of the content.

### Parsing a Commit Object

Now that we can read and decompress any object, let's parse a commit. A commit object looks like this when decompressed:

```
commit 234\0tree a1b2c3d4e5f6...
parent f8g9h0i1j2k3...
author Mrsheerluck <email> 1234567890 +0000
committer Mrsheerluck <email> 1234567890 +0000

Initial commit message
```

The format is key-value headers until a blank line, then the commit message.

```rust
fn parse_commit(content: &[u8]) -> Commit {
let content_str = std::str::from_utf8(content)
    .expect("Invalid commit: not valid UTF-8");
let mut tree_hash = String::new();
let mut parent_hashes = Vec::new();
let mut message = String::new();
let mut in_message = false;

for line in content_str.lines() {
    if in_message {
        if !message.is_empty() {
            message.push('\n');
        }
        message.push_str(line);
        continue;
    }

    if line.is_empty() {
        in_message = true;
        continue;
    }

    if let Some(hash) = line.strip_prefix("tree ") {
        tree_hash = hash.to_string();
    } else if let Some(hash) = line.strip_prefix("parent ") {
        parent_hashes.push(hash.to_string());
    }
}

Commit {
    tree_hash,
    parent_hashes,
    message,
}
}
```

Now, let me explain what we just did. `content_str.lines()` returns an iterator over the lines of the commit text. We walk through them one by one using a `for` loop.

Before we hit the blank line, we are parsing headers. `line.strip_prefix("tree ")` checks if the line starts with `"tree "`. If it does, it returns the rest of the line (the hash) as a `&str`, and we convert it to an owned `String`. Same for parent lines. There can be multiple parent lines in merge commits, so we push each one into a vector.

When we hit an empty line (`if line.is_empty()`), we flip `in_message` to `true`. From that point on, every line is part of the commit message. We build the message string by appending each line. The `if !message.is_empty()` check ensures we add a newline between lines of a multi-line message, but not at the start.

### Reading Commits Recursively

To build the full commit graph, we need to read a commit, then read all its parents, then their parents, and so on. We use a `HashMap` to cache commits we have already read, so we never read the same object twice and we avoid infinite loops.

```rust
fn read_commit(
git_dir: &Path,
hash: &str,
cache: &mut HashMap<String, Commit>,
) {
if cache.contains_key(hash) {
    return;
}

let data = read_object(git_dir, hash);
let (obj_type, content) = parse_object(&data);

match obj_type {
    "commit" => {
        let commit = parse_commit(content);
        for parent in &commit.parent_hashes {
            read_commit(git_dir, parent, cache);
        }
        cache.insert(hash.to_string(), commit);
    }
    _ => {
        panic!("Expected commit object, got {}", obj_type);
    }
}
}
```

Now, let me explain what we just did. First, `cache.contains_key(hash)` checks if we've already read this commit. If it's in the cache, we return immediately. This prevents redundant reads. Git commit graphs are acyclic, but caching still avoids re-reading commits reachable through multiple merge paths.

Then we read the object from disk, parse its header to confirm it is a `"commit"` type, and parse the commit data into our `Commit` struct.

Before inserting the current commit into the cache, we recursively read all its parent commits. We iterate over `commit.parent_hashes` and call `read_commit` for each one. This recursively walks the entire parent graph so every reachable ancestor commit eventually ends up in the cache. The recursion base case is when a commit has no parents (the root commit) or when all parents are already in the cache.

Finally, `cache.insert(hash.to_string(), commit)` takes ownership of both the key and the value. The `hash` parameter is `&str`, so we call `.to_string()` to get an owned `String` for the map key.

### Rendering `git log --oneline`

Now we have a graph of commits. The `git log --oneline` format looks like this:

```
a1b2c3d Initial commit message
e5f6a7b Add new feature
f8g9h0i Fix bug in parser
```

Each line starts with the abbreviated hash (first 7 characters) and the first line of the commit message.

```rust
fn log_oneline(commits: &HashMap<String, Commit>, start_hash: &str) {
let mut hash = start_hash.to_string();

loop {
    match commits.get(&hash) {
        Some(commit) => {
            let short_hash = &hash[..7];
            let first_line = commit.message.lines().next().unwrap_or("");
            println!("{} {}", short_hash, first_line);

            match commit.parent_hashes.first() {
                Some(parent) => hash = parent.clone(),
                None => break,
            }
        }
        None => break,
    }
}
}
```

Now, let me explain what we just did. We start from `start_hash` (the commit that HEAD points to) and walk backwards through the first-parent chain. For each commit, `&hash[..7]` takes the first 7 characters of the hash string. `commit.message.lines()` returns an iterator over the lines of the message. `.next()` gets the first line, returning `Option<&str>`. We use `unwrap_or("")` to handle the edge case of an empty message. Then we move to the first parent by cloning the hash of `parent_hashes.first()`. If there are no parents, we have reached the root commit and we `break`.

This follows only the first parent, which is similar to `git log --first-parent --oneline`. Merge commits have multiple parents, but the first parent is usually the branch you were on when you merged.

### Walking the Tree

Now let's use iterators to do something more interesting. Let's list all files in the latest commit's directory structure. First, we need to parse tree objects.

Git tree objects have a binary format. Each entry is:

```
<mode> <name>\0<20-byte raw hash>
```

The mode is a string like `100644` (regular file) or `040000` (directory). The name is the filename. The hash is 20 raw bytes, not a hex string.

```rust
struct TreeEntry {
mode: String,
name: String,
hash: String,
}

fn parse_tree(content: &[u8]) -> Vec<TreeEntry> {
let mut entries = Vec::new();
let mut pos = 0;

while pos < content.len() {
    let null_pos = content[pos..].iter().position(|&b| b == 0)
        .expect("Invalid tree: missing null byte") + pos;

    let mode_and_name = std::str::from_utf8(&content[pos..null_pos])
        .expect("Invalid tree: not valid UTF-8");
    let space_pos = mode_and_name.find(' ')
        .expect("Invalid tree entry: missing space");
    let mode = &mode_and_name[..space_pos];
    let name = &mode_and_name[space_pos + 1..];

    pos = null_pos + 1;

    let hash_bytes = &content[pos..pos + 20];
    let hash = hash_bytes.iter()
        .map(|b| format!("{:02x}", b))
        .collect::<Vec<_>>()
        .join("");

    entries.push(TreeEntry {
        mode: mode.to_string(),
        name: name.to_string(),
        hash,
    });

    pos += 20;
}

entries
}
```

Now, let me explain what we just did. Tree objects are binary, so we walk through the data byte by byte.

`content[pos..].iter().position(|&b| b == 0)` finds the first null byte starting from the current position. We add `pos` to get the absolute index in the original array. The bytes from `pos` to `null_pos` are the mode and name, separated by a space. So `mode_and_name.find(' ')` finds that space, `&mode_and_name[..space_pos]` is the mode (like `"100644"`), and `&mode_and_name[space_pos + 1..]` is the name (like `"main.rs"`).

After the null byte, the next 20 bytes are the raw SHA-1 hash. `&content[pos..pos + 20]` gives us those bytes. This assumes the tree object is valid and contains at least 20 bytes for the hash. Real parsers would validate bounds more carefully instead of panicking on malformed data. Then we convert each byte to a two-character hex string:

```rust
hash_bytes.iter()
.map(|b| format!("{:02x}", b))
.collect::<Vec<_>>()
.join("")
```

Let me break that iterator chain down. `hash_bytes.iter()` gives us an iterator over each byte. `.map(|b| format!("{:02x}", b))` transforms each byte (like `0xa1`) into a two-character hex string (like `"a1"`). `.collect::<Vec<_>>()` collects all those hex strings into a vector like `["a1", "b2", "c3", ...]`. `.join("")` concatenates them into a single 40-character hash string like `"a1b2c3..."`.

Then we advance `pos` by 20 bytes to move to the next entry.

Now let's write a recursive tree walker that lists every file:

```rust
fn walk_tree(git_dir: &Path, tree_hash: &str, prefix: &str) -> Vec<String> {
let data = read_object(git_dir, tree_hash);
let (_, content) = parse_object(&data);
let entries = parse_tree(content);

entries.iter()
    .flat_map(|entry| {
        let path = if prefix.is_empty() {
            entry.name.clone()
        } else {
            format!("{}/{}", prefix, entry.name)
        };

        if entry.mode == "040000" {
            walk_tree(git_dir, &entry.hash, &path)
        } else {
            vec![path]
        }
    })
    .collect()
}
```

Now, let me explain what we just did. For each entry in the tree, we build the full path. If the prefix is empty (we're at the root), the path is just the entry name. Otherwise, it's `prefix/name`. So if we're in `src/` and the entry is `main.rs`, the path becomes `src/main.rs`.

If the entry's mode is `"040000"`, it's a directory. We recurse into it by calling `walk_tree` again with the subdirectory's tree hash and the new path. If it's a file (mode `"100644"` or similar), we return a vector with just that one path.

The key here is `flat_map`. Each entry produces either a single-element vector (a file) or a multi-element vector (all files in a subdirectory). Without `flat_map`, the result would be `Vec<Vec<String>>` — a vector of file lists, one per entry. With `flat_map`, all those iterators are flattened into one combined `Vec<String>` of every file path.

### Putting It All Together

Now let's wire everything together in `main`:

```rust
fn main() {
let args: Vec<String> = std::env::args().collect();
let git_dir = if args.len() > 1 {
    Path::new(&args[1]).to_path_buf()
} else {
    Path::new(".git").to_path_buf()
};

let head = read_head(&git_dir);
let head_hash = resolve_ref(&git_dir, &head)
    .expect("Failed to resolve HEAD");

let mut cache = HashMap::new();
read_commit(&git_dir, &head_hash, &mut cache);

log_oneline(&cache, &head_hash);

if let Some(commit) = cache.get(&head_hash) {
    println!("\nFiles in latest commit:");
    let files = walk_tree(&git_dir, &commit.tree_hash, "");
    for file in &files {
        println!("  {}", file);
    }
}
}
```

Now, let me explain what we just did. `std::env::args().collect()` reads command line arguments into a `Vec<String>`. If a path argument is provided (`args.len() > 1`), we use it as the git directory. Otherwise, we default to `.git` in the current directory. `Path::new(&args[1]).to_path_buf()` creates an owned `PathBuf` from the string argument.

We read HEAD with `read_head`, which gives us something like `"ref: refs/heads/main"`. Then `resolve_ref` follows that reference to get the actual commit hash.

We create a new, empty `HashMap` as our cache. Then `read_commit` reads the commit graph starting from HEAD, which recursively reads all ancestors. By the time this returns, the cache contains every commit reachable from HEAD.

Finally, we print the log output using `log_oneline`. If the HEAD commit is in the cache (it should always be), we also walk its tree to list all files in the latest snapshot.

### Running The Project

Type this in your terminal:

```bash
cargo run -- /path/to/some/.git
```

Or from inside a git repository:

```bash
cargo run
```

You should see output like this:

```
a1b2c3d Initial commit
e5f6a7b Add feature X
f8g9h0i Set up project structure

Files in latest commit:
  src/main.rs
  src/lib.rs
  Cargo.toml
  README.md
```

That's it, your own `git log --oneline` implementation in less than 200 lines of Rust, using `HashMap`, iterators, and the knowledge of Git's object storage model.

## Conclusion

In this post, you learned about `HashMap`, the `Entry` API for single-lookup insert-or-update, the `Iterator` trait and its three forms (`.iter()`, `.iter_mut()`, `.into_iter()`), consuming adapters like `collect` and `count`, iterator adapters like `map`, `filter`, `flat_map`, and how to chain them lazily. You also learned how Git stores objects (blobs, trees, commits) as zlib-compressed files in `.git/objects/`. And you built a Git Object Store Reader that reads real `.git` directories, parses commits and trees, and renders `git log --oneline` output using nothing but the standard library and `flate2`.

In the next article, we will build something even more interesting. See you soon.

# Learn Rust Async/Await, Tokio, and TCP Networking by Building an HTTP/1.1 Server

> In this post, we are going to learn about async/await in Rust. We will learn about futures, the difference between OS threads and async tasks, the Tokio runtime, async TCP. Once we cover all the concepts, we will build an Async HTTP/1.1 Server

Source: https://blog.sheerluck.dev/posts/rust/learn-rust-async-await-by-building-an-http-server/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-rust-async-await-by-building-an-http-server/index.md

In this post, we are going to learn about async/await in Rust. We will learn about futures, the difference between OS threads and async tasks, the Tokio runtime, async TCP with `tokio::net::TcpListener`, `tokio::spawn` for concurrent tasks, and async file I/O. Once we cover all the concepts, we will build an **Async HTTP/1.1 Server** with a raw TCP listener, manual HTTP request parsing (no framework), request routing, static file serving with proper content types, and concurrent connection handling with `tokio::spawn`. I am really excited for this project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles.

The only prerequisite is that you have read the previous articles in this series, as I will assume you know ownership, borrowing, structs, enums, pattern matching, error handling, generics, traits, lifetimes, HashMap, iterators, closures, smart pointers, and concurrency with threads.

In the last article, we built a Thread Pool from scratch using OS threads, channels, `Arc<Mutex<T>>`, and graceful shutdown. That was preemptive concurrency: the operating system decides when each thread runs. Today we will learn about cooperative concurrency with async/await, where tasks voluntarily yield control at `.await` points. We will build a real HTTP server that handles thousands of concurrent connections using async I/O.

> **A quick note before we begin:** Async Rust introduces several new concepts such as futures, polling, wakers, and runtimes. Don't worry if every detail doesn't immediately click on the first read. Focus on understanding the overall flow of how async programs are structured and how the pieces fit together. We'll revisit many of these concepts in future projects, where repeated exposure will make them feel much more natural.

Get source code from [here](https://github.com/MrSheerluck/async-http-in-rust)

## Threads vs Async

Let's start with a mental model. In the Thread Pool article, each worker was a dedicated OS thread. The OS scheduler preempts threads, meaning it can pause any thread at any instruction and switch to another. Threads are heavyweight: each one has its own stack (typically a few megabytes), and context switching between threads has overhead from the OS kernel.

> Please learn Operating Systems to learn more deeply. It's really interesting

Async is different. An async runtime (like Tokio) runs many tasks on a small number of OS threads, often just one per CPU core. Tasks are lightweight state machines. When a task hits an `.await` point, it yields control back to the runtime, which picks another ready task to run. Tasks are cooperatively scheduled by the runtime. The OS can still preempt the worker thread, but task switching itself does not require a kernel-mediated thread context switch.

Think of threads as a restaurant where each table gets its own dedicated waiter. Expensive but responsive. Think of async as a single waiter handling many tables. The waiter takes an order from table 1, while the kitchen prepares table 1's food the waiter takes table 2's order, then table 3's, then checks if table 1's food is ready. The waiter is never blocked waiting. They switch between tables whenever a table is not ready.

The key insight: async shines when your program spends a lot of time waiting for external operations such as network requests, database queries, timers, or file access. While one task waits for a response, the runtime runs another task. CPU-bound workloads benefit more from threads (and `rayon` for data parallelism, which we won't cover today).

## Futures

In Rust, an async function or block returns a future. A future is a value that represents a computation that is not yet complete. Under the hood, a future is a type that implements the `Future` trait from the standard library:

```rust
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};

pub trait Future {
type Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}
```

Let me read this from the outside in:

- `type Output` is the type the future resolves to. For `async fn foo() -> i32`, `Output` is `i32`.
- `poll` is the heart of the trait. It asks the future: "are you done yet?".
- `Poll<T>` is an enum with two variants: `Poll::Ready(T)` meaning the computation is complete and here is the result, and `Poll::Pending` meaning the computation is not ready yet, try again later.
- `Context` carries a `Waker`, which is a handle the runtime gives to the future. When the future is `Pending`, it must arrange for the waker to be called when progress can be made (for example, when data arrives on a socket).

You almost never implement `Future` by hand. The compiler generates the state machine when you write `async fn` or `async { }` blocks. But understanding the trait helps you understand what `.await` actually does.
## async and .await

An async function is syntactic sugar for a function that returns a future:

```rust
async fn hello() -> String {
String::from("hello")
}
```

This is roughly equivalent to:

```rust
fn hello() -> impl Future<Output = String> {
async { String::from("hello") }
}
```

Calling an async function does not run its body. It returns a future. You need to `.await` the future to run it:

```rust
async fn greet() {
let msg = hello().await;
println!("{}", msg);
}
```

Now, let me explain what we just did.

`hello()` returns a future of type `impl Future<Output = String>`. The `.await` drives that future to completion. When we hit `hello().await`, control yields to the runtime. If the future is ready immediately, execution continues without yielding. If it is pending, the runtime switches to another task.

You can only use `.await` inside another async function or block. This is the "async infection" people talk about: once you go async, everything up the call stack must be async too.

## Async Blocks

You can also create async blocks inline:

```rust
let future = async {
let a = compute_a().await;
let b = compute_b().await;
a + b
};
let result = future.await;
```

An async block is like a closure but for creating futures. It captures variables from the surrounding scope just like closures do.
## The Tokio Runtime

Futures are inert. They don't do anything on their own. You need a runtime to poll them. The runtime has two main components:

1. **The Reactor**: Registers interest in I/O events with the operating system (using epoll on Linux, kqueue on macOS, IOCP on Windows). When a socket has data, the reactor wakes the task that is waiting on it.
2. **The Executor**: Maintains a queue of tasks that are ready to run. It polls tasks, one at a time, on a small number of worker threads.

Tokio is the most popular async runtime for Rust. It provides the reactor, the executor, and async versions of standard library types: `TcpListener`, `TcpStream`, `File`, timers, channels, and more.

Add Tokio to your Cargo.toml:

```toml
[dependencies]
tokio = { version = "1", features = ["full"] }
```

The `features = ["full"]` flag enables all of Tokio's functionality. In production you would pick only the features you need. For learning, full is fine.

##  `#[tokio::main]`

Tokio provides a macro to set up the runtime and run your async main function:

```rust
#[tokio::main]
async fn main() {
println!("Hello from async land");
}
```

This expands to roughly:

```rust
fn main() {
let rt = tokio::runtime::Runtime::new().unwrap();
rt.block_on(async {
    println!("Hello from async land");
});
}
```

`#[tokio::main]` creates a multi-threaded runtime by default. By default, Tokio creates one worker thread per available CPU. You can control this with `#[tokio::main(flavor = "current_thread")]` for a single-threaded runtime, or `#[tokio::main(worker_threads = 4)]` for a specific count.

For an HTTP server, the multi-threaded runtime is appropriate because we want to distribute connections across cores.

## Async TCP with Tokio

Let's write a simple TCP echo server to understand async I/O:

```rust
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;

#[tokio::main]
async fn main() -> tokio::io::Result<()> {
let listener = TcpListener::bind("127.0.0.1:8080").await?;
println!("Listening on 127.0.0.1:8080");

loop {
    let (mut socket, addr) = listener.accept().await?;
    println!("Connection from {}", addr);

    tokio::spawn(async move {
        let mut buf = vec![0u8; 1024];

        loop {
            match socket.read(&mut buf).await {
                Ok(0) => {
                    println!("Connection closed by {}", addr);
                    return;
                }
                Ok(n) => {
                    if socket.write_all(&buf[..n]).await.is_err() {
                        return;
                    }
                }
                Err(_) => return,
            }
        }
    });
}
}
```

Now, let me explain what we just did.

`TcpListener::bind("127.0.0.1:8080").await?` creates a TCP listener bound to port 8080. Notice the `.await` after `bind`. `TcpListener::bind(...).await` creates and registers the socket with Tokio. The operation is usually completed immediately and does not involve waiting for incoming network events.

The `loop` continuously accepts new connections. listener.accept().await suspends the current task until a new TCP connection arrives. When one does, it returns a `(TcpStream, SocketAddr)`.

Each connection is handled inside `tokio::spawn(async move { ... })`. This spawns a new lightweight task on the Tokio runtime. The task runs concurrently with all other tasks. Unlike `thread::spawn`, this does not create a new OS thread. The task is a state machine managed by the executor.

Inside the spawned task, we loop, reading from the socket and writing back whatever we read. `socket.read(&mut buf).await` reads bytes from the socket. If it returns `Ok(0)`, the connection was closed by the peer. `socket.write_all(&buf[..n]).await` writes the same bytes back (echo). Note that both `read` and `write_all` are async methods from `AsyncReadExt` and `AsyncWriteExt`.

A single thread with async I/O can often handle thousands of mostly idle network connections concurrently. The reactor multiplexes I/O events, and the executor switches between ready tasks. Tasks that are waiting for I/O consume almost no resources.
## tokio::spawn and JoinHandle

`tokio::spawn` returns a `JoinHandle<T>` where `T` is the output of the spawned future. This is the async counterpart of `thread::spawn` returning `thread::JoinHandle<T>`:

```rust
#[tokio::main]
async fn main() {
let handle = tokio::spawn(async {
    tokio::time::sleep(std::time::Duration::from_secs(1)).await;
    42
});

println!("Waiting for result...");
let result = handle.await.unwrap();
println!("Got: {}", result);
}
```

Now, let me explain what we just did.

Unlike thread handles where you call `.join()`, you `.await` a tokio `JoinHandle`. This does not block the OS thread. It yields the current task until the spawned task completes. The current thread can do other work while waiting.

The `.unwrap()` on `handle.await` handles the case where the spawned task panicked. Just like threads, if a task panics, the join handle captures the panic.

## tokio::join! and tokio::select!

Sometimes you want to run multiple futures concurrently and wait for all of them. `tokio::join!` does this:

```rust
use tokio::time::{sleep, Duration};

async fn task_a() -> &'static str {
sleep(Duration::from_secs(1)).await;
"A"
}

async fn task_b() -> &'static str {
sleep(Duration::from_secs(2)).await;
"B"
}

#[tokio::main]
async fn main() {
let (a, b) = tokio::join!(task_a(), task_b());
println!("{} {}", a, b);
}
```

Both `task_a` and `task_b` run concurrently. The total time is about 2 seconds (the max), not 3 seconds (the sum). `tokio::join!` returns a tuple of the results in the same order as the futures passed.

`tokio::select!` is different. The losing futures are dropped. Whether that actually cancels the underlying operation depends on the future's implementation.

```rust
use tokio::time::{sleep, Duration};

#[tokio::main]
async fn main() {
tokio::select! {
    _ = sleep(Duration::from_secs(2)) => {
        println!("Timer fired after 2 seconds");
    }
    result = async_operation() => {
        println!("Operation completed: {}", result);
    }
}
}

async fn async_operation() -> &'static str {
sleep(Duration::from_secs(1)).await;
"done"
}
```

Now, let me explain what we just did.

`tokio::select!` runs both futures concurrently. `async_operation` finishes in 1 second. The timer still has 1 second left. The `async_operation` branch wins, prints its message, and the timer future is dropped. For Tokio's sleep future, dropping it effectively cancels the timer. This is the pattern for implementing timeouts, graceful shutdown signals, and racing multiple operations.

## Sharing State Across Tasks

Just like threads, async tasks sometimes need to share data. The same core primitives we learned in the Thread Pool article still apply. `Arc<T>` is used for shared ownership, channels are used for message passing, and mutexes or read-write locks are used to coordinate access to shared mutable state.

One important difference is how locking works. `std::sync::Mutex` blocks the underlying OS thread while waiting for a lock. In async applications, blocking a worker thread can prevent the runtime from making progress on other tasks scheduled on that thread. Tokio therefore provides `tokio::sync::Mutex`, whose `.lock().await` yields the current task instead of blocking the thread.

That said, `std::sync::Mutex` is not forbidden in async code. If lock contention is low and the lock is not held across an `.await` point, it can be a perfectly reasonable choice. The important rule is to avoid blocking the runtime unnecessarily.

For our HTTP server, we do not need any shared mutable state. Each connection is handled independently, and each task works with its own socket. The only shared data is the server's configuration, specifically the directory being served. We will use `Arc<PathBuf>` to share that configuration across all connection-handling tasks.

## What We Are Skipping

We are not covering `Pin` in depth. The short version is: `Pin` guarantees that a value won't be moved in memory, which is required for self-referential types like the state machines generated by async functions. In practice you almost never interact with `Pin` directly. The compiler and the runtime handle it.

We are not covering async streams, async traits, or async iterators. Those are more advanced async patterns.

We are not covering `hyper`, `axum`, `actix-web`, or any HTTP framework. This project is about understanding how HTTP and async work at the lowest level in Rust. Once you understand this, frameworks make sense.

We are not covering TLS/HTTPS. That requires `rustls` or OpenSSL and is a separate topic.

We are not covering HTTP/2 or HTTP/3. We are building HTTP/1.1, which is the simplest and most common version.

## The Project: Async HTTP/1.1 Server

Now that you understand futures, async/await, and Tokio, let's build an Async HTTP/1.1 Server from scratch.

Our program will:

- Bind to a TCP port (default 8080)
- Accept connections concurrently using `tokio::spawn`
- Parse raw HTTP/1.1 request lines and headers (method, path, headers)
- Route GET and HEAD requests to a static file server
- Detect Content-Type from file extensions
- Return proper status codes: 200, 400, 404, 405, 500
- Include Content-Length in every response
- Serve files from a configurable root directory
- Handle URL path traversal attacks (reject paths with `..`)
- Default directory index serving (serve `index.html` for `/` paths)

## Project Setup

Open your terminal and run:

```bash
cargo new async_http
cd async_http
```

Now open `Cargo.toml` and add the dependency:

```toml
[package]
name = "async_http"
version = "0.1.0"
edition = "2024"

[dependencies]
tokio = { version = "1", features = ["full"] }
```

The only dependency is Tokio with all features enabled. Everything else comes from the standard library.

Open `src/main.rs` and delete everything. We will build the server from scratch.

### The Request Struct

```rust
use std::collections::HashMap;

struct Request {
method: String,
path: String,
headers: HashMap<String, String>,
}
```

`Request` holds the parsed request data. The method is something like `GET` or `HEAD`. The path is the URL path like `/index.html` or `/images/cat.png`. The headers are key-value pairs like `Host: localhost:8080` and `Connection: keep-alive`. We store header keys in lowercase for case-insensitive lookup later.

### The Response Struct

```rust
struct Response {
status_code: u16,
status_text: String,
headers: HashMap<String, String>,
body: Vec<u8>,
}
```

`Response` holds everything needed to build an HTTP response. The status code is the numeric code like 200 or 404. The status text is the human-readable version like "OK" or "Not Found". Headers include at minimum `Content-Length`. The body is a byte vector because HTTP bodies can be binary (images, PDFs, etc.)

### Response Builder Methods

Let's implement methods on `Response` to construct responses and serialize them to bytes:

```rust
impl Response {
fn new(status_code: u16, status_text: &str) -> Self {
    Response {
        status_code,
        status_text: status_text.to_string(),
        headers: HashMap::new(),
        body: Vec::new(),
    }
}

fn with_body(mut self, body: Vec<u8>, content_type: &str) -> Self {
    self.headers
        .insert("Content-Length".to_string(), body.len().to_string());
    self.headers
        .insert("Content-Type".to_string(), content_type.to_string());
    self.body = body;
    self
}

fn with_body_text(mut self, body: String) -> Self {
    self.headers
        .insert("Content-Length".to_string(), body.len().to_string());
    self.headers
        .insert("Content-Type".to_string(), "text/plain; charset=utf-8".to_string());
    self.body = body.into_bytes();
    self
}

fn to_bytes(&self) -> Vec<u8> {
    let mut response = format!("HTTP/1.1 {} {}\r\n", self.status_code, self.status_text);

    for (key, value) in &self.headers {
        response.push_str(&format!("{}: {}\r\n", key, value));
    }

    response.push_str("Connection: close\r\n");
    response.push_str("\r\n");

    let mut bytes = response.into_bytes();
    bytes.extend_from_slice(&self.body);
    bytes
}
}
```

Now, let me explain what we just did.

`Response::new` creates a response with just a status code and text, no body and no headers yet. The body is an empty `Vec<u8>`.

`with_body` and `with_body_text` are builder-style methods. They consume `self` and return a new `Response` with the body and appropriate headers set. `with_body` takes a `Vec<u8>` and a content type string. This is for binary responses like images. `with_body_text` takes a `String` and sets `text/plain; charset=utf-8`. This is for error messages and simple text responses. Both methods set the `Content-Length` header automatically from the length of the body.

`to_bytes` serializes the response to a byte vector ready to write to the socket. It builds the status line (`HTTP/1.1 200 OK\r\n`), then each header (`Content-Length: 1234\r\n`), then a blank line to separate headers from body, then the body bytes.

Every response includes `Connection: close`. For simplicity, the server closes the connection after each request. Real HTTP/1.1 servers often support keep-alive, which reuses the connection for multiple requests, but that adds significant complexity (parsing multiple requests from one stream, handling timeouts, etc). We are skipping keep-alive for now.

### Content Type Detection

We need to set the correct `Content-Type` header based on file extension:

```rust
fn guess_content_type(path: &str) -> &str {
let ext = std::path::Path::new(path)
    .extension()
    .and_then(|e| e.to_str())
    .unwrap_or("");

match ext {
    "html" | "htm" => "text/html; charset=utf-8",
    "css" => "text/css; charset=utf-8",
    "js" => "application/javascript; charset=utf-8",
    "json" => "application/json",
    "png" => "image/png",
    "jpg" | "jpeg" => "image/jpeg",
    "gif" => "image/gif",
    "svg" => "image/svg+xml",
    "ico" => "image/x-icon",
    "txt" => "text/plain; charset=utf-8",
    "pdf" => "application/pdf",
    "xml" => "application/xml; charset=utf-8",
    "zip" => "application/zip",
    "woff" => "font/woff",
    "woff2" => "font/woff2",
    "ttf" => "font/ttf",
    "mp3" => "audio/mpeg",
    "mp4" => "video/mp4",
    "wasm" => "application/wasm",
    _ => "application/octet-stream",
}
}
```

Now, let me explain what we just did.

`std::path::Path::new(path).extension()` extracts the file extension. `.and_then(|e| e.to_str())` converts the `OsStr` to a regular `&str`. If there is no extension, we default to an empty string. The `match` maps known extensions to MIME types. Text-based formats get `charset=utf-8` appended. Everything else falls through to `application/octet-stream`, which tells the browser to download the file rather than try to display it.

### Request Parsing

Now we need to parse raw HTTP bytes into a `Request` struct. HTTP/1.1 request format:

```bash
GET /index.html HTTP/1.1\r\n
Host: localhost:8080\r\n
User-Agent: curl/8.0\r\n
Accept: */*\r\n
\r\n
```

The first line is the request line: METHOD, a space, the path, a space, `HTTP/1.1`, then `\r\n`. Each header is `Key: Value\r\n`. The headers end with a blank line `\r\n\r\n`. There may be a body after the blank line if `Content-Length` is present, but for our server we only care about GET and HEAD requests, which have no body.

```rust
fn parse_request(data: &[u8]) -> Option<Request> {
let text = std::str::from_utf8(data).ok()?;
let mut lines = text.lines();

// Parse request line: METHOD PATH HTTP/1.1
let request_line = lines.next()?;
let mut parts = request_line.split_whitespace();
let method = parts.next()?.to_string();
let path = parts.next()?.to_string();
let _version = parts.next()?;

// Parse headers
let mut headers = HashMap::new();
for line in &mut lines {
    if line.is_empty() {
        break; // end of headers
    }
    if let Some((key, value)) = line.split_once(':') {
        headers.insert(key.trim().to_lowercase(), value.trim().to_string());
    }
}

Some(Request {
    method,
    path,
    headers,
})
}
```

Now, let me explain what we just did.

`std::str::from_utf8(data)` converts the raw bytes to a string slice. This parser assumes UTF-8 input for simplicity. Production HTTP servers operate on raw bytes and do not require requests to be valid UTF-8.

`text.lines()` splits the input on `\n` (and strips `\r` automatically). The first line is the request line. We split it on whitespace to extract the method, path, and HTTP version.

The `for line in &mut lines` loop reads header lines until we hit an empty line. `line.split_once(':')` splits on the first colon (header values can contain colons, so we only split on the first one). We store the key in lowercase for case-insensitive lookups. If a header is malformed (no colon), we skip it silently.

### Path Validation

HTTP clients can send malicious paths that attempt to escape the server's root directory. For example, `GET /../../../etc/passwd HTTP/1.1` should be rejected:

```rust
fn validate_path(raw_path: &str) -> Option<String> {
let path = raw_path.trim_start_matches('/');

// Reject paths with ".." to prevent directory traversal
if path.contains("..") {
    return None;
}

// Default to index.html for directory requests
if path.is_empty() || path.ends_with('/') {
    Some(format!("{}index.html", path))
} else {
    Some(path.to_string())
}
}
```

Now, let me explain what we just did.

We strip the leading `/` because the path will be joined with the serve directory later. The `..` check catches obvious traversal attempts, but the canonicalization check later is the actual security boundary. An empty path (the root `/`) or a path ending in `/` is treated as a directory request and defaults to `index.html`. So `GET / HTTP/1.1` serves `/index.html` and `GET /blog/ HTTP/1.1` serves `/blog/index.html`.

### The File Server Function

Now let's build the function that handles GET and HEAD requests. This function is async because it reads files from disk using Tokio's async filesystem:

```rust
use std::path::PathBuf;
use tokio::fs;
use tokio::io::AsyncReadExt;

async fn serve_file(
path: &str,
serve_dir: &PathBuf,
head_only: bool,
) -> Response {
let validated_path = match validate_path(path) {
    Some(p) => p,
    None => {
        let mut resp = Response::new(400, "Bad Request");
        return resp.with_body_text("400 Bad Request: Invalid path".to_string());
    }
};

let file_path = serve_dir.join(&validated_path);

// Canonicalize the path and verify it stays within serve_dir
let canonical = match file_path.canonicalize() {
    Ok(p) => p,
    Err(_) => {
        let mut resp = Response::new(404, "Not Found");
        return resp.with_body_text("404 Not Found".to_string());
    }
};

let serve_dir_canonical = match serve_dir.canonicalize() {
    Ok(p) => p,
    Err(_) => {
        let mut resp = Response::new(500, "Internal Server Error");
        return resp.with_body_text("500 Internal Server Error".to_string());
    }
};

if !canonical.starts_with(&serve_dir_canonical) {
    let mut resp = Response::new(403, "Forbidden");
    return resp.with_body_text("403 Forbidden".to_string());
}

let content_type = guess_content_type(&validated_path);

if head_only {
    // For HEAD, we only need metadata, not the body
    let metadata = match fs::metadata(&canonical).await {
        Ok(m) => m,
        Err(_) => {
            let mut resp = Response::new(404, "Not Found");
            return resp.with_body_text("404 Not Found".to_string());
        }
    };

    let mut resp = Response::new(200, "OK");
    resp.headers
        .insert("Content-Length".to_string(), metadata.len().to_string());
    resp.headers
        .insert("Content-Type".to_string(), content_type.to_string());
    return resp;
}

// For GET, read the file
let mut file = match fs::File::open(&canonical).await {
    Ok(f) => f,
    Err(_) => {
        let mut resp = Response::new(404, "Not Found");
        return resp.with_body_text("404 Not Found".to_string());
    }
};

let mut body = Vec::new();
if let Err(_) = file.read_to_end(&mut body).await {
    let mut resp = Response::new(500, "Internal Server Error");
    return resp.with_body_text("500 Internal Server Error".to_string());
}

Response::new(200, "OK").with_body(body, content_type)
}
```

Now, let me explain what we just did.

This function uses multiple layers of path validation. First, `validate_path` rejects paths with `..`. Then we join the validated path with the serve directory. Then we canonicalize both the requested path and the serve directory. `canonicalize` resolves symlinks and normalizes the path to an absolute form. We then check that the canonical requested path starts with the canonical serve directory. This is defense-in-depth: even if a crafted path somehow bypasses the `..` check, the canonicalization check catches it.

The `head_only` parameter determines whether this is a GET or HEAD request. HEAD should return the same headers as GET would have returned, except without the message body. We return a 200 response with `Content-Length` set but no body. For GET, we open the file with Tokio's `fs::File::open`, read the entire contents into a `Vec<u8>` using `read_to_end`, and return a 200 response with the full body.

Tokio exposes an async API for file operations. Internally these operations are often executed on a dedicated blocking thread pool. These yield the task while the disk I/O is in progress, allowing other connections to be handled.

Note that `file_path.canonicalize()` and `serve_dir.canonicalize()` are blocking operations from `std`. For a simple server this is fine because canonicalization is fast. A production server might use Tokio's `spawn_blocking` for these.

### The Connection Handler

Now we tie everything together. Each incoming connection is handled by this function:

```rust
use std::sync::Arc;
use tokio::io::AsyncWriteExt;
use tokio::net::TcpStream;

async fn handle_connection(mut socket: TcpStream, serve_dir: Arc<PathBuf>) {
let mut buf = vec![0u8; 8192];

// Read data from the socket
let n = match socket.read(&mut buf).await {
    Ok(0) => return, // client closed immediately
    Ok(n) => n,
    Err(_) => return,
};

// Parse the request
let request = match parse_request(&buf[..n]) {
    Some(req) => req,
    None => {
        let response = Response::new(400, "Bad Request")
            .with_body_text("400 Bad Request: Unable to parse request".to_string());
        let _ = socket.write_all(&response.to_bytes()).await;
        return;
    }
};

println!("{} {} {} -> {}", 
    request.method, 
    request.path,
    request.headers.get("host").map(|h| h.as_str()).unwrap_or("-"),
    socket.local_addr().unwrap()
);

// Route the request
let response = match request.method.as_str() {
    "GET" => serve_file(&request.path, &serve_dir, false).await,
    "HEAD" => serve_file(&request.path, &serve_dir, true).await,
    _ => {
        let mut resp = Response::new(405, "Method Not Allowed");
        resp.headers
            .insert("Allow".to_string(), "GET, HEAD".to_string());
        resp.with_body_text("405 Method Not Allowed".to_string())
    }
};

// Send the response
let _ = socket.write_all(&response.to_bytes()).await;
}
```

Now, let me explain what we just did.

The function takes ownership of the `TcpStream` (consuming it so no one else can read or write to it) and an `Arc<PathBuf>` for the serve directory. The `Arc` lets us share the directory path across all connection tasks without cloning the string.

We allocate a fixed-size buffer of 8 KiB on the stack. This should be enough for most HTTP requests (which are typically under 1 KiB). If a request is larger than 8 KiB, `read` will fill the buffer and the rest of the request stays in the socket's kernel buffer. For our simple server, we only need the request line and headers, which are always at the start, so 8 KiB is more than enough.

`socket.read(&mut buf).await` reads whatever data is available into the buffer. This is the async version of the `read` we used in the thread pool article. If the client sends nothing (`Ok(0)`), we return immediately.

We parse the request from the filled portion of the buffer. If parsing fails, we send a 400 Bad Request and return.

We print a simple access log showing the method, path, host header, and local address of the connection.

Finally, we route the request based on the HTTP method. GET and HEAD go to `serve_file`. Everything else gets a 405 Method Not Allowed with an `Allow` header listing supported methods.

The response is serialized with `response.to_bytes()` and written back to the socket with `socket.write_all(...).await`. We ignore the result of `write_all` because if the client has disconnected, there is nothing we can do.

### The Main Function

```rust
use tokio::net::TcpListener;

#[tokio::main]
async fn main() -> tokio::io::Result<()> {
let args: Vec<String> = std::env::args().collect();

let (port, serve_dir) = if args.len() >= 3 {
    (args[1].clone(), PathBuf::from(&args[2]))
} else if args.len() == 2 {
    // Only port provided, serve from current directory
    (args[1].clone(), PathBuf::from("."))
} else {
    eprintln!("Usage: cargo run -- <port> [serve_directory]");
    eprintln!("Example: cargo run -- 8080 ./public");
    std::process::exit(1);
};

// Ensure we have an absolute path for canonicalization
let serve_dir = match serve_dir.canonicalize() {
    Ok(p) => p,
    Err(e) => {
        eprintln!("Error: Cannot access directory '{}': {}", serve_dir.display(), e);
        std::process::exit(1);
    }
};

let serve_dir = Arc::new(serve_dir);

let addr = format!("127.0.0.1:{}", port);
let listener = TcpListener::bind(&addr).await?;

println!("Server listening on http://{}", addr);
println!("Serving files from: {}", serve_dir.display());
println!("Press Ctrl+C to stop.\n");

loop {
    match listener.accept().await {
        Ok((socket, addr)) => {
            let dir = Arc::clone(&serve_dir);
            tokio::spawn(async move {
                println!("Connection accepted from {}", addr);
                handle_connection(socket, dir).await;
            });
        }
        Err(e) => {
            eprintln!("Error accepting connection: {}", e);
        }
    }
}
}
```

Now, let me explain what we just did.

We parse command-line arguments. The first argument is the port number. The second (optional) argument is the directory to serve files from. If no directory is specified, we serve from the current directory.

We canonicalize the serve directory immediately. This converts it to an absolute path (so we can validate that all requests stay within it) and also verifies that the directory exists and is accessible. If it is not, we print an error and exit.

The serve directory is wrapped in `Arc::new(...)` for sharing across spawned tasks. Each connection handler gets its own `Arc::clone(&serve_dir)`. We used this exact pattern in the Thread Pool article. The only difference is we are now sharing a `PathBuf` instead of an `Arc<Mutex<Receiver<Job>>>`.

The main loop is identical in structure to the async echo server we wrote earlier. It calls `listener.accept().await` to wait for the next connection. When a connection arrives, it clones the `Arc` and spawns a new Tokio task to handle it. The spawned task runs `handle_connection`, which reads the request, routes it, and writes the response.

The `loop` runs forever. To stop the server, press Ctrl+C. The Tokio runtime automatically handles the interrupt signal and shuts down.

### Static File Serving Details

Let's trace through an example request to understand the complete flow. A browser requests `http://localhost:8080/css/style.css` from a server serving the `./public` directory:

1. TCP connection arrives. `listener.accept().await` returns a `(TcpStream, SocketAddr)`.
2. `tokio::spawn` creates a new lightweight task.
3. `handle_connection` reads from the socket. The raw bytes look like:
```bash
   GET /css/style.css HTTP/1.1\r\n
   Host: localhost:8080\r\n
   Connection: keep-alive\r\n
   \r\n
```
4. `parse_request` extracts method `GET`, path `/css/style.css`, and headers.
5. `validate_path("/css/style.css")` strips the leading `/`, yielding `css/style.css`. No `..` detected. No trailing slash, so no `index.html` appended.
6. `serve_file` is called with `path = "/css/style.css"`, `serve_dir = "./public"`, `head_only = false`.
7. The file path becomes `./public/css/style.css`.
8. Both paths are canonicalized. If `./public` resolves to `/home/user/project/public` and `./public/css/style.css` resolves to `/home/user/project/public/css/style.css`, the prefix check passes.
9. `guess_content_type("css/style.css")` looks at the `css` extension and returns `text/css; charset=utf-8`.
10. `fs::File::open` opens the file. If it exists, `read_to_end` reads its contents into a `Vec<u8>`.
11. A `Response` is built with status 200, `Content-Type: text/css; charset=utf-8`, `Content-Length: <size>`, and the body.
12. `to_bytes()` serializes everything and writes it back to the socket.
13. The connection closes (because `Connection: close` is always sent).

## Running the Project

Create a test directory with some files:

```bash
mkdir -p www/css www/images
echo '<!DOCTYPE html><html><body><h1>Hello Async!</h1></body></html>' > www/index.html
echo 'body { font-family: sans-serif; }' > www/css/style.css
echo 'Hello, World!' > www/hello.txt
```

Now start the server:

```bash
cargo run -- 8080 ./www
```

You should see:

```bash
Server listening on http://127.0.0.1:8080
Serving files from: /Users/you/async_http/www
Press Ctrl+C to stop.
```

### Testing with curl

Open another terminal and test:

```bash
curl -v http://localhost:8080/
```

Expected output:

```bash
* Connected to localhost (127.0.0.1) port 8080
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Length: 62
< Content-Type: text/html; charset=utf-8
< Connection: close
<
<!DOCTYPE html><html><body><h1>Hello Async!</h1></body></html>
```

The server log shows:

```bash
Connection accepted from 127.0.0.1:54321
GET / localhost:8080 -> 127.0.0.1:8080
```

Test the CSS file:

```bash
curl http://localhost:8080/css/style.css
```

Expected:

```bash
body { font-family: sans-serif; }
```

Test an HTML file:

```bash
curl http://localhost:8080/hello.txt
```

Expected:

```bash
Hello, World!
```

Test a 404:

```bash
curl -v http://localhost:8080/nonexistent.html
```

Expected:

```bash
< HTTP/1.1 404 Not Found
< Content-Length: 14
404 Not Found
```

Test a HEAD request:

```bash
curl -I http://localhost:8080/
```

Expected:

```bash
HTTP/1.1 200 OK
Content-Length: 62
Content-Type: text/html; charset=utf-8
Connection: close
```

No body is returned. Only the headers.

Test method not allowed:

```bash
curl -v -X POST http://localhost:8080/
```

Expected:

```bash
< HTTP/1.1 405 Method Not Allowed
< Allow: GET, HEAD
405 Method Not Allowed
```
### Concurrent Connection Test

Let's verify the server handles concurrent connections. In one terminal, create a small script to make parallel requests:

```bash
for i in $(seq 1 10); do
curl -s -o /dev/null -w "Request $i: HTTP %{http_code}\n" http://localhost:8080/ &
done
wait
```

Or if you have `ab` (Apache Bench) installed:

```bash
ab -n 1000 -c 50 http://localhost:8080/
```

All requests complete successfully. The server handles them concurrently on Tokio's worker threads. The access log shows interleaved connection messages:

```bash
Connection accepted from 127.0.0.1:54322
GET / localhost:8080 -> 127.0.0.1:8080
Connection accepted from 127.0.0.1:54323
GET / localhost:8080 -> 127.0.0.1:8080
Connection accepted from 127.0.0.1:54324
GET / localhost:8080 -> 127.0.0.1:8080
GET / localhost:8080 -> 127.0.0.1:8080
```

Notice the interleaving: new connections are accepted while previous requests are still being processed. The async runtime multiplexes I/O across all concurrent connections.

## How It All Fits Together

Let's trace the lifecycle of a connection through the async runtime:

1. **Binding**: `TcpListener::bind().await` registers the listening socket with the reactor. The reactor uses kqueue (macOS) or epoll (Linux) to watch for new connections.

2. **Accepting**: `listener.accept().await` yields the main task. The runtime can execute other ready tasks. When a TCP handshake completes, the reactor wakes the main task. It resumes and gets the new `TcpStream`.

3. **Spawning**: `tokio::spawn(async move { ... })` creates a new task. The task is placed in the executor's ready queue. The task does not run yet. The main loop goes back to waiting for the next connection.

4. **Reading**: When the executor picks up the spawned task, it runs it until it hits `.await` on `socket.read(...)`. The read future registers interest in the socket with the reactor and yields. The executor picks up another ready task.

5. **Data arrives**: The reactor detects that the socket has data. It wakes the waiting task. The executor puts it back in the ready queue. When the task runs again, `read` returns the bytes.

6. **Parsing**: Parsing the request is CPU-only work. There are no `.await` points, so the task continues running until it reaches the next async operation.

7. File reading: `fs::File::open` and `read_to_end` expose an async API. Tokio typically executes the underlying file operations on a dedicated blocking thread pool while the task yields.

8. **Writing**: `socket.write_all(...).await` registers write interest with the reactor. When the socket is writable, the reactor wakes the task.

9. **Completion**: The spawned task finishes. The `TcpStream` is dropped, closing the socket. The `Connection: close` header informs the client that the connection will not be reused.

Throughout this whole process, there are only a few OS threads. On a 4-core machine, Tokio's default runtime uses 4 worker threads. But it can handle thousands of concurrent connections because most connections spend most of their time waiting for I/O, and during that wait the thread they were running on is free to run other tasks.

Compare this to the Thread Pool article where we needed one OS thread per worker, and each worker could only handle one job at a time. For an HTTP server, the async model is dramatically more scalable.

## What We Skipped

There are a few things I am intentionally skipping in this article:

- **HTTP keep-alive**: Our server closes every connection after one response. Real web servers reuse connections for multiple requests. Implementing this requires buffered reading, timeout management, and tracking connection state across multiple requests. It adds significant complexity for a learning project.
- **Request body parsing**: We only handle GET and HEAD. POST, PUT, and PATCH requests include a body. Handling bodies requires respecting `Content-Length` and `Transfer-Encoding: chunked`.
- **Streaming responses**: For large files, reading the entire file into memory (as we do with `read_to_end`) is wasteful. A production server would use `tokio::io::copy` or `sendfile` to stream the file to the socket without loading it all into RAM.
- **Multipart and URL-encoded form parsing**: Common patterns for web applications but out of scope for this article.
- **Async graceful shutdown**: Our server stops when Ctrl+C is pressed, but it does not drain in-flight requests before exiting. Tokio provides `tokio::signal::ctrl_c()` and `CancellationToken` for this.
- **HTTPS/TLS**: Setting up `rustls` with `tokio-rustls` is a separate topic.
- **HTTP routing**: Real servers match paths against patterns and extract parameters. In our server, paths map directly to filesystem paths plus `index.html` for directories.
- **Custom error pages**: We return plain text for all error responses. A real server would serve custom HTML pages.

> Everything we skipped in this article exists for a reason in real-world servers. Rather than introducing that complexity all at once, we focused on the async runtime, TCP networking, HTTP fundamentals, and static file serving. In the upcoming Rust backend series, we'll build production-style services with Axum and revisit many of these topics, including routing, request extraction, middleware, request bodies, graceful shutdown, and other features commonly used in modern web applications.

## Conclusion

In this post, you learned about async/await in Rust. You learned the difference between OS threads (preemptive concurrency) and async tasks (cooperative concurrency). You learned about futures (the `Future` trait and how the `poll` method works), async functions and blocks (which compile to state machines), and how `.await` yields control back to the runtime.

You learned about the Tokio runtime: the reactor (which watches I/O events using kqueue/epoll) and the executor (which schedules ready tasks on a small pool of worker threads). You learned `#[tokio::main]`, `tokio::spawn`, `tokio::join!`, and `tokio::select!`.

You built an Async HTTP/1.1 Server from scratch. The server listens on a TCP port using `tokio::net::TcpListener`, accepts connections concurrently with `tokio::spawn`, parses raw HTTP/1.1 request lines and headers manually (no framework), routes GET and HEAD requests to a static file server, detects Content-Type from file extensions, validates paths against directory traversal attacks, and returns proper HTTP status codes with Content-Length headers.

This ties together everything from the Thread Pool article. Where threads give us preemptive multitasking at the OS level, async gives us cooperative multitasking within a single thread. The same primitives apply: `Arc` for shared ownership, `spawn` for concurrent execution, and channels for message passing (Tokio provides their async versions).

In the next article, we will learn about SQLx and build a Book Library CLI with SQLite, compile-time checked queries, connection pooling, schema migrations, and full CRUD. See you soon.

If you like reading this, please subscribe and share this with others. It'll really help me and motivate me to keep publishing more such articles.

# Learn Rust Basics By Building a Brainfuck Interpreter

> In this post, we are going to build a brainfuck language interpreter in Rust

Source: https://blog.sheerluck.dev/posts/rust/learn-rust-basics-by-building-a-brainfuck-interpreter/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-rust-basics-by-building-a-brainfuck-interpreter/index.md

Hi, In this series we are going to learn Rust programming language by building exciting projects per article. For every article, we will first learn some concepts and then build a mini project. I promise you these mini projects will be exciting. The only prerequisite is, you should know any one programming language like just basics of it as this series will focus on teaching rust and not programming from zero. 

My motive is to explain how to do a certain thing in Rust, so I won't explain how a loop works or how a function works. I'll just explain how to work with variables, functions, loops etc in Rust but I'll explain Rust specific concepts in detail.

In this post, we are going to **build a brainfuck language interpreter in Rust**. But before that, we will learn about variables and mutability, scalar types, compound types, functions, basic string handling (we will dive deep in the next article), `println!` & basic macros. We will also learn about cargo and control flow in rust like if and else blocks, loops, and match (this is interesting). 

You can get the source code from **[here](https://github.com/MrSheerluck/brainfuck-interpreter-in-rust)**

Let's start, I can't wait.

## How Rust Works and Cargo
### What happens when you write a Rust program?
Rust is a **compiled language**. This means before the program can run, it has to be translated from human readable Rust code into machine code that your CPU can directly execute. This translation is done by the **Rust compiler** called `rustc`.

### Cargo
Cargo is Rust's official build system and package manager. It does multiple things:
- creates new projects with a standard folder structure
- compiles the rust code by invoking `rustc` under the hood
- downloads and manages external libraries(in rust world we call them **crates**)
- run tests, benchmarks and documentation generation

When you'll install Rust (via `rustup`), Cargo comes with it automatically.

Let me show you some of the common commands that you'll be using constantly:

| Command                  | What it does                                  |
| ------------------------ | --------------------------------------------- |
| `cargo new project_name` | creates a new project folder with boilerplate |
| `cargo run`              | compiles and immediately runs the binary      |
| `cargo check`            | checks for errors without producing a binary  |
| `cargo build`            | compiles your project, produces a binary      |

### Project Structure
Let's try to create a project with Cargo:
Open up your terminal and run the following command:
```bash
cargo new hello
```
It'll create a new Rust project called `hello`. Now, open the project folder in your preferred editor and you'll see a folder structure like this:
```bash
hello/
├── Cargo.toml
└── src/
└── main.rs
```
Let me explain what each of these files are:
#### `Cargo.toml`
This is the **manifest file** for your project. It's written in TOML format (a simple config format). It contains:
```toml
[package]
name = "hello"
version = "0.1.0"
edition = "2024"

[dependencies]
```
- `name` - the name of your project
- `version` - your project's version
- `edition` - which edition of Rust you are using
- `[dependencies]` - this is where you list down the external crates your project needs. Initially its empty.

#### `src/main.rs`
This is the entry point of every Rust program. Cargo generates this for you:
```rust
fn main() {
	println!("Hello, world!");
}
```

`fn` is the keyword to define a function in Rust. `main` is the name of the function. The curly braces `{}` defines the body of the function.

`println!("Hello, world!");` - `println!` is a `macro`, not a regular function. Notice there is a `!` right before the parentheses `!()`. Macros are just code that generates other code at compile time. You don't need to understand how they work internally right now. Just keep in mind that `println!` prints a line of text to the terminal followed by a new line.

The text inside the quotes is a **string literal**. if you want to print values inside the string, you use `{}` as a placeholder:
```rust
println!("The value is {}", 42);
```

## Variables and Mutability
### Declaring Variables
In Rust, you declare a variable with `let`:
```rust
let x = 5;
```
This creates a variable named `x` and binds the value 5 to it.

### Variables are Immutable by Default
This is one of Rust's core design decisions. Once you bind a value to a variable, you cannot change it unless you explicitly say you want to.
```rust
let x =5;
x = 6; // ERROR: cannot assign twice to immutable variable
```

To make a variable mutable, you need to add `mut` keyword:
```rust
let mut x = 5;
x = 6; // OK this is fine
```

### Shadowing
Rust allows you to **shadow** a variable. This means you can declare a new variable with the same name which replaces the previous one:
```rust
let x = 5;
let x = x + 1; // this is a new `x`, not mutating the old one
let x = x * 2;
println!("{}", x); // prints 12
```

You need to understand that shadowing is different than `mut`. With shadowing, you are creating a brand new variable (and can even change its type). But with `mut`, you're modifying the same variable in place.

## Data Type
Rust is **statically typed**. This means every variable has a type known at compile time. Most of the time the compiler can infer the type from context, so you don't have to write it explicitly. But you can always annotate it:
```rust
let x: i32 = 5;
```
The `: i32` after the variable name is the **type annotation**.

### Integer Types
Rust has multiple integer types. They differ in size (how many bits they use) and whether they can hold negative numbers:

| Type  | Size            | Range                                         |
| ----- | --------------- | --------------------------------------------- |
| i8    | 8-bit signed    | -128 to 127                                   |
| i16   | 16-bit signed   | -32,768 to 32,767                             |
| i32   | 32-bit signed   | ~-2 billion to ~2 billion                     |
| i64   | 64-bit signed   | very large range                              |
| u8    | 8-bit unsigned  | 0 to 255                                      |
| u16   | 16-bit unsigned | 0 to 65,535                                   |
| u32   | 32-bit unsigned | 0 to ~4 billion                               |
| u64   | 64-bit unsigned | ver large range                               |
| usize | pointer-sized   | depends on your OS (64-bit on 64-bit systems) |

`i32` is the default integer type when Rust infers. `usize` is special because its used for indexing into collections (arrays, vectors etc) because its size matches memory address size of your machine.

### Boolean
```rust
let is_active: bool = true;
let is_done = false;
```
Only two values are there for boolean type in Rust: `true` or `false`

### Character
```rust
let c: char = 'A';
```
Rust's char type represents a single Unicode scalar value. It uses single quotes (double quotes are for strings). A char is 4 bytes in Rust, not 1, because it can hold any unicode character.

### Floating Point
```rust
let f: f64 = 3.14;
```
`f32` and `f64` are 32-bit and 64-bit floating point numbers. The default is `f64` because on modern hardware it's roughly as fast as `f32` but more precise.

## Compound Types
### Tuples
A tuple groups multiple values of potentially different types into one compound value. Let me show you an example:
```rust
let tup: (i32, f64, bool) = (500, 6.4, true);
```
One thing to note about tuples is that they have a fixed length. Once they are declared, then you cannot add or remove elements to the tuple.

But you can access elements from tuple, to do that you can use the **dot notation** with the index:
```rust
let x = tup.0; // 500
let y = tup.1 // 6.4
let x = tup.2 // true
```

You can also destructure a tuple. Destructuring means unpacking the tuple elements in individual variables:
```rust
let (a,b,c) = tup;
println!("{}", a); // 500
```

### Arrays
An Array holds multiple values of the same type, with a fixed length.
```rust
let arr: [i32; 5] = [1,2,3,4,5];
```
The type annotation `[i32; 5]` means its an array of `i32` with exactly 5 elements.

To access elements in array, you can use these `[]`:
```rust
let first = arr[0]; // 1
let second = arr[1]; // 2
```

> If you try to access an index that's out of bounds, then Rust will **panic** ( crash at runtime with an error message )

You can also create an array where every element is the same value:
```rust
let zeros = [0u8; 25]; // 25 elements all set to 0
```
This means, create an array of type `u8` with 25 elements and intialize all the elements to 0.

## Functions
### Defining Functions
```rust
fn add(x: i32, y: i32) -> i32 {
	x + y
}
```
To define a function in rust you need to use the `fn` keyword. `add` is the function's name. `(x: i32, y: i32)` are the parameters. Parameter types are always required. `-> i32` this means the function returns an `i32` type and the body is inside the `{}`.

### Return Values
In Rust, the last expression is a function body is automatically returned, that's why we didn't have a return keyword in the above function.

> **Statements** end with a semicolon `;` and do not produce a value but **Expressions** do not end with a semicolon and produce a value.

If you add a semicolon to the last line, it becomes a statement, produces no value and the function now returns `()` (this is called a unit type, basically nothing) instead of `i32` and this would be a compilation error.

You can also keep it simple and use the return keyword:
```rust
fn check(x: i32) -> i32 {
	if x < 0 {
		return 0; // explicit return
	}
	x // implicit return
}
```

## Control Flow
### If/Else 
```rust
let number = 7;
if number < 5 {
	println!("less than 5");
} else if number == 5 {
	println!("exactly 5");
} else {
	println!("greater than 5");
}
```

In Rust, the condition doesn't need parentheses but the body must be inside the curly braces.

### `If` is an Expression
This is important to remember that in Rust, `if` is not just a statement, its an expression that produces a value:
```rust
let result = if number % 2 == 0 { "even" } else { "odd" };
```
Note that both the branches should produce the same type otherwise the compilation will fail.

### Loop
`loop` runs a block of code forever until you explicitly break out of it:
```rust
let mut count = 0;
loop {
	count += 1;
	if count == 5 {
		break;
	}
}
```

`loop` can also return a value through `break`:
```rust
let result = loop {
	count += 1;
	if count == 10 {
		break count * 2; // returns this value from the loop
	}
}
```

### While Loop
While loop runs as longs as a condition is true:
```rust
let mut n = 3;
while n > 0 {
	println!("{}", n);
	n -= 1;
}
```

### For Loop
The most common loop in Rust. The `for` loop iterates over a collection or a range:
```rust
// iterating over an array
let arr = [10, 20, 30];
for element in arr {
	println!("{}", element);
}

// iterating over a range
for i in 0..5 {
	println!("{}", i); // prints 0, 1, 2, 3, 4
}

// 0..=5 includes 5 (inclusive on both ends)
for i in 0..=5 {
	println!("{}", i); // prints 0, 1, 2, 3, 4, 5
}
```

`0..5` is a range, it starts from 0 and goes up to but not including 5. But `0..=5` is an inclusive range that includes 5.

## Match
`match` is Rust's pattern matching construct. It compares a value against a series of **patterns** and executes the code for the first pattern that matches:
```rust
let x = 3;

match x {
	1 => println!("one"),
	2 => println!("two"),
	3 => println!("three"),
	4 => println!("four"),
	_ => println!("other")
}
```

Each line inside the `match` is called an **arm**: `pattern => code`
`_` is the wildcard pattern, it matches anything. It's used as a catch all default case. One thing you need to remember is that `match` is exhaustive. This means you must cover all the possible values. If you don't the compiler will reject your code.

Like `if`, `match` is also an expression and can return a value:
```rust
let name = match x {
	1 => "one",
	2 => "two",
	_ => "other",
};
```
you can also match multiple patterns with `|`:
```rust
match x {
	1 | 2 => println!("one or two"),
	3..=9 => println!("three through nine"),
	_ => println!("something else"),
}
```

## Vectors (Brief Introduction)
Arrays have a fixed size known at compile time but sometimes you need a collection that can grow. This is where you can use a `Vec<T>`:
```rust
let mut v: Vector<char> = Vec::new();
v.push('a');
v.push('b');
println!("{}", v.len()); // 2
```

- `Vec::new()` creates an empty vector
- `.push(value)` adds an element to the end
- `.pop()` removes and returns the last element
- `.len()` returns the number of elements

## Basic String Types
Rust has two main string types:
- `&str` this is a string slice. It is an immutable reference to a sequence of UTF-8 bytes. String literals like `"hello"` have type `&str`.
- `String` this is a heap allocated growable string.
We will dive deep into these in the next article but for now just understand that if you need to have string literals, just use `&str`. If you need to build a string dynamically, then use the `String` type.

To iterate over each character of a string, you can use `chars()`:
```rust
let s = "hello";
for c in s.chars() {
	println!("{}", c);
}
```

This is great, now you know everything to build your project. We are going to build a **Brainfuck Interpreter** in Rust. 

## The Brainfuck Interpreter
Before writing any code, lets fully understand what we've building.

### What is Brainfuck?
Brainfuck is a programming language with only 8 instructions. The entire language fits in 8 characters. Despite that, it can compute anything a normal programming language can.

A brainfuck program operates on:
- A memory tape - an array of cells, each holding a number (a `u8`, so values 0-255). 
- A data pointer - an index that points to the current cell on the tape. It starts at 0 (the leftmost cell)
- A program - a string of characters, most of which are instructions

### The 8 Instructions

| Character | What it does                                                 |
| --------- | ------------------------------------------------------------ |
| >         | Move the data pointer one cell to the right                  |
| <         | Move the data pointer one cell to the left                   |
| +         | Increment the value at the current cell by 1                 |
| -         | Decrement the values at the current cell by 1                |
| .         | Output the current cell's value as an ASCII character        |
| ,         | Read one byte of input and store it in the current cell      |
| [         | If the current cell is 0, jump forward to the matching ]     |
| ]         | If the current cell is non-zero, jump back to the matching [ |

Every other character in a Brainfuck program is a comment, these gets simply ignored.

### How Loop Works
`[` and `]` together form a loop. Let me explain:
- When you hit `[:` check the current cell. If its `0`, skip everything until the matching `]`. If its non zero, enter the loop body.
- When you hit `]:` check the current cell. If its non-zero, jump back to the matching `[`. If its 0, exit the loop.
This is basically a `while (cell != 0) {...}` loop.

## What We Need to Build
1. A memory tape
2. A data pointer
3. A way to iterate through the program instruction by instruction basically a program counter (pc) as a `usize`
4. Bracket matching when we hit `[` or `]`, we need to find the matching counterpart. We will precompute this into a map before running.

### Create the Project
Let's start build our brainfuck interpreter. Open up your terminal and create a new project:
```shell
cargo new brainfuck-interpreter
cd brainfuck-interpreter
```

Open the project in your preferred editor and open the `src/main.rs` file and delete all the code

### The Memory Tape and Pointers
Type this into `src/main.rs`:
```rust
fn main() {
let program = "++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.";

let mut tape = [0u8; 30000];
let mut dp: usize = 0;
let mut pc: usize = 0;
}
```
Now, let me explain what we just did:
- `let program = "..."` this is a brainfuck program that prints "Hello World!". It's just a `&str`(string literal). Our interpreter will read through it character by character.
- `let mut tape = [0u8; 30000]` this is our memory tape. This creates an array of 30,000 elements, each of them initialized to `0`, and each of them is of type `u8`(values 0-255). We need `mut` because instructions `+` and `-` will modify the cells.
- `let mut dp: usize = 0` dp stands for **data pointer**. It's the index into tape that points to the current cell. It starts at 0. Its `usize` because array indices in Rust must be `usize`.
- `let mut pc: usize = 0` pc stands for **program counter**. Its the index into program pointing at the current instruction. It starts at 0.

### Precomputing Bracket Matches
Before we run the program, we'll precompute where every `[` matches with its `]` and vice versa. This way, when we need to jump, we just do a lookup instead of scanning through the program every time.
We'll store this in a `Vec` where the index is the position of `[` or `]` and the value is the position of its matching counterpart.

Add this before main, as a separate function:
```rust
fn build_bracket_map(program: &str) -> Vec<usize> {
let bytes = program.as_bytes();
let len = bytes.len();
let mut map = vec![0usize; len];
let mut stack: Vec<usize> = Vec::new();

for i in 0..len {
    match bytes[i] {
        b'[' => {
            stack.push(i);
        }
        b']' => {
            let open = stack.pop().expect("Unmatched ]");
            map[open] = i;
            map[i] = open;
        }
        _ => {}
    }
}
if !stack.is_empty() {
    panic!("Unmatched [");
}
map
}
```

Now, let me explain everything:
- `fn build_bracket_map(program: &str) -> Vec<usize>` this function takes a `&str`(the program text) and returns a `Vec<usize>`(the bracket map)
- `let bytes = program.as_bytes()` here `as_bytes()` converts the `&str` into a slice of raw bytes `&[u8]`. This lets us compare characters as byte values using `b'['` syntax(a byte literal). Its slightly more efficient than working with `char` here and brainfuck only uses ASCII characters so its completely valid.
- `let mut map = vec![0usize; len]` here `vec![value; len]` is a macro that creates a `Vec` with `count` elements all set to `value`. So this creates a vector of `len` zeroes. Here, every position starts as `0`.
- `let mut stack: Vec<usize> = Vec::new()` this is our stack for tracking open brackets. When we see a `[`, we push its position. When we see a `]`, we pop the most recent `[` position because that is its matching opening bracket.
- `for i in 0..len` loop here we iterate through every index of the program
	- `match bytes[i]` we match on the byte at position `i`
		- `b'[' => stack.push(i)` when we see an open bracket, we push its index onto the stack.
		- `b']'` when we see a close bracket
			- `stack.pop()` removes and returns the last pushed index(the matching `[`)
			- `.expect("Unmatched ]")` if the stack is empty, there's no matching `[`, so we crash with this error message. `.expect()` is a method on `Option` that either unwraps the value or panics with your message. We'll cover `Option` deeply later, but for now `pop()` returns `None` if the stack is empty and `expect()` handles that.
			- `map[open] = i` this means at the `[` position, the jump target is `]`
			- `map[i] = open` this means at the `]` position, the jump target is `[`
		- `_ => {}` for any other character, we will just ignore and do nothing
- `if !stack.is_empty() {panic!(...)}` after processing the whole program, if the stack still has entries, that means there are unmatched `[`. `panic!` crashes the program with a message

### The Main Execute Loop
Now back inside `main` function, after the variable declarations, call `build_bracket_map` and then write the execution loop:
```rust
fn main() {
let program = "++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.";

let mut tape = [0u8; 30000];
let mut dp: usize = 0;
let mut pc: usize = 0;

let bracket_map = build_bracket_map(program);
let bytes = program.as_bytes();

while pc < bytes.len() {
    match bytes[pc] {
        b'>' => dp += 1,
        b'<' => dp -= 1,
        b'+' => tape[dp] = tape[dp].wrapping_add(1),
        b'-' => tape[dp] = tape[dp].wrapping_sub(1),
        b'.' => print!("{}", tape[dp] as char),
        b',' => { /* input not needed for Hello World */ }
        b'[' => {
            if tape[dp] == 0 {
                pc = bracket_map[pc];
            }
        }
        b']' => {
            if tape[dp] != 0 {
                pc = bracket_map[pc];
            }
        }
        _ => {}
    }
    pc += 1;
}

println!();
}
```
Now, let me explain this:
- `while pc < bytes.len()` this keep executing as long as the program counter hasn't gone past the end of the program.
- `b'>' => dp += 1` moves the data pointer right, just incrementing the index
- `b'<' => dp -= 1` moves the data pointer left, just decrementing the index
- `b'+' => tape[dp].wrapping_add(1)` this increments the current cell. We use `.wrapping_add(1)` instead of `tape[dp] += 1` because our cells are `u8`(0-255). If the value is 255 and you add 1, a normal += would panic in debug mode due to integer overflow. The `.wrapping_add` instead wraps around to 0. This is standard brainfuck behaviour.
- `b'-' =? tape[dp].wrapping_sub(1)` this decrements the current cell. Same idea as add operation.
- `b'.' => print!("{}", tape[dp] as char)` this prints the current cell as an ASCII character. `tape[dp] as char` is a type cast from `u8` to `char` type. For example, the value `72` becomes `H`. `print!`(without `ln`) prints without creating a newline.
- `b','` we leave it as an empty block as we are not taking input dynamically.
- `b'['` if the current cell is `0`, then jump to the matching `]` by setting `pc = bracket_map[pc]`. The `pc += 1` at the bottom of the loop then moves us past the `]`.
- `b']'` if the current cell is non-zero, then jump back to the matching `[`
- `_ => {}` we are using this again to ignore any non instruction character
- `pc += 1` after processing each instruction, move the program counter to the next character
- `println!()` after the program finishes, it prints a newline so your terminal prompt appears on a fresh line.

Finally, we covered a bunch of stuff and also built our project. Now lets try running it and see it in action. In your terminal, type:
```shell
cargo run
```

You should see an output like:
```shell
Compiling brainfuck-interpreter v0.1.0 (/Users/.../.../.../brainfuck-interpreter)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.10s
 Running `target/debug/brainfuck-interpreter`
Hello World!
```

## Conclusion
In this post, you understood most of basic common things in Rust and build a simple brainfuck interpreter in Rust. In the next one, you are going to learn about ownership, borrowing and slices and build a **mini grep clone** that's gonna be fun. I hope to see you soon. Till then, goodbye!

# Learn Rust Closures By Building a Tiny Rule-Based Linter

> In this post, we are going to learn about closures and functional patterns in Rust. Once we cover all the concepts, we will build a Tiny Rule-based Text Linter

Source: https://blog.sheerluck.dev/posts/rust/learn-rust-closures-by-building-a-tiny-linter/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-rust-closures-by-building-a-tiny-linter/index.md

In this post, we are going to learn about closures and functional patterns in Rust. Once we cover all the concepts, we will build a Tiny Rule-based Text Linter where lint rules are closures stored in a `Vec<Box<dyn Fn>>`, pass behaviour as values, and auto-fix code. I'm really excited for this project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles.

The only prerequisite is that you have read the previous articles in this series, as I will assume you know ownership, borrowing, structs, enums, pattern matching, error handling, generics, traits, and lifetimes.

Get the source code [here](https://github.com/MrSheerluck/linter-rust)

## What is `Box<T>`
Before we talk about closures, we need to understand `Box<T>`. You saw `Box` briefly in the JSON parser when we talked about recursive enums. Let me explain it properly now.

A `Box<T>` is a heap-allocated value. When you write `let x = Box::new(42)`, Rust allocates space for `42` on the heap and stores a pointer to it on the stack. The `Box` owns the value, when the `Box` is dropped, the heap memory is freed.

```rust
let x = Box::new(42);
println!("{}", x); // 42
```
`Box::new(42)` allocates `42` on the heap and returns a `Box<i32>` that owns it. You can use `x` just like a regular `i32`. Rust dereferences it automatically when you access it. This is called "deref coercion."

Without `Box`, local bindings themselves are usually stack-allocated, though many types such as `Vec` and `String` may still allocate their contents on the heap internally. With `Box`, you explicitly move a value onto the heap while keeping a pointer on the stack.
## Why Do We Need Box
There are two main reasons to use Box:
1. Recursive types: A struct that contains itself would have infinite size on the stack. Putting it behind a `Box` gives it a fixed size (just the pointer). This is exactly what we did in the JSON parser with recursive `JsonValue` variants.
2. Trait objects: When you want to store values of different types in a single `Vec`, they must all be the same size. `Box` puts them on the heap so the `Vec` only stores fixed-size pointers.
## What Are Trait Objects
A trait object lets us work with values through a trait interface without knowing their concrete type at compile time.

```rust
trait Animal {
fn speak(&self);
}

struct Dog;
struct Cat;

impl Animal for Dog {
fn speak(&self) {
    println!("woof");
}
}

impl Animal for Cat {
fn speak(&self) {
    println!("meow");
}
}
```

`Dog` and `Cat` are completely different concrete types. Normally, a `Vec<T>` can only store a single concrete type:

```rust
let dogs: Vec<Dog> = vec![Dog, Dog];
```

If we try to mix `Dog` and `Cat` directly inside the same vector, Rust rejects it:

```rust
// ERROR
// let animals = vec![Dog, Cat];
```

Rust needs every element in a `Vec` to have the same concrete type and size at compile time.

Trait objects solve this problem.

```rust
let animals: Vec<Box<dyn Animal>> = vec![
Box::new(Dog),
Box::new(Cat),
];
```

`dyn Animal` means "Some type that implements the `Animal` trait."

The concrete type (`Dog` or `Cat`) is erased, and we interact with the value only through the trait methods defined by `Animal`.

Now we can iterate over the vector:

```rust
for animal in animals {
animal.speak();
}
```

Rust will call the correct implementation at runtime, either `Dog::speak` or `Cat::speak` depending on the concrete type behind the trait object.

This is called dynamic dispatch.
Under the hood, a trait object stores a pointer to the actual data along with a pointer to a vtable.
A vtable ("virtual method table") contains function pointers for the trait methods of the concrete type.

When we call:
```rust
animal.speak();
```

Rust looks up the correct function in the vtable and calls the appropriate implementation for the concrete type behind the trait object.

Trait objects are dynamically sized types (DSTs). Rust does not know their size at compile time, so they must be used behind some kind of pointer such as `Box<dyn Trait>`, `&dyn Trait`, or `Rc<dyn Trait>`.

This is why we later use `Box<dyn Fn>` for storing closures of different concrete types inside the same `Vec`.
## Closures
Now that you understand Box and dyn, let's learn about closures.

A closure is an anonymous function that can capture variables from its environment.
```rust
let x = 10;
let add_x = |n| n + x;
println!("{}", add_x(5)); // 15
```
`|n| n + x` is a closure. The parameter `n` goes between the pipes `|n|`, and the body `n + x` follows. The closure captures `x` from the surrounding scope and adds it to `n`. When we call `add_x(5)`, it returns `15`.

This is different from a regular function. A regular `fn` cannot access variables from the enclosing scope

## Closure Syntax
```rust
// Full syntax with type annotations
let closure = |n: i32| -> i32 { n + 1 };
// Inferred types (most common)
let closure = |n| n + 1;
// Multiple parameters
let add = |a, b| a + b;
// No parameters
let greet = || println!("hello");
```
Closures can have type annotations like regular functions, but usually Rust infers the types from how you use the closure. The pipes `||` enclose the parameter list, just like parentheses `()` in a regular function. Unlike functions, closures with a single expression can omit the curly braces.

## How Closures Capture Variables
Closures capture variables from their environment in three ways, matching the three borrowing modes:

| Capture Mode | How It Captures     | Closure Trait |
| ------------ | ------------------- | ------------- |
| `&T`         | Immutable reference | `Fn`          |
| `&mut T`     | Mutable reference   | `FnMut`       |
| `T`          | Ownership (move)    | `FnOnce`      |

Rust infers the capture mode automatically based on what the closure does with the captured variable.

```rust
let s = String::from("hello");
let print_it = || println!("{}", s);

let mut s2 = String::from("hello");
let mut push_world = || s2.push_str(" world");

let s3 = String::from("hello");
let consume = || drop(s3);
```
`print_it` only reads `s`, so Rust captures it by immutable reference and the closure implements `Fn`.

`push_world` mutates `s2`, so Rust captures it by mutable reference and the closure implements `FnMut`.

`consume` takes ownership of `s3` using `drop`, so the closure implements `FnOnce`. After the first call, `s3` is gone and the closure cannot be called again.

The traits form a hierarchy:
- `Fn` closures can be called many times without mutation
- `FnMut` closures can mutate captured state
- `FnOnce` closures may consume captured values
## The `move` Keyword
Sometimes you need to force a closure to take ownership of captured variables. This is essential when the closure will outlive the scope where it was created.
```rust
let s = String::from("hello");
let closure = move || {
println!("{}", s);
};
// println!("{}", s); // ERROR: s was moved into the closure
```
The `move` keyword before the pipes tells Rust: capture everything by value, even if the closure only reads the variable. Without `move`, the closure would borrow `s` by reference. With `move`, it takes ownership of `s`.

## Storing Closures: The Problem
Every closure has a unique type. Two closures with the same signature are different types:
```rust
let c1 = |x: &str| x.len() > 80;
let c2 = |x: &str| x.contains('\t');
// These are DIFFERENT types. You cannot do this:
// let rules: Vec<???> = vec![c1, c2];
```
We cannot name the type of a closure. Even if we could, they would be different sizes.

## Storing Closures: The Solution with `Box<dyn Fn>`
We use `Box<dyn Fn>`, a boxed trait object. `Box` gives the vector a fixed-size pointer representation even though the underlying closure types differ. `dyn Fn(...)` erases the concrete type and uses a vtable for dynamic dispatch.
```rust
let mut rules: Vec<Box<dyn Fn(&str) -> Vec<String>>> = Vec::new();
rules.push(Box::new(|line| {
if line.len() > 80 {
    vec!["Line too long".to_string()]
} else {
    vec![]
}
}));
rules.push(Box::new(|line| {
if line.contains("TODO") {
    vec!["Contains TODO".to_string()]
} else {
    vec![]
}
}));
```
`Box::new(|line| ...)` heap-allocates the closure. `Box::new` allocates space on the heap and moves the closure into it, returning a `Box` pointer. When we push into rules, Rust sees `Box<UniqueClosureType>` and coerces it to `Box<dyn Fn(...)>`.

The `dyn` keyword means the `Box` now holds a fat pointer: one pointer to the heap-allocated closure data, and one pointer to a vtable that tells Rust how to call `Fn::call` on this specific closure. When we call `rule(line)`, Rust looks up the function address in the vtable and calls through it.

## Why Can't We Use Generics in the Struct
Instead of `Box<dyn Fn>`, why not make the `Linter` struct generic?
```rust
struct Linter<F: Fn(&str) -> Vec<Violation>> {
rules: Vec<F>,
}
```
A generic struct is monomorphized: the compiler creates a separate copy for each concrete `F`. But we want to store different closures with different types in the same `Vec`. A `Vec<F>` can only hold one type of closure. With `Box<dyn Fn>`, all closures are erased to the same type (`Box<dyn Fn(...)`) so they fit in the same `Vec`.

This is the fundamental tradeoff:
- Generics (`impl Fn` / `F: Fn`): Static dispatch, faster, but all items must be the same concrete type
- Trait objects (`Box<dyn Fn>`): Dynamic dispatch, slightly slower, but items can be different concrete types

## The Project: Tiny Rule-based Text Linter
Now that you understand closures, `Box`, `dyn` trait objects, and how they work together, let's build our Tiny Rule-based Text Linter.

Our program will:
- Define lint rules as closures: Fn(&str) -> `Vec<Violation>`
- Store rules in a `Vec<Box<dyn Fn(&str) -> Vec<Violation>>>`
- Run all rules over each line of a file
- Report all violations with line numbers
- Support fixers as `Fn(&mut String) -> bool` closures
- Apply fixers to auto-correct issues
## Project Setup
Open your terminal and run:
```bash
cargo new linter
cd linter
```

Now open `src/main.rs` and let's build this step by step.

### The Violation and Linter Types
```rust
use std::fs;

#[derive(Debug)]
struct Violation {
line: usize,
message: String,
}

struct Linter {
rules: Vec<Box<dyn Fn(&str) -> Vec<Violation>>>,
fixers: Vec<Box<dyn Fn(&mut String) -> bool>>,
}
```
`Violation` holds a line number and a message describing the problem. `Linter` holds two vectors of boxed trait objects.

Let's read `Vec<Box<dyn Fn(&str) -> Vec<Violation>>>` from the inside out:
- `&str` is the input: a line of text
- `Vec<Violation>` is the output: all violations found on that line  
- `dyn Fn(&str) -> Vec<Violation>` is a trait object: any closure that takes &str and returns `Vec<Violation>`
- `Box<...>` heap-allocates that trait object so all entries in the `Vec` are the same size (a pointer)
- `Vec<Box<...>>` is the collection holding all our rules
Same for fixers, but fixers take `&mut` String (the entire file) and return `bool` (whether they changed anything).

### Adding Rules and Fixers
```rust
impl Linter {
fn new() -> Linter {
    Linter {
        rules: Vec::new(),
        fixers: Vec::new(),
    }
}
fn add_rule<F>(&mut self, rule: F)
where
    F: Fn(&str) -> Vec<Violation> + 'static,
{
    self.rules.push(Box::new(rule));
}
fn add_fixer<F>(&mut self, fixer: F)
where
    F: Fn(&mut String) -> bool + 'static,
{
    self.fixers.push(Box::new(fixer));
}
}
```

Each `add_*` method is generic over `F`. The `where` clause constrains `F` to implement the appropriate `Fn` trait and be `'static`.

The `'static` bound means the closure cannot contain borrowed references tied to shorter lifetimes. In practice, this usually means the closure either captures nothing or captures owned values using `move`.This is required because `Box<dyn Fn(...)>` does not carry lifetime information.

When we call `Box::new(rule)`, Rust allocates the closure on the heap. The `Box<F>` is then coerced to `Box<dyn Fn(...)>` when pushed into the vector. The concrete type `F` is erased, replaced with a fat pointer that includes a vtable.

### Running Rules and Fixers
```rust
impl Linter {
fn check(&self, content: &str) -> Vec<Violation> {
    let mut all_violations = Vec::new();
    for (line_num, line) in content.lines().enumerate() {
        for rule in &self.rules {
            let violations = rule(line);
            for v in violations {
                all_violations.push(Violation {
                    line: line_num + 1,
                    message: v.message,
                });
            }
        }
    }
    all_violations
}
fn fix(&self, content: &mut String) -> usize {
    let mut fixed_count = 0;
    for fixer in &self.fixers {
        if fixer(content) {
            fixed_count += 1;
        }
    }
    fixed_count
}
}
```
`check` iterates over every line using `content.lines()`, which returns an iterator. `enumerate()` pairs each line with its 0-based index. We adjust to 1-based for human-friendly output.
For each line, we iterate over every rule in `self.rules`. Each rule is a `Box<dyn Fn(&str) -> Vec<Violation>>`. When we call `rule(line)`, Rust:
1. Reads the fat pointer from the Box
2. Looks up `Fn::call` in the vtable
3. Calls the closure's actual function through the vtable pointer
4. Returns the `Vec<Violation>` result

This is dynamic dispatch. The tiny runtime cost is the vtable lookup.
`fix` is similar but passes `&mut` content to each fixer. Fixers return `bool` to indicate whether they changed anything.

### Defining Lint Rules as Closures
Now let's define some lint rules. Each rule is a closure that takes a line and returns violations.
```rust
fn main() {
let mut linter = Linter::new();
// Rule 1: Check for trailing whitespace
linter.add_rule(|line| {
    let mut violations = Vec::new();
    if line.len() > line.trim_end().len() {
        violations.push(Violation {
            line: 0,
            message: "Trailing whitespace".to_string(),
        });
    }
    violations
});
// Rule 2: Check for lines longer than 80 characters
linter.add_rule(|line| {
    let mut violations = Vec::new();
    if line.len() > 80 {
        violations.push(Violation {
            line: 0,
            message: format!("Line too long ({} characters)", line.len()),
        });
    }
    violations
});
// Rule 3: Check for hard tabs
linter.add_rule(|line| {
    let mut violations = Vec::new();
    if line.contains('\t') {
        violations.push(Violation {
            line: 0,
            message: "Hard tab detected, use spaces".to_string(),
        });
    }
    violations
});
// Rule 4: Check for TODO and FIXME comments
linter.add_rule(|line| {
    let mut violations = Vec::new();
    if line.contains("TODO") || line.contains("FIXME") {
        violations.push(Violation {
            line: 0,
            message: "Contains TODO or FIXME marker".to_string(),
        });
    }
    violations
});
}
```
Each `linter.add_rule(...)` passes a closure. These closures don't capture anything from the environment, they only use their `line` parameter. Since they have no captures, Rust infers `Fn` for them, and they satisfy `'static` trivially.

Each rule sets `line: 0` as a placeholder. The `check` method overwrites this with the actual line number when it iterates. The rules don't know their line number, they only analyze a single line.

Let's trace through what happens when we call `linter.check`:
1. `check` opens the file and starts reading lines
2. For line 1, it calls the first closure (trailing whitespace rule). Rust looks up the vtable, calls the closure, gets back a `Vec`. If the line has trailing spaces, the `Vec` has one violation. If not, it is empty.
3. It calls the second closure (line length rule) on the same line. Same vtable lookup process.
4. Continues for all four rules on line 1.
5. Moves to line 2, repeats.
6. All violations are collected into a single `Vec`.

### A Fixer Closure
```rust
fn main() {
// ... rules from above ...
// Fixer: Remove trailing whitespace
linter.add_fixer(|content| {
    let original = content.clone();
    *content = content
        .lines()
        .map(|line| line.trim_end())
        .collect::<Vec<_>>()
        .join("\n");
    if original.ends_with('\n') {
        content.push('\n');
    }
    *content != original
});
}
```
This closure captures nothing. `content.clone()` saves the original so we can compare at the end. `content.lines()` returns an iterator over lines (without trailing newlines). `.map(|line| line.trim_end())` transforms each line by stripping trailing whitespace. `.collect::<Vec<_>>()` collects into a `Vec<&str>`. `.join("\n")` joins them back with newlines.

The tricky part: `lines()` does not include the trailing newline. If the original file ended with `\n`, the joined string won't have one. We check `original.ends_with('\n')` and push a newline if needed.

The closure returns `*content != original`, `true` if we modified the content, `false` if it was already clean.

### The Main Function
```rust
fn main() {
// ... rule definitions and fixer from above ...
let args: Vec<String> = std::env::args().collect();
let file_path = if args.len() > 1 {
    &args[1]
} else {
    eprintln!("Usage: cargo run -- <file>");
    return;
};
let mut content = match fs::read_to_string(file_path) {
    Ok(s) => s,
    Err(e) => {
        eprintln!("Error reading {}: {}", file_path, e);
        return;
    }
};
let violations = linter.check(&content);
if violations.is_empty() {
    println!("No violations found.");
} else {
    for v in &violations {
        println!("{}:{} {}", file_path, v.line, v.message);
    }
    println!("\n{} violation(s) found.", violations.len());
}
let fixed_count = linter.fix(&mut content);
if fixed_count > 0 {
    println!("\n{} fixer(s) applied.", fixed_count);
    match fs::write(file_path, &content) {
        Ok(()) => println!("File updated."),
        Err(e) => eprintln!("Error writing {}: {}", file_path, e),
    }
}
}
```
We read the file into a `String`, run check to collect violations, print them in `filename:line: message` format, then run fix to auto-correct. If any fixer applied changes, we write back to disk.

The check method takes `&self` because rules only read lines. The fix method also takes `&self`, the mutation is on the external content parameter, not on self.

## Closures That Capture
All the closures so far have been "pure" (no captures). Let's make things more interesting by capturing variables

```rust
// Configurable max line length
let max_line_length = 100;
linter.add_rule(move |line| {
let mut violations = Vec::new();
if line.len() > max_line_length {
    violations.push(Violation {
        line: 0,
        message: format!(
            "Line too long ({} chars, max {})",
            line.len(),
            max_line_length
        ),
    });
}
violations
});
```
The closure captures `max_line_length` from the surrounding scope. We use `move` to force ownership transfer into the closure.

Without `move`, the closure would capture `&max_line_length`, a reference to the local variable. But `Box<dyn Fn(...)>` requires the closure to be `'static`. If the closure held a reference to `max_line_length`, what happens when main returns and `max_line_length` is dropped? The reference would dangle. Rust prevents this at compile time.

With `move`, the closure takes ownership of `max_line_length`. The `i32` value is copied into the closure's internal state (on the heap, inside the Box). Now the closure is self-contained and `'static`.

### Forbidden Words Rule
```rust
let forbidden_words = vec!["debug", "println", "unwrap", "todo"];
linter.add_rule(move |line| {
let mut violations = Vec::new();
for word in &forbidden_words {
    if line.contains(word) {
        violations.push(Violation {
            line: 0,
            message: format!("Forbidden word '{}' found", word),
        });
    }
}
violations
});
```
`move` gives ownership of the entire `Vec<&str>` to the closure.The `Vec` stores its elements in a heap-allocated buffer, while the `Vec` struct itself contains pointer/len/cap metadata. With `move`, the closure owns this vector. Inside the closure, for word in `&forbidden_words` borrows from the closure's own captured state.

Closures let us package both behaviour and captured state together as a single value.
## Running The Project
Create a test file:
```bash
cat > test.rs << 'EOF'
fn main() {
println!("hello world");   
let x = 10; // TODO: fix this
println!("{}", x);
}
EOF
```
This contains three trailing whitespace after the `println!("hello world");`.

Now run the linter:
```bash
cargo run -- test.rs
```
You should see output like this:
```bash
test.rs:2 Trailing whitespace
test.rs:3 Contains TODO or FIXME marker
2 violation(s) found.
1 fixer(s) applied.
File updated.
```
The linter found trailing whitespace on line 2 and a TODO marker on line 3. The trailing whitespace fixer ran and cleaned up the file automatically.

# Learn Rust Concurrency By Building a Thread Pool

> In this post, we are going to learn about concurrency and threads in Rust. Once we cover all the concepts, we will build a Thread Pool in Rust

Source: https://blog.sheerluck.dev/posts/rust/learn-rust-concurrency-by-building-a-thread-pool/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-rust-concurrency-by-building-a-thread-pool/index.md

In this post, we are going to learn about concurrency in Rust. We will learn about OS threads, message passing with channels, shared mutable state with `Arc<Mutex<T>>`, and graceful shutdown. Once we cover all the concepts, we will build a **Thread Pool from Scratch**, a fixed number of worker threads, a job queue behind `Arc<Mutex<>>`, an `mpsc` channel for dispatching tasks, graceful shutdown that processes remaining queued jobs before exiting, and backpressure when the queue is full. I am really excited for this project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles.

The only prerequisite is that you have read the previous articles in this series, as I will assume you know ownership, borrowing, structs, enums, pattern matching, error handling, generics, traits, lifetimes, HashMap, iterators, closures, and smart pointers.

In the last article, we learned about `Rc` and `RefCell` for shared mutable state in single-threaded code. I teased that the smart pointer family has one more member: `Arc<T>`, the atomic reference counter for multi-threaded code. Today we will cover `Arc` together with `Mutex` and concurrency primitives, and we will build a thread pool that puts all of it to work.

Get the source code from [here](https://github.com/MrSheerluck/threadpool)
## What Is a Thread Pool

Think of a restaurant kitchen. You have a fixed number of chefs. Orders come in, and each chef picks up the next available order, cooks it, and goes back to wait for the next one. You do not hire a new chef for every order and fire them afterwards, that would be wasteful. You also do not let the order queue grow infinitely, if the kitchen is too backed up, you tell the front of house to wait before sending more orders.

A thread pool is the same idea. You spawn a fixed number of worker threads once, and they live for the lifetime of the pool. Jobs are sent to the pool through a channel. Idle workers receive jobs from the shared receiver. When the pool shuts down, workers continue processing queued jobs until the channel becomes empty and disconnected, then exit cleanly. If the queue is full, the sender blocks until there is space.
## Spawning Threads

Let's start with the basics. Create a new project:

```bash
cargo new threadpool
cd threadpool
```

Open `src/main.rs` and replace everything with:

```rust
use std::thread;
use std::time::Duration;

fn main() {
let handle = thread::spawn(|| {
    for i in 1..=5 {
        println!("spawned thread: {}", i);
        thread::sleep(Duration::from_millis(100));
    }
});

for i in 1..=3 {
    println!("main thread: {}", i);
    thread::sleep(Duration::from_millis(150));
}

handle.join().unwrap();
}
```

Run it:

```bash
cargo run
```

You will see output from both threads interleaved. Something like:

```bash
main thread: 1
spawned thread: 1
spawned thread: 2
main thread: 2
spawned thread: 3
main thread: 3
spawned thread: 4
spawned thread: 5
```

Now, let me explain what we just did.

`thread::spawn` takes a closure and runs it on a brand new OS thread. The OS scheduler decides when each thread runs, so the output is interleaved unpredictably. `thread::spawn` returns a `JoinHandle<T>` where `T` is whatever the closure returns. The handle's `.join()` method blocks the calling thread until the spawned thread finishes. We call `.unwrap()` on `.join()` because if the spawned thread panicked, `join` returns the panic payload as an `Err`. The spawned thread gets five iterations with 100ms sleeps, while the main thread runs three iterations with 150ms sleeps. Even though main finishes its loop first, it blocks on `handle.join()` until the spawned thread is done.
## Threads Need Ownership: `move` Closures

In the example above the closure captures nothing. Let's try capturing a variable:

```rust
fn main() {
let name = String::from("worker");

let handle = thread::spawn(|| {
    println!("hello from {}", name);
});

handle.join().unwrap();
}
```

Try to compile this. You will get an error:

```bash
error[E0373]: closure may outlive the current function, but it borrows `name`,
which is owned by the current function
```

The compiler is telling us something important about threads: the spawned thread could outlive the scope that owns `name`. If `main` finishes and drops `name` while the spawned thread is still running, the reference would dangle. Rust prevents this at compile time.

The fix is the `move` keyword:

```rust
let handle = thread::spawn(move || {
println!("hello from {}", name);
});
```

We already learned about `move` in the closures article. The `move` keyword forces the closure to take ownership of every variable it captures. `name` moves into the closure. It is no longer accessible in `main` after `spawn`. The spawned thread now owns the string and can use it safely for as long as it runs.

This is the core rule of threads: anything a thread uses must either be owned by the thread, or be guaranteed to outlive the thread (like a `&'static str` literal).
## Message Passing with `mpsc`

Threads need to communicate. Rust provides channels in `std::sync::mpsc`. The name stands for multiple producer, single consumer. You can clone the `Sender` and send messages from many threads, but there is only one `Receiver`.

Here is a minimal example:

```rust
use std::sync::mpsc;
use std::thread;

fn main() {
let (tx, rx) = mpsc::channel();

thread::spawn(move || {
    tx.send(String::from("hello from the spawned thread")).unwrap();
});

let msg = rx.recv().unwrap();
println!("received: {}", msg);
}
```

Now, let me explain what we just did.

`mpsc::channel()` returns a `(Sender<T>, Receiver<T>)` tuple. `T` is inferred from the first `send` call. `Sender::send` takes ownership of the message being sent, but only borrows the sender itself. A single sender can be reused to send many messages. If you want multiple threads to send messages concurrently, you can call `clone()` on the sender to create additional sending handles. `Receiver::recv` blocks the calling thread until a message arrives. It returns `Result<T, RecvError>`, where the error means all senders have been dropped and the channel is closed.

There are other methods on `Receiver`:

| Method | Behaviour |
|---|---|
| `recv()` | Blocks until a message is available. Returns `Err(RecvError)` if all senders are dropped. |
| `try_recv()` | Does not block. Returns `Ok(msg)`, `Err(TryRecvError::Empty)`, or `Err(TryRecvError::Disconnected)`. |
| `iter()` | Returns an iterator that yields messages until the channel is closed. This is how workers will loop. |

`Sender::send` also has a counterpart:

```rust
let (tx, rx) = mpsc::sync_channel(2); // bounded channel, capacity 2
```

A `sync_channel` has a fixed capacity. When the channel is full, `send` blocks the calling thread until space is freed. This is how we implement backpressure: the producer is forced to wait if the workers are overwhelmed.
## Multiple Producers

`Sender` implements `Clone`. Here is a pattern with two sending threads:

```rust
use std::sync::mpsc;
use std::thread;

fn main() {
let (tx, rx) = mpsc::channel();

let tx1 = tx.clone();
thread::spawn(move || {
    for i in 0..3 {
        tx1.send(i).unwrap();
    }
});

thread::spawn(move || {
    for i in 10..13 {
        tx.send(i).unwrap();
    }
});

for msg in rx {
    println!("received: {}", msg);
}
}
```

Now, let me explain what we just did.

`tx.clone()` creates a second sender. Both threads send messages. The main thread uses `for msg in rx` to iterate over the receiver. This loop runs until all senders are dropped and the channel is empty, at which point the iterator ends and the loop exits. You do not need to know how many messages there are ahead of time. The channel closing is detected automatically.
## Arc: The Thread-Safe Rc

In the smart pointers article, we learned about `Rc<T>` for shared ownership in single-threaded code. `Rc` uses non-atomic reference counting, which is faster but not safe across threads. The compiler enforces this: `Rc` does not implement `Send` or `Sync`, so you cannot send an `Rc` to another thread.

`Arc<T>` - Atomic Reference Counted is the thread-safe version. It uses atomic operations for the reference count, which are slightly slower than `Rc`'s non-atomic counter, but safe across threads. The usage is identical:

```rust
use std::sync::Arc;
use std::thread;

fn main() {
let data = Arc::new(vec![1, 2, 3]);

let data1 = Arc::clone(&data);
let handle1 = thread::spawn(move || {
    println!("thread 1: {:?}", data1);
});

let data2 = Arc::clone(&data);
let handle2 = thread::spawn(move || {
    println!("thread 2: {:?}", data2);
});

handle1.join().unwrap();
handle2.join().unwrap();
}
```

Now, let me explain what we just did.

`Arc::new` allocates a reference-counted allocation containing the value and returns an `Arc` handle to it. `Arc::clone` increments the atomic reference count and returns another handle to the same allocation. Unlike `Rc`, `Arc::clone` does not actually deep-clone the data - it just increments the counter. `Arc` uses atomic operations for reference counting, making shared ownership safe across threads. `Arc<T>` implements `Send` and `Sync` when `T` itself satisfies the necessary thread-safety requirements. This allows us to move cloned `Arc` handles into threads with `move` closures. The three handles (`data`, `data1`, `data2`) all point to the same heap allocation. When the last handle is dropped, the reference count reaches zero and the data is freed.
## Mutex: Mutual Exclusion

`Arc` gives us shared ownership, but it gives us immutable shared ownership. You cannot mutate through an `Arc`:

```rust
let data = Arc::new(42);
// *data += 1; // ERROR: cannot modify through Arc
```

If multiple threads all hold an `Arc` to the same value, they can all read it safely. But what if they need to write? Or what if the value is something like a `Vec` that needs to be modified?

This is where `Mutex<T>` comes in. A `Mutex` provides mutual exclusion: only one thread can access the data at a time. Other threads block until the lock is released.

```rust
use std::sync::Mutex;
use std::thread;

fn main() {
let counter = Mutex::new(0);

{
    let mut num = counter.lock().unwrap();
    *num += 1;
} // lock released here

println!("counter = {}", *counter.lock().unwrap());
}
```

Now, let me explain what we just did.

`Mutex::new(0)` wraps the integer in a mutex. `counter.lock()` blocks until we can acquire the lock, then returns a `MutexGuard<T>`. The `MutexGuard` derefs to `&mut T`, so we can mutate the value through it. When the guard goes out of scope (at the end of the block), the lock is automatically released. We call `.unwrap()` on `lock()` because it returns a `Result`, the error case is if the mutex is poisoned, which means another thread panicked while holding the lock.

The `Mutex` pattern is similar to `RefCell` which we learned in the previous article. Both provide interior mutability. The difference is: `RefCell` panics at runtime if you violate borrowing rules. `Mutex` blocks the thread until the lock is available. `RefCell` is for single threads. `Mutex` is for multiple threads.
## `Arc<Mutex<T>>`: Shared Mutable State Across Threads

Now combine them. `Arc` provides shared ownership. `Mutex` provides safe mutation. Together, they give us shared mutable state across threads:

```rust
use std::sync::{Arc, Mutex};
use std::thread;

fn main() {
let counter = Arc::new(Mutex::new(0));
let mut handles = Vec::new();

for _ in 0..5 {
    let counter = Arc::clone(&counter);
    let handle = thread::spawn(move || {
        let mut num = counter.lock().unwrap();
        *num += 1;
    });
    handles.push(handle);
}

for handle in handles {
    handle.join().unwrap();
}

println!("counter = {}", *counter.lock().unwrap());
}
```

Now, let me explain what we just did.

We create an `Arc<Mutex<i32>>`. Inside the loop, we clone the `Arc` and move the clone into a thread. Each thread acquires the lock, increments the counter, and releases the lock when the guard goes out of scope. Only one thread can hold the lock at any moment, so the increments are serialised and there are no data races. After all threads finish, the counter is exactly 5.

Let me read `Arc<Mutex<T>>` from the inside out:

- `T` is the actual data
- `Mutex<T>` wraps the data with a lock
- `Arc<Mutex<T>>` stores the `Mutex<T>` inside an `Arc` managed heap allocation and enables multiple owners

You will see this pattern everywhere in multi-threaded Rust code. It is the thread-safe counterpart of `Rc<RefCell<T>>` from the previous article.
## The Problem with Sharing the Receiver

For our thread pool, every worker thread needs to receive jobs from the same queue. The natural approach is to share the `Receiver` from an `mpsc` channel. But let me show you why this does not work directly:

```rust
use std::sync::mpsc;

fn main() {
let (tx, rx) = mpsc::channel::<i32>();

// ERROR: Receiver is not Sync, cannot be shared
// let shared_rx = std::sync::Arc::new(rx);
}
```

`Receiver<T>` does not implement `Sync`. The reason is that receiving from a channel is inherently stateful: only one thread should receive each message. If multiple threads tried to call `recv()` simultaneously on the same `Receiver`, they would race.

The solution is `Arc<Mutex<Receiver<T>>>`. The `Mutex` ensures only one worker can receive at a time. The worker locks the mutex, calls `recv` (which blocks until a message arrives), then releases the lock for the next worker:

```rust
use std::sync::{Arc, Mutex, mpsc};
use std::thread;

fn main() {
let (tx, rx) = mpsc::channel::<i32>();
let rx = Arc::new(Mutex::new(rx));

let rx1 = Arc::clone(&rx);
thread::spawn(move || {
    let msg = rx1.lock().unwrap().recv().unwrap();
    println!("worker 1 got: {}", msg);
});

let rx2 = Arc::clone(&rx);
thread::spawn(move || {
    let msg = rx2.lock().unwrap().recv().unwrap();
    println!("worker 2 got: {}", msg);
});

tx.send(10).unwrap();
tx.send(20).unwrap();
}
```

Now, let me explain what we just did.

`rx` is an `Arc<Mutex<Receiver<i32>>>`. Each worker clones the `Arc` and moves the clone into its thread. To receive a job, the worker calls `rx.lock().unwrap().recv().unwrap()`. The `.lock()` blocks until the mutex is available. The `.recv()` blocks until a message arrives. Both workers are racing to acquire the lock. Whichever wins gets to receive the next message. The other worker blocks on `.lock()` until the first worker releases the lock (which happens when the guard goes out of scope at the end of the statement).

This means exactly one message goes to exactly one worker. No duplication. No missed messages. The mutex serialises access to the receiver. An important detail is that the mutex is held during the blocking `recv()` call. This means only one worker can wait on the receiver at a time. Workers still execute jobs concurrently, but receiving jobs from the channel is serialised through the mutex.
## The Thread Pool Architecture

Now let's design our thread pool. Here is the architecture:

```
main thread                thread pool
|                          |
|-- tx.send(job) --------> |  (mpsc::Sender<Job>)
|                          |
|                          +-- Worker 1 -- rx.lock().recv() -> execute job
|                          +-- Worker 2 -- rx.lock().recv() -> execute job
|                          +-- Worker 3 -- rx.lock().recv() -> execute job
|                          +-- Worker 4 -- rx.lock().recv() -> execute job
|                          |
|-- drop(tx) ------------> |  (workers finish queued jobs, then exit once the channel is empty)
```

The `ThreadPool` struct holds a `Sender<Job>` and a `Vec<JoinHandle<()>>` for the worker threads. When a user calls `pool.execute(closure)`, the closure is boxed into a `Job` and sent through the channel. All workers loop. Each worker acquires the mutex protecting the receiver, waits for and receives the next available job, then releases the mutex before executing the job. When the pool is dropped, the `Sender` is dropped, the channel closes, each worker's `recv()` returns `Err`, and the loop exits.

## The Project: Thread Pool from Scratch

Our program will:
- Define a `Job` type as `Box<dyn FnOnce() + Send + 'static>`
- Create a `Worker` struct that holds a thread handle and an ID
- Create a `ThreadPool` struct that holds a sender and worker handles
- Implement `ThreadPool::new(size)` that spawns `size` workers
- Implement `pool.execute(closure)` that sends a job to the workers
- Implement graceful shutdown: workers continue receiving and executing queued jobs until the channel becomes both disconnected and empty, then exit cleanly
- Use `sync_channel` with a bounded capacity for backpressure
- Write a demo `main` that shows it working
## Project Setup

Delete the test code from `src/main.rs`. We will build the entire pool from scratch:

```bash
cargo new threadpool
cd threadpool
```

We do not need any external crates. Everything is in `std`. Open `src/main.rs` and we will build everything step by step.
## The Job Type

A job is something that can be executed once. The most natural way to represent this in Rust is a trait object:

```rust
type Job = Box<dyn FnOnce() + Send + 'static>;
```

Let me read this from inside out:

- `FnOnce()` means a closure that takes no arguments and returns nothing, and can be called exactly once.
- `Send` means the closure can be transferred safely between threads.
- `'static` means the closure cannot contain borrowed references with shorter lifetimes.
- `Box<dyn ...>` heap-allocates the closure and erases its concrete type.

This is exactly the same pattern we used in the linter article where we stored `Box<dyn Fn(&str) -> Vec<Violation>>`. The only new bits are `Send` (for threads) and `FnOnce` (because a job is consumed when executed).
## The Worker Struct

A `Worker` is a thread that loops forever, receiving and executing jobs:

```rust
use std::thread;

struct Worker {
id: usize,
thread: thread::JoinHandle<()>,
}
```

The `id` field is for debug printing (we will print "worker 2 got a job" etc.). The `thread` field holds the join handle so we can wait for the worker to finish during shutdown.
## The Worker Loop

A worker needs to receive jobs from the shared receiver. The worker runs a loop:

```rust
impl Worker {
fn new(
    id: usize,
    receiver: Arc<Mutex<mpsc::Receiver<Job>>>,
) -> Worker {
    let thread = thread::spawn(move || {
        loop {
            let job = receiver.lock().unwrap().recv();
            match job {
                Ok(job) => {
                    println!("worker {} got a job; executing.", id);
                    job();
                }
                Err(_) => {
                    println!("worker {} shutting down.", id);
                    break;
                }
            }
        }
    });

    Worker { id, thread }
}
}
```

Now, let me explain what we just did.

`Worker::new` takes an `id` and an `Arc<Mutex<Receiver<Job>>>`. Inside `thread::spawn(move || ...)`, the closure starts with a `loop`. In each iteration, the worker:

1. Calls `receiver.lock().unwrap()` to acquire the mutex. This blocks if another worker is currently receiving a message.
2. Calls `.recv()` on the receiver. This blocks if the channel is empty but not closed.
3. Matches on the result. `Ok(job)` means we got a job, so we call `job()` to execute it and print the worker ID. `Err(_)` means all senders have been dropped (the channel is closed), so we `break` out of the loop.
4. When the loop exits, the thread function returns and the thread terminates.

The `move` keyword is essential here. It moves ownership of `id` and `receiver` into the closure. `id` is a `usize` (which is `Copy`), but `receiver` is an `Arc` which must be moved so the thread can share ownership of the receiver.
## The ThreadPool Struct

```rust
use std::sync::{Arc, Mutex, mpsc};

struct ThreadPool {
workers: Vec<Worker>,
sender: Option<mpsc::Sender<Job>>,
}
```

Now, let me explain what we just did.

`workers` is a `Vec<Worker>`. We store the workers so we can join them during shutdown. `sender` is `Option<mpsc::Sender<Job>>`. When we want to shut down, we set `sender` to `None` by calling `take()`. This drops the sender, which closes the channel, which causes every worker's `recv()` to return `Err`, which causes every worker to `break` out of its loop. The `Option` is the standard Rust pattern for dropping something during `Drop` that you also need to access during the struct's normal lifetime.
## ThreadPool::new

Now let's implement `ThreadPool::new`:

```rust
impl ThreadPool {
fn new(size: usize) -> ThreadPool {
    assert!(size > 0, "ThreadPool must have at least 1 worker");

    let (sender, receiver) = mpsc::channel();
    let receiver = Arc::new(Mutex::new(receiver));

    let mut workers = Vec::with_capacity(size);
    for id in 0..size {
        workers.push(Worker::new(id, Arc::clone(&receiver)));
    }

    ThreadPool {
        workers,
        sender: Some(sender),
    }
}
}
```

Now, let me explain what we just did.

We use `mpsc::channel()` to create an unbounded channel for jobs. The receiver is wrapped in `Arc::new(Mutex::new(receiver))` so it can be shared across all workers. For each worker (from 0 to `size`), we clone the `Arc` and pass it to `Worker::new`. Each worker gets its own clone of the `Arc`, so all workers share the same `Mutex<Receiver<Job>>`. The sender is wrapped in `Some` so we can later `take()` it during shutdown.

The `assert!` ensures we never create a pool with zero workers. A pool with zero workers would never execute any jobs.

## The `execute` Method

The `execute` method takes a closure, boxes it into a `Job`, and sends it through the channel:

```rust
impl ThreadPool {
fn execute<F>(&self, f: F)
where
    F: FnOnce() + Send + 'static,
{
    let job = Box::new(f);
    self.sender.as_ref().unwrap().send(job).unwrap();
}
}
```

Now, let me explain what we just did.

`execute` is generic over `F`, which must implement `FnOnce() + Send + 'static`. These are the same bounds as on `Job` itself. Inside, we box the closure into a `Box<dyn FnOnce() + Send + 'static>` and send it through the channel. We call `unwrap()` on both the `as_ref()` (the sender should always exist during normal operation) and `send()` (we assume the channel is healthy). A production implementation would return a `Result`, but for learning purposes `unwrap` is fine.

When `execute` returns, the job is in the queue. One of the workers will eventually pick it up and run it.

## Graceful Shutdown: The Drop Implementation

When the pool is dropped, we want every worker to finish its current job, drain any remaining jobs in the queue, and then shut down. Here is the Drop implementation:

```rust
impl Drop for ThreadPool {
fn drop(&mut self) {
    // Drop the sender to close the channel.
    // Workers will see the channel close after draining remaining messages.
    drop(self.sender.take());

    // Wait for every worker to finish.
    for worker in self.workers.drain(..) {
        println!("shutting down worker {}", worker.id);
        worker.thread.join().unwrap();
    }
}
}
```

Now, let me explain what we just did.

This is a two-part shutdown. First, we call `self.sender.take()` to get the `Option<mpsc::Sender<Job>>` and extract the sender, then immediately `drop` it. This closes the channel. Workers currently blocked on `recv()` will unblock and:

- If there are still messages in the channel, they will receive those messages (the `Ok(job)` arm) and execute them.
- Once the channel is drained, the next `recv()` returns `Err(RecvError)` (the `Err(_)` arm) and the worker breaks out of its loop.

So all remaining jobs in the queue are processed before any worker shuts down.

Second, we iterate over `self.workers.drain(..)`. The `drain(..)` removes every element from the `Vec`, yielding ownership of each `Worker`. For each worker, we call `thread.join().unwrap()` to block until that worker's thread has finished. The order of joining does not matter: some workers may finish sooner than others, but by the time the `for` loop finishes, all workers have exited.

This is true graceful shutdown. No work is lost, and the program waits for everything to finish. This assumes the pool owns the last sender. If other sender clones still exist, the channel remains open and workers will continue waiting for new jobs.

## Adding Backpressure: Bounded Channels

Our current implementation uses `mpsc::channel()`, which creates an unbounded channel. The queue can grow infinitely. If jobs are submitted faster than workers can process them, the program's memory grows without bound.

We solve this with `mpsc::sync_channel(capacity)`, which creates a bounded channel. When the channel is full, `send()` blocks until a worker frees a slot by receiving a job. This is backpressure: the producer is forced to slow down.

Update `ThreadPool::new` to use `sync_channel`:

```rust
impl ThreadPool {
fn new(size: usize) -> ThreadPool {
    assert!(size > 0, "ThreadPool must have at least 1 worker");

    let (sender, receiver) = mpsc::sync_channel(size * 2);

    let receiver = Arc::new(Mutex::new(receiver));

    let mut workers = Vec::with_capacity(size);
    for id in 0..size {
        workers.push(Worker::new(id, Arc::clone(&receiver)));
    }

    ThreadPool {
        workers,
        sender: Some(sender),
    }
}
}
```

Now, let me explain what we just did.

The only change is `mpsc::channel()` becoming `mpsc::sync_channel(size * 2)`. The capacity is twice the number of workers, which gives a small buffer. The choice of capacity is a tuning parameter. A larger capacity means more queueing but smoother throughput. A smaller capacity means tighter backpressure but more blocking. Twice the worker count is a reasonable default.

Note that `sync_channel` returns `SyncSender<Job>` instead of `Sender<Job>`. The two types are different: `SyncSender` is used with bounded channels and `Sender` is used with unbounded channels. Since `Worker::new` is now receiving from a bounded channel, the type in `Worker::new` must also change to `Arc<Mutex<mpsc::Receiver<Job>>>`. The `Receiver` type is the same for both bounded and unbounded channels. The `Sender` field in `ThreadPool` changes from `Option<mpsc::Sender<Job>>` to `Option<mpsc::SyncSender<Job>>`.
## Running the Project

Type this in your terminal:

```bash
cargo run
```

You should see output like this:

```bash
Creating thread pool with 4 workers
All jobs submitted. Pool will shut down when scope ends.
worker 0 got a job; executing.
worker 2 got a job; executing.
worker 1 got a job; executing.
worker 3 got a job; executing.
job 0: starting
job 1: starting
job 2: starting
job 3: starting
job 0: finished
job 1: finished
job 2: finished
worker 0 got a job; executing.
job 3: finished
worker 2 got a job; executing.
worker 1 got a job; executing.
worker 3 got a job; executing.
job 4: starting
job 5: starting
job 6: starting
job 7: starting
job 4: finished
job 5: finished
job 6: finished
job 7: finished
worker 0 shutting down.
shutting down worker 0
worker 2 shutting down.
shutting down worker 2
worker 1 shutting down.
shutting down worker 1
worker 3 shutting down.
shutting down worker 3
```

Now, let me explain what happened.

The first four jobs were picked up immediately by the four idle workers. They all started roughly at the same time. The remaining four jobs were queued in the channel. As soon as a worker finished its job, it picked up the next one from the queue. After all eight jobs were done and the channel was empty, each worker's `recv()` returned `Err`, the loop broke, and the worker printed its shutdown message. The main thread then joined each worker and printed the final shutdown messages.

Four workers processed eight jobs, with no more than four jobs running at any time.

## Running with Only One Worker

Let's test with a single worker to see that jobs are truly serialised. Change `ThreadPool::new(4)` to `ThreadPool::new(1)` and run again:

```bash
Creating thread pool with 1 workers
All jobs submitted. Pool will shut down when scope ends.
worker 0 got a job; executing.
job 0: starting
job 0: finished
worker 0 got a job; executing.
job 1: starting
job 1: finished
worker 0 got a job; executing.
job 2: starting
job 2: finished
worker 0 got a job; executing.
job 3: starting
job 3: finished
worker 0 got a job; executing.
job 4: starting
job 4: finished
worker 0 got a job; executing.
job 5: starting
job 5: finished
worker 0 got a job; executing.
job 6: starting
job 6: finished
worker 0 got a job; executing.
job 7: starting
job 7: finished
worker 0 shutting down.
shutting down worker 0
```

Now, let me explain what happened.

With one worker, jobs are processed one at a time. Each job starts and finishes before the next one begins. No concurrency, but the thread pool abstraction is still useful: it prevents unbounded thread creation and gives you a bounded queue.

## Verifying Backpressure

Let's write a test that proves backpressure works. Replace `main` with:

```rust
fn main() {
println!("Creating thread pool with 2 workers (bounded channel capacity 4)");
let pool = ThreadPool::new(2);

// Submit many long-running jobs to fill the queue and block the main thread
for i in 0..10 {
    println!("main: submitting job {}", i);
    pool.execute(move || {
        println!("job {}: starting", i);
        thread::sleep(Duration::from_secs(1));
        println!("job {}: finished", i);
    });
    println!("main: job {} accepted", i);
}

println!("All jobs submitted.");
}
```

With a `sync_channel` of capacity 4 (2 workers * 2), jobs are accepted until the channel's buffer becomes full. Since the workers are simultaneously consuming jobs, the exact point at which `send()` begins blocking depends on thread scheduling. What is guaranteed is that if jobs are submitted faster than workers can process them, the producer will eventually block once the queue reaches capacity. Once the channel buffer reaches capacity, further calls to `send()` block until a worker receives a job and frees a slot.
## What We Skipped

There are a few things I am intentionally skipping in this article that we will cover in future ones:

- **Async/await**: Threads are OS-level concurrency. Async is cooperative concurrency within a single thread. We will build an async HTTP server in a future article and learn Tokio.
- **`rayon` crate**: Rayon provides data parallelism with work-stealing thread pools. It is a higher-level abstraction built on top of the primitives we used today.
- **Panic handling in workers**: Our implementation uses `unwrap()` on `lock()` and `join()`. A production thread pool would handle poisoned mutexes and panicked threads gracefully.
- **Configurable queue capacity**: The capacity is hardcoded to `size * 2`. A production library would expose this as a parameter.
- **Atomic counters for active jobs**: A production pool might track how many jobs are currently executing using an `AtomicUsize`.

## Conclusion

In this post, you learned about OS threads (`std::thread::spawn`), message passing (`mpsc::channel` and `sync_channel`), shared ownership across threads (`Arc`), mutual exclusion (`Mutex`), and the `Arc<Mutex<T>>` pattern for shared mutable state. You built a Thread Pool from scratch with a fixed number of workers, a bounded job queue, an `execute` method that accepts closures, and graceful shutdown that continues processing queued jobs before exiting.

This project ties together everything from the smart pointers article and the closures article. `Arc` is the thread-safe sibling of `Rc`. `Mutex` is the thread-safe sibling of `RefCell`. `Box<dyn FnOnce() + Send>` is the thread-safe version of the closure stores we built in the linter.

In the next article, we will learn about **async/await** and **Tokio** and **build an HTTP/1.1 server from scratch**. See you soon.

If you like reading this, please subscribe and share this with others. It'll really help me and motivate me to keep publishing more such articles.

# Learn Rust Lifetimes by Building a Generic LRU Cache

> In this post, we are going to learn Rust lifetimes concept by building a generic LRU cache in Rust

Source: https://blog.sheerluck.dev/posts/rust/learn-rust-lifetimes-by-building-a-lru-cache/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-rust-lifetimes-by-building-a-lru-cache/index.md

In this post, we are going to learn about lifetimes in Rust. Once we cover all the concepts, we will build a **Generic LRU Cache**. I'm really excited for this project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles.

The only prerequisite is that you have read the previous articles in this series, as I will assume you know ownership, borrowing, structs, enums, pattern matching, error handling, generics and traits.

In this post, we will first learn what lifetimes are, why they exist, how lifetime annotations work, lifetime elision, lifetimes on structs, and then we will build a generic LRU cache that returns zero-copy references with lifetime annotations.

Get the source code from [here](https://github.com/MrSheerluck/lru_cache_in_rust)

> This is an educational LRU implementation focused on learning lifetimes and borrowing semantics. Production LRU caches typically use a HashMap and a doubly linked list to achieve O(1) operations.

## The Problem Lifetimes Solve
Every reference in Rust has a **lifetime**. A lifetime is just the span of time during which a reference is valid. It starts when the reference is created and ends when the reference is no longer usable. Every reference in Rust has a lifetime, even if you don't see it written anywhere.

Most of the time, lifetimes are invisible. The compiler can figure them out on its own. But sometimes, the compiler needs your help. When the compiler cannot figure out the relationship between references in a function's inputs and outputs, it asks you to annotate the lifetimes. That is what this article is about.

Let's start with a simple example that compiles:
```rust
fn main() {
	let x = 42;
let r = &x;
println!("{}", r);
}
```
`x` is created, then `r` borrows `x`. Both `x` and `r` go out of scope at the end of `main`. `r` is always valid because `x` lives longer than `r`. The compiler can see this trivially.

But what about this?
```rust
fn main() {
let r;
{
    let x = 42;
    r = &x;
}
println!("{}", r); // ERROR: `x` does not live long enough
}
```
Here, `r` is declared before the inner block. Inside the block, `r` borrows `x`. But `x` is dropped when the inner block ends. When we try to print `r` after the block, `r` points to freed memory. This is a **dangling reference**. In C, this compiles and runs, and reads garbage or crashes. In Rust, this does not compile at all. The compiler checks that `r` cannot outlive the data it points to. That check is the borrow checker, and it enforces lifetime rules.

The error message says "`x` does not live long enough." The compiler has computed the lifetimes of both references and determined that the reference `r` would be invalid by the time it is used.

Here is another common example:
```rust
fn longest(x: &str, y: &str) -> &str {
if x.len() > y.len() { x } else { y }
}
```

This seems straightforward. Return the longer of two string slices. But this will not compile. The compiler does not know whether the returned reference points to `x` or `y`. It does not know how the lifetimes of the inputs relate to the lifetime of the output. It cannot guarantee that the returned reference will be valid when the caller uses it. So it rejects the code.
To fix this, we need to tell the compiler: "The returned reference cannot outlive either input reference." That is exactly what lifetime annotations do.

## Lifetime Annotations
A lifetime annotation is written with a single quote followed by a name, just like `'a`. By convention, we use short names like `'a`, `'b`, `'ctx`, `'src`. The annotation does not change how long anything lives. It just names a lifetime so you can describe relationships between references.

Here is the longest function with lifetime annotations:
```rust
fn longest<'a>(x: &'a str, y: &'a str) -> &'a str {
if x.len() > y.len() { x } else { y }
}
```
Let me explain what we just did. `<'a>` introduces a lifetime parameter, just like `<T>` introduces a type parameter. `x: &'a str` means "`x` is a reference to a `str` with lifetime `'a`". `y: &'a str` means the same for `y`. And `-> &'a str` means "the return value is a reference with lifetime `'a`".

The meaning of this signature is: "The function takes two string slices that both live at least as long as `'a`, and returns a string slice that also lives at least as long as `'a`." In other words, the returned reference cannot outlive either input reference.

Now the compiler knows enough to check the caller. Here is a valid call:
```rust
fn main() {
let x = String::from("short");
let y = String::from("loooooong");
let result = longest(&x, &y); // our longest function that we wrote above
println!("{}", result);
}
```
Both `x` and `y` live until the end of `main`. result is used within that scope. 

Everything checks out. But here is what gets rejected:
```rust
fn main() {
let x = String::from("short");
let result;
{
    let y = String::from("loooooong");
    result = longest(&x, &y); // our longest function that we wrote above
}
println!("{}", result); // ERROR: `y` does not live long enough
}
```

The function signature ties the returned reference lifetime to both inputs, so the compiler conservatively assumes the returned reference cannot outlive either one. Since `y` is dropped inside the block, result would be invalid after the block. The compiler catches this.

But what if the function always returns one specific input, regardless of the other?
```rust
fn first<'a, 'b>(x: &'a str, y: &'b str) -> &'a str {
x
}
```
Here `x` has lifetime `'a` and `y` has lifetime `'b`. The return type is `&'a str`, meaning the returned reference has the same lifetime as `x`. This is valid because `first` always returns `x`. The lifetime of `y` is completely unrelated.

At the call site:
```rust
fn main() {
let x = String::from("hello");
let result;
{
    let y = String::from("world");
    result = first(&x, &y); // our first function that we wrote above
}
println!("{}", result); // This works! result borrows from x, not y
}
```

This compiles because result only borrows from `x`, which lives longer. The compiler can see that the return value's lifetime is tied to `x` and not to `y`.

## Lifetime Annotations in Struct Methods
Structs can have lifetimes too, and methods on those structs need them as well.

Consider a struct that holds a reference:
```rust
struct Config<'a> {
name: &'a str,
version: &'a str,
}
```
The `<'a>` on the struct definition means every instance of `Config` has a lifetime parameter. Any reference stored in the struct must live at least as long as `'a`. The struct cannot outlive the data it references.

Now let's add a method:
```rust
impl<'a> Config<'a> {
fn name(&self) -> &str {
    self.name
}
}
```
The `impl<'a>` introduces the lifetime parameter for the impl block. The method `name` returns `&str`. Can you spot why we don't need to annotate the return type's lifetime here? Because of lifetime elision. The compiler sees that the return type is a reference, and that the method takes `&self`. When a method takes `&self`, the compiler assumes the return type has the same lifetime as `&self`. It fills in the blank automatically.

## Lifetime Elision Rules
You might be wondering: why do some functions need explicit lifetime annotations and others do not? The answer is that Rust has three lifetime elision rules. The compiler applies these rules to every function signature. If the rules can figure out the lifetimes, you do not need to write them. If the rules cannot figure it out, the compiler asks you to be explicit.

Here are the three rules for function and method signatures:

| Rule | Description                                                                                       |
| ---- | ------------------------------------------------------------------------------------------------- |
| 1    | Each input reference gets its own lifetime parameter                                              |
| 2    | If there is exactly one input lifetime, it is assigned to all output references                   |
| 3    | If the method takes &self or &mut self, the lifetime of self is assigned to all output references |

Let's see these rules in action.
```rust
fn hello() -> &str { // ERROR: no input references
"hello"
}
```
This fails because there are no input references, so the compiler cannot figure out what lifetime to assign to the output. The fix is to return `&'static str`:
```rust
fn hello() -> &'static str {
"hello"
}
```
String literals have the `'static` lifetime, meaning they live for the entire duration of the program.

```rust
fn first_word(s: &str) -> &str { // Works! Rule 2 applies
// ...
}
```
One input reference, one output reference. Rule 2 assigns the input lifetime to the output.

```rust
fn longest(x: &str, y: &str) -> &str { // ERROR: two input references
// ...
}
```
Two input references, one output reference. Rule 1 creates two lifetimes (`'a`, `'b`). Rule 2 does not apply because there are two inputs, not one. Rule 3 does not apply because this is not a method. So the compiler asks for explicit annotations.

```rust
impl<'a> Config<'a> {
fn get_name(&self) -> &str { // Works! Rule 3 applies
    &self.name
}
}
```

Takes `&self`. Rule 3 assigns the lifetime of self to the output. No explicit annotation needed.
These three rules cover the vast majority of cases. In practice, explicit lifetime annotations on functions are relatively rare. Most of the time, the elision rules handle it.

## The `'static` Lifetime
The `'static` lifetime is special. It means "this reference is valid for the entire program." String literals are `'static`:
```rust
let s: &'static str = "hello world";
```
The string `"hello world"` is embedded directly in the compiled binary. It exists for as long as the program runs, so its lifetime is `'static`.

There is another way to get a `'static` lifetime that surprises beginners:
```rust
fn main() {
let s: &'static str = "hello";
}
```

This works because string literals are stored in the binary. But this does not:
```rust
fn make_string() -> &'static str {
let s = String::from("hello");
&s // ERROR: cannot return reference to local variable
}
```
Even though we wrote `&'static str`, the compiler knows we are lying. `s` is a heap-allocated `String` that will be dropped at the end of the function. It cannot have `'static` lifetime.

The `'static` bound is also commonly seen in trait bounds:
```rust
fn process<T: 'static>(t: T) {
// T does not contain any non-static references
}
```
This means `T` cannot contain any references that are not `'static`. If `T` is `i32`, that is fine because integers do not contain references. If `T` is `&'a str` where `'a` is not `'static`, this will not compile.

Now that we understand how lifetime relationships work in functions and structs, let's start working on our project
## The Project: Generic LRU Cache
Now that you understand lifetimes, let's build a generic LRU cache. An LRU (Least Recently Used) cache is a data structure that stores a fixed number of items. When the cache is full and a new item is added, the least recently used item is evicted to make space.

Our LRU cache will:
- Be generic over key type `K` and value type `V`
- Hold a borrowed `Config` reference with a lifetime annotation
- Provide zero-copy `get` that returns `Option<&V>` with proper lifetime annotations
- Support `put`, `get`, and `contains` operations
- Be split into a separate `src/cache.rs` module

We are using an LRU cache because it gives us a practical way to work with borrowed data and references returned from methods. The cache holds a reference to a configuration struct, and the `get` method returns references to values stored in the cache. Both of these need proper lifetime annotations to satisfy the borrow checker.

### Project Setup
Open your terminal and run:
```shell
cargo new lru_cache
cd lru_cache
```

Now open the `Cargo.toml` file.  We do not need any external dependencies for this project. Everything is standard library.
```toml
[package]
name = "lru_cache"
version = "0.1.0"
edition = "2024"
```

We will create two files: `src/cache.rs` for the cache implementation and `src/main.rs` for the demo. 

> The below code snippets will come under `src/cache.rs` file, please, check the complete code on github if you are coding along while reading

### The Config Struct
Before we build the cache, we need a configuration struct. The cache will borrow from it.
```rust
struct CacheConfig {
max_capacity: usize,
name: String,
}
```
The cache will hold a reference to this config using a lifetime annotation.

### The Cache Struct
Now let's define the cache struct. This is where lifetimes come in.
```rust
struct Cache<'a, K, V> {
config: &'a CacheConfig,
items: Vec<(K, V)>,
access_order: Vec<usize>,
}
```
Now, let me explain what we just did. `Cache<'a, K, V>` has three generic parameters: a lifetime `'a` and two type parameters `K` and `V`. The `config` field is `&'a CacheConfig`, meaning it borrows a `CacheConfig` that must live at least as long as `'a`. The cache cannot outlive the config it borrows from.

`items` is a `Vec<(K, V)>` holding the key-value pairs. `access_order` is a `Vec<usize>` of indices into `items`, ordered from most recently used to least recently used.

### The Constructor
We need a way to create a `Cache`. The constructor takes a reference to a `CacheConfig`:
```rust
impl<'a, K, V> Cache<'a, K, V> {
fn new(config: &'a CacheConfig) -> Self {
    Cache {
        config,
        items: Vec::new(),
        access_order: Vec::new(),
    }
}
}
```
`new` takes `config: &'a CacheConfig` and returns `Cache<'a, K, V>`. The lifetime `'a` is the same for both the input reference and the output struct. This tells the compiler that the cache is borrowing from the config, and the cache cannot outlive the config.

### Capacity Getter
Since the cache borrows from `CacheConfig`, it can access its fields through the reference:
```rust
impl<'a, K, V> Cache<'a, K, V> {
fn capacity(&self) -> usize {
    self.config.max_capacity
}
}
```
Notice that `capacity` returns `usize`, not a reference. It reads the value from the borrowed config and returns an owned copy. No lifetime annotation needed.

But what if we wanted to return a reference to a field of `config`?
```rust
impl<'a, K, V> Cache<'a, K, V> {
fn name(&self) -> &str {
    &self.config.name
}
}
```
This works without explicit lifetime annotations because of elision rule 3. The method takes `&self`, so the output reference is assumed to have the same lifetime as `self`. Since `self` is `&'a CacheConfig` internally, the returned `&str` is actually `&'a str`. But we don't need to write any of that.

### The `bump` Helper Method
The `bump` helper moves a recently accessed index to the front of the access order:
```rust
fn bump(&mut self, index: usize) {
	let pos = self.access_order.iter().position(|&i| i == index).unwrap();
	self.access_order.remove(pos);
	self.access_order.insert(0, index);
}
```
It finds where the index currently is in the access order, removes it, and inserts it at the front. This is called every time a key is accessed via `get` or updated via `put`.
### The `get` Method
The `get` method looks up a key and returns a reference to the value without copying it:
```rust
pub fn get(&mut self, key: &K) -> Option<&V>
where
	K: PartialEq,
{
	let idx = self.items.iter().position(|(k, _)| k == key)?;
	self.bump(idx);
	Some(&self.items[idx].1)
}
```
Now, let me explain what we just did. First, we search for the key using `self.items.iter().position(...)`. The `position` method walks the iterator and returns the index of the first element where the closure returns `true`. If no element matches, it returns `None`, and the `?` operator makes the whole function return `None` immediately.
Once we have the index, we call `self.bump(idx)` to move that index to the front of the access order. Then we return `Some(&self.items[idx].1)`, a reference to the value at that position.
The return type is `Option<&V>`. Notice there is no explicit lifetime annotation on the return value. This works because of elision rule 3: the method takes `&mut self`, so the output reference has the same lifetime as `self`. The important thing is that `get` returns a reference to the value stored inside the cache. No cloning, no copying. The caller gets a direct reference to the cached data. This is zero-copy.

### The `put` Method
The `put` method inserts or updates a key-value pair. If the cache is full, it evicts the least recently used item:
```rust
pub fn put(&mut self, key: K, value: V)
where
	K: PartialEq,
{
	if let Some(idx) = self.items.iter().position(|(k, _)| *k == key) {
		self.items[idx].1 = value;
		self.bump(idx);
		return;
	}
	if self.items.len() >= self.capacity() {
		let evict_index = self.access_order.pop().unwrap();
		self.items.remove(evict_index);
		for idx in self.access_order.iter_mut() {
			if *idx > evict_index {
*idx -= 1;
			}
		}
	}
	self.items.push((key, value));
	self.access_order.insert(0, self.items.len() - 1);
}
```

Now, let me explain what we just did. The method first checks if the key already exists using `position()` again. If it does, we update the value in place with `self.items[idx].1 = value`, bump the access order, and return. 
If the key does not exist, we check if the cache is full by comparing `self.items.len()`against `self.capacity()`. If we are at capacity, we need to evict the least recently used item. The LRU item is at the end of `access_order`, so we `pop()` it to get the eviction index, then `remove` that item from the vector. After removal, any indices in `access_order` that pointed beyond the removed item need to be decremented by one because `remove` shifts all later elements down.
Finally, we push the new key-value pair onto `items` and insert its index (which is `items.len() - 1`) at the front of `access_order`, marking it as most recently used.

### The `contains` Method
A simple check for key existence:
```rust
impl<'a, K: PartialEq, V> Cache<'a, K, V> {
fn contains(&mut self, key: &K) -> bool {
    self.get(key).is_some()
}
}
```
We call `self.get(key)` which bumps the access order, then check if the result is `Some`.

> Notice that `contains` takes `&mut self` instead of `&self`. That is because checking for existence also updates the access order, which mutates the cache state.

### The `len` Method
```rust
impl<'a, K, V> Cache<'a, K, V> {
fn len(&self) -> usize {
    self.items.len()
}
}
```

### Module Split: `src/cache.rs`
Right now, everything is in one file. Let's split the cache implementation into its own module. Create a new file `src/cache.rs`:
```rust
pub struct CacheConfig {
pub max_capacity: usize,
pub name: String,
}
pub struct Cache<'a, K, V> {
config: &'a CacheConfig,
items: Vec<(K, V)>,
access_order: Vec<usize>,
}
impl<'a, K, V> Cache<'a, K, V> {
pub fn new(config: &'a CacheConfig) -> Self {
    Cache {
        config,
        items: Vec::new(),
        access_order: Vec::new(),
    }
}
pub fn capacity(&self) -> usize {
    self.config.max_capacity
}
pub fn name(&self) -> &str {
    &self.config.name
}
pub fn len(&self) -> usize {
    self.items.len()
}
pub fn contains(&mut self, key: &K) -> bool
where
    K: PartialEq,
{
    self.get(key).is_some()
}
pub fn get(&mut self, key: &K) -> Option<&V>
where
    K: PartialEq,
{
    let idx = self.items.iter().position(|(k, _)| k == key)?;
    self.bump(idx);
    Some(&self.items[idx].1)
}
pub fn put(&mut self, key: K, value: V)
where
    K: PartialEq,
{
    if let Some(idx) = self.items.iter().position(|(k, _)| *k == key) {
        self.items[idx].1 = value;
        self.bump(idx);
        return;
    }
    if self.items.len() >= self.capacity() {
        let evict_index = self.access_order.pop().unwrap();
        self.items.remove(evict_index);
        for idx in self.access_order.iter_mut() {
            if *idx > evict_index {
                *idx -= 1;
            }
        }
    }
    self.items.push((key, value));
    self.access_order.insert(0, self.items.len() - 1);
}
fn bump(&mut self, index: usize) {
    let pos = self.access_order.iter().position(|&i| i == index).unwrap();
    self.access_order.remove(pos);
    self.access_order.insert(0, index);
}
}
```

Now, let me explain what we just did. Everything is marked `pub` that needs to be visible outside the module: the structs, `new`, `get`, `put`, `capacity`, `len`, `name`, `contains`. The `bump` method is private because it is an internal helper.
Notice the where `K: PartialEq` bounds on `get`, `put`, and `contains`. These methods need to compare keys, so `K` must implement `PartialEq`. The `where` clause is placed on each method that needs it, not on the `impl` block. This is a stylistic choice. You could also put it on the `impl` block:
```rust
impl<'a, K: PartialEq, V> Cache<'a, K, V> {
// all methods here
}
```
Either is fine. I prefer the method-level bounds when only some methods need the constraint. `capacity`, `len`, `new`, and `name` do not need `PartialEq`.

### The Demo in `src/main.rs`
Now let's wire everything together in `src/main.rs`. First, we declare the module and use the types:
```rust
mod cache;
use cache::{Cache, CacheConfig};
```
`mod cache;` tells Rust to look for `src/cache.rs` and load it as a module. The items inside are accessible as `cache::Cache`, `cache::CacheConfig`, etc.

Now the demo:
```rust
fn main() {
let config = CacheConfig {
    max_capacity: 3,
    name: String::from("demo_cache"),
};
let mut cache = Cache::new(&config);
cache.put("key1", 100);
cache.put("key2", 200);
cache.put("key3", 300);
println!("Cache '{}' has {} items", cache.name(), cache.len());
println!("Capacity: {}", cache.capacity());
if let Some(value) = cache.get(&"key1") {
    println!("key1 = {}", value);
}
cache.put("key4", 400);
println!("After inserting key4:");
println!("Cache has {} items", cache.len());
// key2 should have been evicted (LRU)
if cache.contains(&"key2") {
    println!("key2 is still in cache (unexpected)");
} else {
    println!("key2 was evicted (expected)");
}
if let Some(value) = cache.get(&"key3") {
    println!("key3 = {}", value);
}
}

```

Now, let me explain why this compiles and works despite lifetimes being everywhere. The `config` is created first. It lives until the end of `main`. The `cache` is created with `Cache::new(&config)`, borrowing from `config`. The cache cannot outlive `config`. Since both are dropped at the end of `main` (cache first, then config, in reverse order of creation), the lifetimes are satisfied.

When we call `cache.get(&"key1")`, it returns `Option<&i32>`. The returned reference points into the cache's internal storage and remains valid for the duration of the borrow.

### Running the Project
Type this in your terminal:
```shell
cargo run
```
You should see output like this:

```shell
Cache 'demo_cache' has 3 items
Capacity: 3
key1 = 100
After inserting key4:
Cache has 3 items
key2 was evicted (expected)
key3 = 300
```
The cache evicted `key2` because it was the least recently used item. `key1` stayed in the cache because calling `get` moved it to the front of the access order.

## Conclusion
In this post, you learned about lifetimes, the three elision rules, and the `'static` lifetime. You built a generic LRU cache with a lifetime-annotated config reference and zero-copy get that returns `Option<&V>`, split into `src/cache.rs` and `src/main.rs`.
In the next article, we will learn about `HashMap` and build an **inverted index search engine**. See you soon.

If you like reading this, please subscribe and share this with others. It'll really help me and motivate me to keep publishing more such articles.

# Learn Rust Ownership and Borrowing By Building Mini Grep

> In this post, we are going to build mini grep in Rust by learning about Rust's ownership and borrowing concept

Source: https://blog.sheerluck.dev/posts/rust/learn-rust-ownership-by-building-mini-grep/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-rust-ownership-by-building-mini-grep/index.md

In this post, we are going to learn about the problem Rust is solving, Rust's ownership, borrowing concept and build a mini grep clone in Rust. I'm really excited for the project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles. Let's start.

You can get the complete source code [here](https://github.com/MrSheerluck/mini-grep-in-rust)

## The Problem Rust is Solving
Every program needs memory and you use memory to store data like strings, numbers, lists whatever. There is an important question we need to answer and that is **who is responsible for freeing that memory when you're done with it?**

There are two traditional answers.

### The Programmer Does it Manually
Like in C programming language, you call `malloc` to allocate memory and `free` to release it. The best part of this thing is speed and control but the problem is that we can make mistakes and we might free memory twice or forget to free it or use it after freeing it. 

These bugs are problematic. They're hard to reproduce and hard to debug.

### The Language Does it Automatically
Like in Python programming language. There is a garbage collector runs in the background, figures out what memory is no longer reachable, and frees it. The best part is you basically can't have those dangerous memory bugs but the problem is performance. The garbage collector runs at unpredictable times, pauses your program and add overhead.

When it comes to Rust, it gives you memory safety without a garbage collector. It does this through a system. This system is built on three ideas: ownership, borrowing and slices.

## Ownership
Here's the core idea: in Rust, every piece of data has exactly one variable that "owns" it. When that variable goes out of scope, the data is automatically freed. No GC needed, because the compiler can figure out exactly when each piece of data should be cleaned up just by looking at the code.

```rust
fn main() {
	let s = String::from("hello"); // s owns the string
} // s goes out of scope here and Rust automatically frees the string
```

I hope you understand the above example but now lets get to the interesting part.

What happens when you do this?
```rust
let s1 = String::from("hello");
let s2 = s1;
```
In most languages, you'd expect both `s1` and `s2` to point to the same string or `s2` to be a copy of `s1`. In Rust, neither happens. **Ownership moves from `s1` to `s2`.

After the second line, `s1` no longer exists as far as the compiler is concerned. Try to use it and you get a compile error:
```rust
let s1 = String::from("hello");
let s2 = s1;
println!("{}", s1); // ERROR: value borrowed here after move
```

You might ask, **why does Rust do this?** Because `String` is heap allocated. If both `s1` and `s2` owned it, who would free it? If both tried, you'd have a double free bug.

So by keeping a single owner, Rust keeps it simple.

But for simple types like integers don't have this issue as they live on the stack, copying them is trivial and there's no heap memory to worry about. So, for those, Rust just copies:
```rust
let x = 5;
let y = x;
println!("{}", x); // Fine - integers are copied not moved
```

Technically, integers implement the `Copy` trait. `String` does not, so it moves instead. We will understand what trait is later in the series.

## Borrowing
Now the question is, if ownership moves when you pass data around, how do you actually use data in function? If you pass a `String` into a function, does the function consume it?
```rust
fn print_it(s: String) {
	println!("{}", s);
} // s is dropped here

fn main() {
	let s = String::from("hello");
	print_it(s); // ownership moves into the function
	println!("{}", s); // ERROR: s was moved
}

```
Now, this is a problem. If you can't use `s` anymore once you pass it to a function, then its too much restrictive. You don't want every function call to permanently consume your data. To handle this situation, Rust gives you **borrowing**. This means instead of giving ownership, you lend a *reference* to the data.

```rust
fn print_it(s: &String) {
	println!("{}", s);
} // the reference goes out of scope, but the original string is unaffected

fn main() {
	let s = String::from("hello");
	print_it(&s); // lend a reference, don't move ownership
	println!("{}", s); // still works, s is still the owner
}
```

The `&` means "a reference to". You're not passing the string directly instead you are passing a pointer to the string and the compiler tracks this carefully. The function can look at the data through the reference but it doesn't own it, so it can't free it and ownership stays with the original variable.

This is called a **shared reference** or an **immutable reference**. Through this reference, you can read, but you can't modify. You can have as many of these as you want simultaneously because reading doesn't conflict with reading.

### Mutable References
What if you need to modify data? There's a mutable reference for that:
```rust
fn add_world(s: &mut String) {
	s.push_str(" world");
}

fn main() {
	let mut s = String::from("hello");
	add_world(&mut s);
	println!("{}", s); // "hello world"
}
```
If you want mutable access, then you need to make the variable as `mut` mutable and give a mutable reference to the function. But there's a catch, **you can have one mutable reference at a time, and you cannot have any immutable references at the same time**. Wait, let me give you an example:
```rust
let mut s = String::from("hello");

let r1 = &s; // immutable borrow
let r2 = &mut s; // ERROR: can't borrow as mutable while immutable borrow exists
```

But why can't we do this, what's the issue? Because if you could read and write at the same time, you'd have a data race, you're reading data while something else is modifying it. The result is undefined and Rust makes this entire category of bug impossible at the compile time.

To give you a mental model of this borrowing concept:
> You can have either many readers or one write, never both simultaneously. This is not enforced at runtime but by the compiler before your code runs.

## Slices - Borrowing a Piece of Something
Sometimes you don't want to borrow an entire `String`, you just want to borrow part of it. That's what slices are for.
```rust
let s = String::from("hello world");
let hello = &s[0..5];
let world = &s[6..11];
```

`hello` here is of type `&str`. Note that, its not a new string, instead it's a reference that points directly into the memory of `s`.
> A slice is a borrow. It points into someone else's data and that means while a slice is alive, the borrow checker won't let you modify or drop the original.

```rust
let mut s = String::from("hello world");
let word = &s[0..5]; // word borrows from s
s.clear() // ERROR: can't mutate s while word is borrowing from it
println!("{}", word)
```

If `s.clear()` were allowed, it would wipe out that `word` is pointing and you'd have a dangling reference (a pointer into freed memory). In C, this compiles and runs, and crashes or corrupts data unpredictably. In Rust, it doesn't compile. The borrow checker sees that `word` is still alive and still pointing into s, and it rejects the mutation.

> `&str` vs `&String`
> You'll see both `&str` and `&String` in Rust code, and the distinction matters:
> `&String` is a reference to a heap allocated `String` object
> `&str` is a reference to a sequence of UTF-8 bytes

In practice, when writing functions that just need to read a string, you should prefer `&str` as the parameter type as it accepts both string literals and references to `String` values.

Let's now start building our grep clone but before that lets learn a little bit about mini grep.

## Grep 101
`grep` is a command line tool that searches through text for lines that match a pattern. That's it. The name stands for **Global Regular Expression Print**. It reads input, tests every line against a pattern and prints the lines that match.

Open your terminal and try this:
```shell
echo -e "hello world\ngoodbye world\nhello rust" | grep "hello"
```
Output:
```shell
hello world
hello rust
```

It went through three lines and found two that contained the word "hello", and printed those. The third line didn't match so it was ignored. Now, lets try with a flag:
```shell
echo -e "hello world\ngoodbye world\nhello rust" | grep -v "hello"
```

output:
```shell
goodbye world
```
`-v` means invert, this print lines that do not match

Let's try another flag:
```shell
echo -e "hello world\ngoodbye world\nhello rust" | grep -c "hello"
```

output:
```shell
2
```
`-c` means count, this don't print lines instead just print how many matched 
## What are we Building
Before touching `main.rs`, let me tell you what we are building:
Our program will:
- accept a pattern and a path from the command line
- accept optional flags
- walk the directory (or read a single file) recursively
- for each file, read its contents, search line by line
- print results with filename, line number

## Project Setup
Run this in your terminal:
```bash
cargo new rgrep
cd rgrep
```

Now open the `Cargo.toml` file and add the dependencies with this:
```toml
[package]
name = "rgrep"
version = "0.1.0"
edition = "2024"

[dependencies]
regex = "1"
walkdir = "2"
```

We have added three dependencies:
- `regex` compiles and runs regular expression patterns against text
- `walkdir` recursively walks a directory tree, giving you every file one by one

## Imports
Open `src/main.rs` and delete everything. Write this:
```rust
use regex::Regex;
use std::env;
use std::fs;
use walkdir::WalkDir;
```
- `use regex::Regex` is used for regex operations
- `use std::env` is used access process environment like command line arguments and environment variables
- `use std::fs` here `fs` stands for file system and we are using this to read files, write files, checking if path exists etc.
- `use walkdir::WalkDir` this helps us to do recursive directory walk. Basically we can give it a directory and it'll return the every entry inside

### Reading Command Line Arguments
```rust
fn main() {
	let args: Vec<String> = env::args().collect();
}
```
Here, `env::args()` returns an **iterator** over the command line arguments. For example, when a user runs:
```shell
cargo run -- -v "hello" ./src
```
the arguments are: `["rgrep", "-v", "hello", "./src"]`. The first element is always the program's own name.

> An iterator is something you can step through one element at a time. The `.collect()` exhausts the iterator and gathers every element into a collection.

The type annotation `Vec<String>` tells Rust what kind of collection you want. `.collect()` can produce several types, so you have to specify.

> `Vec<String>` means a growable list of owned strings. You can check my previous article for more info.

#### Why `String` and not `&str`?
You might ask, why use `String` and not `&str`? To answer this, you need to understand what `&str` actually is. A `&str` is a reference, it does not own data, it points to data that already exists somewhere in memory and has a defined lifetime. String literals like `"hello"` work as `&str` because the compiler compiles them directly into the binary, so they live for the entire duration of the program.

Command line arguments are different, they come from the operating system at runtime. There is no pre-existing place in your program's memory where they live. Rust has to allocate heap memory to hold each argument string as it reads it. `String` is the type for heap-allocated, owned string data, it carries the data with it and is responsible for freeing it when it goes out of scope. `&str` cannot do this because it is just a pointer, and a pointer needs something to point to.

So `Vec<String>` is the only option here. Each `String` in the vector owns its argument data, and the vector owns all the `String`s. When `args` goes out of scope, everything is cleaned up automatically.

### Validating Argument Count
```rust
if args.len() < 3 {
	println!("Usage: rgrep [OPTIONS] <pattern> <path>");
	eprintln!("Options: -v -c -l");
	std::process::exit(1);
}
```

`args.len()` returns how many arguments we have. The minimum valid call is:
```shell
rgrep "pattern" ./path
```
This gives us `["rgrep", "pattern", "./path"]` a total of 3 elements. If there are fewer than three, the user didn't provide enough and we print an error and exit.

> `eprintln!` is like `println!` but writes to stderr instead of stdout. Errors and usage messages conventionally go to stderr so they don't pollute the output when piping.

Finally, `std::process::exit(1)` terminates the program with exit code 1.

### Parsing Flags
```rust
let mut invert = false;
let mut count_only = false;
let mut files_only = false;
let mut pattern_index = 1;

for i in 1..args.len() {
	match args[i].as_str() {
		"-v" => invert = true,
		"-c" => count_only = true,
		"-l" => files_only = true,
		_ => {
			pattern_index = i;
			break;
		}
	}
}
```
We start with four mutable variables. The first three tracks which flags were passed. `pattern_index` tracks where in `args` the pattern is. This starts with `1` because `args[0]` is always the program name

We are iterating over all the arguments and checking which flag is being passed by the user and then setting that to `true`. The `_` is the catch-all, if its none of the above that means we have hit the pattern and we are recording its index and then stop the loop with `break`.

So, if the user runs `rgrep -v -c "hello" ./src`, after the loop: `invert` is true, `count_only` is true, `pattern_index` is 3 (pointing at "hello")

### Extracting Pattern and Path
```rust
if pattern_index + 1 >= args.len() {
	eprintln!("Error: missing pattern or path");
	std::process::exit(1);
}

let pattern = &args[pattern_index];
let path = &args[pattern_index + 1];
```
After the flag loop, `pattern_index` is pointing to the pattern. The paths comes right after it at `pattern_index + 1`. If that index is out of bounds, the user forgot to provide one of them.

`let pattern = &args[pattern_index]` this is the borrowing concept. We don't need our own copy of the pattern, we just need to read it and act accordingly. So, `pattern` is a `&String` type, a reference that borrows from `args`.
Same thing happens for `path`.

### Compiling The Regex
```rust
let regex = match Regex::new(pattern) {
	Ok(r) => r,
	Err(e) => {
		eprintln!("Invalid pattern: {}", e);
		std::process::exit(1);
	}
};
```
`Regex::new(pattern)` takes the pattern string and compiles it into a `Regex` object. "Compiling" here means parsing the pattern syntax and building an internal state machine that can efficiently test strings against it. 

`Regex::new` returns a `Result<Regex, Error>`, this is a enum with two variants: `Ok(value)` meaning success and `Err(error)` meaning failure. This is the way Rust handles operations that can fail.

`match` on a `Result` is the standard way to handle it:
- `Ok(r)` - the pattern is compiled successfully, `r` is the `Regex` object
- `Err(e)` - the pattern was invalid (malformed regex syntax). `e` is the error and we just print it and exit

> If you don't understand clearly, no worries, we will learn more about it in a later article, for now just follow and finish the project

### Walking the Directory
```rust
for entry in WalkDir::new(path).into_iter() {
	let entry = match entry {
		Ok(e) => e,
		Err(_) => continue,
	};

	if !entry.file_type().is_file() {
		continue;
	}

	let file_path = entry.path();
}
```
`WalkDir::new(path).into_iter()` starts a recursive walk at path and gives us entries one at a time. Each entry is a `Result<DirEntry>` because reading a directory can fail may be due to permissions, broken symlinks, etc.

We `match` on each entry. Success case gives us `DirEntry` and in case of failure, we continue to the next iteration

If `!entry.file_type().is_file()` gives us both files and directories, we only care about files so we skip anything that is not a file with `continue`

`entry.path()` gives us a `&Path`, its a borrowed reference to the file's path. `Path` is Rust's type for filesystem paths.

### Reading Files
Once we get to know that we got a file and not a directory, we are keeping its file path and then we are now going to read that file's content:
```rust
let contents = match fs::read_to_string(file_path) {
	Ok(c) => c,
	Err(_) => continue,
};
```
`fs::read_to_string(file_path)` reads the entire file and returns `Result<String>`. On success, `contents` has the file's text and on failure case (again may be due to permission or something else), we just continue to the next iteration.

### The Search Function
We want to search through the content and get back the matching lines. The question we need to ask is what should the search function take and what should it return?

It should take `contents` as a borrow that means `&str` because it only needs to read the text, not own it. 

For the return value, each matching line is a slice of `contents`. This means a `&str` pointing directly into file's text but returning those slices requires lifetime annotations, which we haven't covered yet. So instead we return an owned `String` copy of each matching line. It's of course less efficient but its correct and simple for now.

```rust
fn search(contents: &str, regex: &Regex, invert: bool) -> Vec<(usize, String)> {
let mut results: Vec<(usize, String)> = Vec::new();
let mut line_number = 1;

for line in contents.lines() {
    let is_match = regex.is_match(line);

    let should_include = if invert { !is_match } else { is_match };

    if should_include {
        results.push((line_number, line.to_string()));
    }

    line_number += 1;
}
results
}
```

We are borrowing `contents`, `regex` as we only need to read them. The function is returning `Vec<(uszie, String)>`, this is a vector of tuples. Each tuple is a line number. `contents.lines()` gives each line as a `&str` slice pointing into `contents`. 

Inside the loop, we are matching the regex with that line, then checking the `invert` flag and accordingly decides if we need to include the line and line number in the result or not.

`line.to_string()` converts the borrowed `&str` slice into an owned `String`. This is the copy that we take so we don't need to worry about the slice outliving `contents`. Finally, we are just returning the `results`

### Calling the Search Function

Now we have all the pieces. We have walked the directory, read the file content, and we have our `search` function. Now we just need to call it and handle the output based on the flags.

```rust
let matches = search(&contents, &regex, invert);
```

We are passing `&contents` and `&regex` as borrows. The `search` function only needs to read them, not own them. `invert` is a `bool` which implements `Copy`, so it's just copied over directly.

`matches` is now a `Vec<(usize, String)>` holding every line that matched (or didn't match if `-v` was passed).

#### Handling `-l` (files only)

```rust
if files_only {
if !matches.is_empty() {
    println!("{}", file_path.display());
}
continue;
}
```

If the user passed `-l`, we only care about whether the file had any matches at all, not what those lines are. `.is_empty()` checks if the vector has zero elements. If it does have matches, we print the file path and move on with `continue` to the next file.

`file_path.display()` converts the `Path` into something printable. `Path` doesn't implement `Display` directly but `.display()` returns a wrapper that does.

#### Handling `-c` (count only)

```rust
if count_only {
println!("{}: {}", file_path.display(), matches.len());
continue;
}
```

If the user passed `-c`, we don't print the actual lines either. We just print the file path and the number of matches. `.len()` on a `Vec` returns the number of elements as a `usize`.

#### The Default Case

```rust
for i in 0..matches.len() {
let line_num = matches[i].0;
let line = &matches[i].1;
println!("{}:{}: {}", file_path.display(), line_num, line);
}
```

If no output flag was passed, we print every matching line with its filename and line number. `matches[i].0` accesses the first element of the tuple (the line number) and `matches[i].1` accesses the second (the line text).

`let line = &matches[i].1` is a borrow again. We just need to read the string to print it, no reason to take ownership or clone it.

The output format `filename:line_number: line` is the standard grep format that most editors understand. If you run this in a terminal that supports it, you can click those results and jump directly to the file and line.

## Run The Project
```shell
cargo run -- "fn" ./src
```
You should get something like this:
```shell
./src/main.rs:8: fn search(contents: &str, regex: &Regex, invert: bool) -> Vec<(usize, String)> {
./src/main.rs:27: fn main() {
```

You can try using flags too like this:
```shell
cargo run -- -c "fn" ./src
```
you should get something like this:
```shell
./src/main.rs: 2
```

## Conclusion
This was a long one too but I think now you have some practical knowledge on ownership and borrowing. I'm not going too much theoretical as that'll be just boring. We'll learn more about `Result` in a later article. In the next one, we will **build a JSON Parser** by learning about structs, enums and pattern matching. See you soon.

# Learn Rust Smart Pointers and Interior Mutability by Building Git Commit Graph Viewer

> In this post, we are going to learn about smart pointers and interior mutability in Rust. Once we cover all the concepts, we will build a Git Commit Graph Viewer

Source: https://blog.sheerluck.dev/posts/rust/learn-rust-smart-pointers-and-interior-mutability-by-building-git-commit-graph-viewer/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-rust-smart-pointers-and-interior-mutability-by-building-git-commit-graph-viewer/index.md

In this post, we are going to learn about smart pointers and interior mutability in Rust. Once we cover all the concepts, we will build a **Git Commit Graph Viewer** that reads a real `.git` directory, parses every commit, builds a connected DAG (directed acyclic graph) using `Rc<RefCell<CommitNode>>` with `Weak` edges, and renders an ASCII graph of your commit history. I am really excited for this project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles.

The only prerequisite is that you have read the previous articles in this series, as I will assume you know ownership, borrowing, structs, enums, pattern matching, error handling, generics, traits, lifetimes, HashMap, iterators, and closures.

Get the source code from [here](https://github.com/MrSheerluck/git-commit-graph)

## What is a Smart Pointer

A pointer is a variable that holds a memory address. A smart pointer is a data structure that acts like a pointer but also has additional metadata and capabilities. In Rust, `String` and `Vec<T>` are technically smart pointers. They own heap-allocated data and manage it automatically.

But when people say "smart pointers" in Rust, they usually mean the types in the standard library that provide ownership and borrowing patterns beyond the basic reference: `Box<T>`, `Rc<T>`, `RefCell<T>`, `Weak<T>`, and later `Arc<T>` and `Mutex<T>`.

We will start with `Box` (which you have already seen) and build up to the more complex ones.

## Box Recap

You saw `Box<T>` in the article on closures. A `Box<T>` is the simplest smart pointer. It allocates a value on the heap and owns it. When the `Box` is dropped, the heap memory is freed.

```rust
let x = Box::new(42);
```

This stores `42` on the heap. The `Box` on the stack holds a pointer to it. You use `x` like a regular `i32` thanks to deref coercion.

The two main use cases for `Box` are recursive types (like our JSON parser's recursive `JsonValue` enum) and trait objects (`Box<dyn Trait>`). Both require heap allocation because the compiler cannot know the size of the type at compile time.

## Rc: Reference-Counted Smart Pointer

Now let's talk about `Rc<T>`. `Rc` stands for "Reference Counted." It enables multiple ownership. Multiple parts of your code can share ownership of the same value, and the value is dropped only when all owners are gone.

```rust
use std::rc::Rc;

let a = Rc::new(42);
let b = Rc::clone(&a);
let c = Rc::clone(&a);
```

Here, `a`, `b`, and `c` all point to the same heap allocation containing `42`. `Rc::clone` does not deep-copy the data. It increments a reference counter. When `a`, `b`, and `c` all go out of scope, the counter reaches zero and the data is freed.

Let me explain what `Rc` gives us. Without `Rc`, each value has exactly one owner. If we want to share a value between multiple structs, we could use references with lifetimes, but lifetimes force us to think about who outlives whom. `Rc` says: "Instead of enforcing a single owner, I will keep a runtime reference count and free the data when the last owner disappears."

```rust
use std::rc::Rc;

let data = Rc::new(vec![1, 2, 3]);
let shared1 = data.clone();
let shared2 = data.clone();

println!("Reference count: {}", Rc::strong_count(&data));
```

`Rc::strong_count` tells you how many handles point to the same allocation. In this case, it would print `3` because `data`, `shared1`, and `shared2` all share ownership.

There is an important limitation: `Rc` is not thread-safe. It uses non-atomic reference counting, which is faster but unsafe across threads. For multi-threaded code, you use `Arc` (Atomic Reference Counted), which we will cover in a future article.

## RefCell: Interior Mutability

Now we get to the interesting one. `RefCell<T>` provides interior mutability. "Interior mutability" is the ability to mutate data even when the `RefCell` itself is immutable.

Normally, Rust's borrow rules are enforced at compile time:
- You can have either many immutable references or one mutable reference
- These rules are checked and enforced before your program runs

`RefCell<T>` moves these checks from compile time to runtime. You can always call `borrow` or `borrow_mut` on a `RefCell`. If you violate the rules, your program will panic at runtime.

```rust
use std::cell::RefCell;

let x = RefCell::new(42);

let r1 = x.borrow();
let r2 = x.borrow();
// Both immutable borrows are fine
println!("{} {}", r1, r2);

let mut r3 = x.borrow_mut();
*r3 = 100;
// This would panic: already immutably borrowed
// let r4 = x.borrow();
```

`borrow` returns a `Ref<T>`, which derefs to `&T`. `borrow_mut` returns a `RefMut<T>`, which derefs to `&mut T`. Both types automatically release the borrow when they go out of scope.

The runtime checking is why this pattern is called "interior mutability." The `RefCell` can be accessed through an immutable binding or immutable reference, yet the value inside it can still be mutated through `borrow_mut`.

Why would you want this? Consider a struct with a method that takes `&self` but needs to modify internal state:

```rust
use std::cell::RefCell;

struct Counter {
count: RefCell<u32>,
}

impl Counter {
fn increment(&self) {
    *self.count.borrow_mut() += 1;
}

fn value(&self) -> u32 {
    *self.count.borrow()
}
}
```

The `increment` method takes `&self` (an immutable reference), but it can still mutate `count` through `RefCell`. Without `RefCell`, we would need `&mut self` everywhere, which would make the code harder to use.

## `Rc<RefCell<T>>`: The Shared Mutable Pattern

Now combine `Rc` and `RefCell`. `Rc` gives us shared ownership. `RefCell` gives us interior mutability. Together, they give us shared mutable state.

This is one of the most common patterns in single-threaded Rust code.

```rust
use std::rc::Rc;
use std::cell::RefCell;

let shared = Rc::new(RefCell::new(42));

let a = Rc::clone(&shared);
let b = Rc::clone(&shared);

*a.borrow_mut() += 1;

println!("{}", shared.borrow()); // 43
println!("{}", a.borrow());      // 43
println!("{}", b.borrow());      // 43
```

All three handles see the same value because they share the same `RefCell`. Modifying through any handle affects all of them.

Let me read `Rc<RefCell<T>>` from the inside out:

- `T` is the actual data
- `RefCell<T>` wraps the data with runtime borrow checking
- `Rc<RefCell<T>>` stores a `RefCell<T>` inside an `Rc`-managed heap allocation and enables multiple owners.

You will see this pattern everywhere in Rust code that needs shared mutable state.

## The Problem: Rc Cycles

`Rc` has a problem. If two `Rc` values reference each other, they form a cycle. Neither will ever have a reference count of zero. The memory leaks.

```rust
use std::rc::Rc;
use std::cell::RefCell;

struct Node {
value: i32,
next: Option<Rc<RefCell<Node>>>,
}

let a = Rc::new(RefCell::new(Node { value: 1, next: None }));
let b = Rc::new(RefCell::new(Node { value: 2, next: None }));

a.borrow_mut().next = Some(Rc::clone(&b));
b.borrow_mut().next = Some(Rc::clone(&a));
// Now a and b point to each other. Neither will ever be dropped.
```

This is a memory leak. `Rc` cannot detect cycles.

Now, a Git commit graph is a DAG (directed acyclic graph), but that does not automatically mean `Rc` is safe to use in every direction. The important distinction is between the commit graph and the ownership graph created by our smart pointers.

Suppose a parent stores an `Rc` to its child, and the child stores an `Rc` back to its parent. The commit graph is still a DAG, but the ownership graph now contains a cycle. Neither node can ever reach a strong reference count of zero, so the memory leaks.

`Weak<T>` solves this problem. We make the graph edges non-owning references. The `HashMap` is the sole owner of every node through `Rc`, while the parent/child relationships are represented using `Weak`. When the `HashMap` is dropped, every strong reference disappears and the entire graph is freed correctly.

## Weak: Non-Owning References

`Weak<T>` is the solution. `Weak` is a version of `Rc` that does not own the value. It does not increment the strong reference count. The value can be dropped even while `Weak` references exist.

```rust
use std::rc::{Rc, Weak};
use std::cell::RefCell;

let strong = Rc::new(42);
let weak = Rc::downgrade(&strong);
// strong_count = 1, weak_count = 1

drop(strong);
// strong_count = 0, data is freed

match weak.upgrade() {
Some(value) => println!("Still alive: {}", value),
None => println!("Data was dropped"),
}
```

`Rc::downgrade` creates a `Weak` reference from an `Rc`. `Weak::upgrade` tries to get back an `Option<Rc<T>>`. If the data has been dropped (all strong references are gone), `upgrade` returns `None`.

The naming comes from strong references vs weak references. A strong reference keeps the value alive. A weak reference allows the value to be dropped.

| Pointer | Ownership | Can access data | Prevents drop | Use case |
|---|---|---|---|---|
| `Rc<T>` | Shared | Yes | Yes | Multiple owners |
| `Weak<T>` | None | Via `upgrade` | No | Break cycles, caches, graph edges |

This is exactly what we need for our commit graph. A parent commit can be referenced by many children through `Weak` pointers. The HashMap holds the strong references. When we finish traversing the graph, the HashMap is dropped, all strong counts hit zero, and all nodes are freed, even though they still have `Weak` edges pointing between them.

## The Project: Git Commit Graph Viewer

Now that you understand `Box`, `Rc`, `RefCell`, and `Weak`, let's build a Git Commit Graph Viewer.

Our program will:

- Accept a path to a `.git` directory
- Read and decompress every commit object from `.git/objects/`
- Build a connected graph of `CommitNode` structs using `Rc<RefCell<>>`
- Wire up parent and child edges as `Weak<>` references
- Render an ASCII graph showing the commit history with branch labels

You already built a Git Object Store Reader in Phase 8. That project parsed commits and rendered `git log --oneline` output. In this project, we build on that code, but instead of a flat HashMap of disconnected commits, we build a **fully connected graph** where each node knows its parents and children through smart pointers.

## Project Setup

Open your terminal and run:

```bash
cargo new commit_graph
cd commit_graph
```

Now open `Cargo.toml` and add the dependency:

```toml
[package]
name = "commit_graph"
version = "0.1.0"
edition = "2024"

[dependencies]
flate2 = "1"
```

The only external dependency is `flate2` for zlib decompression. Everything else `Rc`, `RefCell`, `Weak`, `HashMap` comes from the standard library.

Open `src/main.rs` and delete everything.

### Imports

```rust
use flate2::read::ZlibDecoder;
use std::cell::RefCell;
use std::collections::{HashMap, HashSet};
use std::fs;
use std::io::Read;
use std::path::Path;
use std::rc::{Rc, Weak};
```

- `flate2::read::ZlibDecoder` decompresses Git's zlib-compressed object files
- `std::cell::RefCell` provides interior mutability for our graph edges
- `std::collections::HashMap` stores the node lookup (hash → `Rc<RefCell<CommitNode>>`)
- `std::rc::{Rc, Weak}` provides shared ownership and non-owning references

### The CommitNode Struct

```rust
struct CommitNode {
hash: String,
short_hash: String,
message: String,
timestamp: u64,
parents: RefCell<Vec<Weak<RefCell<CommitNode>>>>,
children: RefCell<Vec<Weak<RefCell<CommitNode>>>>,
}
```

Let me read this struct from the inside out:

- `CommitNode` is a vertex in our graph. It represents one git commit.
- `hash` is the full 40-character SHA-1 hash. `short_hash` is the first 7 characters for display.
- `message` is the first line of the commit message.
- `timestamp` is the Unix timestamp from the commit's author line.

Now the key parts: `parents` and `children` are both `RefCell<Vec<Weak<RefCell<CommitNode>>>>`.

- `Weak<RefCell<CommitNode>>` is a non-owning reference to another node in the graph. The node lives in a `HashMap` behind an `Rc`. The `Weak` points to it without keeping it alive.
- `Vec<Weak<...>>` holds multiple such edges (a commit can have multiple parents and multiple children).
- `RefCell<Vec<...>>` enables interior mutability. We can push edges into these `Vec`s even when we only have an immutable reference to the node.

Why `RefCell`? Because after we create a `CommitNode` and store it behind `Rc`, we need to wire up its edges. The node already exists, the `Rc` is in the HashMap. Without `RefCell`, we would need `&mut CommitNode` to push into the `Vec`s, and you cannot get `&mut` from an `Rc`. With `RefCell`, we call `borrow_mut()` on the `RefCell`, which gives us runtime-checked mutable access to the `Vec`, and we push our `Weak` edges into it.

### The Raw Commit

We need a temporary struct to hold the raw data we parse from git objects:

```rust
struct RawCommit {
parent_hashes: Vec<String>,
author: String,
timestamp: u64,
message: String,
}
```

`RawCommit` holds the data as it comes out of the object file. It stores parent hashes as `String`s (not references) because after we parse all commits, we do a second pass to resolve those strings into actual `Weak` references to the corresponding nodes.

### Reading Git Objects

These functions are the same as in Phase 8. I will include them here with brief explanations so the code is self-contained.

```rust
fn read_head(git_dir: &Path) -> Option<String> {
let content = fs::read_to_string(git_dir.join("HEAD")).ok()?;
Some(content.trim().to_string())
}

fn resolve_ref(git_dir: &Path, reference: &str) -> Option<String> {
if reference.starts_with("ref: ") {
    let ref_path = &reference[5..];
    fs::read_to_string(git_dir.join(ref_path))
        .ok()
        .map(|s| s.trim().to_string())
} else {
    Some(reference.to_string())
}
}

fn read_object(git_dir: &Path, hash: &str) -> Vec<u8> {
let object_path = git_dir
    .join("objects")
    .join(&hash[..2])
    .join(&hash[2..]);
let compressed = fs::read(&object_path)
    .unwrap_or_else(|_| panic!("Failed to read object: {}", hash));
let mut decoder = ZlibDecoder::new(&compressed[..]);
let mut decompressed = Vec::new();
decoder.read_to_end(&mut decompressed)
    .unwrap_or_else(|_| panic!("Failed to decompress object: {}", hash));
decompressed
}
```

Now, let me explain what we just did. `read_head` reads the `.git/HEAD` file to find out which branch we are on. `resolve_ref` follows a symbolic reference like `ref: refs/heads/main` to its actual commit hash. `read_object` constructs the path `.git/objects/XX/YYYY...`, reads the file, and decompresses it with zlib.

### Parsing Object Headers and Commits

```rust
fn parse_object(data: &[u8]) -> (&str, &[u8]) {
let header_end = data.iter().position(|&b| b == 0)
    .expect("Invalid object: no null byte");
let header = std::str::from_utf8(&data[..header_end])
    .expect("Invalid object: header not valid UTF-8");
let content = &data[header_end + 1..];
let mut parts = header.splitn(2, ' ');
let obj_type = parts.next().unwrap();
let _size = parts.next().unwrap();
(obj_type, content)
}

fn parse_commit(content: &[u8]) -> RawCommit {
let content_str = std::str::from_utf8(content)
    .expect("Invalid commit: not valid UTF-8");
let mut parent_hashes = Vec::new();
let mut author = String::new();
let mut timestamp: u64 = 0;
let mut message = String::new();
let mut in_message = false;

for line in content_str.lines() {
    if in_message {
        if !message.is_empty() {
            message.push('\n');
        }
        message.push_str(line);
        continue;
    }
    if line.is_empty() {
        in_message = true;
        continue;
    }
    if let Some(hash) = line.strip_prefix("parent ") {
        parent_hashes.push(hash.to_string());
    } else if let Some(author_line) = line.strip_prefix("author ") {
        author = author_line.to_string();
        if let Some(last_space) = author_line.rfind(' ') {
            if let Some(prev_space) = author_line[..last_space].rfind(' ') {
                let ts_str = &author_line[prev_space + 1..last_space];
                if let Ok(ts) = ts_str.parse::<u64>() {
                    timestamp = ts;
                }
            }
        }
    }
}

RawCommit {
    parent_hashes,
    author,
    timestamp,
    message,
}
}
```

Now, let me explain what we just did. `parse_object` splits the decompressed bytes into the header (before the null byte) and the content (after it). The header contains the object type and size. `parse_commit` walks through the lines of the commit content, collecting parent hashes, the author timestamp, and the commit message. We skip the `tree` line since we only care about the graph structure. The timestamp parsing extracts the Unix seconds from the author line, which looks like `author Name <email> 1234567890 +0000`.

### Reading All Commits

```rust
fn read_all_commits(git_dir: &Path, head_hash: &str) -> HashMap<String, RawCommit> {
let mut raw_commits = HashMap::new();
read_commit_recursive(git_dir, head_hash, &mut raw_commits);
raw_commits
}

fn read_commit_recursive(
git_dir: &Path,
hash: &str,
raw_commits: &mut HashMap<String, RawCommit>,
) {
if raw_commits.contains_key(hash) {
    return;
}
let data = read_object(git_dir, hash);
let (obj_type, content) = parse_object(&data);
match obj_type {
    "commit" => {
        let commit = parse_commit(content);
        for parent in &commit.parent_hashes {
            read_commit_recursive(git_dir, parent, raw_commits);
        }
        raw_commits.insert(hash.to_string(), commit);
    }
    _ => panic!("Expected commit object, got {}", obj_type),
}
}
```

Now, let me explain what we just did. `read_commit_recursive` walks the entire commit graph from a starting hash, recursively reading every parent. The `HashMap` acts as a cache: if we have already read a commit hash, we return immediately so we never read the same object twice. This handles the case where two branches share common ancestors, the shared history is parsed only once. After this function returns, `raw_commits` contains every commit reachable from `head_hash`.

### Reading All Branches

To support rendering branch labels, we also read every branch reference:

```rust
fn read_all_branches(git_dir: &Path) -> Vec<(String, String)> {
let mut branches = Vec::new();
let heads_dir = git_dir.join("refs").join("heads");
if let Ok(entries) = fs::read_dir(&heads_dir) {
    for entry in entries.flatten() {
        let path = entry.path();
        if let Some(name) = path.file_name().and_then(|n| n.to_str()) {
            if let Ok(hash) = fs::read_to_string(&path) {
                branches.push((name.to_string(), hash.trim().to_string()));
            }
        }
    }
}
branches
}
```

Now, let me explain what we just did. `read_all_branches` reads loose local branch references from `.git/refs/heads/`. In many repositories, each local branch appears as a file containing the branch tip's commit hash. Git can also store references in `packed-refs`, but for simplicity this implementation only reads loose branch references.

### Building the Graph: First Pass - Creating Nodes

Now we get to the core of the article: building the graph using smart pointers. This happens in two passes.

In the first pass, we create an `Rc<RefCell<CommitNode>>` for every raw commit and store it in a `HashMap`:

```rust
fn build_graph(
raw_commits: &HashMap<String, RawCommit>,
) -> HashMap<String, Rc<RefCell<CommitNode>>> {
let mut nodes: HashMap<String, Rc<RefCell<CommitNode>>> = HashMap::new();

for (hash, raw) in raw_commits {
    let node = Rc::new(RefCell::new(CommitNode {
        hash: hash.clone(),
        short_hash: hash[..7].to_string(),
        message: raw.message.lines().next().unwrap_or("").to_string(),
        timestamp: raw.timestamp,
        parents: RefCell::new(Vec::new()),
        children: RefCell::new(Vec::new()),
    }));
    nodes.insert(hash.clone(), node);
}

nodes
}
```

Now, let me explain what we just did. For each raw commit, we create an `Rc::new(RefCell::new(CommitNode { ... }))`. The `RefCell` enables interior mutability so we can later wire up edges. The `Rc` enables shared ownership: the HashMap holds one strong reference, and later each child will hold a `Weak` reference pointing back to the parent. Both `parents` and `children` start as empty `Vec`s, we populate them in the second pass.

 Let me trace the memory layout. The `HashMap` key is a `String` (the commit hash). The value is an `Rc<RefCell<CommitNode>>`. The `Rc` stored in the HashMap points to a heap allocation containing the `RefCell<CommitNode>`. The `CommitNode` itself lives inside that allocation.

 Fields such as `String` and `Vec` are stored inline inside the `CommitNode`, but they manage their own heap allocations for their contents. For example, the `hash` field contains a `String` value inside the struct, while the actual string bytes live in a separate heap allocation. Likewise, `parents` and `children` contain `Vec`s whose elements live in separate heap allocations.

### Building the Graph: Second Pass - Wiring Edges

In the second pass, we walk through every node and wire up its parent and child edges using `Weak` references:

```rust
fn wire_edges(
nodes: &HashMap<String, Rc<RefCell<CommitNode>>>,
raw_commits: &HashMap<String, RawCommit>,
) {
for (hash, node) in nodes {
    if let Some(raw) = raw_commits.get(hash) {
        for parent_hash in &raw.parent_hashes {
            if let Some(parent_node) = nodes.get(parent_hash) {
                // Add parent as Weak reference to current node
                node.borrow()
                    .parents
                    .borrow_mut()
                    .push(Rc::downgrade(parent_node));

                // Add current node as child (Weak) to parent node
                parent_node
                    .borrow()
                    .children
                    .borrow_mut()
                    .push(Rc::downgrade(node));
            }
        }
    }
}
}
```

Now, let me explain what we just did. This is where `RefCell`'s interior mutability shines.

We iterate over every node in the graph. For each node, we look at its raw commit to find its parent hashes. For each parent hash, we look up the parent's `Rc<RefCell<CommitNode>>` in the `nodes` HashMap.

Then we do two things:

1. **Parent edge**: We push a `Weak` reference to the parent into the current node's `parents` Vec. We get mutable access to the Vec through `node.borrow().parents.borrow_mut()`. The first `.borrow()` gets an immutable reference to the `RefCell<CommitNode>`. The second `.borrow_mut()` gets mutable access to the `Vec` inside it.

2. **Child edge**: We push a `Weak` reference to the current node into the parent node's `children` Vec. This is the reverse direction. After this second pass, every node knows both its parents and its children.

`Rc::downgrade` takes an `&Rc<T>` and returns a `Weak<T>`. The strong count does not change. The weak count increments. This is important: the `HashMap` is the only thing keeping nodes alive through strong references. The edges are all `Weak`, so they do not prevent any node from being dropped if we remove it from the `HashMap`.

Let me also highlight why we need `RefCell` here. We already have an `Rc<RefCell<CommitNode>>` from the HashMap. We could also get the node by borrowing the HashMap: `nodes.get(parent_hash)` gives us an `Option<&Rc<RefCell<CommitNode>>>`. But we cannot call `parent_node.borrow_mut()` directly on the CommitNode because we only have a shared reference through `Rc`. The `RefCell` layer is what gives us the ability to call `borrow()` and `borrow_mut()` on a shared reference.

### Walking the Graph

Now let's write functions that walk the graph using `Weak::upgrade`:

```rust
fn walk_first_parent_chain(
nodes: &HashMap<String, Rc<RefCell<CommitNode>>>,
start_hash: &str,
) -> Vec<String> {
let mut path = Vec::new();
let mut current_hash = start_hash.to_string();

loop {
    if let Some(node) = nodes.get(&current_hash) {
        let n = node.borrow();
        path.push(current_hash.clone());

        match n.parents.borrow().first() {
            Some(first_parent) => match first_parent.upgrade() {
                Some(parent) => current_hash = parent.borrow().hash.clone(),
                None => break,
            },
            None => break,
        }
    } else {
        break;
    }
}

path
}
```

Now, let me explain what we just did. This function walks the first-parent chain from a starting hash to the root commit. For each commit, it:

1. Looks up the node in the `nodes` HashMap
2. Borrows it to access `parents`
3. Takes the first parent (the one in the `first()` position)
4. Calls `upgrade()` on the `Weak` reference. If the parent node still exists (strong count > 0), `upgrade` returns `Some(Rc<RefCell<CommitNode>>)`. If the parent was dropped (strong count == 0), it returns `None`.
5. Extracts the parent's hash and continues the loop

The `upgrade()` call is the bridge between the `Weak` world and the `Rc` world. Since the HashMap holds all strong references, every node is guaranteed to exist. But the type system forces us to handle the `Option` case anyway, because `Weak<T>` makes no compile-time guarantee that the allocation is still alive.

### Rendering the Graph

Now let's render the commit graph as ASCII art. The output is inspired by `git log --graph --oneline --all`:

```bash
* 7a8b9c0 (HEAD -> main) Latest commit
* 1b2c3d4 Add feature X
|\
| * 5e6f7g8 (feature) Fix bug in sidebar
|/
* 1a2b3c4 Initial commit
```

The `*` marks a commit. The `|\` marks a merge point. The `| *` lines represent commits that belong to a merged side branch. The `|/` marks where that branch reconnects to the main line in our simplified renderer.

```rust
fn render_graph(
nodes: &HashMap<String, Rc<RefCell<CommitNode>>>,
head_hash: &str,
branches: &[(String, String)],
) {
let branch_map: HashMap<&str, &str> = branches
    .iter()
    .map(|(name, hash)| (hash.as_str(), name.as_str()))
    .collect();

let main_line = walk_first_parent_chain(nodes, head_hash);
let main_set: HashSet<&str> = main_line.iter().map(|h| h.as_str()).collect();
let mut shown: HashMap<String, bool> = HashMap::new();

for hash in &main_line {
    if let Some(node) = nodes.get(hash) {
        let n = node.borrow();
        shown.insert(hash.clone(), true);

        // Check if this commit has additional parents (merge commit)
        let parents: Vec<Rc<RefCell<CommitNode>>> = n
            .parents
            .borrow()
            .iter()
            .filter_map(|w| w.upgrade())
            .collect();

        if parents.len() > 1 {
            // This is a merge commit. Show merged branches before showing the merge.
            for parent in parents.iter().skip(1) {
                let p = parent.borrow();
                let side_line = walk_first_parent_chain(nodes, &p.hash);

                println!("|\\");
                for side_hash in &side_line {
                    if main_set.contains(side_hash.as_str()) {
                        break;
                    }
                    let label = branch_map
                        .get(side_hash.as_str())
                        .map(|name| format!(" ({})", name))
                        .unwrap_or_default();

                    if let Some(side_node) = nodes.get(side_hash) {
                        let sn = side_node.borrow();
                        println!("| * {} {}{}", sn.short_hash, sn.message, label);
                        shown.insert(side_hash.clone(), true);
                    }
                }
                println!("|/");
            }
        }

        let label = branch_map
            .get(hash.as_str())
            .map(|name| format!(" ({})", name))
            .unwrap_or_default();

        println!("* {} {}{}", n.short_hash, n.message, label);
    }
}

// Show any branches whose tips weren't on the main line
for (branch_name, branch_hash) in branches {
    if !shown.contains_key(branch_hash.as_str()) {
        if let Some(_node) = nodes.get(branch_hash.as_str()) {
            let side_line = walk_first_parent_chain(nodes, branch_hash);
            println!("\n  (unconnected branch '{}')", branch_name);
            for side_hash in &side_line {
                if let Some(side_node) = nodes.get(side_hash) {
                    let sn = side_node.borrow();
                    println!("    * {} {}", sn.short_hash, sn.message);
                }
                if shown.contains_key(side_hash) {
                    break;
                }
            }
        }
    }
}

// Print graph statistics
println!("\n{}", "-".repeat(50));
println!("Graph statistics:");
println!("  Total commits: {}", nodes.len());
println!("  Branch refs: {}", branches.len());

let merge_count = nodes
    .values()
    .filter(|node| node.borrow().parents.borrow().len() > 1)
    .count();
println!("  Merge commits: {}", merge_count);

let root_count = nodes
    .values()
    .filter(|node| node.borrow().parents.borrow().is_empty())
    .count();
println!("  Root commits:   {}", root_count);

let timestamps: Vec<u64> = nodes
    .values()
    .map(|node| node.borrow().timestamp)
    .collect();
if let (Some(min), Some(max)) = (timestamps.iter().min(), timestamps.iter().max()) {
    println!("  Oldest commit:  {} (Unix epoch)", min);
    println!("  Newest commit:  {} (Unix epoch)", max);
}
}
```

Now, let me explain what we just did. The renderer walks the first-parent chain from HEAD (the "main line"). For each commit on the main line, it checks if the commit is a merge commit (has more than one parent). If it does, it walks the first-parent chain of each additional parent and prints them as side branches with pipe characters.

The key detail is `main_set: HashSet<&str>`, a set of all hashes on the main line. When rendering side branches, if we hit a commit that's already in the main line (the merge base), we `break`. This prevents shared ancestors from appearing twice. Only commits unique to the feature branch are shown in the `| *` section.

The `branch_map` is a lookup from commit hash to branch name. We use it to label commits that are at the tip of a branch. For example, if `refs/heads/main` points to hash `7a8b9c0`, we label that commit `(main)`.

After rendering the main line, we check for branches whose tips weren't on the main line. This can happen if you have an unmerged branch. We render those separately.

Finally, we print graph statistics and use the `timestamp` field (extracted from the commit's author line) to show the oldest and newest commit.

### Demonstrating RefCounts

Let's write a small function that demonstrates the reference counting in action:

```rust
fn print_refcounts(nodes: &HashMap<String, Rc<RefCell<CommitNode>>>) {
println!("Reference counts (nodes with at least one weak edge):");
for (hash, node) in nodes {
    let strong = Rc::strong_count(node);
    let weak = Rc::weak_count(node);
    if weak > 0 {
        println!(
            "  {} has {} strong, {} weak references",
            &hash[..7], strong, weak
        );
    }
}
}
```

Now, let me explain what we just did. `Rc::strong_count` tells us how many `Rc` handles point to this allocation. In our case, this should always be 1 because only the HashMap holds an `Rc`. All edges are `Weak`. `Rc::weak_count` tells us how many `Weak` handles point to this allocation. For example, a commit with one parent and two children may have three weak references associated with it: one stored in its own `parents` list and two stored in its children's `parents` lists. The exact weak count depends on the structure of the graph.
### Putting It All Together in main

```rust
fn main() {
let args: Vec<String> = std::env::args().collect();
let git_dir = if args.len() > 1 {
    Path::new(&args[1]).to_path_buf()
} else {
    Path::new(".git").to_path_buf()
};

let head = match read_head(&git_dir) {
    Some(h) => h,
    None => {
        eprintln!("Error: {} is not a git repository (no HEAD file)", git_dir.display());
        return;
    }
};
let head_hash = match resolve_ref(&git_dir, &head) {
    Some(h) => h,
    None => {
        eprintln!("Error: failed to resolve HEAD reference '{}'", head);
        return;
    }
};
let branches = read_all_branches(&git_dir);

println!("Reading commits from {}...", git_dir.display());
let raw_commits = read_all_commits(&git_dir, &head_hash);
println!("Parsed {} commits.\n", raw_commits.len());

// Step 1: Create nodes (first pass)
let graph = build_graph(&raw_commits);

// Step 2: Wire edges (second pass, uses interior mutability)
wire_edges(&graph, &raw_commits);

// Show ref counts before rendering
print_refcounts(&graph);

// Render the graph
println!("\nCommit Graph:\n");
render_graph(&graph, &head_hash, &branches);
}
```

Now, let me explain what we just did. `main` reads command-line arguments, resolves HEAD, and reads all branches. If the `.git` directory doesn't exist or HEAD can't be resolved, it prints a clear error and exits gracefully instead of panicking. Then it parses all commits, builds the graph in two passes, prints reference counts, and renders the graph. The two-pass approach for `build_graph` and `wire_edges` is the key pattern: you cannot wire edges until all nodes exist, because a parent hash can only be resolved to a `Weak` reference if the parent node's `Rc` already exists in the HashMap.

### Running The Project

Type this in your terminal:

```bash
cargo run -- /path/to/some/.git
```

Or from inside a git repository:

```bash
cargo run
```

You should see output like this:

```bash
Reading commits from .git...
Parsed 12 commits.

Reference counts:
  a1b2c3d has 1 strong, 3 weak references
  d4e5f6g has 1 strong, 2 weak references
  7a8b9c0 has 1 strong, 1 weak references

Commit Graph:

* 7a8b9c0 (main) Latest commit
* 1b2c3d4 Add feature X
|\
| * 5e6f7g8 (feature) Fix bug in sidebar
|/
* 1a2b3c4 Refactor utils
* f8e7d6c Initial commit

--------------------------------------------------
Graph statistics:
  Total commits: 12
  Branch refs:    2
  Merge commits:  1
  Root commits:   1
  Oldest commit:  1746316800 (Unix epoch)
  Newest commit:  1748736000 (Unix epoch)
```

The first parent of the merge commit `1b2c3d4` is `1a2b3c4` (the main line), so that's where the main line continues. The second parent is `5e6f7g8` (the feature branch), which is rendered as a side branch. The `main_set` check ensures only the unique feature branch commits appear under `| *`, shared ancestors stay on the main line.

## How It All Fits Together

Let's trace through the lifecycle of a commit node in our graph:

1. **Creation**: `Rc::new(RefCell::new(CommitNode { ... }))` allocates the node on the heap. The `Rc` lives in the HashMap, which is the sole strong owner. Strong count = 1.

2. **Wiring**: During `wire_edges`, for a commit with one parent and two children, `Rc::downgrade` is called three times: once for the parent edge in this node, and twice for the child edges from the parent and child nodes into this node. Strong count stays at 1. Weak count becomes 3.

3. **Walking**: When `render_graph` traverses the graph, each call to `walk_first_parent_chain` calls `upgrade()` on `Weak` handles. Each `upgrade()` temporarily increments the strong count (so the node cannot be dropped mid-traversal), gives us an `Rc<RefCell<CommitNode>>`, lets us borrow and read it, then the temporary `Rc` is dropped and the strong count returns to 1.

4. **Cleanup**: When `main` returns, the `HashMap` is dropped, which drops all its keys (Strings) and values (Rcs). The last strong reference to each node is dropped. Strong count hits 0 for every node. All nodes are freed. The `Weak` handles in parents and children become invalid. Any future `upgrade()` would return `None`.

## Conclusion

In this post, you learned about Rust's smart pointers: `Box<T>`, `Rc<T>`, `RefCell<T>`, and `Weak<T>`. You learned how `Rc<RefCell<T>>` provides shared mutable state in single-threaded code, and how `Weak<T>` enables non-owning references that do not prevent deallocation. You built a Git Commit Graph Viewer that reads real `.git` directories, parses commit objects, builds a connected graph where each node knows its parents and children through `Weak` edges, and renders an ASCII visualization of your commit history.

The smart pointer family has one more member we did not cover: `Arc<T>`, the atomic reference counter for multi-threaded code. We will cover `Arc` together with `Mutex` and concurrency in the next article, where we build a Thread Pool from Scratch.

If you like reading this, please subscribe and share this with others. It will really help me and motivate me to keep publishing more such articles.

# Build a JSON Parser in Rust from Scratch

> Build a JSON Parser in Rust from Scratch Learn structs, enums, and pattern matching by tokenizing and parsing JSON into a recursive data structure

Source: https://blog.sheerluck.dev/posts/rust/learn-rust-structs-enums-pattern-matching-by-building-a-json-parser/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-rust-structs-enums-pattern-matching-by-building-a-json-parser/index.md

In this post, we are going to learn about structs, enums and pattern matching in Rust. Once we cover all these concepts, we will build a **JSON Parser in Rust from scratch**. I'm really excited about this project. Let's start.

Get the source code from [here](https://github.com/MrSheerluck/json-parser-in-rust)

## Structs in Rust
### What a Struct Is?
A struct is a way to define a new type of grouping related pieces of data together under a single name. Each piece of data inside a struct is called a **field**, and every field has a name and a type.

Let's say you are building a product and you want to keep track of all the data for a user like name, age, etc. This is where you can use a struct to group together all the related data for a **User** type. Don't worry if you don't understand this, once we start working on the project this will be clear.

### Defining a Struct in Rust
```rust
struct User {
	name: String,
	age: u32,
	active: bool,
}
```
This defines a new type called `User`. It has three fields. The definition itself doesn't create any data, its just a definition or schema of the type like what kind  of data this struct can hold. No memory is allocated at this point.

There are some rules to define a struct:
- The struct name uses `PascalCase` by convention
- Each field is `name: Type`
- Fields are separated by commas
- A trailing comma after the last field is allowed and idiomatic in Rust

### Creating an Instance of a Struct in Rust
To actually create a value of type `User`, you create an instance:
```rust
let user = User{
	name: String::from("alice"),
	age: 30,
	active: true,
};
```
This is called a **struct literal**. You provide values for every field by name. The order you write the fields doesn't matter, Rust matches them by bame and not by position. But remember that you must provide all the fields. If you skip any field then it'll show a compile error.

### Accessing Fields
You access fields with dot notation:
```rust
println!("{}", user.name); // alice
println!("{}", user.age); // 30
println!("{}", user.active); // true
```

### Mutability
Rust's mutability applies to the entire struct instance. You can't make just once field mutable. If you want to modify any field, the whole binding must be `mut`:
```rust
let mut user = User{
	name: String::from("alice"),
	age: 30,
	active: true,
};

user.age = 31; // this is fine as user is a mutable struct
user.active = false // this is fine too for the same reason
```
If `user` is not `mut`, no field can be modified. Let's not go to why this is the case, I might explain it at some point later in this series.

### Field Shorthand
When you're creating a struct instance and you have a local variable with the same name as a field, you don't need to repeat yourself:
```rust
fn build_user(name: String, age: u32) -> {
	User {
		name, // shorthand for name: name
		age, // shorthand for age: age
		active: true,
	}
}
```
This only works when the variable name and the field name are identical.

### Struct Update Syntax
If you want to create a new instance of a struct that's mostly same to an existing one, but with some fields changed, you can use struct update syntax:
```rust
let user2 = User {
	age: 25,
	...user
};
```
The `...user` part means "fill in all remaining fields from `user`". Fields you specify explicitly take precedence. Fields not mentioned are copied or moved from `user`. 

> There's an important ownership nuance here: if any field being moved out of user is a non-Copy type (like `String`), then `user` is partially moved and you can no longer use it as a whole after this. You can still use fields that were not moved (like `age` and `active`, which are Copy types), but `user.name` would be moved into `user2.name`.

### `impl` Blocks (Giving a Struct Behaviour)
A struct definition only describes data but to give a struct behaviour that means functions that operate on that data, you write an `impl` block:
```rust
impl User {
	// methods and assocuated functions go here
}
```
`impl` stands for "implementation". Everything inside an `impl User` block belongs to the `User` type. You can have multiple `impl` blocks for the same type and Rust will them as one, though conventionally you group everything into one block.

### Methods (Functions with `self`)
A `method` is a function defined inside an `impl` block that takes `self` as its first parameter. `self` refers to the specific instance the method is being called on. There are three forms of `self` and each of them has a distinct meaning:

#### `&self` Immutable Borrow of an Instance
```rust
impl User {
	fn greeting(&self) -> String {
		format!("Hello, my name is {} and I am {} years old", self.name, self.age)
	}
}
```
`&self` is shorthand for `self: &User`. The method borrows the instance immutably. It can read fields but cannot modify them. The caller retains the ownership of the instance after the call.
```rust
let user = User{
	name: String::from("alice"),
	age: 30,
	active: true,
};

println!("{}", user.greeting()); // Hello, my name is alice and I am 30 year old
// user is still valid here
```
This is by far the most common form. Use it whenever a method only needs to read data.

#### `&mut self` Mutable Borrow of an Instance
```rust
impl User {
	fn deactivate(&mut self) {
		self.active = false;
	}
	fn have_birthday(&mut self) {
		self.age += 1;
	}
}
```
`&mut self` is shorthand for `self: &mut User`. The method borrows the instance mutably. It can read and modify fields. The caller must have a `mut` binding and the normal borrow rules apply (no other borrows of the instance can exist while this method runs).
```rust
let mut user = User{
	name: String::from("alice"),
	age: 30,
	active: true,
};

user.have_birthday();
println!("{}", user.age); // 31

user.decativate();
println!("{}", user.active); // false
```

#### `self` Taking Ownership of the Instance
```rust
impl User {
	fn into_name(self) -> String {
		self.name // self is consumed, we return just the name
	}
}
```
`self` (no reference) means the method takes the ownership of the instance. After calling it, the original binding is gone, it is moved into the method.
```rust
let user = User {
	name: String::from("alice"),
	age: 30,
	active: true,
};

let name = user.into_name();
// user is no longer valid here, it was moved to that method
println!("{}", name); // alice
```
This is used when a method is meant to consume the instance and produce something else from it may be a transformation or conversion. It's less common than the borrow forms.

### Associated Functions
An associated function is defined inside an `impl` block but does not take self in any form. It belongs to the type, not to any specific instance.
```rust
impl User {
	fn new(name: String, age: u32) -> User {
		User {
			name,
			age,
			active: true,
		}
	}
}
```
You call it with `::` syntax on the type name, not on an instance:
```rust
let user = User::new(String::from("alice"), 30);
```
This is the conventional way to write constructors in Rust. Rust has no special constructor syntax. `new` is just a naming convention for an associated function that produces an instance. You can name it anything.

Another common pattern: associated functions that act as alternative constructors with specific defaults or validation logic:
```rust
impl User {
	fn guest() -> {
		User {
			name: String::from("guest"),
			age: 0,
			active: false,
		}
	}
}

let g = User::guest();
```

## Enums in Rust
### What an Enum Is?
An enum (short for enumeration) is a type that defines a fixed set of possible variants. A value of an enum type is exactly one of those variants at any given time.
You've already used enums without thinking about it. `Option<T>` and `Result<T, E>` are both enums defined in Rust's standard library. But before we get to the advanced forms, we need to build up from the ground up.

### Basic enums
The simplest form of an enum is a set of named variants that carry no data:
```rust
enum Direction {
	North,
	South,
	East,
	West,
}
```
This defines a new type called `Direction`. A value of type `Direction` is one of exactly those four things. You create a value using `::` syntax:
```rust
let d = Direction::North;
```
Each variant is namespaced under the enum name. You can't just write `North`, you need to write it like `Direction::North`. This prevents name collisions and makes code immediately readable.

At the memory level, a basic enum with no data is stored as an integer. Rust chooses the smallest integer type that fits all variants. With four variants, it fits in a `u8`.

### Data Carrying Variants
This is where Rust enums become genuinely powerful and unlike enums in most other languages. Each variant of an enum can carry its own data, and each variant can carry different data with a different shape:
```rust
enum Shape {
	Circle(f64),
	Rectangle(f64, f64),
	Triangle(f64, f64, f64),
}
```
`Circle` carries one `f64` (the radius). `Rectangle` carries two `f64`s (width and height). `Triangle` carries three `f64`s (the three sides). These are called **tuple variants** because the data is positional just like a tuple.

You can create them like this:
```rust
let c = Shape::Circle(5.0);
let r = Shape::Rectangle(4.0, 6.0);
let t = Shape::Triangle(3.0, 4.0, 5.0);
```
Variants can also carry named fields, like a struct:
```rust
enum Shape {
	Circle{ radius: f64 },
	Rectangle{ width: 64, height: f64 },
	Triangle{ a: f64, b: f64, c:f64 },
}

let r = Shape::Rectangle{ width: 4.0, height: 6.0 };
```
These are called **struct variants**. Named fields are more verbose but more readable, especially when a variant carries several values that could be confused with each other.

### What an Enum Value Actually Looks Like in Memory
This is important to understand. An enum value in memory is laid out as a **tagged union**. It consists of:
1. A **tag** (the discriminant) - an integer identifying which variant this value is
2. **Space for the largest variant's data** - all variants share the same memory region, sized to fit whichever variant is biggest
So if you have:
```rust
enum Message {
	Quit,
	Move { x: i32, y: i32 },
	Write(String),
	Color(u8, u8, u8),
}
```
The size of `Message` value in memory is the size of the largest variant (`Write(String)`) is the biggest here since `String` is 24 bytes on 64-bit machines plus the tag. Every `Message` value occupies the same amount of space, regardless of which variant it actually is. If it's `Quit`, most of those bytes are unused but they are still there, reserved.

### Enums with Mixed Variant Shapes
Nothing forces all variants to have the same kind of data, you can feely mix:
```rust
enum Event {
	KeyPress(char),
	MouseClick{ x: u32, y: u32 },
	Resize(u32, u32),
	Quit,
}
```

`KeyPress` is a tuple variant with one value. `MouseClick` is a struct variant with named fields. `Resize` is a tuple variant with two values. `Quit` has no data. This is completely valid.

### Enums in `impl` Blocks
Just like structs, enums can have `impl` blocks with methods:
```rust
enum Direction {
North,
South,
East,
West,
}

impl Direction {
fn opposite(&self) -> Direction {
    match self {
        Direction::North => Direction::South,
        Direction::South => Direction::North,
        Direction::East => Direction::West,
        Direction::West => Direction::East,
    }
}

fn is_vertical(&self) -> bool {
    match self {
        Direction::North | Direction::South => true,
        Direction::East | Direction::West => false,
    }
}
}
```
The `match` expression here is the primary tool for working with enum values. We'll cover it in depth in the next section, but notice already: you can't access the "contents" of an enum variant without pattern matching. There are no fields you can dot into directly. The match forces you to say "if it's this variant, do this; if it's that variant, do that."

### Recursive Enums and `Box<T>`
Here's where enums become essential for building data structures like parsers, ASTs and trees.
Suppose you want to define a type that represents a simple expression like a number, or an addition of two expressions:
```rust
enum Expr {
	Number(f64),
	Add(Expr, Expr), // ERROR: recursive type has infinite size
}
```
This won't compile because the compiler needs to know the size of `Expr` at compile time. But `Expr::Add` contains two `Expr` values, each of which might be `Add` themselves which contain more `Expr` values. The could go on till infinity.

The fix is `Box<T>`. A `Box<T>` is a heap allocated values basically its a pointer and a pointer always has a fixed, known size (8 bytes on 64-bit).So instead of storing the `Expr` directly inside `Add`, you store a pointer to it on the heap:
```rust
enum Expr {
	Number(f64),
	Add(Box<Expr>, Box<Expr>),
}
```
Now the compiler knows the size of `Expr`: its the size of the largest variant. `Number(f64)` is 8 bytes and `Add(Box<Expr>, Box<Expr>)` is two pointers, so 16 bytes. The size is fixed and finite even though the actual tree in memory can be arbitrarily deep.

Creating a recursive value with `Box`:
```rust
let expr = Expr::Add(
	Box::new(Expr::Number(1.0)),
	Box::new(Expr::Add(
		Box::new(Expr::Number(2.0)),
		Box::new(Expr::Number(3.0)),
	)),
);
```
This represents `1 + (2 + 3)`. The outer `Add` holds two boxed `Expr` values on the heap. The inner `Add` is also heap-allocated. The leaves (`Number`) hold their data directly.

## Pattern Matching
Pattern matching is a mechanism for simultaneously inspecting the structure of a value and binding names to the parts inside it. A switch statement checks a condition and jumps to a branch. Pattern matching does that plus it destructures the value, extracts its internals, binds those internals to names you can use and the compiler verifies that every possible case is handled.

### The `match` Expression
```rust
match some_value {
	pattern1 => expression1,
	pattern2 => expression2,
	pattern3 => expression3,
}
```
`match` takes a value and compares it against a series of **arms**. Each arm has a pattern on the left of `=>` and an expression on the right. Rust tries each pattern from top to bottom and executes the first one that matches. The entire `match` is an expression that means it produces a value and all arms must produce the same type of value.

The simplest possible match is on a plain value:
```rust
let x:i32 = 3;

let description = match x {
	1 => "one",
	2 => "two",
	3 => "three",
	_ => "something else",
};

println!("{}", description); // three
```
`_` is the wildcard. It matches anything and binds nothing. Every `match` on a type with more possibilities than your explicit arms must include a catch-all or the compiler will complain.

### Matching on Enum Variants With Data
When a variant carries data, the pattern names bindings to extract that data:
```rust
enum Shape {
	Circle(f64),
	Rectangle(f64, f64),
}

let s = Shape::Rectangle(4.0, 6.0);

match s {
	Shape::Circle(radius) => {
		println!("Circle with radius {}", radius);
	}
	Shape::Rectangle(width, height) => {
		println!("Rectangle {}x{}", width, height);
	}
}
```
`Shape::Circle(radius)` is a pattern that matches any `Circle` variant and binds the inner `f64` to the name `radius`. Inside the arm, `radius` is a local variable holding that value. Same for width and height in the rectangle arm.

For struct variants with named fields:
```rust
enum Shape {
Circle { radius: f64 },
Rectangle { width: f64, height: f64 },
}

let s = Shape::Circle { radius: 5.0 };

match s {
Shape::Circle { radius } => println!("Circle, radius = {}", radius),
Shape::Rectangle { width, height } => println!("{}x{}", width, height),
}
```

The pattern `Shape::Circle { radius }` matches any `Circle` variant and binds the `radius` field to a local variable also called `radius`. If you want to rename it, you write `Shape::Circle { radius: r }`

### How ownership works in match
This is subtle and important. When you match on a value, what happens to ownership depends on whether you match by value or by reference.

#### Matching by value
```rust
let s = Shape::Circle(5.0);

match s {
Shape::Circle(r) => println!("{}", r),
Shape::Rectangle(w, h) => println!("{} {}", w, h),
}
// s is moved — no longer usable
```
The entire value `s` is moved into the match. The matched arm receives ownership of the extracted data.

#### Matching by reference
```rust
let s = Shape::Circle(5.0);

match &s {
Shape::Circle(r) => println!("{}", r),
Shape::Rectangle(w, h) => println!("{} {}", w, h),
}
// s is still valid here
```
When you match on `&s`, the patterns automatically match against references. The bindings `r`, `w`, `h` become references to the data inside the variant (`&f64` in this case)

In practice, when you have a method taking `&self`, you match on `self` (which is already `&Shape` inside the method), and your bindings are references. If you're in a context where you own the value and want to consume it, you match on the value directly.

### Binding with `@`
Sometimes you want to test whether a value matches a pattern and also give that value a name to use in the arm. The `@` operator does this:
```rust
let n: u32 = 7;

match n {
small @ 1..=5 => println!("{} is small", small),
medium @ 6..=10 => println!("{} is medium", medium),
large => println!("{} is large", large),
}
```
`medium @ 6..=10` means: match any value in the range 6 to 10, and bind that value to the name `medium`. Without `@`, a range pattern matches but gives you no name for the value. With `@`, you get both the test and the binding.

### Range Patterns
Numeric and character types support range patterns:
```rust
let grade: u32 = 85;

let letter = match grade {
90..=100 => "A",
80..=89  => "B",
70..=79  => "C",
60..=69  => "D",
_        => "F",
};
```
`90..=100` is an **inclusive range pattern**,  it matches any value from 90 through 100 inclusive. The `..=` syntax is required for patterns (exclusive ranges with `..` are not allowed in match patterns because the compiler can't always verify exhaustiveness with them).

### Multiple Patterns with `|`
A single arm can match multiple patterns using `|`:
```rust
let x: i32 = 3;

match x {
1 | 2 => println!("one or two"),
3 | 4 => println!("three or four"),
_     => println!("something else"),
}
```
This is OR, the arm firsts if the value matches any of the listed patterns. You can combine this with ranges:
```rust
match x {
1 | 2 | 3..=5 => println!("between 1 and 5"),
_ => println!("out of range"),
}
```

### Match Guards
A **match guard** is an additional `if` condition attached to an arm. The arm only fires if both the pattern matches and the guard is true:
```rust
let pair = (2, -3);

match pair {
(x, y) if x == y        => println!("equal"),
(x, y) if x + y == 0    => println!("sum is zero"),
(x, _) if x > 0         => println!("x is positive"),
_                        => println!("anything else"),
}
// prints: sum is zero
```
The guard `if x + y == 0` has access to the bindings created by the pattern. Bindings are established first, then the guard is evaluated. If the guard fails, Rust moves on to the next arm

> One important caveat: **the compiler does not consider guards when checking exhaustiveness**. Guards are runtime conditions, and the compiler can't reason about their runtime truth. So even if your guards collectively cover all cases, the compiler will still require a wildcard arm if the patterns themselves don't cover all cases.

### Wildcard Patterns in Depth
`_` is not just a catch-all for the whole value. It can appear inside patterns to ignore specific parts:
```rust
let triple = (1, -2, 3);

match triple {
(0, _, _) => println!("starts with zero"),
(x, 0, _) => println!("middle is zero, first is {}", x),
(x, y, _) => println!("first two: {} and {}", x, y),
}
```
The `_` in position means "match anything here, but don't bind it to a name". This is different from a named binding, which would move or copy the value. `_` explicitly discards it. If you are coming from languages like go or python, this should be familiar.

You can also use `..` to ignore multiple fields at once in a struct pattern:
```rust
struct Point {
x: i32,
y: i32,
z: i32,
}

let p = Point { x: 1, y: 2, z: 3 };

match p {
Point { x, .. } => println!("x is {}", x),
}
```
`..` means "ignore all remaining fields". This is especially useful for structs with many fields when you only care about one or two.

### Matching on Structs Directly
You don't have to put a struct inside an enum to match on it. Structs support pattern matching directly in a `match` or a `let` binding:
```rust
struct Point {
x: i32,
y: i32,
}

let p = Point { x: 3, y: 7 };

match p {
Point { x: 0, y } => println!("on y-axis at {}", y),
Point { x, y: 0 } => println!("on x-axis at {}", x),
Point { x, y }    => println!("at ({}, {})", x, y),
}
```
`Point { x: 0, y }` matches only when `x` is exactly `0`, and binds whatever `y` is to the name `y`. `Point { x, y }` matches any `Point` and binds both fields.

## `if let` And `while let`
`match` is exhaustive by design, it forces you to handle every variant. That's exactly what you want most of the time. But sometimes you only care about one specific variant and want to do nothing for all other cases. Writing a full `match` for that feels like unnecessary:
```rust
enum Message {
Quit,
Write(String),
Move { x: i32, y: i32 },
}

let msg = Message::Write(String::from("hello"));

match msg {
Message::Write(text) => println!("{}", text),
_ => (), // do nothing — feels like noise
}
```
The `_ => ()` arm is doing nothing meaningful. It's just there to satisfy exhaustiveness. `if let` exists precisely for this situation.

### `if let`
`if let` combines a pattern match with a conditional. It executes the body only if the pattern matches, and does nothing otherwise:
```rust
let msg = Message::Write(String::from("hello"));

if let Message::Write(text) = msg {
println!("{}", text);
}
```
Read this as: "if `msg` matches the pattern `Message::Write(text)`, bind the inner value to `text` and execute the block." If `msg` is `Message::Quit` or `Message::Move { .. }`, the block is skipped entirely

The binding `text` only exists inside the `if let` block. Outside it, `text` doesn't exist.

### `if let` with an `else` branch
You can attach an `else` to handle the non-matching case:
```rust
let msg = Message::Quit;

if let Message::Write(text) = msg {
println!("Message: {}", text);
} else {
println!("Not a write message");
}
```

And you can chain `else if let` to check multiple patterns sequentially:
```rust
let msg = Message::Move { x: 3, y: 7 };

if let Message::Write(text) = msg {
println!("Write: {}", text);
} else if let Message::Move { x, y } = msg {
println!("Move to ({}, {})", x, y);
} else {
println!("Something else");
}
```
This is a chain of independent pattern tests, tried in order. Unlike `match`, this does not have exhaustiveness checking. Use `match` when you need exhaustiveness, use `if let` when you genuinely only care about one case.

### Ownership in `if let`
The same ownership rules that apply to `match` apply here. If you write:
```rust
let msg = Message::Write(String::from("hello"));

if let Message::Write(text) = msg {
println!("{}", text);
}
// msg is moved — no longer usable
```
`msg` is moved into the pattern. To avoid moving, match on a reference:
```rust
let msg = Message::Write(String::from("hello"));

if let Message::Write(text) = &msg {
println!("{}", text); // text is &String
}
// msg is still valid
```
Same thing as `match`

### `while let`
`while let` runs a loop for as long as a pattern keeps matching. The moment the pattern stops matching, the loop exits:
```rust
let mut stack = vec![1, 2, 3];

while let Some(top) = stack.pop() {
println!("{}", top);
}
```
`Vec::pop()` returns `Option<T>`.  `Some(value)` if the vector has elements, `None` when it's empty. The `while let` keeps looping and binding `top` to each popped value as long as `pop()` returns `Some`. The instant it returns `None`, the pattern doesn't match and the loop exits.

Without `while let`, you'd write this as:
```rust
loop {
match stack.pop() {
    Some(top) => println!("{}", top),
    None => break,
}
}
```
`while let` is just cleaner syntax for this exact pattern

### What `if let` and `while let` actually are
They are not new features with new rules. They are shorthand for specific shapes of `match` that are common enough to deserve their own syntax. The compiler desugars them into `match` expressions before doing anything else.

`if let` = `match` with one meaningful arm and `_ => ()` for everything else, plus an optional `else`.

`while let` = `loop` containing a `match` that breaks on the non-matching case.

### Guards in `if let`
You can attach a guard to `if let` the same way as in `match`:
```rust
let number = Some(7);

if let Some(n) = number && n > 5 {
println!("{} is greater than 5", n);
}
```
This syntax (using `&&` directly after the pattern) is available in Rust 1.65+

## Destructuring in `let` Bindings and `for` Loops
Destructuring is using a pattern on the left side of a `let` binding or in a `for` loop to simultaneously unpack a value and bind its parts to names. 

### Destructuring tuples in `let`
The simplest case is a tuple:
```rust
let (x, y, z) = (1, 2, 3);

println!("{} {} {}", x, y, z); // 1 2 3
```
The left side `(x, y, z)` is a pattern. Rust matches the right side `(1, 2, 3)` against it, binding `1` to `x`, `2` to `y`, `3` to `z`. All three bindings are created simultaneously in a single statement.

You can ignore parts with `_`:
```rust
let (first, _, third) = (10, 20, 30);

println!("{} {}", first, third); // 10 30
```

Or ignore a trailing portion with `..`:
```rust
let (head, ..) = (1, 2, 3, 4, 5);

println!("{}", head); // 1
```

### Destructuring Structs in `let`
Struct patterns work directly in `let` bindings too:
```rust
struct Point {
x: i32,
y: i32,
}

let p = Point { x: 10, y: 20 };
let Point { x, y } = p;

println!("{} {}", x, y); // 10 20
```
`let Point { x, y } = p` destructures `p` into two bindings `x` and `y`. After this, `p` is moved (since `Point` doesn't implement `Copy`), and `x` and `y` are the extracted values

To rename while destructuring:
```rust
let Point { x: horizontal, y: vertical } = p;

println!("{} {}", horizontal, vertical); // 10 20
```
The field `x` is bound to the local name `horizontal`. The field `y` is bound to `vertical`. The struct field name is on the left of `:`, the local binding name is on the right.

To avoid moving, destructure from a reference:
```rust
let p = Point { x: 10, y: 20 };
let Point { x, y } = &p;

// x and y are &i32 here
println!("{} {}", x, y);
// p is still valid
```

### Destructuring Enums in `let`
You can destructure an enum variant directly in a `let` binding, but only if you're certain which variant it is because `let` patterns are **irrefutable**. An irrefutable pattern is one that is guaranteed to match for any possible value of the type.

A tuple `(x, y)` is irrefutable for any tuple with two elements, it always matches. A struct pattern `Point { x, y }` is irrefutable for a `Point`, it always matches. But `Some(x)` is refutable for `Option<T>`, it doesn't match `None`.

If you try:
```rust
let Some(value) = some_option; // ERROR: refutable pattern
```
The compiler rejects this. It can't guarantee the pattern matches. For refutable patterns, you must use `match` or `if let`.

This restriction exists because a failing `let` binding has nowhere to go, there's no else branch, no other arm. The compiler catches this at compile time rather than letting it panic at runtime.

### Destructuring in `for` loops
A `for` loop iterates over a sequence and binds each element to a pattern. Most of the time people write:
```rust
for item in collection {
// use item
}
```
But `item` is actually a pattern, not just a name. You can put any irrefutable pattern there:
```rust
let pairs = vec![(1, 'a'), (2, 'b'), (3, 'c')];

for (number, letter) in pairs {
println!("{}: {}", number, letter);
}
```
Each element of `pairs` is a tuple `(i32, char)`. The pattern `(number, letter)` destructures each tuple on the spot, giving you both values named and ready to use inside the loop body. Without destructuring, you'd need to write `pair.0` and `pair.1`, which is less clear.

Destructuring structs in `for` loops works the same way:
```rust
struct Point {
x: i32,
y: i32,
}

let points = vec![
Point { x: 1, y: 2 },
Point { x: 3, y: 4 },
];

for Point { x, y } in &points {
println!("({}, {})", x, y);
}
```

> Note `&points` iterating over a reference to the vector gives you references to each element, so `x` and `y` are `&i32`. The vector remains valid after the loop.

### Nested Destructuring
Patterns can be nested to match nested structures:
```rust
let nested = ((1, 2), (3, 4));
let ((a, b), (c, d)) = nested;

println!("{} {} {} {}", a, b, c, d); // 1 2 3 4
```
The outer pattern `((a, b), (c, d))` matches a tuple of two tuples. Rust unpacks both levels simultaneously. This scales to arbitrary depth.

### Destructuring Function Parameters
Patterns also work directly in function parameter position:
```rust
fn print_point(&(x, y): &(i32, i32)) {
println!("x={}, y={}", x, y);
}

let p = (3, 7);
print_point(&p);
```
`&(x, y)` in the parameter position is a pattern that matches a reference to a tuple and immediately destructures it. Inside the function, `x` and `y` are `i32` values, not references. This is occasionally useful but less common than destructuring in `let` or `for`.

Now, its time to start working on our JSON parser. If you think you don't understand all the above concepts thats fine. I would say just start building the project and read the required concept again. It's too much to take at once, so take your time.

## What a JSON Parser Actually Does
Parsing is the process of taking raw text and turning it into structured data your program can work with. For JSON, that means taking a string like this:
```json
{"name": "alice", "age": 30, "scores": [95, 87, 100], "active": true}
```
and producing a value in memory that your Rust code can inspect, traverse and query. Something like: "this is an object with four keys, the first key `name` maps to the string `alice`, the second key `age` maps to the number `30`" and so on

A parser typically works in two stages:
1. Tokenization (lexing) - walk the raw input character by character and group characters into meaningful units called tokens. A token might be `{` or `"alice"` or `30` or `true` or `[`. At this stage you are not thinking about structure, you are just identifying the atoms of the language.
2. Parsing - Take the stream of tokens and build a structured value from them according to the grammar rules of JSON. This is where you recognise that `{` starts an object, that an object contains key-value pairs separated by ",", that each key is a string followed by `:` followed by a value and so on.

### The Grammar of JSON
JSON is simple, precise grammar. A JSON value is exactly one of:
- An object: `{` followed by zero or more `"key": value` pairs separated by `,`, followed by `}`
- An array: `[` followed by zero or more values separated by `,`, followed by `]`
- A string: `"` followed by characters followed by `"`
- A number: an optional `-`, digits, optional decimal point and more digits
- A boolean: exactly `true` or `false`
- null: exactly `null`

Objects and arrays contain values and values can themselves be objects or arrays. This is what makes JSON recursive, and why recursive enum is the right representation.

### The `JSONValue` type
Before writing any parsing code, we define the type that represents a parsed JSON value:
```rust
enum JSONValue {
Object(Vec<(String, JSONValue)>),
Array(Vec<JSONValue>),
Str(String),
Number(f64),
Bool(bool),
Null,
}
```
Let's understand each and every variant:
- `Object(Vec<(String, JsonValue)>)` - A JSON object is an unordered collection of key-value pairs. Each key is always a string. Each value is any `JsonValue`. We represent this as a `Vec` of `(String, JsonValue)` tuples. We're using a `Vec` of tuples rather than a `HashMap` because we haven't covered hashmap concept yet. A `Vec` of tuples works correctly and is simpler.
- `Array(Vec<JsonValue>)` - A JSON array is an ordered sequence of values. Each element is any `JsonValue`. Straightforward.
- `Str(String)` - A JSON string. We name the variant `Str` rather than `String` to avoid conflicting with Rust's built-in `String` type name.
- `Number(f64)` - JSON makes no distinction between integers and floats. The spec says all numbers are IEEE 754 doubles, so `f64` is the correct choice.
- `Bool(bool)` - JSON booleans `true` and `false`.
- `Null` - JSON null. No data needed.

Notice that `Object` contains `JsonValue` inside it, and `Array` contains `JsonValue` inside it. This is the recursive structure. But unlike the `Expr` example from the enums section, we don't need explicit `Box` here because `Vec` already heap-allocates its contents. The `Vec` itself is a fixed-size struct on the stack (a pointer, a length, and a capacity of 24 bytes). The actual `JsonValue` elements live on the heap inside the `Vec`'s buffer. So the indirection is already there.

### The Token Type
The tokenizer will produce a sequence of tokens. Here's what the token type looks like:
```rust
enum Token {
LeftBrace,        // {
RightBrace,       // }
LeftBracket,      // [
RightBracket,     // ]
Colon,            // :
Comma,            // ,
StringToken(String),
NumberToken(f64),
True,
False,
Null,
}
```

Each variant represents one meaningful unit from the input. Single-character punctuation maps to simple variants with no data. Strings and numbers carry their parsed value. `True`, `False`, and `Null` are keyword tokens.

### Project Setup
Create the project:
```bash
cargo new json_parser
cd json_parser
```

Open `src/main.rs`. We'll put everything in one file for simplicity, we will later learn about modularising our code. Here's the skeleton we'll fill in:

```rust
// The token type
enum Token {
LeftBrace,
RightBrace,
LeftBracket,
RightBracket,
Colon,
Comma,
StringToken(String),
NumberToken(f64),
True,
False,
Null,
}

// The JSON value type
enum JsonValue {
Object(Vec<(String, JsonValue)>),
Array(Vec<JsonValue>),
Str(String),
Number(f64),
Bool(bool),
Null,
}

// Stage 1: tokenizer
fn tokenize(input: &str) -> Vec<Token> {
todo!()
}

// Stage 2: parser
fn parse(tokens: &[Token]) -> JsonValue {
todo!()
}

fn main() {
let input = r#"{"name": "alice", "age": 30, "active": true}"#;
let tokens = tokenize(input);
let value = parse(&tokens);
}
```

`todo!()` is a macro that panics with "not yet implemented". It lets the code compile while functions are incomplete.

The `r#"..."#` syntax is a raw string literal. The `#` delimiters mean backslashes and inner quotes are not treated as escape sequences. This makes it easier to write JSON strings in Rust source code without escaping every `"`.

### The Tokenizer
The tokenizer's job is to walk the input string character by character and produce a `Vec<Token>`. It needs to:

- Skip whitespace
- Recognise single-character punctuation (`{`, `}`, `[`, `]`, `:`, `,`)
- Recognise string literals (everything between `"` and `"`, handling escape sequences)
- Recognise numbers (digits, optional minus sign, optional decimal)
- Recognise the keywords `true`, `false`, `null`
- Panic on anything unexpected

### How We'll Walk the Input
We need to walk through the characters of the input string with the ability to:
- Look at the current character
- Advance past it
- Sometimes look ahead without advancing

The right tool for this is a **struct** that holds the input as a slice of bytes and tracks the current position:
```rust
struct Lexer {
input: Vec<u8>,
pos: usize,
}
```
`input` is the raw bytes of the input string, owned by the lexer. `pos` is the current position, its the index of the next character to be processed.

> We could use a concept called lifetime here but lets not go there as we haven't understood that concept as of now. We will keep it simple by letting the lexer own its data

Now the `impl` block:
```rust
impl Lexer {
fn new(input: &str) -> Lexer {
    Lexer {
        input: input.as_bytes().to_vec(),
        pos: 0,
    }
}

fn current(&self) -> Option<u8> {
    if self.pos < self.input.len() {
        Some(self.input[self.pos])
    } else {
        None
    }
}

fn advance(&mut self) {
    self.pos += 1;
}

fn peek(&self) -> Option<u8> {
    if self.pos + 1 < self.input.len() {
        Some(self.input[self.pos + 1])
    } else {
        None
    }
}
}
```
- `new` converts the `&str` to a `Vec<u8>`,  we work with raw bytes because JSON is ASCII-compatible for all structural characters, and byte indexing is simpler than character indexing for this purpose.
- `current` returns the byte at the current position, wrapped in `Option`,  `None` if we've reached the end.
- `advance` moves the position forward by one.
- `peek` looks one position ahead without moving. We'll need this for number parsing to distinguish `-3` (a negative number) from `-` followed by something invalid.

### The Main Tokenize Loop
Now the `tokenize` function creates a `Lexer` and drives a loop:
```rust
fn tokenize(input: &str) -> Vec<Token> {
let mut lexer = Lexer::new(input);
let mut tokens = Vec::new();

loop {
    match lexer.current() {
        None => break,

        Some(b' ') | Some(b'\t') | Some(b'\n') | Some(b'\r') => {
            lexer.advance();
        }

        Some(b'{') => { tokens.push(Token::LeftBrace);    lexer.advance(); }
        Some(b'}') => { tokens.push(Token::RightBrace);   lexer.advance(); }
        Some(b'[') => { tokens.push(Token::LeftBracket);  lexer.advance(); }
        Some(b']') => { tokens.push(Token::RightBracket); lexer.advance(); }
        Some(b':') => { tokens.push(Token::Colon);        lexer.advance(); }
        Some(b',') => { tokens.push(Token::Comma);        lexer.advance(); }

        Some(b'"') => {
            let s = lexer.read_string();
            tokens.push(Token::StringToken(s));
        }

        Some(b't') => {
            lexer.read_keyword("true");
            tokens.push(Token::True);
        }
        Some(b'f') => {
            lexer.read_keyword("false");
            tokens.push(Token::False);
        }
        Some(b'n') => {
            lexer.read_keyword("null");
            tokens.push(Token::Null);
        }

        Some(b'-') | Some(b'0'..=b'9') => {
            let n = lexer.read_number();
            tokens.push(Token::NumberToken(n));
        }

        Some(c) => {
            panic!("Unexpected character: {}", c as char);
        }
    }
}

tokens
}
```
A few things to understand here:

`b' '` is a **byte literal** - the ASCII value of the space character as a `u8`. This is how you write character comparisons when working with `&[u8]` instead of `&str`.

`b'0'..=b'9'` is a byte range pattern - it matches any byte whose value is between the ASCII value of `'0'` and the ASCII value of `'9'` inclusive. This is how you check "is this character a digit" without calling any methods.

The `Some(b't')`, `Some(b'f')`, `Some(b'n')` arms peek at the first character to decide which keyword we're reading, then delegate to `read_keyword` which consumes the full keyword and panics if it doesn't match.

`Some(c)` at the bottom is the catch-all - if we hit any character we don't recognise, we panic with a message showing what it was.

### Reading Keywords
```rust
impl Lexer {
fn read_keyword(&mut self, keyword: &str) {
    for expected in keyword.as_bytes() {
        match self.current() {
            Some(c) if c == *expected => self.advance(),
            Some(c) => panic!(
                "Unexpected character '{}' while reading keyword '{}'",
                c as char, keyword
            ),
            None => panic!("Unexpected end of input while reading keyword '{}'", keyword),
        }
    }
}
}
```
We iterate over the bytes of the expected keyword string. For each expected byte, we check the current character. If it matches, we advance. If it doesn't match or we hit the end of input, we panic.

The pattern `Some(c) if c == *expected` is a match guard. `expected` is `&u8` (a reference to a byte in the keyword slice), so we dereference it with `*expected` to compare against `c` which is `u8`

### Reading Strings
String reading is the most involved part of the tokenizer because JSON strings support escape sequences: `\"`, `\\`, `\/`, `\n`, `\t`, `\r`, and Unicode escapes `\uXXXX`.
```rust
impl Lexer {
fn read_string(&mut self) -> String {
    // consume the opening "
    self.advance();

    let mut result = String::new();

    loop {
        match self.current() {
            None => panic!("Unterminated string"),

            Some(b'"') => {
                self.advance(); // consume closing "
                return result;
            }

            Some(b'\\') => {
                self.advance(); // consume the backslash
                match self.current() {
                    Some(b'"')  => { result.push('"');  self.advance(); }
                    Some(b'\\') => { result.push('\\'); self.advance(); }
                    Some(b'/')  => { result.push('/');  self.advance(); }
                    Some(b'n')  => { result.push('\n'); self.advance(); }
                    Some(b't')  => { result.push('\t'); self.advance(); }
                    Some(b'r')  => { result.push('\r'); self.advance(); }
                    Some(b'b')  => { result.push('\x08'); self.advance(); }
                    Some(b'f')  => { result.push('\x0C'); self.advance(); }
                    Some(b'u')  => {
                        self.advance(); // consume 'u'
                        let codepoint = self.read_unicode_escape();
                        let ch = char::from_u32(codepoint)
                            .unwrap_or_else(|| panic!("Invalid unicode codepoint: {}", codepoint));
                        result.push(ch);
                    }
                    Some(c) => panic!("Invalid escape sequence: \\{}", c as char),
                    None    => panic!("Unterminated escape sequence"),
                }
            }

            Some(c) => {
                result.push(c as char);
                self.advance();
            }
        }
    }
}

fn read_unicode_escape(&mut self) -> u32 {
    let mut value: u32 = 0;
    for _ in 0..4 {
        match self.current() {
            Some(c) => {
                let digit = match c {
                    b'0'..=b'9' => (c - b'0') as u32,
                    b'a'..=b'f' => (c - b'a' + 10) as u32,
                    b'A'..=b'F' => (c - b'A' + 10) as u32,
                    _ => panic!("Invalid hex digit in unicode escape: {}", c as char),
                };
                value = value * 16 + digit;
                self.advance();
            }
            None => panic!("Unterminated unicode escape"),
        }
    }
    value
}
}
```
The outer loop runs until we hit the closing `"` or the end of input. Three cases:

- `Some(b'"')` - closing quote, we advance past it and return the accumulated string.
- `Some(b'\\')` - backslash, meaning an escape sequence follows. We advance past the backslash, then look at the next character to determine which escape it is. Each escape character maps to its actual value. The `\uXXXX` case reads exactly four hex digits and converts them to a Unicode codepoint.
- `Some(c)` - any other character, push it directly onto the result string. `c as char` converts the `u8` byte to a Rust `char`.

In `read_unicode_escape`, the hex digit parsing uses byte arithmetic: `c - b'0'` gives the numeric value of a decimal digit (e.g., `b'3' - b'0'` = `3`). `c - b'a' + 10` gives the numeric value of a lowercase hex letter (e.g., `b'c' - b'a' + 10` = `12`). We accumulate the four digits into a `u32` using `value = value * 16 + digit`, shifting left one hex place each iteration.

### Reading Numbers
JSON numbers can be negative, can have a decimal point, and can have an exponent (`1.5e10`). We'll support negative, integer, and decimal forms. Exponent notation is rare in practice and we'll leave it out to keep the code focused.

```rust
impl Lexer {
fn read_number(&mut self) -> f64 {
    let mut s = String::new();

    // optional minus sign
    if let Some(b'-') = self.current() {
        s.push('-');
        self.advance();
    }

    // integer part
    loop {
        match self.current() {
            Some(c @ b'0'..=b'9') => {
                s.push(c as char);
                self.advance();
            }
            _ => break,
        }
    }

    // optional decimal part
    if let Some(b'.') = self.current() {
        s.push('.');
        self.advance();

        loop {
            match self.current() {
                Some(c @ b'0'..=b'9') => {
                    s.push(c as char);
                    self.advance();
                }
                _ => break,
            }
        }
    }

    s.parse::<f64>().unwrap_or_else(|_| panic!("Invalid number: {}", s))
}
}
```
We accumulate the number's characters into a `String`, then use Rust's built-in `str::parse::<f64>()` to convert it. This avoids reimplementing float parsing from scratch.

`Some(c @ b'0'..=b'9')` is an `@` binding combined with a range pattern, it matches any digit byte and simultaneously binds it to `c` so we can push it onto the string. Without `@`, we'd match the digit but have no name for it.

The `if let Some(b'.') = self.current()` checks for a decimal point without consuming it first, if it's there, we push it and advance into the fractional digits loop.

### The Parser
The parser takes the `Vec<Token>` the tokenizer produced and turns it into a `JsonValue`. It works through the token list using the same cursor pattern as the lexer,  a current position that advances as tokens are consumed.

### The parser Struct
```rust
struct Parser {
tokens: Vec<Token>,
pos: usize,
}

impl Parser {
fn new(tokens: Vec<Token>) -> Parser {
    Parser { tokens, pos: 0 }
}

fn current(&self) -> &Token {
    if self.pos < self.tokens.len() {
        &self.tokens[self.pos]
    } else {
        panic!("Unexpected end of token stream");
    }
}

fn advance(&mut self) {
    self.pos += 1;
}

fn expect(&mut self, description: &str) -> &Token {
    if self.pos < self.tokens.len() {
        let token = &self.tokens[self.pos];
        self.pos += 1;
        token
    } else {
        panic!("Expected {} but reached end of input", description);
    }
}
}
```
`current` returns a reference to the token at the current position without advancing. Unlike the lexer's `current` which returned `Option<u8>`, this one panics on end-of-input, by the time we're parsing, a well-formed token stream always has a next token when the grammar says there should be one.

`expect` is a convenience method that returns the current token and advances past it in one step. We'll use it when we need to consume a token we know must be there like the `:` between a key and value in an object.

Notice `current` and `expect` both return `&Token` a reference into the parser's token vector. The parser owns the tokens, and we borrow them out. This means when we pattern match on `self.current()`, we're matching on a `&Token`. Keep that in mind we'll see it immediately.

### The Core `parse_value`
The parser is built around one central method: `parse_value`. It looks at the current token and dispatches to the appropriate parsing logic:
```rust
impl Parser {
fn parse_value(&mut self) -> JsonValue {
    match self.current() {
        Token::LeftBrace    => self.parse_object(),
        Token::LeftBracket  => self.parse_array(),
        Token::True         => { self.advance(); JsonValue::Bool(true)  }
        Token::False        => { self.advance(); JsonValue::Bool(false) }
        Token::Null         => { self.advance(); JsonValue::Null        }
        Token::NumberToken(n) => {
            let value = *n;
            self.advance();
            JsonValue::Number(value)
        }
        Token::StringToken(s) => {
            let value = s.clone();
            self.advance();
            JsonValue::Str(value)
        }
        Token::RightBrace   => panic!("Unexpected '}'"),
        Token::RightBracket => panic!("Unexpected ']'"),
        Token::Colon        => panic!("Unexpected ':'"),
        Token::Comma        => panic!("Unexpected ','"),
    }
}
}
```
This is the grammar of JSON encoded directly as Rust code. A value is one of six things and the first token tells us which one.

There are two important ownership details here that need explaining.

**Why `*n` for the number:** `self.current()` returns `&Token`. When we match `Token::NumberToken(n)` on a `&Token`, the binding `n` is `&f64`, a reference to the `f64` inside the token. We write `let value = *n` to dereference it and copy the `f64` out. `f64` is `Copy`, so this is a copy, not a move. We need `value` as an owned `f64` to put inside `JsonValue::Number(value)`.

**Why `s.clone()` for the string:** Same situation, `s` is `&String`, a reference to the string inside the token. We can't move out of it because we only have a reference. `String` is not `Copy`. So we call `.clone()` to produce an owned `String` we can put inside `JsonValue::Str(value)`. The token keeps its copy, our new `JsonValue` gets its own copy.

The `parse_object` and `parse_array` calls are where the recursive structure lives, we'll write those next.

### Parsing Objects
A JSON object looks like:
```json
{"key1": value1, "key2": value2}
```

The grammar is: `{`, then zero or more `string : value` pairs separated by `,`, then `}`.

```rust
impl Parser {
fn parse_object(&mut self) -> JsonValue {
    self.advance(); // consume '{'

    let mut pairs: Vec<(String, JsonValue)> = Vec::new();

    // handle empty object
    if let Token::RightBrace = self.current() {
        self.advance();
        return JsonValue::Object(pairs);
    }

    loop {
        // parse the key, must be a string
        let key = match self.expect("object key") {
            Token::StringToken(s) => s.clone(),
            other => panic!("Expected string key, got something else"),
        };

        // consume the colon
        match self.expect("colon") {
            Token::Colon => {}
            _ => panic!("Expected ':' after object key"),
        }

        // parse the value, recursive call
        let value = self.parse_value();

        pairs.push((key, value));

        // after a pair, expect either ',' or '}'
        match self.current() {
            Token::Comma => {
                self.advance(); // consume comma, loop for next pair
            }
            Token::RightBrace => {
                self.advance(); // consume '}', we're done
                break;
            }
            _ => panic!("Expected ',' or '}}' in object"),
        }
    }

    JsonValue::Object(pairs)
}
}
```
Walk through the logic step by step:

We advance past the `{`. Then we immediately check for `}`, an empty object `{}` is valid and we return early with an empty `Vec`.

For non-empty objects we enter a loop. Each iteration parses one key-value pair. We call `self.expect("object key")` which gives us the current token and advances. We match it, if it's a `StringToken` we clone the string out. If it's anything else, we panic.

Then we expect a `Colon`. The match arm `Token::Colon => {}` matches and does nothing, we just needed to consume the token and verify it was a colon.

Then `self.parse_value()`, this is the recursive call. The value after the colon can itself be any JSON value, including a nested object or array. `parse_value` handles all of that. This recursion is what makes the parser work for arbitrarily deep JSON.

After parsing the pair, we look at the current token. A comma means there's another pair, consume it and loop. A `}` means we're done, consume it and break. Anything else is malformed JSON.

One thing to note: the `other` binding in `match self.expect("object key")`, we write it as `other` rather than `_` so we have a name if we wanted to include it in the panic message. With our current conceptual understanding we can't easily format arbitrary token types, so we just use a generic message. In later articles we'll learn to make error messages much richer

### Parsing Arrays
A JSON array looks like:
```json
[value1, value2, value3]
```

The grammar is: `[`, then zero or more values separated by `,`, then `]`. It's structurally identical to object parsing but simpler
```rust
impl Parser {
fn parse_array(&mut self) -> JsonValue {
    self.advance(); // consume '['

    let mut elements: Vec<JsonValue> = Vec::new();

    // handle empty array
    if let Token::RightBracket = self.current() {
        self.advance();
        return JsonValue::Array(elements);
    }

    loop {
        let element = self.parse_value(); // recursive call
        elements.push(element);

        match self.current() {
            Token::Comma => {
                self.advance(); // consume comma, loop for next element
            }
            Token::RightBracket => {
                self.advance(); // consume ']', we're done
                break;
            }
            _ => panic!("Expected ',' or ']' in array"),
        }
    }

    JsonValue::Array(elements)
}
}
```

Same pattern as `parse_object`, advance past the opening bracket, handle the empty case, loop parsing values separated by commas, break on the closing bracket.

The `self.parse_value()` call inside `parse_array` is also recursive. An array element can be another array, or an object, or anything else. The mutual recursion between `parse_value`, `parse_object`, and `parse_array` is what handles arbitrary nesting depth correctly.

### The Top-Level `parse` Function
Now we wire everything together:
```rust
fn parse(tokens: Vec<Token>) -> JsonValue {
let mut parser = Parser::new(tokens);
let value = parser.parse_value();
value
}
```

Note we changed the signature from `parse(tokens: &[Token])` to `parse(tokens: Vec<Token>)`, we're passing ownership into the parser, which is cleaner since the parser owns its token list.

### Displaying Results
We need a way to verify our parser works. Let's write a function that converts a `JsonValue` back into a JSON string. This is recursive too and it's a great demonstration of how naturally the recursive enum structure enables recursive processing:

```rust
fn display(value: &JsonValue) -> String {
match value {
    JsonValue::Null        => String::from("null"),
    JsonValue::Bool(true)  => String::from("true"),
    JsonValue::Bool(false) => String::from("false"),
    JsonValue::Number(n)   => format!("{}", n),
    JsonValue::Str(s)      => format!("\"{}\"", s),

    JsonValue::Array(elements) => {
        let mut result = String::from("[");
        let mut first = true;
        for element in elements {
            if !first {
                result.push_str(", ");
            }
            result.push_str(&display(element));
            first = false;
        }
        result.push(']');
        result
    }

    JsonValue::Object(pairs) => {
        let mut result = String::from("{");
        let mut first = true;
        for (key, value) in pairs {
            if !first {
                result.push_str(", ");
            }
            result.push_str(&format!("\"{}\": {}", key, display(value)));
            first = false;
        }
        result.push('}');
        result
    }
}
}
```
`display` takes `&JsonValue`,  a reference, so it doesn't consume the value. For the simple variants it produces strings directly. For `Array` and `Object` it loops over the children, recursively calling `display` on each one, building up the output string with proper separators.

The `first` boolean flag is how we handle comma separation without a trailing comma,  we skip the comma before the first element and add it before every subsequent one.

## Updating Main function
Lets update the main function to add some test json and check if our parser is working correctly or not.
```rust
fn main() {
let tests = vec![
    r#"null"#,
    r#"true"#,
    r#"42"#,
    r#"-3.14"#,
    r#""hello world""#,
    r#"[]"#,
    r#"{}"#,
    r#"[1, 2, 3]"#,
    r#"{"name": "alice", "age": 30, "active": true}"#,
    r#"{"scores": [95, 87, 100], "info": {"city": "delhi", "zip": null}}"#,
    r#"["hello\nworld", "tab\there", "\u0041\u0042\u0043"]"#,
];

for input in &tests {
    let tokens = tokenize(input);
    let value = parse(tokens);
    println!("Input:  {}", input);
    println!("Output: {}", display(&value));
    println!();
}
}
```

Run it with `cargo run` and you should see:
```bash
Input:  null
Output: null

Input:  true
Output: true

Input:  42
Output: 42

Input:  -3.14
Output: -3.14

Input:  "hello world"
Output: "hello world"

Input:  []
Output: []

Input:  {}
Output: {}

Input:  [1, 2, 3]
Output: [1, 2, 3]

Input:  {"name": "alice", "age": 30, "active": true}
Output: {"name": "alice", "age": 30, "active": true}

Input:  {"scores": [95, 87, 100], "info": {"city": "delhi", "zip": null}}
Output: {"scores": [95, 87, 100], "info": {"city": "delhi", "zip": null}}

Input:  ["hello\nworld", "tab\there", "\u0041\u0042\u0043"]
Output: ["hello
world", "tab	here", "ABC"]
```

The last test shows escape sequences working,  `\n` becomes a real newline, `\t` becomes a real tab, and `\u0041\u0042\u0043` decodes to `ABC`.

## Conclusion
That's it. You've built a real, working JSON parser from scratch. It was a long one but I hope you learned a lot. In the next one we are going to build a TOML config parser and learn about error handling in detail, we will understand about `Result<T, E>` and stop using `unwrap` calls. See you soon.

# Learn SQL and SQLx by Building a Book Library CLI in Rust

> Learn complete SQL and SQLx 0.9 by building a book library cli in Rust

Source: https://blog.sheerluck.dev/posts/rust/learn-sql-and-sqlx-by-building-a-book-library-cli-in-rust/ · Markdown: https://blog.sheerluck.dev/posts/rust/learn-sql-and-sqlx-by-building-a-book-library-cli-in-rust/index.md

In this post, we are going to learn SQL from the ground up, no prior database knowledge required. We will learn about tables, data types, constraints, `SELECT`, `INSERT`, `UPDATE`, `DELETE`, filtering with `WHERE`, pattern matching with `LIKE`, sorting with `ORDER BY`, grouping with `GROUP BY`, aggregate functions like `COUNT` and `AVG`, `JOIN`s across multiple tables, subqueries, transactions, and indexes. 

We will practice every concept hands-on in the `sqlite3` terminal first. Then we will learn how to bring all of that into Rust with SQLx: compile-time checked queries, connection pooling, schema migrations, and the `query!`, `query_as!`, and `query_scalar!` macros. Once we cover all the concepts, we will build a **Book Library CLI** backed by SQLite with full CRUD, rich search, genre statistics, pagination, and bulk import with transactions. I am really excited for this project and I hope you are too. I won't go too deep in theory, just practical and we will build our knowledge of these concepts over time with more articles.

The only prerequisite is that you have read the previous articles in this series, as I will assume you know ownership, borrowing, structs, enums, pattern matching, error handling, generics, traits, lifetimes, HashMap, iterators, closures, smart pointers, concurrency, and async/await with Tokio.

Get source code from [here](https://github.com/MrSheerluck/bookcli)

> **A quick note before we begin:** This article is in three parts. Part 1 teaches SQL from scratch using the `sqlite3` command-line tool, no Rust, just SQL. Part 2 introduces SQLx and explains how its compile-time checked queries work. Part 3 builds the full Book Library CLI. If you already know SQL well, you can skim Part 1. But I recommend at least glancing through it because the examples use the exact same schema we will build in the project, and we cover some patterns (GROUP BY on genres, subqueries for rankings) that you will use in the CLI.

## Part 1: Learning SQL from Scratch

### What Is a Database

A database is an organised collection of data stored on disk. You interact with it using SQL (Structured Query Language). SQL is a declarative language. You describe *what* you want, not *how* to get it. The database engine figures out the how.

Think of it like this. If you stored books in a JSON file, finding all science fiction books by a specific author would require you to read the entire file into memory, loop through every entry, and filter manually. At 10 books, this is fine. At 10,000 books, it is slow. At 10,000,000 books, it does not fit in memory. A database handles this by keeping data organised on disk, using indexes to find things without scanning everything, and returning only what you asked for.

### Why SQLite

The database engine we will use is SQLite. Unlike PostgreSQL or MySQL, SQLite is not a separate server process. It is a library that reads and writes directly to a single file on disk. This makes it perfect for learning, for CLI tools, for mobile apps, for desktop applications, and for embedded systems. SQLite is the most deployed database engine in the world. It runs inside every iPhone, every Android device, every Chrome browser, and every Python installation.

### Getting Started with sqlite3

Most systems come with `sqlite3` pre-installed. Check:

```bash
sqlite3 --version
```

If you see something like `3.43.0`, you are good. If not, install it (`brew install sqlite` on macOS, `apt install sqlite3` on Ubuntu).

Create a database and enter the interactive shell:

```bas
sqlite3 library.db
```

You are now inside the SQLite shell. Every command you type is SQL and ends with a semicolon `;`. To exit, type `.quit` (with a dot, dot commands are sqlite3-specific, not SQL). Let's create our first table.

### Creating Tables

A table is like a spreadsheet. Columns have names and types. Rows are individual records. Here is our first table:

```sql
CREATE TABLE books (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title TEXT NOT NULL,
author TEXT NOT NULL,
genre TEXT NOT NULL,
year INTEGER,
read INTEGER NOT NULL DEFAULT 0
);
```

Now, let me explain what we just did.

`CREATE TABLE books (...)` creates a new table called `books`. Inside the parentheses, we define columns. Each column has a name, a type, and optional constraints.

`id INTEGER PRIMARY KEY AUTOINCREMENT`. `INTEGER` is the type (a whole number). `PRIMARY KEY` means this column uniquely identifies each row, no two rows can have the same ID. `INTEGER PRIMARY KEY` already tells SQLite to automatically generate row IDs when you insert a row without specifying an ID. Adding `AUTOINCREMENT` changes the allocation strategy so previously used IDs are never reused, even if rows are deleted. It has a small performance cost, so most applications only use it when they specifically need that guarantee.

`title TEXT NOT NULL`. `TEXT` stores strings. `NOT NULL` means this column cannot be empty. You must provide a title for every book.

`author TEXT NOT NULL`. Same as title, every book must have an author.

`genre TEXT NOT NULL`. Every book must have a genre. We will use this for filtering and grouping later.

`year INTEGER`. No `NOT NULL` here, so `year` can be `NULL`. Some books have unknown publication years.

`read INTEGER NOT NULL DEFAULT 0`. `read` is an integer that acts as a boolean because SQLite does not have a separate `BOOLEAN` type. `0` means unread, `1` means read. `DEFAULT 0` means new books start as unread unless specified otherwise. This is our convention. We will map `0` and `1` to Rust's `bool` later with SQLx.

SQLite has five storage classes:

|Storage Class|What It Stores|Example|
|---|---|---|
|`INTEGER`|Whole numbers|`42`, `0`, `-7`|
|`TEXT`|Text strings|`'hello'`, `'world'`|
|`REAL`|Floating-point numbers|`3.14`, `-0.5`|
|`BLOB`|Binary data|`X'89504E47'`|
|`NULL`|The absence of a value|`NULL`|

### Inspecting Tables

To see all tables in the database:

```
.tables
```

To see the schema of a specific table:

```
.schema books
```

Output:

```sql
CREATE TABLE books (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title TEXT NOT NULL,
author TEXT NOT NULL,
genre TEXT NOT NULL,
year INTEGER,
read INTEGER NOT NULL DEFAULT 0
);
```

These dot commands are sqlite3-specific shortcuts. They are not SQL. Real SQL queries go through a different table called `sqlite_master`, but `.tables` and `.schema` are faster to type.

### Inserting Data

Now let's add some books. The `INSERT` statement adds rows:

```sql
INSERT INTO books (title, author, genre, year) VALUES
('The Rust Programming Language', 'Steve Klabnik', 'Programming', 2018);
```

Now, let me explain what we just did.

`INSERT INTO books` says we are adding to the `books` table. The parentheses `(title, author, genre, year)` list which columns we are providing values for. `VALUES (...)` provides the corresponding values. We did not specify `id` because `AUTOINCREMENT` handles it. We did not specify `read` because it defaults to `0`. The single quotes `'...'` denote string literals in SQL. Double quotes are for identifiers (table names, column names), single quotes are for string values.

Insert more books:

```sql
INSERT INTO books (title, author, genre, year) VALUES
('Project Hail Mary', 'Andy Weir', 'Science Fiction', 2021);

INSERT INTO books (title, author, genre, year, read) VALUES
('The Martian', 'Andy Weir', 'Science Fiction', 2011, 1);

INSERT INTO books (title, author, genre, year) VALUES
('Atomic Habits', 'James Clear', 'Self-Help', 2018);

INSERT INTO books (title, author, genre, year, read) VALUES
('Dune', 'Frank Herbert', 'Science Fiction', 1965, 1);

INSERT INTO books (title, author, genre, year) VALUES
('Neuromancer', 'William Gibson', 'Cyberpunk', 1984);

INSERT INTO books (title, author, genre, year) VALUES
('Snow Crash', 'Neal Stephenson', 'Cyberpunk', 1992);

INSERT INTO books (title, author, genre, year, read) VALUES
('The Name of the Wind', 'Patrick Rothfuss', 'Fantasy', 2007, 1);

INSERT INTO books (title, author, genre, year) VALUES
('A Wise Man's Fear', 'Patrick Rothfuss', 'Fantasy', 2011);

INSERT INTO books (title, author, genre, year, read) VALUES
('Deep Work', 'Cal Newport', 'Self-Help', 2016, 1);
```

Now we have 10 books to work with.

### Selecting Data

The `SELECT` statement retrieves data. It is the most used statement in SQL.

#### SELECT All Columns

```sql
SELECT * FROM books;
```

Output:

```
id  title                        author             genre            year  read
--  ---------------------------  -----------------  ---------------  ----  ----
1   The Rust Programming Langua  Steve Klabnik      Programming      2018  0
2   Project Hail Mary            Andy Weir          Science Fiction  2021  0
3   The Martian                  Andy Weir          Science Fiction  2011  1
4   Atomic Habits                James Clear        Self-Help        2018  0
5   Dune                         Frank Herbert      Science Fiction  1965  1
6   Neuromancer                  William Gibson     Cyberpunk        1984  0
7   Snow Crash                   Neal Stephenson    Cyberpunk        1992  0
8   The Name of the Wind         Patrick Rothfuss   Fantasy          2007  1
9   A Wise Man's Fear             Patrick Rothfuss   Fantasy          2011  0
10  Deep Work                    Cal Newport        Self-Help        2016  1
```

`*` means "all columns." Read this as "select all columns from the books table."

#### SELECT Specific Columns

Selecting everything is wasteful if you only need titles. Be specific:

```sql
SELECT title, author FROM books;
```

#### SELECT with Aliases

You can rename columns in the output with `AS`:

```sql
SELECT title AS book_title, author AS written_by FROM books;
```

#### SELECT DISTINCT

To get unique values without duplicates:

```sql
SELECT DISTINCT genre FROM books;
```

Output:

```
genre
---------------
Programming
Science Fiction
Self-Help
Cyberpunk
Fantasy
```

`DISTINCT` removes duplicates. There are 10 books but only 5 distinct genres.

### Filtering with WHERE

`WHERE` filters rows based on conditions. It comes after `FROM` and before `ORDER BY`.

#### Equality and Comparison

```sql
SELECT title, year FROM books WHERE author = 'Andy Weir';
```

Output:

```
title               year
------------------  ----
Project Hail Mary   2021
The Martian         2011
```

Comparison operators: `=` (equal), `!=` or `<>` (not equal), `<` (less than), `>` (greater than), `<=`, `>=`.

```sql
SELECT title, year FROM books WHERE year >= 2015;
```

Output:

```
title                        year
---------------------------  ----
The Rust Programming Langua  2018
Project Hail Mary            2021
Atomic Habits                2018
Deep Work                    2016
```

#### AND, OR, NOT

Combine conditions with `AND` and `OR`:

```sql
SELECT title, year FROM books WHERE genre = 'Science Fiction' AND year < 2020;
```

Output:

```
title          year
-------------  ----
The Martian    2011
Dune           1965
```

Both conditions must be true for `AND`. For `OR`, only one needs to be true:

```sql
SELECT title, genre FROM books WHERE genre = 'Cyberpunk' OR genre = 'Fantasy';
```

You can group conditions with parentheses:

```sql
SELECT title, author, year FROM books
WHERE (genre = 'Science Fiction' OR genre = 'Fantasy') AND year > 2000;
```

Now, let me explain what we just did.

This finds Science Fiction or Fantasy books published after the year 2000. Without the parentheses, `AND` has higher precedence than `OR`, so `genre = 'Fantasy' AND year > 2000` would be evaluated first, changing the meaning. Always use parentheses when mixing `AND` and `OR`.

`NOT` negates a condition:

```sql
SELECT title FROM books WHERE NOT read;
```

Output:

```
title
---------------------------
The Rust Programming Langua
Project Hail Mary
Atomic Habits
Neuromancer
Snow Crash
A Wise Man's Fear
```

In SQLite, `read` is `0` or `1`. `NOT 0` is true, `NOT 1` is false. This shows all unread books. You could also write `WHERE read = 0`.

#### BETWEEN

`BETWEEN` checks if a value is within a range (inclusive on both ends):

```sql
SELECT title, year FROM books WHERE year BETWEEN 1990 AND 2020;
```

This is equivalent to `WHERE year >= 1990 AND year <= 2020`.

#### IN

`IN` checks if a value matches any value in a list:

```sql
SELECT title, genre FROM books WHERE genre IN ('Cyberpunk', 'Fantasy');
```

This is equivalent to `WHERE genre = 'Cyberpunk' OR genre = 'Fantasy'`. `IN` is much cleaner when you have many values.

#### LIKE and Pattern Matching

`LIKE` does pattern matching on text. Two wildcards:
- `%` matches any sequence of characters (including zero)
- `_` matches exactly one character

```sql
SELECT title FROM books WHERE title LIKE '%Rust%';
```

Output:

```
title
---------------------------
The Rust Programming Langua
```

`%Rust%` means "anything, then Rust, then anything." It finds any title containing the word "Rust."

```sql
SELECT title FROM books WHERE author LIKE 'A%';
```

`A%` means "starts with A." This matches Andy Weir.

```sql
SELECT title FROM books WHERE title LIKE 'The %';
```
In SQLite, `LIKE` is case-insensitive for ASCII characters by default. For example, `LIKE '%rust%'` also matches `"Rust"`.
#### IS NULL and IS NOT NULL

To check for NULL values, you cannot use `=` because `NULL = NULL` is false in SQL (NULL means "unknown," and two unknowns are not equal). Use `IS NULL` and `IS NOT NULL`:

```sql
SELECT title FROM books WHERE year IS NULL;
```

This would return books with no publication year. We do not have any yet, so it returns nothing.

```sql
SELECT title, year FROM books WHERE year IS NOT NULL;
```

This returns all books (all our books have years).

### Sorting with ORDER BY

`ORDER BY` sorts the result set. It comes after `WHERE` (and after `GROUP BY` if present).

```sql
SELECT title, year FROM books ORDER BY year;
```

Ascending (oldest first) is the default. For descending, add `DESC`:

```sql
SELECT title, year FROM books ORDER BY year DESC;
```

You can sort by multiple columns:

```sql
SELECT title, author, year FROM books ORDER BY author, year DESC;
```

This sorts by author alphabetically. For books by the same author, it sorts by year descending (newest first).

```sql
SELECT title, year FROM books ORDER BY read, year DESC;
```

Now, let me explain what we just did.

Sorting by `read` first groups unread books together (read = 0 comes before read = 1 in ascending order). Within each read group, books are sorted by year descending. This is a common pattern: sort by a category first, then by a numeric value within each category.

### Limiting and Paginating with LIMIT and OFFSET

`LIMIT` restricts how many rows are returned. `OFFSET` skips rows at the beginning:

```sql
SELECT title, year FROM books ORDER BY year DESC LIMIT 3;
```

This returns the 3 most recent books.

```sql
SELECT title, year FROM books ORDER BY year DESC LIMIT 3 OFFSET 3;
```

This skips the first 3 and returns books 4 through 6. This is pagination. Page 1: `LIMIT 3 OFFSET 0`. Page 2: `LIMIT 3 OFFSET 3`. Page 3: `LIMIT 3 OFFSET 6`.

SQLite also supports a shorthand:

```sql
SELECT title, year FROM books ORDER BY year DESC LIMIT 3, 3;
```

`LIMIT 3, 3` means `LIMIT 3 OFFSET 3`. The first number is the offset, the second is the limit. I find this confusing, so I always write `LIMIT x OFFSET y`.

### Aggregate Functions

Aggregate functions compute a single value from multiple rows.

#### COUNT

```sql
SELECT COUNT(*) FROM books;
```

Returns `10`, the total number of rows.

```sql
SELECT COUNT(*) FROM books WHERE read = 1;
```

Returns `4`, the number of read books.

```sql
SELECT COUNT(DISTINCT author) FROM books;
```

Returns the number of unique authors. In our sample data there are 8 unique authors: Steve Klabnik, Andy Weir, James Clear, Frank Herbert, William Gibson, Neal Stephenson, Patrick Rothfuss, and Cal Newport.

#### SUM, AVG, MAX, MIN

```sql
SELECT MAX(year) FROM books;
```

Returns `2021`, the most recent publication year.

```sql
SELECT MIN(year) FROM books;
```

Returns `1965`.

```sql
SELECT AVG(year) FROM books;
```

Returns the average publication year.

```sql
SELECT SUM(read) FROM books;
```

Returns `4`, since `read` is `0` or `1`, summing gives the count of read books.

Because `read` stores `0` and `1`, `SUM(read)` gives the number of read books. This is a perfectly valid SQL technique. Some developers prefer `COUNT(*)` with a `WHERE` clause because it more clearly expresses the intent.

### Grouping with GROUP BY

`GROUP BY` groups rows that have the same values in specified columns. You then use aggregate functions on each group.

```sql
SELECT genre, COUNT(*) AS book_count FROM books GROUP BY genre;
```

Output:

```
genre            book_count
---------------  ----------
Programming      1
Science Fiction  3
Self-Help        2
Cyberpunk        2
Fantasy          2
```

Now, let me explain what we just did.

`GROUP BY genre` creates one row per unique genre. `COUNT(*)` counts how many books are in each group. `AS book_count` gives the count column a name. Without `AS`, the column would be named `COUNT(*)`, which is ugly.

The rule of `GROUP BY`: every column in the `SELECT` list must either be in the `GROUP BY` clause or be wrapped in an aggregate function. If you select `genre` and `title`, but only group by `genre`, which title should SQLite show? It cannot decide, so most databases reject this query. SQLite is lenient and picks an arbitrary title, but you should not rely on this.

#### GROUP BY with Multiple Columns

```sql
SELECT author, genre, COUNT(*) FROM books GROUP BY author, genre;
```

This groups by the combination of author and genre. If an author writes in multiple genres, they appear in multiple rows.

#### HAVING - Filtering Groups

`WHERE` filters rows before grouping. `HAVING` filters groups after grouping:

```sql
SELECT genre, COUNT(*) AS cnt FROM books
GROUP BY genre
HAVING cnt > 1;
```

Output:

```
genre            cnt
---------------  ---
Science Fiction  3
Self-Help        2
Cyberpunk        2
Fantasy          2
```

Now, let me explain what we just did.

`GROUP BY genre` creates the groups. `HAVING cnt > 1` keeps only groups with more than one book. Programming is excluded because it has only one book. The order of operations is: `FROM` → `WHERE` → `GROUP BY` → `HAVING` → `SELECT` → `ORDER BY` → `LIMIT`.

#### GROUP BY with Multiple Aggregates

```sql
SELECT
genre,
COUNT(*) AS total,
SUM(read) AS read_count,
ROUND(AVG(year)) AS avg_year
FROM books
GROUP BY genre
ORDER BY total DESC;
```

Now, let me explain what we just did.

Each genre group gets three computed values: the total number of books, the count of read books (by summing the `read` column, which is 0 or 1), and the rounded average publication year. `ROUND()` is a scalar function that rounds to the nearest integer. The result is sorted by total books, most popular genres first. This kind of query is the backbone of any dashboard or reporting feature.

### Updating Data

`UPDATE` modifies existing rows:

```sql
UPDATE books SET read = 1 WHERE id = 1;
```

Now, let me explain what we just did.

This marks book 1 as read. The `WHERE` clause is critical. If you omit it:

```sql
UPDATE books SET read = 1;  -- DANGER: updates ALL rows
```

Every book would be marked as read. Always double-check your `WHERE` clause before running an `UPDATE`.

You can update multiple columns:

```sql
UPDATE books SET read = 1, year = 2020 WHERE id = 6;
```

The `WHERE` clause can use any condition we have learned:

```sql
UPDATE books SET read = 1 WHERE author LIKE 'Andy%';
```

This marks all books by authors whose name starts with "Andy" as read.

### Deleting Data

`DELETE` removes rows:

```sql
DELETE FROM books WHERE id = 10;
```

Again, the `WHERE` clause is critical. Without it, every row is deleted:

```sql
DELETE FROM books;  -- DANGER: deletes ALL rows from the table
```

If you want to delete all rows in test data, a faster alternative is `DELETE FROM books` (which logs each deletion) or `TRUNCATE TABLE books` (not supported by SQLite, use `DELETE FROM books` instead).

### Subqueries

A subquery is a query nested inside another query. The inner query runs first, and its result is used by the outer query.

#### Subquery in WHERE

Find all books published in a year where at least one book was read:

```sql
SELECT title, year FROM books
WHERE year IN (SELECT year FROM books WHERE read = 1);
```

Now, let me explain what we just did.

The inner query `SELECT year FROM books WHERE read = 1` returns the years of all read books. The outer query finds all books published in any of those years, whether read or not. This is a semi-join pattern.

#### Subquery in SELECT

Show each book's publication year relative to the average:

```sql
SELECT
title,
year,
year - (SELECT AVG(year) FROM books) AS diff_from_avg
FROM books
ORDER BY diff_from_avg DESC;
```

Now, let me explain what we just did.

The subquery `SELECT AVG(year) FROM books` computes the average publication year once. For each book, we compute how far its year is from that average. Books after the average have positive values. Books before the average have negative values. The subquery in `SELECT` must return exactly one row and one column (a scalar value).

#### Subquery in FROM

Find the genre with the most books:

```sql
SELECT genre, cnt FROM (
SELECT genre, COUNT(*) AS cnt FROM books GROUP BY genre
) AS genre_counts
ORDER BY cnt DESC
LIMIT 1;
```

Now, let me explain what we just did.

The inner query creates a temporary table (`genre_counts`) with genre names and counts. The outer query takes that temporary result and sorts it to find the most common genre. Subqueries in `FROM` are called derived tables. They must have an alias (`AS genre_counts`). This pattern is useful for multi-step analysis where you need to query the result of an aggregation.

> There is a cleaner way to write this using a Common Table Expression (CTE or `WITH` clause), but we will save CTEs for a future article.

### Joining Tables

So far we have worked with a single table. Real databases have multiple related tables. Let's create two more tables to demonstrate joins.

#### Creating Related Tables

```sql
CREATE TABLE authors (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL UNIQUE
);

CREATE TABLE genres (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL UNIQUE
);
```

Now, let me explain what we just did.

`UNIQUE` means no two rows can have the same value in this column. If we try to insert "Andy Weir" twice into `authors`, the second insert fails. This prevents duplicate authors and genres.

Insert data into these tables:

```sql
INSERT INTO authors (name) VALUES
('Steve Klabnik'),
('Andy Weir'),
('James Clear'),
('Frank Herbert'),
('William Gibson'),
('Neal Stephenson'),
('Patrick Rothfuss'),
('Cal Newport');

INSERT INTO genres (name) VALUES
('Programming'),
('Science Fiction'),
('Self-Help'),
('Cyberpunk'),
('Fantasy');
```

Now we have three tables. But they are not connected. We need to link books to authors and genres.

#### Creating a Better Books Table

Let's drop our old books table and create one with foreign keys:

```sql
DROP TABLE books;

CREATE TABLE books (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title TEXT NOT NULL,
author_id INTEGER NOT NULL,
genre_id INTEGER NOT NULL,
year INTEGER,
read INTEGER NOT NULL DEFAULT 0,
FOREIGN KEY (author_id) REFERENCES authors(id),
FOREIGN KEY (genre_id) REFERENCES genres(id)
);
```

Now, let me explain what we just did.

`DROP TABLE books` deletes the entire table. This is irreversible. In a real application, you would use migrations to add columns to an existing table instead of dropping it. We are doing this for the learning exercise.

`author_id INTEGER NOT NULL` stores the ID from the `authors` table instead of the author's name directly. `genre_id` does the same for genres. `FOREIGN KEY (author_id) REFERENCES authors(id)` tells SQLite that `author_id` must match an existing `id` in the `authors` table. If you try to insert a book with `author_id = 999` when no author has that ID, the insert fails. This is called referential integrity.

> **Want to learn how to design relational databases?**
> We intentionally designed this schema to be simple so we can focus on learning SQL. In a real application, deciding **what tables to create, how they relate to each other, when to normalize data, which constraints to use, and which columns should be indexed** is an important skill on its own.
> I'll cover all of that in a dedicated follow-up article where we'll design relational databases from scratch, starting from application requirements and gradually evolving them into well-structured, production-ready schemas.

Insert books with the new schema:

```sql
INSERT INTO books (title, author_id, genre_id, year) VALUES
('The Rust Programming Language', 1, 1, 2018);

INSERT INTO books (title, author_id, genre_id, year) VALUES
('Project Hail Mary', 2, 2, 2021);

INSERT INTO books (title, author_id, genre_id, year, read) VALUES
('The Martian', 2, 2, 2011, 1);

INSERT INTO books (title, author_id, genre_id, year) VALUES
('Atomic Habits', 3, 3, 2018);

INSERT INTO books (title, author_id, genre_id, year, read) VALUES
('Dune', 4, 2, 1965, 1);

INSERT INTO books (title, author_id, genre_id, year) VALUES
('Neuromancer', 5, 4, 1984);

INSERT INTO books (title, author_id, genre_id, year) VALUES
('Snow Crash', 6, 4, 1992);

INSERT INTO books (title, author_id, genre_id, year, read) VALUES
('The Name of the Wind', 7, 5, 2007, 1);

INSERT INTO books (title, author_id, genre_id, year) VALUES
('A Wise Man's Fear', 7, 5, 2011);

INSERT INTO books (title, author_id, genre_id, year, read) VALUES
('Deep Work', 8, 3, 2016, 1);
```

Now if we `SELECT * FROM books`, we see IDs instead of names. That is not helpful for humans. We need to join the tables.

#### INNER JOIN

An `INNER JOIN` combines rows from two tables based on a matching condition. It only returns rows where the match exists in both tables:

```sql
SELECT books.title, authors.name AS author, genres.name AS genre
FROM books
INNER JOIN authors ON books.author_id = authors.id
INNER JOIN genres ON books.genre_id = genres.id;
```

Output:

```
title                        author             genre
---------------------------  -----------------  ---------------
The Rust Programming Langua  Steve Klabnik      Programming
Project Hail Mary            Andy Weir          Science Fiction
The Martian                  Andy Weir          Science Fiction
Atomic Habits                James Clear        Self-Help
Dune                         Frank Herbert      Science Fiction
Neuromancer                  William Gibson     Cyberpunk
Snow Crash                   Neal Stephenson    Cyberpunk
The Name of the Wind         Patrick Rothfuss   Fantasy
A Wise Man's Fear             Patrick Rothfuss   Fantasy
Deep Work                    Cal Newport        Self-Help
```

Now, let me explain what we just did.

`FROM books` starts with the books table. `INNER JOIN authors ON books.author_id = authors.id` takes each book row, looks up the matching author row where `authors.id` equals `books.author_id`, and combines them into a single result row. `INNER JOIN genres ON books.genre_id = genres.id` does the same for genres. The result looks like our original flat table, but the data is normalised, author names and genre names are stored only once, in their own tables. If an author's name changes, we update one row in `authors`, not every book by that author.

When joining, you can prefix column names with the table name (`books.title`, `authors.name`) to avoid ambiguity. If two tables have columns with the same name, you must qualify them.

#### LEFT JOIN

An `INNER JOIN` excludes rows that do not have a match. A `LEFT JOIN` includes all rows from the left table, filling unmatched columns with `NULL`:

Let's add an author with no books:

```sql
INSERT INTO authors (name) VALUES ('Isaac Asimov');
```

Now compare:

```sql
-- INNER JOIN: only authors who have books
SELECT authors.name, COUNT(books.id) AS book_count
FROM authors
INNER JOIN books ON authors.id = books.author_id
GROUP BY authors.name
ORDER BY book_count DESC;
```

This only shows the 8 authors who have books. Isaac Asimov is excluded.

```sql
-- LEFT JOIN: all authors, even those with no books
SELECT authors.name, COUNT(books.id) AS book_count
FROM authors
LEFT JOIN books ON authors.id = books.author_id
GROUP BY authors.name
ORDER BY book_count DESC;
```

Isaac Asimov appears with `book_count = 0`.

Now, let me explain what we just did.

`LEFT JOIN` keeps every row from the left table (`authors`). For Isaac Asimov, there are no matching `books` rows, so `books.id` is `NULL` for that joined row. `COUNT(books.id)` counts only non-NULL values, so it returns `0`. This is how you find authors with no books, genres with no books, or any "missing" relationship.

#### Join with Filtering and Sorting

You can combine joins with everything else we have learned:

```sql
SELECT
authors.name AS author,
books.title,
books.year,
genres.name AS genre
FROM books
INNER JOIN authors ON books.author_id = authors.id
INNER JOIN genres ON books.genre_id = genres.id
WHERE books.read = 0
AND genres.name = 'Science Fiction'
ORDER BY books.year DESC;
```

This finds all unread science fiction books, sorted by publication year (newest first). The query reads almost like English: "Select author name, book title, year, and genre name from books, joined with authors and genres, where the book is unread and the genre is Science Fiction, sorted by year descending."

### Transactions

Start a transaction with `BEGIN`, run your statements, then `COMMIT` to save or `ROLLBACK` to undo:

```sql
BEGIN;
INSERT INTO books (title, author_id, genre_id, year) VALUES
('Foundation', 9, 2, 1951);
INSERT INTO books (title, author_id, genre_id, year) VALUES
('Foundation and Empire', 9, 2, 1952);
INSERT INTO books (title, author_id, genre_id, year) VALUES
('Second Foundation', 9, 2, 1953);
COMMIT;
```

Now, let me explain what we just did.

All three inserts happen within a single transaction. If the computer crashes after the first insert but before `COMMIT`, the database rolls back to the state before `BEGIN`. Other database connections do not observe the changes until the transaction commits, subject to SQLite's transaction isolation rules.

Try a rollback:

```sql
BEGIN;
INSERT INTO books (title, author_id, genre_id, year) VALUES
('I, Robot', 9, 2, 1950);
ROLLBACK;
```

The insert is undone. `I, Robot` does not appear in the table.

Transactions are essential for data integrity. Without them, a crash during a bulk import leaves your database in an inconsistent state. With them, it is all or nothing.

### Indexes

When you run `SELECT * FROM books WHERE author_id = 5`, SQLite scans every row in the table checking the condition. With 10 rows, this is instant. With 10,000,000 rows, it is slow. An index speeds up lookups:

```sql
CREATE INDEX idx_books_author_id ON books(author_id);
CREATE INDEX idx_books_genre_id ON books(genre_id);
```

Now, let me explain what we just did.

An index is a separate data structure (a B-tree) that maps values to row locations. When you query `WHERE author_id = 5`, SQLite looks up `5` in the index (using a B-tree) instead of scanning every row in the table, making lookups, joins, and many sorts much more efficient on large datasets. Indexes also speed up `ORDER BY` and `JOIN` operations on the indexed columns.

Indexes have a cost. Every `INSERT`, `UPDATE`, and `DELETE` must also update the indexes. And indexes take up disk space. Index columns you frequently filter or join on. Do not index columns that are rarely queried.

To see what indexes exist:

```
.indexes
```

SQLite also provides `EXPLAIN QUERY PLAN` to show how it intends to execute a query. Other database systems provide similar commands, although the syntax differs:

```sql
EXPLAIN QUERY PLAN SELECT * FROM books WHERE author_id = 5;
```

### SQL Summary

We have covered a lot in this first part of the article. Here is a quick reference of the SQL concepts you now know.

|Clause / Keyword|What It Does|Example|
|---|---|---|
|`CREATE TABLE`|Defines a new table|`CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT)`|
|`INSERT INTO`|Adds rows|`INSERT INTO t (name) VALUES ('Alice')`|
|`SELECT`|Retrieves data|`SELECT * FROM t`|
|`SELECT DISTINCT`|Returns unique values|`SELECT DISTINCT name FROM t`|
|`WHERE`|Filters rows|`WHERE age > 18`|
|`AND`, `OR`, `NOT`|Combines conditions|`WHERE a AND (b OR NOT c)`|
|`BETWEEN`|Range check|`WHERE x BETWEEN 1 AND 10`|
|`IN`|Checks membership in a list|`WHERE color IN ('red', 'blue')`|
|`LIKE`|Pattern matching|`WHERE name LIKE 'A%'`|
|`IS NULL`, `IS NOT NULL`|Checks for NULL values|`WHERE email IS NULL`|
|`ORDER BY`|Sorts results|`ORDER BY year DESC`|
|`LIMIT`, `OFFSET`|Paginates results|`LIMIT 10 OFFSET 20`|
|`COUNT`, `SUM`, `AVG`|Aggregate functions|`SELECT COUNT(*) FROM t`|
|`MAX`, `MIN`|Finds extreme values|`SELECT MAX(score) FROM t`|
|`GROUP BY`|Groups rows for aggregation|`GROUP BY category`|
|`HAVING`|Filters grouped results|`HAVING COUNT(*) > 5`|
|`UPDATE`|Modifies existing rows|`UPDATE t SET x = 1 WHERE id = 1`|
|`DELETE`|Removes rows|`DELETE FROM t WHERE id = 1`|
|`INNER JOIN`|Combines matching rows from multiple tables|`... INNER JOIN b ON a.id = b.a_id`|
|`LEFT JOIN`|Keeps all rows from the left table|`... LEFT JOIN b ON a.id = b.a_id`|
|`FOREIGN KEY`|Enforces relationships between tables|`FOREIGN KEY (x) REFERENCES t(id)`|
|`BEGIN`, `COMMIT`|Starts and commits a transaction|`BEGIN; ... COMMIT;`|
|`ROLLBACK`|Cancels a transaction|`BEGIN; ... ROLLBACK;`|
|`CREATE INDEX`|Speeds up lookups|`CREATE INDEX idx ON t(col)`|
|`EXPLAIN QUERY PLAN`|Shows how SQLite executes a query|`EXPLAIN QUERY PLAN SELECT ...`|

At this point, you know enough SQL to build real applications. You can create tables, modify data, query it in different ways, combine related tables with joins, and write efficient queries using indexes and transactions.

The next step is bringing those SQL queries into Rust. In Part 2, we'll learn SQLx, a library that lets us write raw SQL while giving us compile-time query checking, asynchronous database access, and a powerful migration system. Once we've learned SQLx, we'll put everything together in Part 3 by building our complete Book Library CLI.

## Part 2: SQLx - SQL Meets Rust

### Why SQLx

Rust has several libraries for working with databases, each with a different philosophy.

|Library|Style|Async|Compile-Time Query Checks|
|---|---|:-:|:-:|
|Diesel|ORM with its own query DSL|No (synchronous)|Yes|
|SeaORM|ORM built on top of SQLx|Yes|Via SQLx|
|rusqlite|Raw SQL|No (synchronous)|No|
|SQLx|Raw SQL|Yes|Yes|

For this series, we'll use **SQLx**.

There are three reasons.

First, it is fully asynchronous and integrates directly with Tokio, making it suitable for modern Rust applications.

Second, it lets us write **real SQL** instead of learning another query language. We just spent the entire first half of this article learning SQL, so it makes sense to use that knowledge directly.

Finally and this is SQLx's biggest feature, it verifies your SQL queries at compile time against your actual database schema. If you misspell a column name, reference a table that doesn't exist, or try to map an SQL type to an incompatible Rust type, the compiler reports the error before your application ever runs.

That combination of asynchronous I/O, raw SQL, and compile-time verification makes SQLx one of the most popular choices for database access in modern Rust applications.

### Compile-Time Checked Queries

This is SQLx's signature feature. When you write:

```rust
let books = sqlx::query!("SELECT id, title, author FROM books")
.fetch_all(&pool)
.await?;
```

At compile time, SQLx:

1.  Reads your `DATABASE_URL` environment variable
2.  Connects to the database
3. Asks the database to validate the query and report information about the result columns and parameter types. The exact mechanism depends on the database backend.
4.  Checks that every column you are selecting actually exists
5.  Checks that the types match (TEXT maps to `String`, INTEGER maps to `i64`, etc.)
6.  Generates a struct with the correct field names and types

If you mistype a column name:

```rust
let books = sqlx::query!("SELECT id, title, authr FROM books")
.fetch_all(&pool)
.await?;
```

The compiler gives you an error:

```
error: no such column: authr
```

If you mismatch the number of parameters:

```rust
sqlx::query!("INSERT INTO books (title, author) VALUES (?, ?, ?)", ...)
```

The compiler tells you that you have 2 columns but 3 parameters.

### Setting Up SQLx

Add to `Cargo.toml`:

```toml
[dependencies]
sqlx = { version = "0.9", features = ["runtime-tokio", "sqlite", "macros", "migrate"] }
tokio = { version = "1", features = ["full"] }
```

`runtime-tokio` tells SQLx to use Tokio for async I/O. `sqlite` enables the SQLite driver.

Set the database URL environment variable. You need this set whenever you compile because the macros connect to the database:

```bash
export DATABASE_URL=sqlite:books.db
```

`sqlite:books.db` means "SQLite database in the file `books.db`." You can also use `sqlite::memory:` for an in-memory database (ephemeral), or `sqlite:/absolute/path/to/db.sqlite` for a specific location.

### Connection Pooling with SqlitePool

A connection pool holds multiple open connections. Tasks check out connections, use them, and return them. This gives concurrent database access without serialising everything through a mutex:

```rust
use sqlx::sqlite::SqlitePoolOptions;

let pool = SqlitePoolOptions::new()
.max_connections(5)
.connect("sqlite:books.db")
.await?;
```

Now, let me explain what we just did.

`SqlitePoolOptions::new()` creates a builder. `.max_connections(5)` sets the pool size. Five is a reasonable default for many SQLite applications. SQLite supports many concurrent readers but only one writer at a time, so increasing the pool size does not necessarily improve write throughput. `.connect("sqlite:books.db").await` opens the file and creates the initial connections. The pool is cheap to clone (it uses `Arc` internally). Pass clones to different async tasks.

### Running Migrations

Migrations are versioned SQL files that define your schema. SQLx keeps track of which migrations have already been applied so each migration is executed only once. When you run migrations, SQLx applies any that have not been applied yet. Create a `migrations` directory and add numbered SQL files:

`migrations/0001_initial.sql`:

```sql
CREATE TABLE IF NOT EXISTS books (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title TEXT NOT NULL,
author TEXT NOT NULL,
genre TEXT NOT NULL,
read INTEGER NOT NULL DEFAULT 0
);
```

`migrations/0002_add_rating.sql`:

```sql
ALTER TABLE books ADD COLUMN rating INTEGER;
```

In Rust:

```rust
sqlx::migrate!("./migrations").run(&pool).await?;
```

The `migrate!` macro embeds the SQL files into the binary at compile time. At runtime, `.run(&pool)` applies pending migrations. This is idempotent, running it multiple times is safe.

### The query! Macros

SQLx provides three macros for different return shapes.

#### `query!` - Ad-Hoc Structs

The macro generates an anonymous struct with fields matching the selected columns:

```rust
let rows = sqlx::query!("SELECT id, title, author FROM books")
.fetch_all(&pool)
.await?;

for row in rows {
println!("{}. {} by {}", row.id, row.title, row.author);
}
```

The generated struct has fields `id: i64`, `title: String`, `author: String`. You cannot name this type or store it in a struct field. Use it for quick queries where you consume the result immediately.

#### `query_as!` - Maps to Your Structs

Define a struct and derive `FromRow`:

```rust
#[derive(Debug, sqlx::FromRow)]
struct Book {
id: i64,
title: String,
author: String,
genre: String,
read: bool,
}

let books = sqlx::query_as!(Book, "SELECT id, title, author, genre, read FROM books")
.fetch_all(&pool)
.await?;
```

The macro checks that every field in `Book` has a matching column, that the types are compatible, and that no required column is missing. `read: bool` maps `0`/`1` integers automatically.

Although this example derives `sqlx::FromRow`, the `query_as!` macro does not actually require it because the macro generates the mapping at compile time. `FromRow` is mainly used with the runtime `query_as::<_, T>()` API.

#### `query_scalar!` - Single Value

For queries that return one column and one row:

```rust
let count = sqlx::query_scalar!("SELECT COUNT(*) as count FROM books")
.fetch_one(&pool)
.await?;
```

`fetch_one` expects exactly one row. If there are zero or more than one, it returns an error.

#### Parameterised Queries

All three macros support `?` placeholders with parameters:

```rust
sqlx::query!(
"INSERT INTO books (title, author, genre) VALUES (?, ?, ?)",
title,
author,
genre,
)
.execute(&pool)
.await?;
```

The macro checks at compile time that the number of `?` matches the number of parameters, and that each parameter's Rust type is compatible with the SQL column type. Parameters are bound separately, not interpolated into the SQL string, this prevents SQL injection.

## Part 3: The Project - Book Library CLI

Now that you know SQL and SQLx, let's build a Book Library CLI. Our program will:

-   Store books in a SQLite database with title, author, genre, publication year, read status, and rating
-   Run two migrations: create the `books` table, then add a `rating` column
-   Add, list, search, update, rate, and delete books
-   Search by title, author, or genre with `LIKE` patterns
-   Show genre statistics with `GROUP BY` and `COUNT`
-   Sort results by title, year, or rating with `ORDER BY`
-   Support pagination with `LIMIT` and `OFFSET`
-   Import books in bulk from JSON using transactions
-   Every query is type-safe and checked against the live database at compile time

### Project Setup

```bash
cargo new bookcli
cd bookcli
mkdir -p migrations
```

`Cargo.toml`:

```toml
[package]
name = "bookcli"
version = "0.1.0"
edition = "2021"

[dependencies]
sqlx = { version = "0.9", features = ["runtime-tokio", "sqlite", "macros", "migrate"] }
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "2"
```

Set the database URL. Create a `.env` file so the macros can find it at compile time:

```bash
echo 'DATABASE_URL=sqlite:books.db' > .env
```

You can also export it in your shell (the `.env` file is the most reliable approach since it is checked before every compile):

```bash
export DATABASE_URL=sqlite:books.db
```

### Migration Files

`migrations/0001_initial.sql`:

```sql
CREATE TABLE IF NOT EXISTS books (
id INTEGER PRIMARY KEY AUTOINCREMENT,
title TEXT NOT NULL,
author TEXT NOT NULL,
genre TEXT NOT NULL,
year INTEGER,
read INTEGER NOT NULL DEFAULT 0
);
```

`migrations/0002_add_rating.sql`:

```sql
ALTER TABLE books ADD COLUMN rating INTEGER;
```

### Bootstrap the Database

Before the macros can verify queries, the database must exist with the schema applied. Write a minimal `src/main.rs` to run migrations:

```rust
use sqlx::sqlite::SqlitePoolOptions;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let pool = SqlitePoolOptions::new()
    .max_connections(5)
    .connect("sqlite:books.db")
    .await?;

sqlx::migrate!("./migrations").run(&pool).await?;

println!("Database set up successfully.");
Ok(())
}
```

Run it once:

```
cargo run
```

Now `books.db` exists with the schema applied. Delete the temporary main and let's build the real CLI.

(Alternatively, you can use `sqlx database create && sqlx migrate run` instead of the temporary main.rs. This requires `sqlx-cli` to be installed: `cargo install sqlx-cli`.)

### The Book Struct, Error Type, and Display

```rust
use sqlx::FromRow;
use thiserror::Error;
use std::fmt;

#[derive(Debug, FromRow)]
struct Book {
id: i64,
title: String,
author: String,
genre: String,
year: Option<i64>,
read: i64,
rating: Option<i64>,
}

#[derive(Error, Debug)]
enum AppError {
#[error("Database error: {0}")]
Db(#[from] sqlx::Error),

#[error("Migration error: {0}")]
Migrate(#[from] sqlx::migrate::MigrateError),

#[error("IO error: {0}")]
Io(#[from] std::io::Error),

#[error("JSON error: {0}")]
Json(#[from] serde_json::Error),

#[error("{0}")]
Message(String),
}

impl fmt::Display for Book {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
    let status = if self.read != 0 { "[x]" } else { "[ ]" };
    let year_str = self.year.map(|y| y.to_string()).unwrap_or_else(|| "????".to_string());
    write!(f, "{} {:3}. {} by {} ({}) [{}]",
        status, self.id, self.title, self.author, self.genre, year_str)?;
    if let Some(r) = self.rating {
        write!(f, " ★{}", r)?;
    }
    Ok(())
}
}
```

Now, let me explain what we just did.

`Book` has all the columns from our schema. `year` is `Option<i64>` because some books might not have a publication year. `read` is `i64`, SQLite stores booleans as `0`/`1` integers, and the `query_as!` macro sees `INTEGER` in the schema and requires `i64`. We treat `0` as unread and any non-zero value as read. `rating` is `Option<i64>` because the second migration added it later and existing rows default to `NULL`.

Note that when you *bind* a Rust `bool` as a query parameter (e.g., in `mark_book`), SQLx's `Encode` trait converts `false` → `0` and `true` → `1` automatically. The asymmetry is that *reading* a column back must match the Rust field type, while *writing* a parameter can convert freely.

`AppError` has variants for database errors, migration errors, I/O errors, JSON parsing errors, and custom messages. The `#[from]` attributes let us use `?` on `sqlx::Error`, `MigrateError`, `std::io::Error`, and `serde_json::Error`, they auto-convert.

`Display` for `Book` uses `[x]` for read, `[ ]` for unread, and shows the year (or `????` if unknown). If a rating exists, it shows stars.

### Database Initialisation

```rust
use sqlx::sqlite::{SqlitePool, SqlitePoolOptions};

async fn init_db() -> Result<SqlitePool, AppError> {
let pool = SqlitePoolOptions::new()
    .max_connections(5)
    .connect("sqlite:books.db")
    .await?;

sqlx::migrate!("./migrations").run(&pool).await?;

Ok(pool)
}
```

### Adding a Book

```rust
async fn add_book(
pool: &SqlitePool,
title: &str,
author: &str,
genre: &str,
year: Option<i64>,
) -> Result<(), AppError> {
sqlx::query!(
    "INSERT INTO books (title, author, genre, year) VALUES (?, ?, ?, ?)",
    title,
    author,
    genre,
    year,
)
.execute(pool)
.await?;

println!("Added: \"{}\" by {}", title, author);
Ok(())
}
```

Now, let me explain what we just did.

`year: Option<i64>` is passed directly as a parameter. SQLx maps `None` to SQL `NULL` and `Some(n)` to the integer `n`. The `?` placeholder accepts either. The macro verifies that the type (`Option<i64>`) is compatible with the column type (`INTEGER`).

### Listing Books with Sorting and Pagination

```rust
async fn list_books(
pool: &SqlitePool,
unread_only: bool,
sort_by: &str,
descending: bool,
limit: Option<i64>,
offset: Option<i64>,
) -> Result<(), AppError> {
let order_col = match sort_by {
    "year" => "year",
    "rating" => "rating",
    "author" => "author",
    "genre" => "genre",
    _ => "id",
};

let direction = if descending { "DESC" } else { "ASC" };

let books = if unread_only {
    // Because the SQL is built dynamically, we use the non-macro query!
    // for the WHERE clause variation. But for compile-time checking,
    // we use the macro on the base query pattern.
    sqlx::query_as!(
        Book,
        "SELECT id, title, author, genre, year, read, rating FROM books WHERE read = 0 ORDER BY id"
    )
    .fetch_all(pool)
    .await?
} else {
    sqlx::query_as!(
        Book,
        "SELECT id, title, author, genre, year, read, rating FROM books ORDER BY id"
    )
    .fetch_all(pool)
    .await?
};

// Sorting and pagination are applied in Rust since the ORDER BY
// and LIMIT/OFFSET vary at runtime. This is fine for CLI-scale data.
let mut books = books;
books.sort_by(|a, b| {
    let cmp = match order_col {
        "year" => a.year.cmp(&b.year).then(a.title.cmp(&b.title)),
        "rating" => a.rating.cmp(&b.rating).then(a.title.cmp(&b.title)),
        "author" => a.author.cmp(&b.author).then(a.title.cmp(&b.title)),
        "genre" => a.genre.cmp(&b.genre).then(a.title.cmp(&b.title)),
        _ => a.id.cmp(&b.id),
    };
    if descending { cmp.reverse() } else { cmp }
});

let start = offset.unwrap_or(0) as usize;
let end = limit.map(|l| start + l as usize).unwrap_or(books.len());
let page: Vec<&Book> = books.iter().skip(start).take(end.saturating_sub(start)).collect();

if page.is_empty() {
    println!("No books found.");
} else {
    for book in &page {
        println!("{}", book);
    }
    println!("\n{} book(s) shown (total: {})", page.len(), books.len());
}

Ok(())
}
```

Now, let me explain what we just did.

Dynamic sorting and pagination are tricky with compile-time checked macros because `ORDER BY` and `LIMIT` vary at runtime. One approach is to use `sqlx::QueryBuilder` for fully dynamic SQL. Another approach, which we use here, is to fetch all matching rows with a compile-time checked query and then sort and paginate in Rust. For a personal book library with hundreds or thousands of books, this is perfectly fast. For millions of rows, you would use dynamic SQL with `QueryBuilder` or separate compile-time queries for each sort order.

The `.sort_by()` closure compares fields based on the `order_col` string. `Option<i64>` comparison puts `None` before `Some(n)`. For pagination, we `skip(offset)` and `take(limit)` using standard iterator methods. We show the page count and total count so the user knows where they are.

An alternative approach which demonstrates more SQL is to build the query with `sqlx::query_as` (runtime-checked, not compile-time checked) and string interpolation for the `ORDER BY` and `LIMIT` clauses. But the parameters must still be bound through placeholders, never string-interpolated:

```
// Example of dynamic ORDER BY using runtime query (no compile-time check on sort column):
let query_str = format!(
"SELECT id, title, author, genre, year, read, rating FROM books ORDER BY {} {} LIMIT ? OFFSET ?",
order_col, direction
);
// This loses compile-time checking for the ORDER BY clause.
// For a learning project, sorting in Rust is simpler and safer.
```

We will stick with sorting in Rust for clarity.

### Searching Books

```rust
async fn search_books(pool: &SqlitePool, query: &str) -> Result<(), AppError> {
let pattern = format!("%{}%", query);

let books = sqlx::query_as!(
    Book,
    "SELECT id, title, author, genre, year, read, rating FROM books
     WHERE title LIKE ? OR author LIKE ? OR genre LIKE ?
     ORDER BY id",
    pattern,
    pattern,
    pattern,
)
.fetch_all(pool)
.await?;

if books.is_empty() {
    println!("No books matching \"{}\".", query);
} else {
    for book in &books {
        println!("{}", book);
    }
    println!("\n{} book(s) found", books.len());
}

Ok(())
}
```

Now, let me explain what we just did.

`LIKE` with `%query%` does a substring search across title, author, and genre. All three columns use the same pattern. The macro checks that three `?` placeholders match three parameters. This is the same `LIKE` we practiced in Part 1.

### Searching by Author or Genre

```rust
async fn search_by_author(pool: &SqlitePool, author: &str) -> Result<(), AppError> {
let pattern = format!("%{}%", author);

let books = sqlx::query_as!(
    Book,
    "SELECT id, title, author, genre, year, read, rating FROM books WHERE author LIKE ? ORDER BY year DESC",
    pattern,
)
.fetch_all(pool)
.await?;

if books.is_empty() {
    println!("No books by author matching \"{}\".", author);
} else {
    for book in &books {
        println!("{}", book);
    }
    println!("\n{} book(s) found", books.len());
}

Ok(())
}

async fn search_by_genre(pool: &SqlitePool, genre: &str) -> Result<(), AppError> {
let pattern = format!("%{}%", genre);

let books = sqlx::query_as!(
    Book,
    "SELECT id, title, author, genre, year, read, rating FROM books WHERE genre LIKE ? ORDER BY year DESC",
    pattern,
)
.fetch_all(pool)
.await?;

if books.is_empty() {
    println!("No books in genre matching \"{}\".", genre);
} else {
    for book in &books {
        println!("{}", book);
    }
    println!("\n{} book(s) found", books.len());
}

Ok(())
}
```

The search by author and genre both sort by year descending, newest first. This is useful: when you browse an author's work, you typically want to see their latest book first.

### Getting Genre Statistics with GROUP BY

This is where we use the SQL we learned in Part 1 to build a useful feature:

```rust
async fn genre_stats(pool: &SqlitePool) -> Result<(), AppError> {
#[derive(Debug)]
struct GenreStat {
    genre: String,
    total: i64,
    read_count: i64,
    avg_rating: Option<f64>,
}

let rows = sqlx::query!(
    "SELECT
        genre,
        COUNT(*) as total,
        SUM(read) as read_count,
        AVG(CAST(rating AS REAL)) as avg_rating
     FROM books
     GROUP BY genre
     ORDER BY total DESC, genre ASC"
)
.fetch_all(pool)
.await?;

if rows.is_empty() {
    println!("No books in the library.");
    return Ok(());
}

println!("{:<20} {:>6} {:>6} {:>10}", "Genre", "Books", "Read", "Avg Rating");
println!("{}", "-".repeat(46));

for row in rows {
    let avg = row.avg_rating
        .map(|r| format!("{:.1}", r))
        .unwrap_or_else(|| "  -".to_string());
    println!(
        "{:<20} {:>6} {:>6} {:>10}",
        row.genre,
        row.total,
        row.read_count,
        avg,
    );
}

Ok(())
}
```

Now, let me explain what we just did.

This query uses `GROUP BY genre` with three aggregate functions: `COUNT(*)` for total books, `SUM(read)` for read books (summing 0s and 1s), and `AVG(CAST(rating AS REAL))` for the average rating. The `CAST(rating AS REAL)` is needed because `AVG` on an `INTEGER` column causes the sqlx macro to infer `Option<i64>` instead of `Option<f64>`, which leads to wrong formatting (no decimal places) or a runtime type mismatch crash. `ORDER BY total DESC, genre ASC` puts the most popular genres first and breaks ties alphabetically.

`query!` generates an anonymous struct. We access fields as `row.genre`, `row.total`, etc. `row.read_count` is `i64` (not optional) because sqlx infers `SUM(read)` on a `NOT NULL` column as non-nullable over a non-empty group. No `.unwrap_or(0)` needed. `row.avg_rating` is `Option<f64>`,  `AVG` can return `NULL` when there are no ratings. We format it to one decimal place or show `-` if no ratings exist.

### Showing Author Statistics

```rust
async fn author_stats(pool: &SqlitePool) -> Result<(), AppError> {
let rows = sqlx::query!(
    "SELECT
        author,
        COUNT(*) as total,
        SUM(read) as read_count,
        MIN(year) as first_published,
        MAX(year) as latest_published
     FROM books
     GROUP BY author
     HAVING total > 0
     ORDER BY total DESC, author ASC"
)
.fetch_all(pool)
.await?;

if rows.is_empty() {
    println!("No books in the library.");
    return Ok(());
}

println!("{:<20} {:>6} {:>6} {:>14} {:>14}", "Author", "Books", "Read", "First Pub", "Latest Pub");
println!("{}", "-".repeat(66));

for row in rows {
    let first = row.first_published.map(|y| y.to_string()).unwrap_or_else(|| "  ?".to_string());
    let latest = row.latest_published.map(|y| y.to_string()).unwrap_or_else(|| "  ?".to_string());
    println!(
        "{:<20} {:>6} {:>6} {:>14} {:>14}",
        row.author.as_deref().unwrap_or("?"),
        row.total,
        row.read_count,
        first,
        latest,
    );
}

Ok(())
}
```

Now, let me explain what we just did.

`GROUP BY author` with `MIN(year)` and `MAX(year)` shows the span of each author's work. This uses `HAVING total > 0` even though it is redundant (no author has 0 books because they are grouped from the books table). I include it to show the `HAVING` syntax in a real query. `MIN` and `MAX` on `year` (which is `INTEGER`) return the earliest and latest publication years. If all `year` values are `NULL`, they return `NULL`.

### Marking Books, Rating, and Deleting

```rust
async fn mark_book(pool: &SqlitePool, id: i64, read: bool) -> Result<(), AppError> {
let result = sqlx::query!(
    "UPDATE books SET read = ? WHERE id = ?",
    read,
    id,
)
.execute(pool)
.await?;

if result.rows_affected() == 0 {
    println!("No book found with ID {}.", id);
} else {
    let status = if read { "read" } else { "unread" };
    println!("Marked book {} as {}.", id, status);
}

Ok(())
}

async fn rate_book(pool: &SqlitePool, id: i64, rating: i64) -> Result<(), AppError> {
if rating < 1 || rating > 5 {
    return Err(AppError::Message("Rating must be between 1 and 5.".to_string()));
}

let result = sqlx::query!(
    "UPDATE books SET rating = ? WHERE id = ?",
    rating,
    id,
)
.execute(pool)
.await?;

if result.rows_affected() == 0 {
    println!("No book found with ID {}.", id);
} else {
    println!("Rated book {} with {} stars.", id, rating);
}

Ok(())
}

async fn delete_book(pool: &SqlitePool, id: i64) -> Result<(), AppError> {
let result = sqlx::query!("DELETE FROM books WHERE id = ?", id)
    .execute(pool)
    .await?;

if result.rows_affected() == 0 {
    println!("No book found with ID {}.", id);
} else {
    println!("Deleted book {}.", id);
}

Ok(())
}
```

`rows_affected()` tells us whether any row was actually updated or deleted. If the ID does not exist, we get `0` and print a message.

### Counting with query_scalar!

```rust
async fn show_counts(pool: &SqlitePool) -> Result<(), AppError> {
let total = sqlx::query_scalar!("SELECT COUNT(*) as count FROM books")
    .fetch_one(pool)
    .await?;

let read_count = sqlx::query_scalar!("SELECT COUNT(*) as count FROM books WHERE read = 1")
    .fetch_one(pool)
    .await?;

let has_ratings = sqlx::query_scalar!("SELECT COUNT(*) as count FROM books WHERE rating IS NOT NULL")
    .fetch_one(pool)
    .await?;

let avg_rating = sqlx::query_scalar!("SELECT AVG(CAST(rating AS REAL)) as avg_rating FROM books WHERE rating IS NOT NULL")
    .fetch_one(pool)
    .await?;

println!("Total books:     {}", total);
println!("Read:            {} ({}%)", read_count,
    if total > 0 { (read_count * 100) / total } else { 0 });
println!("Unread:          {}", total - read_count);
println!("With ratings:    {}", has_ratings);
if let Some(avg) = avg_rating {
    println!("Average rating:  {:.1} / 5", avg);
}

Ok(())
}
```

Now, let me explain what we just did.

Four separate `query_scalar!` calls. `COUNT(*)` never returns `NULL` in SQL, so the macro infers `i64` directly, not `Option<i64>`. We do **not** call `.unwrap_or(0)` because the type is already `i64`.

`AVG(CAST(rating AS REAL))` uses `CAST` because without it, sqlx infers `AVG(rating)` on an `INTEGER` column as `Option<i64>` (it reasons from the input column type rather than the SQL return type of `AVG`). At runtime, SQLite returns a `REAL` value, causing a type mismatch crash. With `CAST`, sqlx infers `Option<f64>` correctly, and we handle it with `if let Some(avg)`.

### Bulk Import with Transactions

```rust
use serde::Deserialize;

#[derive(Deserialize)]
struct BookImport {
title: String,
author: String,
genre: String,
year: Option<i64>,
}

async fn import_books(pool: &SqlitePool, file_path: &str) -> Result<(), AppError> {
let content = tokio::fs::read_to_string(file_path).await?;
let imports: Vec<BookImport> = serde_json::from_str(&content)
    .map_err(|e| AppError::Message(format!("Invalid JSON: {}", e)))?;

println!("Importing {} books...", imports.len());

let mut tx = pool.begin().await?;

for (i, book) in imports.iter().enumerate() {
    sqlx::query!(
        "INSERT INTO books (title, author, genre, year) VALUES (?, ?, ?, ?)",
        book.title,
        book.author,
        book.genre,
        book.year,
    )
    .execute(&mut *tx)
    .await
    .map_err(|e| AppError::Message(format!("Row {}: {}", i + 1, e)))?;
}

tx.commit().await?;

println!("Successfully imported {} books.", imports.len());
Ok(())
}
```

Now, let me explain what we just did.

`pool.begin().await?` starts a transaction. Inside the loop, we pass `&mut *tx` to `.execute()`,  this tells SQLx to run within the transaction. If any insert fails, the `?` returns early, the `Transaction` is dropped before commit, and SQLx automatically rolls back. `tx.commit().await?` makes all inserts permanent.

The error is wrapped with the row number for debugging. If row 42 has a typo in the JSON, the error tells you exactly which row.

### CLI Parsing

```rust
use std::env;

async fn run() -> Result<(), AppError> {
let pool = init_db().await?;

let args: Vec<String> = env::args().collect();

if args.len() < 2 {
    print_usage();
    return Ok(());
}

match args[1].as_str() {
    "add" => {
        if args.len() < 5 {
            println!("Usage: cargo run -- add <title> <author> <genre> [year]");
            return Ok(());
        }
        let year: Option<i64> = if args.len() > 5 {
            Some(args[5].parse().map_err(|_| AppError::Message("Invalid year.".to_string()))?)
        } else {
            None
        };
        add_book(&pool, &args[2], &args[3], &args[4], year).await?;
    }
    "list" => {
        let mut unread_only = false;
        let mut sort_by = "id";
        let mut descending = false;
        let mut limit: Option<i64> = None;
        let mut offset: Option<i64> = None;

        let mut i = 2;
        while i < args.len() {
            match args[i].as_str() {
                "--unread" => unread_only = true,
                "--sort" => {
                    i += 1;
                    if i < args.len() { sort_by = &args[i]; }
                }
                "--desc" => descending = true,
                "--limit" => {
                    i += 1;
                    if i < args.len() { limit = args[i].parse().ok(); }
                }
                "--offset" => {
                    i += 1;
                    if i < args.len() { offset = args[i].parse().ok(); }
                }
                _ => {}
            }
            i += 1;
        }
        list_books(&pool, unread_only, sort_by, descending, limit, offset).await?;
    }
    "search" => {
        if args.len() != 3 {
            println!("Usage: cargo run -- search <query>");
            return Ok(());
        }
        search_books(&pool, &args[2]).await?;
    }
    "search-author" => {
        if args.len() != 3 {
            println!("Usage: cargo run -- search-author <author>");
            return Ok(());
        }
        search_by_author(&pool, &args[2]).await?;
    }
    "search-genre" => {
        if args.len() != 3 {
            println!("Usage: cargo run -- search-genre <genre>");
            return Ok(());
        }
        search_by_genre(&pool, &args[2]).await?;
    }
    "read" => {
        if args.len() != 3 {
            println!("Usage: cargo run -- read <id>");
            return Ok(());
        }
        let id: i64 = args[2].parse().map_err(|_| AppError::Message("Invalid ID.".to_string()))?;
        mark_book(&pool, id, true).await?;
    }
    "unread" => {
        if args.len() != 3 {
            println!("Usage: cargo run -- unread <id>");
            return Ok(());
        }
        let id: i64 = args[2].parse().map_err(|_| AppError::Message("Invalid ID.".to_string()))?;
        mark_book(&pool, id, false).await?;
    }
    "rate" => {
        if args.len() != 4 {
            println!("Usage: cargo run -- rate <id> <1-5>");
            return Ok(());
        }
        let id: i64 = args[2].parse().map_err(|_| AppError::Message("Invalid ID.".to_string()))?;
        let rating: i64 = args[3].parse().map_err(|_| AppError::Message("Invalid rating.".to_string()))?;
        rate_book(&pool, id, rating).await?;
    }
    "delete" => {
        if args.len() != 3 {
            println!("Usage: cargo run -- delete <id>");
            return Ok(());
        }
        let id: i64 = args[2].parse().map_err(|_| AppError::Message("Invalid ID.".to_string()))?;
        delete_book(&pool, id).await?;
    }
    "import" => {
        if args.len() != 3 {
            println!("Usage: cargo run -- import <file.json>");
            return Ok(());
        }
        import_books(&pool, &args[2]).await?;
    }
    "genres" => {
        genre_stats(&pool).await?;
    }
    "authors" => {
        author_stats(&pool).await?;
    }
    "stats" => {
        show_counts(&pool).await?;
    }
    _ => {
        println!("Unknown command: {}", args[1]);
        print_usage();
    }
}

Ok(())
}

fn print_usage() {
println!("Book Library CLI");
println!();
println!("Commands:");
println!("  add <title> <author> <genre> [year]");
println!("                            Add a new book");
println!("  list [--unread] [--sort <col>] [--desc] [--limit N] [--offset N]");
println!("                            List books with optional filters");
println!("  search <query>            Search by title, author, or genre");
println!("  search-author <author>    Search by author");
println!("  search-genre <genre>      Search by genre");
println!("  read <id>                 Mark a book as read");
println!("  unread <id>               Mark a book as unread");
println!("  rate <id> <1-5>           Rate a book (1-5 stars)");
println!("  delete <id>               Delete a book");
println!("  import <file.json>        Import books from a JSON array");
println!("  genres                    Show genre statistics (GROUP BY)");
println!("  authors                   Show author statistics");
println!("  stats                     Show library summary");
}

#[tokio::main]
async fn main() {
if let Err(e) = run().await {
    eprintln!("Error: {}", e);
    std::process::exit(1);
}
}
```

Now, let me explain what we just did.

The CLI has grown from previous projects. The `list` command supports `--sort` (by `id`, `title`, `author`, `genre`, `year`, `rating`), `--desc`, `--unread`, and `--limit`/`--offset` for pagination. The `genres` and `authors` commands give us statistical views of the library using the `GROUP BY` queries we practiced in Part 1. The `stats` command gives an overview. The `add` command accepts an optional year. The `import` command reads JSON arrays with transactions.

### Running the Project

Make sure `DATABASE_URL` is set before compiling:

```
export DATABASE_URL=sqlite:books.db
```

Add some books:

```
cargo run -- add "The Rust Programming Language" "Steve Klabnik" "Programming" 2018
cargo run -- add "Project Hail Mary" "Andy Weir" "Science Fiction" 2021
cargo run -- add "The Martian" "Andy Weir" "Science Fiction" 2011
cargo run -- add "Atomic Habits" "James Clear" "Self-Help" 2018
cargo run -- add "Dune" "Frank Herbert" "Science Fiction" 1965
cargo run -- add "Neuromancer" "William Gibson" "Cyberpunk" 1984
cargo run -- add "Snow Crash" "Neal Stephenson" "Cyberpunk" 1992
cargo run -- add "The Name of the Wind" "Patrick Rothfuss" "Fantasy" 2007
cargo run -- add "A Wise Mans Fear" "Patrick Rothfuss" "Fantasy" 2011
cargo run -- add "Deep Work" "Cal Newport" "Self-Help" 2016
```

List all books:

```
cargo run -- list
```

Expected output:

```
[ ]   1. The Rust Programming Language by Steve Klabnik (Programming) [2018]
[ ]   2. Project Hail Mary by Andy Weir (Science Fiction) [2021]
[ ]   3. The Martian by Andy Weir (Science Fiction) [2011]
[ ]   4. Atomic Habits by James Clear (Self-Help) [2018]
[ ]   5. Dune by Frank Herbert (Science Fiction) [1965]
[ ]   6. Neuromancer by William Gibson (Cyberpunk) [1984]
[ ]   7. Snow Crash by Neal Stephenson (Cyberpunk) [1992]
[ ]   8. The Name of the Wind by Patrick Rothfuss (Fantasy) [2007]
[ ]   9. A Wise Mans Fear by Patrick Rothfuss (Fantasy) [2011]
[ ]  10. Deep Work by Cal Newport (Self-Help) [2016]

10 book(s) shown (total: 10)
```

List with sorting and pagination:

```
cargo run -- list --sort year --desc --limit 3
```

Expected output (3 most recent books):

```
[ ]   2. Project Hail Mary by Andy Weir (Science Fiction) [2021]
[ ]   1. The Rust Programming Language by Steve Klabnik (Programming) [2018]
[ ]   4. Atomic Habits by James Clear (Self-Help) [2018]

3 book(s) shown (total: 10)
```

Search:

```
cargo run -- search "Science Fiction"
```

Expected output:

```
[ ]   2. Project Hail Mary by Andy Weir (Science Fiction) [2021]
[ ]   3. The Martian by Andy Weir (Science Fiction) [2011]
[ ]   5. Dune by Frank Herbert (Science Fiction) [1965]

3 book(s) found
```

Mark books as read and rate them:

```
cargo run -- read 1
cargo run -- read 3
cargo run -- read 5
cargo run -- rate 1 5
cargo run -- rate 3 4
cargo run -- rate 5 5
```

View genre statistics (the GROUP BY query):

```
cargo run -- genres
```

Expected output:

```
Genre                 Books   Read Avg Rating
----------------------------------------------
Science Fiction           3      2        4.5
Cyberpunk                 2      0          -
Fantasy                   2      0          -
Self-Help                 2      0          -
Programming               1      1        5.0
```

View author statistics (MIN and MAX years):

```
cargo run -- authors
```

Expected output:

```
Author                Books   Read      First Pub     Latest Pub
------------------------------------------------------------------
Andy Weir                 2      1           2011           2021
Patrick Rothfuss          2      0           2007           2011
Cal Newport               1      0           2016           2016
Frank Herbert             1      1           1965           1965
James Clear               1      0           2018           2018
Neal Stephenson           1      0           1992           1992
Steve Klabnik             1      1           2018           2018
William Gibson            1      0           1984           1984
```

View overall stats:

```
cargo run -- stats
```

Expected output:

```
Total books:     10
Read:            3 (30%)
Unread:          7
With ratings:    3
Average rating:  4.7 / 5
```

### Bulk Import with a JSON File

Create `more_books.json`:

```
[
{ "title": "Foundation", "author": "Isaac Asimov", "genre": "Science Fiction", "year": 1951 },
{ "title": "Hyperion", "author": "Dan Simmons", "genre": "Science Fiction", "year": 1989 },
{ "title": "The Hobbit", "author": "J.R.R. Tolkien", "genre": "Fantasy", "year": 1937 }
]
```

Import:

```
cargo run -- import more_books.json
```

Expected output:

```
Importing 3 books...
Successfully imported 3 books.
```

### Pagination in Action

After importing, we have 13 books. Paginate through them:

```
cargo run -- list --sort year --limit 5 --offset 0
```

Output shows the 5 oldest books. Then:

```
cargo run -- list --sort year --limit 5 --offset 5
```

Output shows books 6–10. Then:

```
cargo run -- list --sort year --limit 5 --offset 10
```

Output shows the remaining 3.

```
13 book(s) shown (total: 13)
```

## Conclusion

You now know enough SQL to build real applications with Rust. We covered table creation, querying, filtering, joins, aggregation, transactions, indexes, and then used SQLx to integrate those concepts into a complete CLI application with compile-time checked queries.

In the next article, we'll revisit this project from a different perspective. Instead of focusing on SQL and SQLx, we'll design the database itself, starting from requirements, modeling relationships, normalizing the schema, choosing constraints and indexes, and implementing the final design using both SQL and SQLx. See you soon.

# No BS Guide to Vim Motions

> In this post, we are going to learn about vim motions

Source: https://blog.sheerluck.dev/posts/tools/no-bs-guide-to-vim-motions/ · Markdown: https://blog.sheerluck.dev/posts/tools/no-bs-guide-to-vim-motions/index.md

Hello there, in this article I’m going to explain vim motions and most of the common commands and motions that you need to know to start using it on your day to day work

# Vim Basics

## Playground

We’ll use a simple Markdown file throughout this guide. Create a file called `notes.md` and paste the following into it.

```markdown
# Vim Playground

## Todo

- Learn Vim
- Write an article
- Push to GitHub

## Languages

- Rust
- Go
- Python

Vim is a modal text editor.

Learning by doing is the fastest way to get comfortable with Vim.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
```

Everything in this guide can be tried inside this file.

# Vim Concepts

Let’s first learn a few terminologies before diving into vim motions

## Buffer

A buffer is a representation of a file in memory

## Window

A window displays a buffer. Even after you close a window the buffer can still remain in memory. Only the view disappear

## Split

A split is simply another window. Using splits you can work with multiple files at the same time

## Opening Vim

Open your terminal and run:

```
vim notes.md
```

This will open the file inside vim

You should see something like this appears in your terminal:

!Screenshot 2026-08-05 at 16.35.53.png

Now, to quit vim, you can type `:q` and press enter. Now, you should be back in your terminal view.

> If there are unsaved changes, you need to save them first by typing `:w` , this will save the changes, then you can type `:q` to quit. You can also do this thing in a single command `:wq`, this means you want to save and quit
> 

If you want to quit without saving, you can type `:q!` and press enter.

## Modes

There are different modes in vim. I’ll just mention the ones that you’ll use the most:

- Normal Mode
- Insert Mode
- Visual Mode
- Visual Line Mode

When you start vim, it starts in normal mode

### Insert Mode

You can see, after you open that markdown file, you can’t type anything. That’s because you are in normal mode. So, lets first get into **insert mode** by typing `i`

Now you can just type like any other editor. Once you are done typing, you can go back to normal mode by pressing `Esc` key.

Ok, so there are multiple ways to enter insert mode, I’ll list them down here but for the sake of simplicity, just remember to use `i` to get into insert mode. 

| i | Insert before cursor |
| --- | --- |
| a | insert after cursor |
| I | insert at beginning of line |
| A | insert at the end of line |
| o | new line below |
| O | new line above |

## Basic Navigation

To navigate, you need to be in normal mode. This is the reason, you should always get back to normal mode, once you are done editing your text or code.

Let’s start with the 4 basic directions:

| Left | h |
| --- | --- |
| Down | j |
| Up | k |
| Right | l |

These basic navigations will help us to move character by character but ofcourse these are too slow and annoying. There are some better ways to fast forward navigations. These are word motions

| w | to move to the beginning of the next word |
| --- | --- |
| b | to move back one word |
| e | to move to the end of the current word |

## Horizontal Navigation

Until now we've been moving one character at a time using `h` and `l`. That works, but it's rarely the fastest way to move around a line.

Let's look at a few commands you'll use all the time.

Suppose you have:

```rust
let username="sheerluck";
```

### Beginning and End of Line

To jump to the beginning of the line:

| 0 | beginning of line |
| --- | --- |
| ^ | first non-whitespace character |
| $ | end of line |

There are navigations to move around the file as well

| gg | go to first line |
| --- | --- |
| G | go to last line |

### Find a Character

Suppose your cursor is here:

```
let username="sheerluck";
^
```

and you want to jump to the `=`.

Instead of pressing `l` multiple times, type:

```
f=
```

`f` means **find**.

It jumps to the next occurrence of that character.

Some useful examples:

```
f"
f(
f{
f,
f:
```

### Before a Character

Sometimes you don't want to land on the character itself.

Suppose you want to stop just before `"`.

Use:

```
t"
```

`t` means **till**.

It's similar to `f`, but stops one character before.

### Searching Backwards

Both commands also work backwards.

| Command | Action |
| --- | --- |
| `Fx` | Find previous `x` |
| `Tx` | Move before previous `x` |

### Repeat Find

Suppose you used:

```
f,
```

To jump to the next comma again, press:

```
;
```

To go back to the previous one:

```
,
```

This is extremely useful when editing function arguments.

## Vertical Navigation

### Line Navigation

Move to a specific line:

```
42G
```

Go to the first line:

```
gg
```

Go to the last line:

```
G
```

### Scrolling

Sometimes you don't want to move the cursor one line at a time.

Instead, scroll by pages.

| Command | Action |
| --- | --- |
| `Ctrl-d` | Half page down |
| `Ctrl-u` | Half page up |
| `Ctrl-f` | Full page down |
| `Ctrl-b` | Full page up |

### Keep Cursor Centered

After jumping around, your cursor often ends up at the top or bottom of the screen.

Press:

```
zz
```

to center the current line.

You can also use:

| Command | Action |
| --- | --- |
| `zt` | Cursor at top |
| `zz` | Cursor in middle |
| `zb` | Cursor at bottom |

`zz` is one of those commands you'll end up pressing constantly.

## Delete, Copy and Paste

| dd | delete current line |
| --- | --- |
| yy | to copy (or yank) the current line |
| p | paste below the cursor |
| P | paste above the cursor |
| u | undo the last change |
| Ctrl-r | redo it |

## Visual Mode

In visual mode, we can select or highlight text. To get to visual mode, press `v`. Now you can move around with the navigation commands you learned earlier and you can see text will be selected/highlighted.

Again, press `Esc` to get back from visual to normal mode

Once, you have selected some text, you can:

- press `y` to copy it
- press `d` to delete it
- press `c` to replace it

It’s basically the same how you would operate in any other text editor. The only thing is you are using your keyboard instead of mouse.

## Visual Line Mode

Sometimes instead of selecting randomly you want to select entire lines. To do that, you can press `V` and now you can move up and down to select multiple lines and you can again to the same operations as visual mode.

> To indent lines, you can select them in visual line mode and then press `>` to indent and `<` to unindent them
> 

## Searching

Searching is also very common when we use any code or text editor. To search in vim, you can just stay in normal mode and type `/word` . Whatever you type after the `/` will be searched and then press enter

Now, once you do that, you can navigate around using `n` for the next match and `N` for the previous match.

You can also place the cursor on a word and press `*` to search for the next occurrence and press `#` for previous occurrence. This really helps when searching for a variable or function name

# Operators

Until now we've only been moving around the file.

Now let's actually start editing it.

Most editing commands in Vim follow the same pattern:

```
operator + motion
```

An **operator** is simply an action.

For example:

| Operator | Action |
| --- | --- |
| `d` | Delete |
| `c` | Change |
| `y` | Copy (Yank) |

The **motion** tells Vim what the action should apply to.

Let's see a few examples.

| Command | Meaning |
| --- | --- |
| `dw` | Delete a word |
| `d$` | Delete to the end of the line |
| `dd` | Delete the current line |
| `cw` | Change a word |
| `c$` | Change to the end of the line |
| `yy` | Copy the current line |
| `yw` | Copy a word |

## Delete

Suppose you have:

```rust
let username = "sheerluck";
```

To delete just the word `username`, place the cursor on it and type:

```
dw
```

To delete everything from the cursor to the end of the line:

```
d$
```

To delete the entire line:

```
dd
```

## Change

Changing is one of the commands you'll use the most.

Instead of deleting something and entering Insert mode manually, use `c`.

For example:

```
cw
```

changes the current word and immediately puts you into Insert mode.

Suppose you have:

```rust
let username = "sheerluck";
```

Place the cursor on `username`.

Type:

```
cw
```

Now type:

```
email
```

Press `Esc`.

The result becomes:

```rust
let email = "sheerluck";
```

To change everything until the end of the line:

```
C
```

## Copy (Yank)

Copy works exactly the same way.

Copy the current line:

```
yy
```

Copy the next word:

```
yw
```

Copy to the end of the line:

```
y$
```

Paste below:

```
p
```

Paste above:

```
P
```

## Counts

Most commands accept a number.

Delete five lines:

```
5dd
```

Copy three words:

```
3yw
```

Move down ten lines:

```
10j
```

Move forward five words:

```
5w
```

## Repeat the Last Change

One of Vim's best commands is:

```
.
```

It repeats your last edit.

For example, change one variable using `cw`.

Now move to another variable and press:

```
.
```

Vim performs the same edit again.

# Text Objects

Motions work great, but sometimes you don't want to move from one place to another.

Sometimes you want to edit **the thing your cursor is already inside**.

That's where text objects come in.

## Inner Word

Suppose your cursor is anywhere inside:

```rust
let username = "sheerluck";
```

Type:

```
ciw
```

This stands for:

- `c` → change
- `iw` → inner word

No matter where your cursor is inside `username`, the whole word is selected and replaced.

To delete the current word instead:

```
diw
```

To copy it:

```
yiw
```

## Inside Quotes

Suppose you have:

```rust
println!("Hello World");
```

Place the cursor anywhere inside `"Hello World"`.

Type:

```
ci"
```

Everything inside the quotes is removed and you're placed into Insert mode.

You don't have to move to the beginning or end of the string.

To delete the contents:

```
di"
```

To copy them:

```
yi"
```

## Inside Parentheses

The same idea works with brackets.

```rust
println!("Hello", name);
```

Place the cursor anywhere inside the parentheses.

Type:

```
ci(
```

Everything inside the parentheses is replaced.

Likewise:

```
di(
yi(
```

work exactly as you'd expect.

The same commands work for:

```
()
[]
{}
<>
```

## Around Objects

So far we've used **inside**.

You can also use **around**.

For example:

```
ci"
```

changes only the text inside the quotes.

```
ca"
```

changes the text **and** the quotes.

The same works for brackets.

| Command | Action |
| --- | --- |
| `ci(` | Inside parentheses |
| `ca(` | Around parentheses |
| `ci{` | Inside braces |
| `ca{` | Around braces |
| `ci[` | Inside brackets |
| `ca[` | Around brackets |

# Working with Multiple Files

You might have to open multiple files at once. Although I don’t often do this but just putting it out here incase you need it

To open another file:

```
:e main.rs
```

To open it in a vertical split:

```
:vsp main.rs
```

Or a horizontal split:

```
:sp main.rs
```

Move between splits:

| Command | Action |
| --- | --- |
| `Ctrl-w h` | Left |
| `Ctrl-w j` | Down |
| `Ctrl-w k` | Up |
| `Ctrl-w l` | Right |

Close the current split:

```
Ctrl-w q
```

Equalize all splits:

```
Ctrl-w =
```

We can wrap this guide here. I think I’ve shared enough that you can start using vim motions in your day to day writing and find more useful stuff regarding vim motions. See you soon

# Rust Playground Demo

> Interactive demo, multiple editable Rust playgrounds on one page via play.rust-lang.org.

Source: https://blog.sheerluck.dev/rust-playground-demo/ · Markdown: https://blog.sheerluck.dev/rust-playground-demo/index.md

import { RustPlayground } from "@/components/ui/rust-playground";

## 1. Hello, World!

```rust playground title="Hello World"
fn main() {
println!("Hello, world!");
}
```

Try editing `println!` to print your name and hit **Run**.

## 2. With variables and mutability

```rust playground title="Variables & Mutability" edition=2024 channel=stable mode=debug
fn main() {
let mut x = 5;
println!("x is {x}");
x = 6;
println!("x is now {x}");
}
```

## 3. Second example on same page — proves multiple embeds work

```rust playground title="Second Example — Loops"
fn main() {
for i in 0..5 {
    println!("i = {}", i);
}
}
```

## 4. Error handling — see compiler output

Try running, then fix the error and run again.

```rust playground title="Compilation Error Demo"
fn main() {
let x = 5;
println!("{}", x);
// Uncomment next line to see an error:
// let y: i32 = "oops";
}
```

```rust playground title="Fix Me — Error"
fn main() {
let y: i32 = "oops";
println!("{}", y);
}
```

## 5. Nightly + 2024 edition + release mode

```rust playground title="Nightly • 2024 • Release" channel=nightly edition=2024 mode=release
fn main() {
println!("Running on nightly, edition 2024, release mode");
}
```

## 6. Explicit component usage (import)

<RustPlayground
  code={`fn main() {
    let s = String::from("from component");
    println!("Hello {s}!");
}`}
  title="Via <RustPlayground> Component"
  edition="2024"
  channel="stable"
  mode="debug"
/>

## Static code stays static

This block is plain ````rust` without `playground` — it remains a highlighted, non-runnable code block:

```rust
fn main() {
    println!("This one is static — no Run button");
}
```

---

**Fence syntax:**

````md
```rust playground title="My Example" edition=2024 channel=stable mode=debug
fn main() { println!("hi"); }
```
````

For MDX you can also use the component:

```mdx
import { RustPlayground } from "@/components/ui/rust-playground";

```