About url-shortener

url-shortener is a high-performance URL shortening service built with Rust using the Axum async web framework. It delivers fast, secure, and scalable link management with native support for deep linking tailored to mobile platforms. The architecture integrates PostgreSQL for persistent storage and Redis for high-speed caching using MessagePack serialization to optimize memory usage and reduce payload sizes by 30-50 percent compared to JSON. Security is enforced through JWT-based authentication and rate limiting, while unique short keys are generated using a hybrid Base62 encoding and random prefix-suffix strategy to prevent sequential guessing and ensure collision-free IDs. Key features include intelligent platform detection that redirects iOS and Android users to custom deep links with automatic fallback to standard URLs if deep links are unavailable. The system supports asynchronous webhook notifications to trigger external events upon link access. It utilizes xxHash for efficient duplicate detection and mi

l

Published by

lee-lou2

Visit View Profile

README.md

View on GitHub

URL Shortener

A service that shortens URLs. My first practice with gRPC & microservices in event-driven system with distributed tracing.

Architecture

Diagram

Architecture Diagram

Main service (shortener), gRPC

To shorten URL, this service stores original URL in PostgreSQL and encodes ID into base62.

To batch shorten URLs, service runs workers for fast shortening.

To unshorten URL, it tries to get original URL by code from cache (Valkey). If not in cache, it decodes base62 and queries the PostgreSQL.

It is a Kafka producer for topics shortener.shortened and shortener.unshortened.

It is a Kafka consumer for topic shortened.top_unshortened.

Caching

This service gets a top of URLs by clicks for the last time (configured in .env) from Kafka and stores these URLs in Valkey with configured TTL.

Gateway, REST

This service communicates with the shortener by gRPC.

To shorten URL, it gets base62 encoded id from shortener and constructs final URL using PUBLIC_HOST and base62. For example, base62 is 1z, PUBLIC_HOST is https://sh.some/. Final URL is https://sh.some/1z.

To unshorten URL, it queries the shortener and gets original URL by base62 in the path param in the request. Then, it redirects with 302 to the original URL.

Bot, Telegram inline mode

This service also communicates with the shortener by gRPC.

Bot only shortens the URL, unshortening process is on gateway.

Shortening process is the same as the one in gateway: walk to the shortener, get base62, connect public host.

It takes the URL in inline mode. For example, @mybot https://github.com/misshanya/url-shortener. And you will get the shortened URL.

Statistics

This service is a Kafka consumer for 2 topics: shortener.shortened and shortener.unshortened.

It gets events, logs them, stores in ClickHouse and increments Prometheus counters.

Statistics service has a background goroutine to get top of the URLs by clicks from ClickHouse for the last time. It sends this top to Kafka.

This strategy helps to cache URLs in the shortener service.

Tech stack

Go
PostgreSQL
Valkey (Cache)
ClickHouse
Kafka (UI is available on port 8088)
Prometheus
Grafana (available on port 8089)
OpenTelemetry
Jaeger (UI is available on port 16686)

How to

Prerequisites

Docker
Go 1.24+ (if you want to build without Docker)
Telegram bot token

Run

Clone repo and cd into it

git clone https://github.com/misshanya/url-shortener
cd url-shortener

Copy .env.example to the .env file and edit it with your preferences (like public url and bot token)

cp .env.example .env

Run!

docker compose up -d

Gateway's usage

Shorten - POST /shorten with the following body:

 {
   "url": "your-url-to-shorten"
 }

Batch shorten - POST /shorten/batch with the following body:

{
  "urls": [
    { "url": "your-url-to-shorten_1" },
    { "url": "your-url-to-shorten_2" },
    { "url": "your-url-to-shorten_X" }
  ]
}

Unshorten - GET /{base62}

License

This project is licensed under the MIT license. See the LICENSE file for details.

url-shortener