Kafka Version Compare

Taking inspiration from Umbraco compare web site.

Overview

This ASP.NET Core application provides version comparison functionality for:

1. Apache Kafka - Compares release notes between different Kafka versions using the Apache Kafka releases page
2. Confluent Platform - Compares release notes between different Confluent Platform versions using a deterministic HTML parser that produces a structured component-by-component comparison table

Features

• Apache Kafka Comparison: Parses Kafka release pages to show differences between versions including bugs, improvements, new features, and tasks
• Confluent Platform Comparison: Deterministic HTML parser extracts structured component-level changes from each CP release page and generates a Markdown comparison table directly — no AI or AWS required
• Streaming Output: Results are streamed to the browser in real time via Server-Sent Events
• Smart Caching:
  • Memory cache for fast in-session access
  • File-based cache for persistence across application restarts
  • Release page caching to avoid repeated web crawling
• Deprecation Warnings: Automatically highlights deprecation notices in a dedicated summary section below the comparison table

How the Confluent Platform diff engine works

The CP comparison uses a deterministic parser — no AI is involved:

1. The application crawls each CP release notes page from the Confluent docs site
2. CpReleaseParser walks the HTML and extracts a Component list — one entry per h2/h3 section (e.g. "Kafka brokers / Confluent Server")
3. For each component, change items are extracted from <p> and <ul><li> elements and rendered as Markdown, preserving links and bold text
4. DeterministicDiffService takes the component lists for all selected versions and builds a Markdown comparison table, with one row per component and one column per version
5. Deprecation notices are collected into a separate summary section at the end of the table

Prerequisites

• .NET 9.0 SDK
• Docker (optional, for containerized deployment)

Running the Application

Using Docker

1. Copy and populate the .env file in the KafkaVersionCompare/ directory:

cp KafkaVersionCompare/.env.example KafkaVersionCompare/.env

Edit .env and fill in the required values:

Variable	Description
Admin__Password	Password for the /Admin page
AWS_ACCESS_KEY_ID	AWS access key (optional — only needed if AWS features are used)
AWS_SECRET_ACCESS_KEY	AWS secret key
AWS_SESSION_TOKEN	AWS session token (if using temporary credentials)
AWS__Region	AWS region (default: eu-west-1)
OpenTelemetry__OtlpExporter	Set to false if no OTLP collector is running

1. Build and start:

cd KafkaVersionCompare
docker compose up --build

Or to run without compose:

docker build -t kafka-version-compare . && \
docker run --env-file .env -e ASPNETCORE_ENVIRONMENT=Development -p 80:80 kafka-version-compare

Open a browser at: http://localhost

Using .NET CLI

cd KafkaVersionCompare
dotnet run

Open a browser at: https://localhost:7252 or http://localhost:5252

Caching

The application uses a multi-layer caching strategy:

Cache Type	Location	Expiration	Purpose
Memory Cache	In-process	Session-based	Fast access during runtime
File Cache (Comparisons)	/tmp/KafkaVersionCompare/cache/	90 days	Persist comparison results
File Cache (Releases)	/tmp/KafkaVersionCompare/release-pages/	1 day	Persist crawled release data

To clear all caches:

rm -rf /tmp/KafkaVersionCompare

Screenshots

License

See LICENSE file for details.