LLM and AI-Infused Applications with Java & Spring AI
Samples showing how to build Java applications powered by Generative AI and Large Language Models (LLMs) using Spring AI.
🛠️ Pre-Requisites
- Java 25
- Podman/Docker
💡 Use Cases
-
Chatbot Chatbot using LLMs via Ollama.
-
Question Answering Question answering with documents (RAG) using LLMs via Ollama and PGVector.
-
Semantic Search Semantic search using LLMs via Ollama and PGVector.
-
Structured Data Extraction
Structured data extraction using LLMs via Ollama. -
Text Classification Text classification using LLMs via Ollama.
🧠 Models
Chat Models
Chat completion with LLMs via different model providers:
Embedding Models
Vector transformation (embeddings) with LLMs via different model providers:
Image Models
Image generation with LLMs via different model providers:
Audio Models
Speech generation with LLMs via different model providers:
Speech transcription with LLMs via different model providers:
Moderation Models
Coming soon
📐 Patterns
Prompts, Messages, and Templates
Prompting using simple text:
Prompting using structured messages and roles:
Prompting using templates:
Structured Output
Converting LLM output to structured JSON and Java objects:
Multimodality
Including various media in prompts with LLMs:
Tool Calling
Tool calling with LLMs via different model providers:
Memory
Using chat memory to preserve context in conversations with LLMs:
Guardrails
Guardrails for input and output with LLMs via different model providers:
📥 Data Ingestion
Document Readers
Reading and vectorizing documents with LLMs via Ollama:
Document Transformers
Document transformation with LLMs via Ollama:
- Metadata
Enrich documents with keywords and summary metadata for enhanced retrieval. - Splitters
Divide documents into chunks to fit the LLM context window.
🔢 Vector Stores
Coming soon
🔄 Retrieval Augmented Generation (RAG)
Question answering with documents using different RAG flows (with Ollama and PGVector):
Sequential RAG
Branching RAG
Conditional RAG
📊 Observability
LLM Observability
LLM Observability for different model providers:
Vector Store Observability
Vector Store Observability for different vector stores:
⚙️ Model Context Protocol
Integrations with MCP Servers for providing contexts to LLMs.
📋 Evaluation
Coming soon
🤖 Agents
Coming soon
📚 References and Additional Resources
Conferences
- Introducing Spring AI by Christian Tzolov and Mark Pollack (Spring I/O 2024)
- Spring AI Is All You Need by Christian Tzolov (GOTO Amsterdam 2024)
- Concerto for Java and AI - Building Production-Ready LLM Applications by Thomas Vitale (Devoxx UK 2025)
- Modular RAG Architectures with Java and Spring AI by Thomas Vitale (Spring I/O 2025)
Videos
- Building Intelligent Applications With Spring AI by Dan Vega (JetBrains Live Stream)
- Spring AI Series by Dan Vega
- Spring AI Series by Craig Walls
- Spring AI Series by Josh Long
Demos
- Airline Customer Support (Marcus Hellberg)
- Composer Assistant (Thomas Vitale)
- Document Assistant (Marcus Hellberg)
- Flight Booking (Christian Tzolov)