Content:

The course will consist of the following topics and may be extended or adapted based on the audience.
The examples in the course will focus on widely used AI/ML frameworks and platform engineering practices.
For in-house courses there is a selection of technologies and deployment scenarios which can be chosen to better fit the audience.

– Basics and Theory of AI and Large Language Models:
... - History of Artificial Intelligence and Neural Networks
... - LLM architectures (Transformer, Attention Mechanism)
... - Model types (Instruct, Chat, Code, Embedding)
... - Wording: Tokens, Quantization, Weights, Layers, Context Window
... - Open Source vs. Open Weight vs. Commercial Models
... - Multimodality and other Neural Networks
– AI/ML Frameworks and Standards:
... - PyTorch and TensorFlow basics
... - ONNX and model interchange formats
... - Hugging Face Transformers and Model Hub
... - Standards: MCP (Model Context Protocol), OpenAI API compatibility
– Self-Hosted Model Inference and Deployment:
... - Ollama for quick model deployment
... - On-premises model weight storage
... - Model download and management
... - First inference tests and prompting
– Inference Engines and Performance Optimization:
... - Inference engines: vLLM, TensorRT-LLM, Triton Inference Server
... - Batching strategies (Static vs. Dynamic Batching)
... - Caching strategies (KV-Cache, Prompt Caching)
... - Parallelization (Tensor Parallelism, Pipeline Parallelism)
... - Quantization (INT8, INT4, GPTQ, AWQ, GGUF)
... - Performance metrics (Latency, Throughput, TTFT, TPOT)
– GPU Infrastructure and Resource Management:
... - GPU selection and sizing for LLM workloads
... - CUDA, ROCm and GPU driver management
... - GPU sharing and multi-tenancy
... - Kubernetes GPU scheduling and resource quotas
... - Cost optimization for GPU workloads
– LLM Gateways and API Management:
... - LiteLLM: Installation, configuration and backends
... - Model load balancing and fallback strategies
... - Rate limiting and cost tracking
... - API key management and authentication
... - Multi-model routing and A/B testing
– Guardrails and Content Filtering:
... - Concepts and necessity of guardrails
... - Content filter implementation
... - Input/output validation
... - Safety and compliance
– Container Orchestration for AI Workloads:
... - Docker images for LLM deployment
... - Kubernetes deployments for inference servers
... - Helm charts for AI applications
... - Autoscaling strategies for AI workloads
... - Service mesh for model serving
– Model Serving Patterns and Architectures:
... - REST API and gRPC for model serving
... - Synchronous vs. asynchronous inference
... - Streaming responses and server-sent events
... - Model versioning and canary deployments
... - Blue-green deployments for models
– Monitoring and Observability for AI Workloads:
... - Metrics: Inter-token latency, throughput, token/s
... - Prometheus integration for LLM metrics
... - Grafana dashboards for AI monitoring
... - OpenTelemetry for distributed tracing
... - Log aggregation for inference logs
... - Cost tracking and resource utilization
– Retrieval Augmented Generation (RAG) Infrastructure:
... - Vector databases (Pinecone, Weaviate, Qdrant, Milvus)
... - Embedding models and deployment
... - Chunking strategies and indexing
... - Hybrid search and reranking
... - RAG pipeline orchestration
– Fine-Tuning and Model Adaptation:
... - Fine-tuning concepts and use cases
... - LoRA (Low-Rank Adaptation) and QLoRA
... - Full fine-tuning vs. parameter-efficient fine-tuning
... - Training infrastructure and GPU requirements
... - Model evaluation and validation
– MLOps for LLMs:
... - Model registry and versioning (MLflow, Weights & Biases)
... - Experiment tracking and reproducibility
... - Model lifecycle management
... - CI/CD for model deployment
... - Automated model testing and validation
– Tool-Calling and Function-Calling:
... - Tool-calling concepts and implementation
... - Function-calling APIs
... - Integration of external tools and services
... - Agentic workflows (distinction from agentic coding)
– Security and Compliance for AI Workloads:
... - Model security and supply chain
... - Data privacy and GDPR compliance
... - Secret management for API keys
... - Network security for model serving
... - Audit logging and compliance reporting
– Cloud Providers and Managed Services:
... - AWS SageMaker, Bedrock and EC2 for LLMs
... - Azure OpenAI Service and Azure ML
... - Google Cloud Vertex AI and GKE
... - Hybrid and multi-cloud strategies
– Best Practices and Future Trends:
... - Emerging technologies in AI infrastructure
... - Cost optimization and sustainability
... - Performance tuning and benchmarking
... - Team organization for AI platform teams

The course combines theoretical foundations with intensive practical exercises. Participants work with real AI infrastructure projects and learn how to professionally operate LLM workloads.

Disclaimer: The actual course content may vary from the above, depending on the trainer, implementation, duration and constellation of participants.

Whether we call it training, course, workshop or seminar, we want to pick up participants at their point and equip them with the necessary practical knowledge so that they can apply the technology directly after the training and deepen it independently.