Beyond the Model: Securing AI from HPC to Applications¶
Matthias Hellmann (ITCC)
Abstract¶
Securely operating AI services requires more than protecting the models themselves. Security must extend across the entire stack, from HPC infrastructure and model inference to Kubernetes-hosted applications and user access.
Using the University of Cologne’s RAMSES infrastructure as a practical example, this talk presents an architecture combining Slurm, Kubernetes, and open-source AI technologies. It highlights key security mechanisms, including authentication and authorization, workload isolation, and encryption of data at rest, in transit, and in use.
The talk demonstrates how these technologies enable the secure provision of AI services, including chatbots and RAG applications, even for sensitive research data.