K8s Cluster Documentation#

Welcome to the documentation for our Kubernetes (K3s) cluster setup. This documentation covers both the infrastructure setup and the services running on our clusters.

Overview#

This repository contains all the configuration files for deploying our research team’s computational environment across our campus-based workstations. We use a Kubernetes-based approach (specifically K3s) that provides containerized software abstractions along with hardware orchestration and resource management capabilities.

Quick Links#

Infrastructure Setup#

• K3s Installation & Configuration
• NVIDIA GPU Support
• Storage with OpenEBS
• Certificate Manager
• External DNS

Services#

• JupyterHub
• PostgreSQL
• MinIO
• GitHub Actions Runners
• vLLM

Administration#

• User Access Management
• Secrets Management
• Tips & Tricks

Getting Started#

If you’re new to this cluster:

1. For Users: Start with the User Access Management guide to get your credentials
2. For Administrators: Begin with K3s Installation to understand the base setup
3. For Service Deployment: Check the specific service documentation in the Services section

Architecture#

Our cluster is built on:

• K3s: Lightweight Kubernetes distribution
• Traefik: Built-in ingress controller
• Cert-Manager: Automatic SSL/TLS certificate management
• External-DNS: Automatic DNS record management
• OpenEBS ZFS: Persistent storage with disk quotas
• NVIDIA Device Plugin: GPU resource management with time-slicing

Support#

For issues or questions:

• Check the Tips & Tricks section
• Review the relevant service documentation
• Consult the GitHub repository