Astronomical research computing platform designed as a skill multiplier laboratory

The Proxmox-Astronomy-Lab organization develops enterprise-grade astronomical research computing infrastructure that creates cascading skill development across systems engineering, DevOps, security, automation, machine learning, and AI. Every infrastructure decision drives research capability, which informs better automation, which teaches deeper technical skills - creating an upward spiral of capability that makes us better researchers, engineers, and scientists.

Our work focuses on computational analysis of high-quality public astronomical datasets, positioning for upcoming surveys like Vera Rubin Observatory while building reproducible workflows that demonstrate how modern enterprise practices enable rather than hinder scientific research.

We practice radical transparency in both research and infrastructure development. Every workflow, decision, failure, and success is documented with the intent that our entire approach - from lab architecture to research pipelines - is completely reproducible. This Open Science/Open IT philosophy ensures that:

• Research methodologies are fully documented and repeatable
• Infrastructure configurations are version-controlled and automated
• Security implementations demonstrate enterprise practices in research environments
• Learning processes are captured and shared for community benefit

Our production research platform runs on a 7-node Proxmox cluster (144 cores, 700GB RAM) built from MS-A1 small form factor workstations that fit on a closet shelf while delivering enterprise-grade computational capability. The platform implements:

• Hybrid Kubernetes + VM Architecture: RKE2 orchestration with strategic static VMs for databases and persistent services
• Enterprise Security Baseline: CIS Controls implementation as both security research and operational requirement
• Secure Remote Access: Entra E5 hybrid identity with Cloudflare ZTNA for staff and external researcher collaboration
• Open Source Toolchain: GitOps automation, container orchestration, and scientific computing workflows

This architecture enables secure, scalable astronomical research while serving as a learning laboratory for modern infrastructure practices.

graph TD
    A[Management Switch<br/>10.16.207.x] --> C[Node01<br/>12900H]
    A --> D[Node02<br/>12600H]
    A --> E[Node03<br/>12600H]
    A --> F[Node04<br/>12900H]
    A --> G[Node05<br/>12600H]
    A --> H[Node06<br/>12900H]
    A --> I[Node07<br/>12600H]
    
    B[Data Switch<br/>10.25.x.x] --> C
    B --> D
    B --> E
    B --> F
    B --> G
    B --> H
    B --> I
    
    C --> J[RKE2 K8s Cluster]
    D --> J
    E --> J
    F --> J
    G --> J
    H --> J
    I --> J
    
    style A fill:#e1f5fe
    style B fill:#f3e5f5
    style J fill:#e8f5e8

We are actively running research projects on production infrastructure while continuing platform expansion:

DESI Cosmic Void Analysis - Our flagship research project focuses on analyzing the large-scale structure of the universe using DESI Data Release 1, specifically examining galaxy populations within cosmic voids. This work involves processing 30GB+ PostgreSQL datasets containing spectroscopic observations of void galaxies, implementing statistical analysis pipelines to understand how galaxy properties correlate with void environments, and developing visualization tools for three-dimensional void mapping. The project demonstrates enterprise-scale data management while contributing to our understanding of structure formation in the universe.

DESI AGN Outflow Studies - This project investigates active galactic nucleus outflows through semi-automated spectral fitting techniques combined with Cloudy photoionization modeling. We're developing automated pipelines that can identify and characterize AGN-driven outflows in massive spectroscopic datasets, using machine learning to classify spectral features and systematic modeling to understand the physical conditions of outflowing material. The work advances both astronomical understanding of galaxy evolution and demonstrates sophisticated data processing workflows on our enterprise infrastructure.

DESI Anomalous Quasar Detection - Leveraging machine learning for large-scale anomaly detection across millions of quasar spectra in the DESI survey. This project implements distributed computing frameworks using Ray clusters on our Kubernetes infrastructure to process massive datasets, develop 1D convolutional variational autoencoders for spectral analysis, and identify statistically unusual quasars that may represent new physics or rare astrophysical phenomena. The project showcases both AI/ML capabilities and the power of modern infrastructure for scientific discovery.

• CIS Baseline Implementation: Documenting the tension and solutions for running research workloads on security-hardened infrastructure
• AI-Enhanced Workflows: Spec-driven development using Claude for accelerated infrastructure automation
• Enterprise Integration: Hybrid cloud identity management and secure collaboration frameworks

• Production: 5 nodes operational, running real research workloads
• Expansion: 2 additional nodes in transit, targeting full 7-node capacity
• Documentation: Comprehensive knowledge base covering all implementations and decisions

Our multi-domain approach creates diverse opportunities for collaboration and skill development:

• Kubernetes optimization for scientific workloads
• Enterprise security implementation in research environments
• Hybrid cloud architecture and identity management
• Infrastructure as Code and GitOps automation

• Public dataset analysis and pipeline development
• Large-scale data processing and visualization
• Preparation for Vera Rubin Observatory data access
• Scientific publication and methodology development

• CIS Controls implementation and validation
• Research environment threat modeling
• Secure collaboration frameworks
• Zero Trust Network Access for research teams

• Spec-driven development methodologies
• Agentic coding and infrastructure automation
• Machine learning pipeline development
• AI-enhanced research workflows

• Open Science documentation practices
• Enterprise-grade knowledge base development
• Reproducible research methodology capture
• Technical writing and community engagement

Our organization reflects the multi-tier documentation and development approach:

1. Explore Documentation: Start with our centralized documentation for comprehensive platform understanding
2. Choose Your Domain: Select areas of interest from astronomy, infrastructure, security, or AI/automation
3. Join Discussions: Engage in repository discussions and issue tracking for active projects
4. Contribute Skills: Submit pull requests, documentation improvements, or research contributions
5. Collaborate Remotely: Request access for secure remote research collaboration

All contributions follow our Open Science principles and must align with our commitment to reproducible, well-documented research and infrastructure practices.

The Proxmox-Astronomy-Lab organization demonstrates how enterprise security practices enhance rather than impede scientific research:

• CIS Controls Baseline: Full enterprise security implementation with documented research workflow accommodations
• Zero Trust Access: Secure remote collaboration for distributed research teams
• Open Source Security: Transparent security implementations and audit capabilities
• Responsible AI: Ethical AI practices in both research automation and scientific analysis
• Data Stewardship: Proper handling of public datasets with full provenance tracking

• Documentation Hub: docs.radioastronomy.io - Comprehensive platform and project documentation
• GitHub Discussions: Technical discussions, research collaboration, and community support
• Issue Tracking: Project-specific development and research milestone tracking
• Knowledge Base: Extensive documentation of all workflows, decisions, and implementations

All projects operate under open source licenses (primarily MIT) to ensure maximum reproducibility and community benefit. Our commitment to Open Science means that not only are our findings and tools freely available, but our entire infrastructure approach is documented and reproducible.

Building the future of astronomical research through enterprise infrastructure and skill multiplication