Principal Solutions Architect

Principal Solutions Architect

About the Role

Rafay seeks a Principal Solutions Architect to enable enterprise customers/Neo clouds in deploying, operating, and scaling AI/ML workloads on our GPU Platform-as-a-Service offering. This is a hybrid technical leadership and people-management role: alongside hands-on, customer-facing architecture work, this person will build, lead, and grow a team of Solutions Architects, collaborating with platform engineering, MLOps, data science, and infrastructure teams to architect production-ready AI infrastructure solutions built on Kubernetes and GPU-accelerated environments.

Key Responsibilities

Team Leadership & People Management

Recruit, hire, and onboard Solutions Architects as the team scalesDirectly manage a team of Solutions Architects, including workload allocation, coaching, and day-to-day supportSet individual and team goals; conduct regular 1:1s and performance reviewsOwn career development planning for direct reports, including skills growth, promotion readiness, and succession planningFoster an inclusive, high-performing team culture aligned with Rafay's valuesManage team capacity, prioritization, and staffing against customer and project demandPartner with sales, engineering, and executive leadership on hiring plans and team structureMentor and upskill both direct reports and junior team members across the broader organizationTechnical & Customer-Facing Responsibilities

Design comprehensive AI/ML platform architectures covering inference, training, and data pipelinesDevelop reference architectures for GPU cluster deployment and LLM serving infrastructureEvaluate inference serving frameworks including vLLM, TGI, and TritonAdvise on GPU fabric topology options for distributed training scenariosDesign observability strategies using DCGM, OpenTelemetry, and eBPFTranslate infrastructure requirements into actionable platform designsDeliver technical presentations, workshops, and proof-of-concept engagementsServe as trusted advisor on AI infrastructure strategy, cost optimization, and scalingPartner with customer stakeholders to understand workload requirementsArchitect networking, identity management, observability, and security integrationsMonitor and troubleshoot production environments for GPU utilization and cluster healthLead root cause analysis for complex customer issuesDocument reference architectures and implementation best practicesRequired Qualifications

10+ years in infrastructure, platform, or solutions engineering roles3+ years focused on AI/ML infrastructure or MLOps2+ years of direct people-management experience, including hiring, performance management, and career development of technical staffDemonstrated ability to lead and grow a technical team while remaining hands-on with customers and architectureDeep Kubernetes expertise including cluster lifecycle and RBACHands-on experience with NVIDIA GPU infrastructure (H100/H200/B200 preferred)Proficiency with distributed training concepts (NCCL, tensor parallelism)Experience with LLM inference serving and optimizationFamiliarity with GPU Operator, MIG, SR-IOV, and network fabricsStrong scripting and automation skills (Python, Bash, Go preferred)Ability to communicate complex technical concepts to diverse audiences, including executive stakeholdersExperience with AWS, Azure, or GCP platformsFamiliarity with monitoring tools like Prometheus, Grafana, and OpenTelemetryUnderstanding of GPU-based workloads and model servingProven troubleshooting capabilities for infrastructure issuesExcellent communication, coaching, and customer-facing skillsPreferred Qualifications

Experience building a Solutions Architecture or technical pre-sales team from the ground upFormal people-management training or leadership certificationEnterprise customer support experience in cloud-native environmentsFamiliarity with PyTorch and TensorFlow frameworksExperience with Run:AI and SlurmGPU scheduling and autoscaling expertiseMulti-tenant Kubernetes environment knowledgeMLOps platform experienceTechnical workshop leadership experienceRelevant certifications (CKA, CKAD, AWS/Azure/GCP Solutions Architect)Understanding of multi-tenant GPU isolation technologiesWhy Join Rafay?

Rafay is at the forefront of GPU PaaS technologies and Kubernetes and we offer unique opportunities to join a winning team working on foundational technology for cloud and AI/ML services and enterprises. We work in a collaborative environment that rewards creative thinking and provides opportunities to advance professional careers in advanced technology development. On top of this we offer a fun and dynamic work environment, a competitive salary, robust benefits and attractive stock options. As the first of our kind, we are truly in a class of our own.