AI Operations Architect

AI Operations Architect & AWS Engineer, Life Sciences Company: Norstella Location: Remote, United States Date Posted: Jul 27, 2026 Employment Type: Full Time Job ID: R-2070 Description Job Description – AI Operations Architect & AWS Engineer, Life Sciences About Norstella Norstella is a premier and critical global life sciences data and AI solutions platform dedicated to improving patient access to life-saving therapies. Norstella supports pharmaceutical and biotech companies across the full drug development lifecycle — from pipeline to patient. Our mission is simple: to help our clients bring therapies to market faster and more efficiently, ultimately impacting patient lives. Norstella unites five market-leading brands (Citeline, Evaluate, MMIT, Panalgo, and The Dedham Group) that all have a shared vision of improving patient access to life-saving therapy. Norstella delivers must-have answers and insights for critical strategic, clinical, and commercial decision-making. We help our clients: • Accelerate the drug development cycle • Make data driven commercial and financial decisions • Match and recruit patients for clinical trials • Assess competition and bring the right drugs to market • Identify and address barriers to patient access • Leverage the latest AI to turn data into insights for faster and better decision making Norstella serves almost every pharmaceutical and biotech company in the world, along with regulators like the FDA, and payers. By providing critical proprietary data supporting AI-driven workflows, Norstella helps clients make decisions faster and with greater confidence. Norstella’s investments in AI are transforming how data is consumed and decisions are made, disrupting inefficient legacy workflows and helping the industry become more efficient, innovative, and responsive to patient needs. About The Position: We are seeking an LLM Operations Architect & AWS Engineer to design, deploy, and operate production-grade LLM systems on AWS infrastructure at Norstella. You will build and maintain the agentic AI pipelines and cloud architectures that power large-scale clinical data extraction from real-world clinical notes, directly accelerating RWD-driven drug development and treatment access for patients with unmet need. This role sits at the intersection of ML infrastructure, cloud engineering, and applied NLP — you will own the operational backbone that turns LLM capabilities into reliable, scalable data products. Key Responsibilities: • Design and maintain AWS cloud architectures optimized for LLM inference and fine-tuning workloads, including GPU instance management, auto-scaling, and cost optimization across spot and on-demand capacity. • Build and operate agentic AI pipelines — multi-step LLM orchestration workflows with tool use, structured output extraction, retrieval-augmented generation (RAG), and automated quality control loops. • Deploy and manage open-weight LLMs (Llama, Gemma, Qwen, and successors) in production, including model serving infrastructure, batching strategies, and latency/throughput optimization. • Implement robust evaluation and monitoring frameworks for LLM-based extraction systems: automated accuracy measurement, drift detection, prompt regression testing, and human-in-the-loop QC integration. • Develop and maintain CI/CD pipelines for prompt versioning, model updates, and configuration management across development, staging, and production environments. • Collaborate with NLP scientists, data engineers, and software developers to translate prototype extraction logic into production-hardened, recurrent processing feeds. • Ensure compliance with data security requirements (HIPAA, PHI handling) in all LLM deployment and data processing workflows, working closely with IT security and compliance teams. • Analyze and optimize computational resource utilization — GPU hours, storage, network throughput — balancing cost efficiency against processing SLAs. • Evaluate and integrate emerging LLM serving technologies, orchestration frameworks, and inference optimization techniques (quantization, speculative decoding, structured generation) to maintain operational edge. Required Qualifications • Master's in Computer Science, Data Science, or related field. • 5 years in production NLP/ML systems development and operations on AWS. • Hands-on experience deploying and serving open-weight LLMs (Llama, Gemma, Qwen families) at scale, using frameworks such as vLLM, TGI, or equivalent serving infrastructure. • Expert Python proficiency and work