Senior Architect, Databricks

Are you ready to make an impact? West Monroe is seeking a Senior Architect to join our thriving Technology and Experience Practice. This is an excellent opportunity to work with clients developing Big Data strategy and roadmaps with emphasis on Databricks, while gaining exposure to a variety of industries including Healthcare, Financial Services, Insurance, and Utilities. Responsibilities: Design data flow and processing pipelines for analysis using modern toolsets such as Spark on Scala, Storm, Flume, SqoopDesign data structures for ingestion and reporting, specific to use case and technologyProvide data management expertise to our clients in evaluating requirements and developing data architecture and performing platform selection and design. Data management includes appropriate structuring, stewardship of data, semantics/syntax of data attributes, coding structures, and mapping schemesCollaborate with data science and engineering teams to productionize machine learning and Generative AI workloads within modern cloud ecosystemsEstablish best practices for AI data readiness, feature engineering, model monitoring, observability, and governanceLead technical discussions around AI-enabled modernization strategies including semantic layers, vector databases, AI agents, and knowledge retrieval systemsEvaluate emerging AI technologies and recommend fit-for-purpose tooling aligned to client objectives, scalability, security, and compliance requirementsGuide and coach consultants, share best practices, perform code reviewsCoordinate activities with data source application owners to ensure integration with analytics platform and source systemsEnsure that data integrity issues are addressedLiaise and attend meetings necessary to perform duties and aid business and technical development Qualifications: 10+ years of professional experience and 3+ years of expeirence with Databricks8+ years of prior consulting experienceVery strong technical expertise in Databricks, including architecture, performance optimization, Delta Lake, Unity Catalog, workflow orchestration, security, governance, and cloud-native data engineering best practicesExperience architecting and deploying AI/ML or Generative AI solutions within enterprise environmentsHands-on experience with LLM ecosystems and frameworks such as OpenAI, Azure OpenAI, LangChain, Semantic Kernel, MLflow, or Databricks Mosaic AIKnowledge of Retrieval-Augmented Generation (RAG), vector databases, embeddings, prompt engineering, and AI orchestration patternsFamiliarity with AI governance, model lifecycle management, responsible AI principles, and enterprise security considerationsExperience integrating AI capabilities into data platforms, analytics solutions, or customer-facing applicationsKnowledge of standard methodologies, concepts, best practices, and procedures within a Big Data environmentA commitment to inclusion and diversity, and openness to new ideas and perspectivesTravel requirements up to 30 to 50%Hybrid role: 1 to 2 days a week in office

Create an account to see the full posting, access our search engine, and more.