Azure Databricks Architect

Dice is the leading career destination for tech experts at every stage of their careers. Our client, Reliable Software Resources, is seeking the following. Apply via Dice today!

Job Role: Azure Databricks Architect

Location: Remote and Travel

Hire-Type: Long-term

Key Responsibilities

Responsibilities below apply across the team. Individual engineers own the areas aligned to their role above.

Security Platform — Pipeline Operations & Data QualitySecurity Platform — Customer EnablementSecurity Platform — New Source OnboardingSecurity Platform — Pipeline DevelopmentSOX Platform — Source Onboarding & Pipeline ManagementSOX Platform — Alert ManagementSOX Platform — Monitoring Team SupportSOX Platform — Audit Support & EvidenceSOX Platform — Application Development & Release ManagementCross-Platform Responsibilities

Experience

Required Qualifications

5+ years of data engineering experience with at least 2 years on Azure Databricks or equivalent cloud lakehouse platforms.Hands-on experience with Medallion Architecture pipelines, Delta Lake (MERGE, OPTIMIZE, VACUUM, Z-ORDER, schema enforcement), and Databricks Structured Streaming.Familiarity with SOX compliance, IT General Controls (ITGC), or regulated-environment data engineering; experience supporting audits or evidence collection.

Security Domain Knowledge

Familiarity with OCSF (Open Cybersecurity Schema Framework) or willingness to develop deep expertise rapidly.Understanding of security log source types (EDR, firewall, IAM/PAM, CASB) and SOC/SIEM/SOAR workflows; awareness of MITRE ATT&CK framework.

Preferred Qualifications

OCSF normalization implementation across multiple security log sources.Experience with Azure API Management (APIM) for data platform API governance.UEBA, threat scoring, or ML-backed security analytics (MLflow, Databricks Feature Store, Mosaic AI).SOX audit participation — auditor walkthroughs, evidence packages, IT General Controls testing.Certifications: Databricks Certified Data Engineer Associate/Professional, Azure Data Engineer Associate (DP-203), or equivalent.Graph-based correlation (Databricks GraphX); Delta Sharing.