Big Data Architect Contractor

Position Summary

Contractor

Remote

Work you'll do

We are seeking a Big Data Architect Contractor to design, build, and optimize large-scale data solutions on Azure and Databricks. The role will focus on developing robust ETL/ELT pipelines, enabling high-performance data processing, and supporting end-to-end architecture across development, UAT, and production environments. The ideal candidate will work closely with client stakeholders and engineering teams to deliver scalable, secure, and cost-efficient data platforms.

Roles and Responsibilities

Design, develop, and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and DatabricksBuild and tune PySpark and Spark SQL notebooks for large-scale data transformationSupport end-to-end data solutions across dev, UAT, and prod environments using Unity CatalogParticipate in technical design discussions with client architects and cross-functional counterpartsCollaborate with teams on data contracts and schema agreementsSupport the design and optimization of high-volume data pipelinesDefine and enforce engineering standards, including naming conventions, partitioning strategies, cluster configurations, and Spark tuningDrive performance optimization through AQE tuning, liquid clustering, broadcast joins, and shuffle partition managementSupport Databricks cluster policies, autoscaling configurations, and cost optimization strategiesPerform root cause analysis on production incidents and implement permanent fixesSupport junior and mid-level engineers through code reviews and pair programmingEvaluate emerging technologies and recommend adoption where relevant, such as DABs, DLT, Auto Loader, Serverless Compute, and Event HubsKey Skills

Databricks - Advanced (6-9 years of experience)Strong hands-on experience with Azure Data FactoryAdvanced proficiency in Python, PySpark, and Spark SQLStrong background in big data architecture and scalable data platform designExperience with Unity Catalog and multi-environment deliveryKnowledge of Spark performance tuning and optimization techniquesExperience with cluster configuration, autoscaling, and cost managementStrong analytical and troubleshooting skills for production supportAbility to collaborate with technical teams and client stakeholdersExperience supporting engineering standards and best practicesThe expected pay range for this contract assignment is $65 - $70 per hour will vary based on skills, experience, and location and will be determined by the third-party whose employees provide services to Deloitte.

Candidates interested in applying for this opportunity must be geographically based in the United States and must be legally authorized to work in the United States without the need for employer sponsorship.

We do not accept agency resumes and are not responsible for any fees related to unsolicited resumes.

Deloitte is not the employer for this role.

#LI-AR2

#Remote

Expected Work Schedule

Approximate hours per week

About Deloitte

Our inclusive culture empowers our people to be who they are, contribute their unique perspectives, and make a difference individually and collectively. It makes Deloitte one of the most rewarding places to work.

Skills:
Acceptance Testing, Accounts Receivable, Analysis Skills, Apache Spark, Autoscaling, Best Practices, Big Data, Broadcasting, Cisco Unity, Code Reviews, Cost Control, Cross-Functional, Data Management, Data Processing, DataArchitect Data Modeling Tool, Database Extract Transform and Load (ETL), Emerging Technology, Hubs, Identify Issues, Microsoft Windows Azure, Performance Tuning/Optimization, Process Improvement, Production Support, Production Systems, Python Programming/Scripting Language, Root Cause Analysis, SQL (Structured Query Language), System Architecture, Team Player, Technical Support, Technical/Engineering Design

About the Company:
Deloitte Touche Tohmatsu Ltd