Data Analytics Engineer – CL3.5
Role Overview: As a Data Analytics Engineer, you will actively engage in your engineering craft, taking a hands-on approach to multiple high-visibility projects. Your expertise will be pivotal in delivering solutions that delight customers and users, while also driving tangible value for Deloitte's business investments. You will leverage your extensive engineering craftmanship across multiple programming languages and modern frameworks, consistently demonstrating your strong track record in delivering high-quality, outcome-focused solutions. The ideal candidate will be a dependable team player, collaborating with cross-functional teams to design, develop, and deploy advanced software solutions.
Work you will do
This is a unique opportunity to be a part of growing data operations and enterprise controls team that works on new and existing high-end data management platforms within Deloitte. You will be responsible for development and implementation of Databricks Unity Catalog solutions to support data governance capabilities in a complex hybrid, multi-cloud environment. Responsibilities include:
Catalog and Schema Design
- Configuring and managing Unity Catalog resources: catalogs, schemas, and tables.
- Organizing data assets efficiently for security, lineage, and usability.
Data Governance & Security
- Implementing robust access controls using Unity Catalog’s attribute-based access control (ABAC) and role-based access control (RBAC).
- Auditing data access and ensuring compliance with enterprise policies.
Data Lineage and Metadata Management : Utilizing Unity Catalog to track data lineage and metadata for audit and discoverability.
Migration Experience: Migrating legacy metastore assets to Unity Catalog, including knowledge of migration best practices and potential pitfalls.
Testing and Documentation: Unit-testing data transformations, documenting data pipelines, and writing clear technical documentation.
Collaboration with Stakeholders: Work closely with business analysts, data stewards, and other stakeholders to understand data quality requirements. Collaborate with IT teams to integrate data quality solutions into existing data workflows.
Issue Resolution and Root Cause Analysis: Identify and resolve data quality issues in a timely manner. Perform root cause analysis to prevent recurring data quality problems.
Data Cataloging and Discovery: Use Databricks Unity Catalog to create and maintain a comprehensive data catalog. Facilitate data discovery and access for business users.
Key Responsibilities:
Outcome-Driven Accountability: Embrace and drive a culture of accountability for customer and business outcomes. Develop engineering solutions that solve complex problems with valuable outcomes, ensuring high-quality, lean designs and implementations.
Technical Leadership and Advocacy: Serve as the technical advocate for products, ensuring code integrity, feasibility, and alignment with business and customer goals. Lead requirement analysis, component design, development, unit testing, integrations, and support.
Engineering Craftsmanship: Maintain accountability for the integrity of code design, implementation, quality, data, and ongoing maintenance and operations. Stay hands-on, self-driven, and continuously learn new approaches, languages, and frameworks. Create technical specifications, and write high-quality, supportable, scalable code ensuring all quality KPIs are met or exceeded. Demonstrate collaborative skills to work effectively with diverse teams.
Customer-Centric Engineering: Develop lean engineering solutions through rapid, inexpensive experimentation to solve customer needs. Engage with customers and product teams before, during, and after delivery to ensure the right solution is delivered at the right time.
Incremental and Iterative Delivery: Adopt a mindset that favors action and evidence over extensive planning. Utilize a learning-forward approach to navigate complexity and uncertainty, delivering lean, supportable, and maintainable solutions.
Cross-Functional Collaboration and Integration: Work collaboratively with empowered, cross-functional teams including product management, experience, and delivery. Integrate diverse perspectives to make well-informed decisions that balance feasibility, viability, usability, and value. Foster a collaborative environment that enhances team synergy and innovation.
Advanced Technical Proficiency: Possess deep expertise in modern software engineering practices and principles, including Agile methodologies and DevSecOps to deliver daily product deployments using full automation from code check-in to production with all quality checks through SDLC lifecycle. Strive to be a role model, leveraging these techniques to optimize solutioning and product delivery. Demonstrate understanding of the full lifecycle product development, focusing on continuous improvement, and learning.
Domain Expertise: Quickly acquire domain-specific knowledge relevant to the business or product. Translate business/user needs, architectures, and data designs into technical specifications and code. Be a valuable, flexible, and dedicated team member, supportive of teammates, and focused on quality and tech debt payoff.
Effective Communication and Influence: Exhibit exceptional communication skills, capable of articulating complex technical concepts clearly and compellingly. Inspire and influence teammates and product teams through well-structured arguments and trade-offs supported by evidence. Create coherent narratives that align technical solutions with business objectives.
Engagement and Collaborative Co-Creation: Engage and collaborate with product engineering teams at all organizational levels, including customers as needed. Build and maintain constructive relationships, fostering a culture of co-creation and shared momentum towards achieving product goals. Align diverse perspectives and drive consensus to create feasible solutions.
The team: US Deloitte Technology Product Engineering has modernized software and product delivery, creating a scalable, cost-effective model that focuses on value/outcomes that leverages a progressive and responsive talent structure. As Deloitte’s primary internal development team, Product Engineering delivers innovative digital solutions to businesses, service lines, and internal operations with proven bottom-line results and outcomes. It helps power Deloitte’s success. It is the engine that drives Deloitte, serving many of the world’s largest, most respected companies. We develop and deploy cutting-edge internal and go-to-market solutions that help Deloitte operate effectively and lead in the market. Our reputation is built on a tradition of delivering with excellence.
Key Qualifications:
- Bachelor’s degree in computer science, software engineering, or a related discipline. An advanced degree (e.g., MS) is preferred but not required. Experience is the most relevant factor.
- 2-4 years of experience in Databricks Unity Catalog.
- Strong in writing complex SQL queries, procedures etc. using MS SQL Server database.
- Deep hands-on experience with Databricks notebooks, jobs, clusters, and Delta Lake.
- Familiarity with Databricks Unity Catalog concepts: managed tables, metastore, access controls, data lineage, and catalog automation.
- Experience developing data pipelines using PySpark or Spark APIs.
- Advanced -level Python programming
- Experience with metadata management, data lineage, and impact analysis.
- Strong verbal and written communication skills, with an ability to express complex business concepts in non-technical terms.
- Experience working with cross-functional teams in delivery of new products or services.
- Strong collaboration skills and self-motivated, taking ownership of deliverables and responsibilities.
- Strong systems and application delivery ability.
Good to have skills:
1. Informatica CDGC and CDMP (Marketplace).
How You will Grow: At Deloitte, our professional development plans focus on helping people at every level of their career to identify and use their strengths to do their best work every day and excel in everything they do.