Job Requirements

We are looking for a Senior AI Data Analytics Engineer with strong Data Engineering expertise, analytical thinking, and AI enablement capabilities to build scalable data solutions that power analytics, dashboards, recommendations, and AI-driven use cases.

The role involves designing and evolving data products within a modern Azure + Databricks Lakehouse architecture, enabling business insights and AI solutions through curated, consumption-ready datasets. The ideal candidate will own the end-to-end data lifecycle and work closely with business, product, and engineering teams to deliver scalable and maintainable solutions.

Qualification

• BE / B.Tech / MCA / ME / M.Tech

• 8+ years of experience in Data Engineering / Analytics Engineering

Key Responsibilities

• Lead data quality and governance initiatives, including root-cause analysis, remediation of data inconsistencies, reliability improvements, and exploratory data analysis (EDA)

• Collaborate with business and analytics stakeholders to define ownership, standardized definitions, and calculation methodologies for KPIs and core business metrics

• Establish consistent sources of truth across systems and ensure the accuracy, consistency, and trustworthiness of datasets, metrics, and analytics outputs

• Design, develop, and optimize scalable data pipelines using Databricks and Azure data services

• Integrate internal and external data sources and build reusable, modular data components

• Develop curated datasets and data products for analytics, dashboards, recommendations, and AI applications

• Design batch and streaming solutions; optimize Spark workloads, Delta tables, and low-latency data processing

• Prepare and structure datasets for AI Agents / GenAI and support Azure AI Foundry integration patterns

• Implement AI-driven workflows to automate data analysis, reporting, insight generation, and identification of business risks, inefficiencies, and performance gaps

• Design semantic and metadata-driven datasets and enable downstream AI and BI consumption

• Ensure dashboards, reports, insights, and recommendations are actionable, aligned with business priorities, and support measurable outcomes

• Support Qlik / BI performance optimization and consistent consumption of governed business metrics

• Implement testing, CI/CD, version control, and engineering best practices using Azure DevOps

• Participate in agile delivery including planning, estimation, releases, and cross-functional collaboration

Required Skills

Data Engineering & Platform

• Spark 3.x (DataFrames, SQL, Batch & Structured Streaming)

• Databricks (Workflows, SQL Warehouses, DLT, Unity Catalog, Auto Loader, Pipelines)

• Azure Data Services and Lakehouse / Medallion Architecture

• Parquet / Delta, partitioning, compaction, and performance optimization

Programming & Analytics

• Strong Python and SQL (Spark SQL, TSQL, HiveQL)

• Data quality, EDA, KPI-driven analytical modeling

• Understanding of statistical concepts and data readiness for analytics/recommendation use cases

• Experience building reusable, analytics-ready, and AI-ready datasets

Enterprise Data Governance & Metric Management

• Establishing and enforcing enterprise data governance, data quality standards, and consistent sources of truth across systems

• Defining ownership, standardized definitions, and calculation methodologies for core business metrics

• Leading data quality initiatives to identify and remediate inconsistencies across data sources and pipelines, ensuring accurate, consistent, and trusted analytics outputs

Business Partnership & Outcome Accountability

• Act as the bridge between the engineering team and the analytics/product consumers.

• Partnering with business and analytics stakeholders to align data, reporting, dashboards, and AI solutions with business priorities and measurable outcomes

• Using AI-driven workflows to automate data analysis, reporting, insight generation, and identification of risks, inefficiencies, and performance gaps

• Translating analytical findings into actionable, data-driven recommendations and risk mitigation strategies

AI, BI & Delivery

• Azure AI Foundry integration and AI/Agent data preparation

• Experience supporting Qlik / Power BI / Tableau workloads

• Testing frameworks (pytest, Great Expectations, Acceptance Testing)

• CI/CD with Azure DevOps and YAML pipelines

• Agile/Scrum development practices

Good to Know

• ADLS, Managed Identity, Azure AI Foundry

• Feature engineering concepts

• Airflow / ADF / Synapse Pipelines

• Scala or Java

• Data Catalogs (Purview, Unity Catalog, Apache Atlas)

• Healthcare domain experience (preferred)

Work Experience