AI/ML/Gen AI Engineer
Ascendion
- Location
- Onsite (Hyderabad, TS,IN)
- Employment
- Full-time
- Level
- Senior Level
Posted 3 weeks ago
About the role
Ascendion is seeking a Senior Big Data Engineer to design and deliver scalable, high-quality data solutions for Housing products. The role involves building large-scale data pipelines, implementing modern architectural patterns like Medallion Architecture, and leveraging AI tools to accelerate development and ensure data quality.
Skills
Big Data Engineering
Python
SQL
PySpark
AWS Glue
Apache Airflow
Data Modeling
CI/CD
Data Pipelines
Generative AI
AWS Athena
AWS Redshift
NoSQL
Data Quality
Observability
Medallion Architecture
Full job details
AWS Data Engineer
What To Expect-
We are seeking a skilled and collaborative Big Data Engineer to join our team. As a Senior Big Data
Engineer, you will play a critical role in designing and delivering scalable, high-quality data solutions
across Housing (RentBureau and Mortgage) products. You will leverage your expertise to design and
implement large-scale data pipelines and integrated data solutions using the latest technologies,
ensuring efficient data storage, processing, and presentation. You will also build and utilize advanced
data modeling frameworks and adopt modern architectural patterns such as Medallion Architecture to
deliver well-structured, efficient, and future-ready data solutions.
This role requires expertise in big data architectures, data processing, and the ability to build modular,
reusable components that can scale.
What You'll Do:
• Design and implement robust, scalable solutions for processing huge volume of data
• Develop and maintain high-performance data pipelines using Spark, Glue, Airflow, SageMaker,
and other modern data technologies
• Build and optimize batch data processing systems
• Develop aggregate and summary tables for various engineering teams that span across multiple
product lines and geographies
• Design and develop new systems and tools to enable cross-functional teams to consume and
understand data faster
• Implement and maintain CI/CD pipelines for data applications
• Research and incorporate emerging data infrastructures, tools, and technologies
Primary Responsibilities
• Experience in Requirement Gathering, Estimating, Managing large scale Data Engineering
Projects.
• Data Modeling, Data Pipeline Design and Integration. Create robust and scalable data pipelines
and data products in a variety of domains.
• Testing: Define and manage the data load procedures to reject or discard datasets or data points
that do not meet the defined business rules and quality thresholds.
• Operations: Able to independently manage the data load operations of the solutions or products
in scope and be able to debug
• Understanding of and ability to Implement Data Engineering principles and best practices.
• Continuous Integration and Continuous Deployment (CI/CD): Implement and maintain CI/CD
pipelines for data solutions, ensuring rapid, reliable, and streamlined updates to the data
environment.
• Leverage AI on day to day tasks to bring in efficiency in implementation and quicker time to
market.
• Use Github copilot or Claude Code to generate and refactor boilerplate PySpark jobs, SQL, Airflow
DAGs, and documentation
• Define and maintain agents and skills for recurring workflows such as requirement intake, code
review, PySpark job generation
• Build and evolve data quality checks, observability signals, and operational runbooks to improve
reliability of the datasets
• Drive engineering best practices code reviews, automated tests, schema controls.
Qualifications
Preferred Qualifications:
• 5 or more years of work experience with a bachelor’s degree
• Hands-on data engineering experience building and operating production data pipelines.
• Extensive experience in Big Data ecosystem and associated technologies, (For e.g. Spark, Python,
AWS Glue, Athena, Redshift), Open table formats (e.g. Iceberg)
• Proficiency in Python, SQL, and PySpark.
• Experience with data pipeline and workflow management tools like Airflow, AWS Step Functions
etc.
• Experience with relational SQL and NoSQL databases, such as Postgres, mySQL
• Practical experience with AWS data engineering services, especially S3, EMR, and Glue Catalog.
• Experience with building processes supporting data transformation, data structures, metadata,
dependency, and workload management.
• Experience designing for data quality, resiliency, and observability, including validation checks,
monitoring, alerting, and runbooks
• Practical experience using generative AI tools such as Cursror, Claude code or similar systems to
accelerate coding, documentation, or incident analysis while applying appropriate
review controls.
• Strong communication skills and the ability to convert ambiguous business requirements into
clear technical implementation plans.
• Experience with platforms such as databricks or snowflake