Vice President, AI / Machine Learning Software Engineer

Role Overview 

We are seeking a senior‑level engineer to design, build, and operate production‑grade GenAI and Retrieval‑Augmented Generation (RAG) platforms at scale. This role focuses on industrializing LLM‑based systems with strong guardrails, observability, evaluation frameworks, and operational rigor, ensuring reliability, safety, and cost efficiency across the full AI lifecycle. This role is located in Jersey City, NJ. 

Key Responsibilities 

• Design and build production‑ready RAG pipelines, including retrieval, ranking, prompt orchestration, and response generation, with comprehensive guardrails, tracing, and observability.
• Implement offline and online evaluation frameworks for prompts, models, and datasets, including quality, safety, latency, and cost metrics.
• Own end‑to‑end lifecycle management for GenAI systems, covering prompt versions, model versions, datasets, and configurations.
• Establish and maintain CI/CD pipelines for prompts, models, and data, enabling safe, repeatable, and auditable releases.
• Implement cost and performance monitoring, including token usage, inference latency, throughput, and spend optimization.
• Build and enforce safety mechanisms, such as content filtering, policy enforcement, red‑teaming feedback loops, and abuse detection.
• Define and operationalize incident management workflows, including alerting, triage, rollback mechanisms, and post‑incident analysis.
• Partner closely with product, platform, and governance teams to ensure GenAI solutions meet enterprise reliability, security, and compliance standards.
• Mentor engineers and influence best practices for building scalable, trustworthy AI systems.

What Success Looks Like

• GenAI systems that are observable, measurable, and resilient, not “black boxes.”
• Safe and cost‑efficient RAG pipelines running reliably in production.
• Fast iteration cycles with strong controls, enabling teams to ship GenAI features with confidence.

Required Qualifications

• Advanced degree in STEM engineering degree, or equivalent work experience with experience preferred in related fields. 7-9 years of related experience required; experience in the securities or financial services industry is a plus
• Strong experience building and operating production ML or GenAI systems in enterprise environments.
• Deep hands‑on expertise with LLM orchestration frameworks, such as LangChain and/or LlamaIndex.
• Experience with model registries and experiment tracking, such as MLflow or equivalent.
• Solid understanding of Kubernetes‑based deployments and cloud‑native architectures.
• Familiarity with feature stores, data pipelines, and retriever/index lifecycle management.
• Proven experience implementing telemetry, logging, metrics, and distributed tracing for ML/AI workloads.
• Strong knowledge of CI/CD practices for ML, GenAI, and data‑driven systems.

Preferred Qualifications

• Experience operating LLM systems at scale, including multi‑model or multi‑provider strategies.
• Exposure to AI safety, governance, and compliance frameworks in regulated environments.
• Background in SRE, platform engineering, or MLOps, with a reliability‑first mindset.
• Ability to translate ambiguous GenAI use cases into robust, production‑grade architectures.

BNY Mellon is an Equal Employment Opportunity/Affirmative Action Employer. Minorities/Females/Individuals with Disabilities/Protected Veterans. Our ambition is to build the best global team-one that is representative and inclusive of the diverse talent, clients and communities we work with and serve-and to empower our team to do their best work. We support wellbeing and a balanced life, and offer a range of family-friendly, inclusive employment policies and employee forums.

BNY assesses market data to ensure a competitive compensation package for our employees. The base salary for this position is expected to be between $120,000 and $210,000 per year at the commencement of employment. However, base salary if hired will be determined on an individualized basis, including as to experience and market location, and is only part of the BNY total compensation package, which, depending on the position, may also include commission earnings, discretionary bonuses, short and long-term incentive packages, and Company-sponsored benefit programs. 
This position is at-will and the Company reserves the right to modify base salary (as well as any other discretionary payment or compensation) at any time, including for reasons related to individual performance, change in geographic location, Company or individual department/team performance, and market factors.