# Lead Data Engineer

**Company:** [Weekday AI](http://jobs.workable.com/companies/pxG9rDgnvZm2c86JUchT1j.md)
**Location:** Pune, India
**Workplace:** on site
**Employment type:** Full-time
**Department:** Weekday's Client via platform

[Apply for this job](http://jobs.workable.com/view/58604d94-261f-4061-8a60-05764fa1fc5a)

## Description

**This role is for one of the Weekday's clients**

**Salary range: Rs 1500000 - Rs 2500000 (ie INR 15-25 LPA)**

Experience: 7+ yrs

Location: Chennai, Coimbatore, Bangalore, Pune

Jobtype: full-time

We are looking for a highly skilled Lead Data Engineer to design, build, and optimize scalable cloud-based data platforms and processing pipelines for enterprise-grade applications. This role is ideal for someone who enjoys solving large-scale data engineering challenges and has deep expertise in Google Cloud Platform (GCP), Apache Beam/Dataflow, Java, and BigQuery.

As a Lead Data Engineer, you will play a critical role in building robust batch and real-time data processing systems that enable analytics, reporting, and data-driven decision-making across the organization. You will work closely with architects, analysts, DevOps teams, and business stakeholders to develop reliable, scalable, and high-performance data solutions.

The ideal candidate is passionate about modern cloud data architectures, distributed systems, and building efficient ETL/ELT pipelines capable of handling large-scale enterprise workloads. This role requires strong technical expertise, problem-solving ability, and the capability to drive engineering best practices across data platforms.

## Requirements

### Key Responsibilities

-   Design and develop scalable ETL/ELT pipelines using Google Cloud Platform services
-   Build and maintain real-time and batch data processing pipelines using Apache Beam and Google Dataflow
-   Develop backend processing components and data transformation services using Java
-   Work extensively with BigQuery for data warehousing, analytics, querying, and performance optimization
-   Integrate data from multiple sources including APIs, relational databases, streaming systems, and cloud platforms
-   Build reliable and scalable streaming data pipelines using technologies such as Kafka and cloud-native services
-   Optimize pipeline performance, scalability, reliability, and cloud infrastructure costs
-   Ensure high standards of data quality, governance, monitoring, security, and operational excellence
-   Collaborate with cross-functional teams including Architects, Analysts, DevOps, and business stakeholders to deliver data-driven solutions
-   Troubleshoot production issues, perform root-cause analysis, and implement long-term scalable fixes
-   Implement CI/CD practices, version control workflows, and automated deployment pipelines for data engineering solutions
-   Design and maintain scalable data models, schemas, and warehouse structures
-   Participate in architecture discussions and contribute to improving overall cloud data platform capabilities
-   Drive best practices in distributed data processing, pipeline optimization, and cloud-native engineering

### What Makes You a Great Fit

-   Strong hands-on experience with Google Cloud Platform (GCP) services
-   Deep expertise in Apache Beam and Google Dataflow for large-scale data processing
-   Strong programming skills in Java with experience building backend processing systems
-   Hands-on experience with BigQuery and cloud-based data warehousing solutions
-   Experience building and maintaining batch and real-time streaming data pipelines
-   Strong understanding of SQL, data modeling, and distributed data processing concepts
-   Experience working with Kafka or similar streaming technologies
-   Familiarity with CI/CD pipelines, Git workflows, and Agile development methodologies
-   Strong analytical, troubleshooting, and debugging capabilities
-   Understanding of scalable cloud architecture, performance optimization, and reliability engineering
-   Ability to work collaboratively with cross-functional engineering and business teams
-   Experience handling enterprise-scale datasets and complex integration requirements
-   Strong communication skills with the ability to translate technical solutions into business impact
-   Passion for building modern cloud-native data platforms and solving large-scale engineering challenges