# Senior Data Engineer - (Flink/Kafka/PySpark)

**Company:** [GSSTech Group](http://jobs.workable.com/companies/cu9XaZB8i9r9X8CdwN62f7.md)
**Location:** Bengaluru, India
**Workplace:** on site

[Apply for this job](http://jobs.workable.com/view/39ec5a86-4e7a-4543-80f5-6ec73c9a49e5)

## Description

We are looking for an experienced Data Engineer with strong expertise in real-time data streaming and distributed data processing technologies. The ideal candidate should have hands-on experience in building scalable, event-driven data platforms using Apache Flink, Kafka, Java/Scala, and PySpark. The candidate will be responsible for designing, developing, and maintaining high-performance data engineering solutions within large-scale enterprise environments.

## Requirements

Key Responsibilities:

-   Design, develop, and maintain real-time streaming data pipelines and event-driven architectures.
-   Build scalable and fault-tolerant data engineering solutions using Apache Flink, Kafka, and Java/Scala.
-   Develop and optimize distributed data processing applications for enterprise-grade systems.
-   Implement best practices for performance tuning, resiliency, security, and compliance requirements.
-   Collaborate with cross-functional teams to design technical and application architectures.
-   Ensure production-grade monitoring, troubleshooting, and operational excellence of distributed services.
-   Contribute to CI/CD implementation and DevOps best practices.
-   Work closely with engineering teams to drive technical improvements and architectural decisions.

Mandatory Technical Skills:

-   Minimum 4+ years of development and design experience in:

-   Apache Flink
-   Java or Scala
-   Apache Kafka
-   PySpark
-   Real-time data streaming technologies
-   Event-driven architectures

-   Hands-on experience with:

-   Apache Flink (Beam or Spark Streaming experience is also valuable)
-   Kafka ecosystem and streaming platforms
-   JVM tuning and performance optimization
-   Distributed systems design and implementation
-   Docker and Kubernetes containerization
-   Linux OS administration and Shell scripting
-   SQL and NoSQL databases
-   CI/CD tools such as GitHub and Jenkins
-   Design patterns and their implementation
-   Production monitoring and troubleshooting of distributed services

Nice to Have Skills:

-   Redis or other caching technologies.
-   Experience with Spark Streaming or Apache Beam.
-   Experience working with banking or fintech platforms.

Data Engineering & Security Requirements:

-   Strong understanding of data security principles and controls.
-   Experience implementing secure data transfer mechanisms including:

-   CRON jobs
-   ETL processes
-   JDBC and ODBC scripts

-   Knowledge of encryption, anonymization, data integrity, and policy controls in large-scale infrastructures.
-   Understanding of data sensitivity requirements, including:

-   Protection of PII data
-   Secure logging practices
-   Secure in-memory data handling

-   Ability to identify security design gaps and recommend enhancements.
-   Experience implementing wrapper solutions for legacy or third-party components to ensure compliance requirements are met.

Infrastructure & Networking Knowledge:

-   Working knowledge of:

-   DNS
-   Proxy servers
-   ACLs
-   Networking policies
-   Troubleshooting network-related issues

Functional Requirements:

-   Experience working in Agile environments.
-   Ability to design and implement scalable technical architectures.
-   Conduct technology research and benchmarking activities.
-   Ability to influence engineering teams on technical best practices.
-   Experience working in enterprise-scale environments.
-   Banking, Financial Services, or FinTech domain experience is preferred.

Soft Skills:

-   Excellent communication and interpersonal skills.
-   Strong problem-solving and analytical abilities.
-   Self-driven and capable of working independently.
-   Ability to collaborate effectively with cross-functional teams and stakeholders.
-   Strong presentation and listening skills.
-   Positive attitude and ability to foster a collaborative team environment.
-   Passionate about engineering excellence and continuous improvement.