Cloud Services - Advanced- Senior Java Spark Engineer
ATR International

Job Info

---

<b>Job Description:</b><br><br> We are seeking a Senior Java Spark Engineer for a very important client <br><br>Key Responsibilities:<br>1 Design & Develop spark data processing pipelines that can apply complex transformations on large volumes of data<br>2 Help teams with Performance tuning of complex spark jobs processing huge volumes of data in limited time<br>3 Develop and enhance common data processing frameworks focusing on efficiency, scalability and code reusability<br>4 Implement and optimize streaming data processing pipelines using Kafka, Flink<br>5 Provide mentorship and technical guidance to junior team members, promoting best practices in data processing & Streaming<br>6 Stay current with the latest trends and technologies in Java, Spark, streaming and big data processing, AWS services<br>7 Lead code reviews, ensuring high coding standards and practices<br><br><b>Requirement:</b><br><br>The ideal candidate will possess the following qualifications: <br>1 Bachelor's or Master's degree in computer science, engineering or related field<br>2 10+ years of relevant professional experience in software development with a focus on Java, spark, streaming and batch high volume data processing<br>3 Proven experience in performance tuning of Spark jobs in large-scale data environments<br>4 Strong background in streaming data technologies and real-time data processing using kafka, spark, flink<br>5 Experience building common frameworks/libraries, particularly for data processing<br>6 Exceptional problem solving skills and algorithmic thinking<br>7 Experience with AWS cloud platform and services (including SNS, SQS, Event bridge, lambda, glue, lake formation etc.)<br>8 Knowledge of Docker, Kubernetes and other containerization and orchestration tools<br>9 Candidate little short on above experience can still apply if they have experience in working with MLOps with specific expertise in implementing and managing offline/online/in-line feature stores using vendor products like Sagemaker, Tecton, Feast, Databricks etc.<br><br>Must have hands on experience :<br>min 10 year programing experience (Strong Java) <br>Cloud - AWS<br>Spark - performance tuning<br>Machine learning ops<br>Data Engineering<br><br>Nice to Have:<br>MLOps knowledge<br>Flink<br>#PandoPandoLogic. , Location: Jersey City, NJ - 07311

This job has expired.