Lead Data Platform Engineer – Apache Iceberg & PySpark

Our requirements

• 7+ years of experience in data engineering or related roles.
• Strong hands-on expertise in PySpark, including DataFrame APIs, performance tuning, and job optimization.
• Practical experience with Apache Iceberg, especially in optimizing file layout, partitioning, and understanding internal architecture.
• Proven ability to automate maintenance operations and operate production data workloads independently.
• Solid software engineering fundamentals with a focus on reliable, maintainable systems.
• Excellent communication skills in English and the ability to influence across geographies.
• Ownership mindset with a focus on end-to-end delivery, discipline, and initiative.

Optional

• Certifications or additional skills in Big Data, Cloud platforms, or related fields.

Your responsibilities

• Lead the design, development, and implementation of data pipelines and table management workflows using PySpark and Apache Iceberg.
• Drive performance and cost optimization for Apache Iceberg by refining partitioning strategies, file sizing, and query efficiency.
• Manage and improve Apache Iceberg maintenance operations, including snapshot management, compaction, and metadata cleanup.
• Troubleshoot platform issues by applying deep understanding of Apache Iceberg internals such as metadata layout, schemas, and schema evolution.
• Promote engineering excellence through code reviews, automation, and best practices in CI/CD and observability.
• Support operational reliability by participating in on-call duties, incident response, and root cause analysis.
• Collaborate with cross-functional teams, influence architectural decisions, and mentor less experienced engineers.
• Partner with stakeholders to ensure solutions are secure, scalable, and compliant.

About the project

This is how we organize our work

#GETREADY to meet with us!

ITDS’s Whistleblower Procedure

What we offer

• Stable and long-term cooperation with very good conditions
• Enhance your skills and develop your expertise in the financial industry
• Work on the most strategic projects available in the market
• Define your career roadmap and develop yourself in the best and fastest possible way by delivering strategic projects for different clients of ITDS over several years
• Participate in Social Events, training, and work in an international environment
• Access to attractive Medical Package
• Access to Multisport Program
• Access to Pluralsight
• Flexible hours

Benefits

• sharing the costs of sports activities
• private medical care
• flexible working time
• fruits
• integration events
• corporate gym
• saving & investment scheme
• no dress code
• coffee / tea
• drinks
• christmas gifts
• birthday celebration
• sharing the costs of a streaming platform subscription
• access to +100 projects
• access to Pluralsight

Recruitment stages

• 1.
  online interview
• 2.
  online interview

ITDS Polska Sp. z o.o.