Lead Data Engineer (Databricks)

Our requirements

• Experience as a Lead / Principal Data Architect in multinational environments.
• Deep knowledge of data modelling, integration, warehousing, and modern data platforms.
• Strong hands-on Python and complex SQL, with production Spark experience on Databricks (Structured Streaming, Auto Loader, Delta Lake, Unity Catalog) to support analytics, AI/ML, and enterprise application use cases.
• Practice working with structured and unstructured data across the full lifecycle: ingestion, transformation, enrichment, indexing, and retention, delivered through disciplined engineering practice including infrastructure-as-code, CI/CD, and automated testing of pipelines.
• Proven experience ingesting from heterogeneous operational sources including relational (Oracle, PostgreSQL) and document stores (MongoDB), using Full load + CDC or equivalent replication patterns, with practical handling of schema drift, deletes, and late or out-of-order data.
• Skills in data modelling across a layered architecture, with clear contracts between raw, curated and serving layers, supporting cross-entity queries and contextual linking, and performant data access patterns.

Optional

• Experience building unstructured document pipelines: parsing and extraction from PDF, Office and scanned formats, chunking, embedding generation, and maintaining searchable indexes at scale.

Your responsibilities

• Build, evolve, and operate data ingestion and processing capabilities for structured, semi-structured, and unstructured data, supporting the transition from early prototypes through early adoption and general release.
• Implement and maintain rich metadata and data quality practices that enable cross- record querying, traceability, and AI-ready data access across experiments, files, inventory, and workflows.
• Work with architects, AI engineers, workflow engineers, and domain experts to ensure data is usable, performant, and trustworthy for downstream GenAI and analytics use cases.
• Support engineering quality through code reviews and mentoring of junior/mid-level engineers.
• Lead architectural decisions and work with senior business and technology stakeholders.Guide and mentor data and engineering teams

About the project

This is how we organize our work

This is how we work on a project

• Continuous Deployment
• Continuous Integration
• DevOps

What we offer

• Private medical care
• Multisport card
• Friendly, informal atmosphere, and direct contact with everyone in the company
• Modern technologies

Benefits

• sharing the costs of sports activities
• private medical care
• remote work opportunities
• flexible working time
• extra social benefits
• baby layette
• school layette
• employee referral program
• charity initiatives
• Gift vouchers for kids (birthdays, Christmas, Child's Day)

Recruitment stages

• 1.
  HR call (max 15 min.)
• 2.
  Technical call with our team (+/-60 min.)
• 3.
  Meeting with our Client (30 - 60 min.)

ACAISOFT POLAND Sp. z o.o.