Data Engineer

Our requirements

• Strong hands-on experience with Azure Databricks: Spark, Delta Lake
• Build and maintain multi-layer pipelines: Landing → Raw → Enriched → Curated
• Schema enforcement and schema-drift handling
• Data quality checks at ingestion
• Orchestration via Databricks Jobs / Workflows: dependencies, retries, alerts
• Batch and API-based data ingestion
• Strong hands-on experience with dbt
• Monitor job failures, data freshness, and pipeline health
• SLA-based data delivery
• Alerts, logging, and retry strategies
• Access control models
• Audit trails, lineage, and traceability
• Logging and evidence generation
• Segregation of duties
• Data contracts
• Pipeline documentation
• Controlled environments (Dev/Test/Prod)
• Traceability to requirements

Optional

• CI/CD & DevOps for Data — Git-based development (branching), CI/CD pipelines for data workflows, promoting code across environments, integrating testing into deployment pipelines
• AI/MLOps — MLOps / AI pipelines on Databricks
• Exposure to regulatory domains (GxP, clinical trials, pharmacovigilance)
• Strong SQL engineering / modular, reusable dbt models, tests, and documentation at Analytics Engineer depthDimensional modeling: star schema, OBT, curated marts
• Tabular Editor (metadata-as-code concepts)
• Power BI dataset structure, performance, and KPI/semantic consistency
• Experience in the pharmaceutical or biotechnology industry

Your responsibilities

• Build and maintain multi-layer pipelines (Landing → Raw → Enriched → Curated) on Azure Databricks (Spark, Delta Lake)
• Orchestrate pipelines via Databricks Jobs / Workflows: dependencies, retries, alerting
• Implement batch and API-based data ingestion from multiple source systems
• Apply schema enforcement and handle schema drift; run data quality checks at ingestion
• Work hands-on with dbt for pipeline observability and data governance tasks
• Monitor job failures, data freshness, and pipeline health against SLA-based delivery targets
• Set up alerts, logging, and retry strategies for production pipelines
• Work within a controlled Dev/Test/Prod environment: access control, audit trails, pipeline documentation, traceability to requirements
• Contribute to CI/CD and MLOps/AI pipelines on Databricks where the project needs it

Benefits

• sharing the costs of sports activities
• private medical care
• sharing the costs of foreign language classes
• fruits
• integration events
• corporate gym
• coffee / tea
• drinks
• birthday celebration
• employee referral program

Recruitment stages

• 1.
  Rozmowa telefoniczna – wstępna rozmowa z Rekruterem
• 2.
  Rozmowa kwalifikacyjna – spotkanie z przedstawicielem firmy

Integral Solutions