Data Engineer

Our requirements

• Working hours: 12:00 PM – 8:00 PM (CEST)
• Bachelor’s degree in computer science, Engineering, Information Systems, or related field.
• 6-8 years of experience in data engineering, software development, or related roles.
• Proficiency in programming languages is commonly used in data engineering (e.g., Python, Java, Scala, etc.).
• Strong knowledge of database systems, data modeling techniques, and SQL proficiency.
• Proficiency with ETL tools commonly used in data engineering (e.g., Databricks, Azure Data Factory).
• Experience with big data technologies and frameworks (e.g., Hadoop, Spark, Kafka, etc.).
• Familiarity with cloud platforms and services (e.g., AWS, Azure, GCP, Snowflake, etc.).
• Excellent problem-solving skills and attention to detail.

Optional

• Experience with TBM Studio and Cloudability Data Platform.

Your responsibilities

• Build a FinOps Data Platform – design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data to produce a cloud cost chargeback dataset.
• Develop robust ETL (Extract, Transform, Load) processes to integrate data from diverse sources into our data ecosystem.
• Implement data validation and quality checks to ensure accuracy and consistency.
• Design and maintain data models, schemas, and database structures to support analytical and operational use cases.
• Optimize data storage and retrieval mechanisms for performance and scalability.
• Evaluate and implement data storage solutions, including relational databases, NoSQL databases, data lakes, and cloud storage services.
• Build and maintain integrations with internal and external data sources and APIs.
• Implement APIs and web services for data access and consumption.
• Ensure compatibility and interoperability between different systems and platforms.
• Ability to build reliable, observable data workflows.
• Knowledge with cloud platform (Azure data factory, Synapse, AWS S3, BiqQuery, etc.)
• Scheduling, dependencies, retries, backfills and data validation.
• Configure and manage data infrastructure components, including databases, data warehouses, data lakes, and distributed computing frameworks.
• Monitor system performance, troubleshoot issues, and implement optimizations to enhance reliability and efficiency.
• Implement data security controls and access management policies to protect sensitive information.
• Collaborate with data analytics, and other stakeholders to understand data requirements and deliver tailored solutions.
• Document technical designs, workflows, and best practices to facilitate knowledge sharing and maintain system documentation.
• Provide technical guidance and support to team members and stakeholders as needed.

This is how we work on a project

• Continuous Deployment
• Continuous Integration
• documentation
• issue tracking tools

Client

Benefits

• private medical care

PTT CONSULTING sp. z o.o.