We are seeking a talented Data Engineer to join our team.
You will play a pivotal role in understanding business and technical challenges, developing and deploying robust data pipelines, and ensuring adherence to industry and internal best practices.
This role requires a combination of technical expertise, communication skills, and a proactive mindset to drive data engineering excellence.
Tasks Projects Understanding and Communication Collaborate with users to understand challenges and align solutions with their needs.
Ensure clarity in architectural designs provided by the Data Architect.
Maintain active communication with peers, Data Architects, and Project Managers regarding progress and solutions.
Development Define and communicate interface contracts for new or updated systems.
Build data pipelines using Databricks and ensure they adhere to best practices.
Deploy necessary infrastructure using Terraform .
Conduct peer reviews and seek feedback for code improvements.
Testing Define test criteria in collaboration with Project Managers.
Perform rigorous testing and document results comprehensively.
Deployments Present developed solutions during deployment reviews.
Monitor and address issues during post-deployment periods.
Ensure compliance with deployment, logging, and monitoring standards.
Requirements 3+ Years of proficiency in PySpark and Spark SQL for data processing.
3+ Years of hands-on experience with Databricks , including Unit Catalog and Delta Live Tables (DLT) .
Knowledge of Azure Data Lake Storage and data lifecycle management.
Familiarity with orchestration tools (e.g., Apache Airflow ) for ETL/ELT pipelines.
Practical knowledge of data security and privacy in cloud environments.
At least 1 year of experience with Terraform , adhering to GitOps best practices.
The role is primarily remote, with onboarding and occasional office visits in Leiria (approximately once a month).
Preferred Skills Experience with Databricks Asset Bundles.
Familiarity with Kubernetes , Apache Kafka , and Vault .
Personal Traits Strong adaptability to various contexts and stakeholders.
Proactive ownership of deliverables.
A continuous improvement mindset.
Clear communication and teamwork skills.
Analytical problem-solving and stress management abilities.
Benefits Opportunity to work on global projects and grow professionally.
Inclusive, collaborative, and supportive work environment.
Regular team events, celebrations and team building actions.
Follow-ups tailored to your profile and career goals.
Flex Benefits - You can chose what benefits are more suitable for you.
Support for certifications and advanced learning programs to enhance your expertise.
Boost IT is a Portuguese technology consultancy company, we are integrated into one of the most entrepreneurial groups in Portugal, with investment in more than 30 companies.
We want to be known for being the most dynamic, energetic and reliable company to operate in the market and, for that, we want to count on you.
If you're passionate about technology and want to work on the most relevant technology projects, then this ad could be for you!
Boost IT.
Doing IT.
Better