Senior Data Engineer (Remote)We are looking for a Senior Data Engineer to join a team to work on data pipeline development.
You would work closely with small, highly collaborative Operations and Development teams to architect and deploy high-impact security solutions.
You must be able to work US time zones (UTC-8 to UTC-5/UTC-7 to UTC-4).
Job ResponsibilitiesBuild data management pipelines from heterogeneous data sources into Data Marts and other repositories for subsequent analysis and data miningCode, test, and support Python-based services, implementing ETL/ELT, data cleansing, etc.Design and use RESTful APIs, develop shippable code, documentation, and unit test new features for Cloud-based productsCollaborate with Quality, Product, and other Engineering teamsProvide code reviews, design feedback, demos, technical requirements & documentationScope projects, provide accurate estimates for reliable delivery of projects Work closely with the DevOps team on the IAC setup for BIKey Applicant RequirementsProfessional development experience using Python, including an understanding of functional programming: 5+ yearsCloud infrastructure experience in AWS, Docker: 3+ yearsWeb-scale data engineering experience using enterprise relational databases: 3+ years Experience with Apache Spark, especially SparkCore and SparkSQL: 3+ yearsCI/CD experience and DevSecOps: 2+ yearsInfrastructure As Code (Terraform) experience: 1+ yearsExperience in Apache Airflow to orchestrate Spark jobsStrong proficiency in Python for Data Science applicationsProficient with relational databases such as Oracle, MySQL, PostgreSQLExperience with AWS services such as RDS, Redshift, S3, EMRExperience with designing and building Data Marts and ETL/ELTProficient with Jira, Confluence, and GIT toolset; IDEs like PyCharm or Visual Studio CodeExperience with CI/CD solutions, message queue systems, and background task managementSkilled in Git and use it for version controlHands-on experience with Agile/ScrumExcellent written & verbal communication skills Ability to coordinate between teamsBachelor's degree or higher in Computer Science/related field or equivalent experiencePlusesDesign, implementation, and/or maintenance of CICD, Bash scriptsExperience with using AWS EMR ServerlessAWS Certified Data Analytics - SpecialtyDatabricks Certified Associate Developer for Apache SparkExperience with serverless technology such as Amazon Lambda or Google Compute EngineExperience with TeamCity and OctopusInterview ProcessWe will assist you with preparation, including mock interviews, and coaching to succeed!
The steps typically are:Prescreen with recruitersTechnical interview with the development team
