Responsibilities: Builds and maintains relevant and reliable data products that support the business needs. Develops and implements new technology solutions as needed to ensure ongoing improvement with data reliability and observability in-view. Participates in new software development engineering. Helps to define business rules that determine the quality of data, assists the product owner in writing test scripts that validate business rules, and performs detailed and rigorous testing to ensure data quality. Develops a solid understanding of the technical details of data domains, and clearly understands what business problems are being solved. Designing and developing data pipelines and ETL processes to extract, transform, and load data from various sources into AWS data storage solutions (e.g., S3, Redshift, Glue). Implementing and maintaining scalable data architectures that support efficient data storage, retrieval, and processing. Collaborating with data scientists and analysts to understand data requirements and ensure data accuracy, integrity, and availability. Building and optimizing data integration workflows to connect data from different systems and platforms. Monitoring and troubleshooting data pipelines, identifying and resolving performance issues and bottlenecks. Ensuring data security and compliance with data governance policies and regulations. Managing data infrastructure on AWS, including capacity planning, cost optimization, and resource allocation. Staying up to date with emerging data engineering technologies, trends, and best practices, and evaluating their applicability to improve data systems and processes. Documenting data engineering processes, workflows, and solutions for knowledge sharing and future reference. Ability and flexibility to coordinate and work with teams distributed across time zones, as needed. Qualifications: Bachelor's or Master's degree in Computer Science or related engineering field and deep experience with AWS infrastructure. Strong experience in data engineering, specifically with AWS backend tech stack, including but not limited to S3, Redshift, Glue, Lambda, EMR, and Athena. Proficiency in programming languages commonly used in data engineering, such as Python. Hands-on experience with big data processing frameworks, such as Apache Spark. Hands-on experience with data modeling, ETL development, and data integration techniques. Working knowledge of relational and dimensional data design and modeling in a large multi-platform data environment. Solid understanding of SQL and database concepts. Expert knowledge of quality functions like cleansing, standardization, parsing, de-duplication, mapping, hierarchy management, etc. Expert knowledge of data, master data, and metadata related standards, processes, and technology. Ability to drive continuous data management quality (i.e. timeliness, completeness, accuracy) through defined and governed principles. Ability to perform extensive data analysis (comparing multiple datasets) using a variety of tools. Demonstrated experience in data management & data governance capabilities. Familiarity with data warehousing principles and best practices. Excellent problem solver - use of data and technology to solve problems or answer complex data-related questions. Excellent communication and collaboration skills to work effectively in cross-functional teams. Preferred Requirements: Experience with JIRA and Confluence as part of project workflow and documentation tools is a plus. Experience with Agile project management methods and terminology a plus. Experience with Prometheus, Grafana. Job Types: Full-time, Temporary, Contract
Contract length: 12 months
Pay: $2,000,000.00 - $2,500,000.00 per month
Application Question(s): How many years of overall experience do you have in the data engineering industry? Are you okay with the salary range which is mentioned in JD?
#J-18808-Ljbffr