Description We are looking for experienced data engineers to join our team. You will use various methods to transform raw data into useful data systems.
To succeed in this data engineering position, you should have strong analytical skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages and knowledge of learning machine methods.
If you are detail-oriented, with excellent organizational skills and want to be part of an exciting team building the best in class connected cars, then this is the perfect opportunity for you.
Responsibilities Experience in building and managing data pipelines. Experience with development and operations of data pipelines in the cloud (Preferably Azure). Experience with distributed data/computing tools: Map/Reduce, Hadoop, Hive, Spark. Deep expertise in architecting and data pipelines in cloud using cloud native technologies. Good experience in both ETL and ELT Ingestion patterns. Hands-on experience working on large volumes of data (Petabyte scale) with distributed compute frameworks. Good understanding of container platforms Kubernetes and Docker. Excellent knowledge and experience with object-oriented programming. Familiarity developing with RESTful API interfaces. Experience in markup languages such as JSON and YAML. Proficient in relational database design and development. Good knowledge on Data warehousing concepts. Working experience with agile scrum methodology. Technical Skills Experience with stream-processing systems: Kafka, Apache Storm, Spark-Streaming, Apache Flink, etc. Hands-on working knowledge in cloud data lake stores like Azure Data Lake Storage. Data pipeline orchestration with Azure Data Factory, Amazon Data Pipeline. Good knowledge on File Formats like ORC, Parquet, Delta, Avro, etc. Good experience in using SQL and No-SQL databases like MySQL, Elasticsearch, MongoDB, PostgreSQL, and Cassandra running huge volumes of data. Strong experience in networking and security measures. Proficiency with CI/CD automation, and specifically with DevOps build and release pipelines. Proficiency with Git, including branching/merging strategies, Pull Requests, and basic command line functions. Job Responsibilities Cloud Analytics, Storage, security, resiliency, and governance. Building and maintaining the data architecture for data engineering and data science projects. Extract Transform and Load data from source systems to data lake or Data Warehouse leveraging a combination of various IaaS or SaaS components. Perform compute on huge volumes of data using open-source projects like Databricks/Spark or Hadoop. Define table schema and quickly adapt with the pipeline. Working with high volume unstructured and streaming datasets. Responsible for managing NoSQL databases on cloud (AWS, Azure, etc.). Architect solutions to migrate projects from on-premises to cloud. Research, investigate and implement newer technologies to continually evolve security capabilities. Identify valuable data sources and automate collection processes. Implement adequate networking and security measures for the data pipeline. Implement monitoring solutions for the data pipeline. Support the design and implement data engineering solutions. Maintain excellent documentation for understanding and accessing data storage. Work independently as well as in teams to deliver transformative solutions to clients. Be proactive and constantly pay attention to the scalability, performance, and availability of our systems. Establish privacy/security hierarchy and regulate access. Collaborate with engineering and product development teams. Systematic problem-solving approach with strong communication skills and a sense of ownership and drive. Organization Mercedes-Benz Research and Development India Private Limited
Primary Location India-Karnataka-Bangalore
Work Locations Brigade Tech Gardens, Katha No. 119, Kundalahalli Village, K.R. Puram Hobli, Ward No. 85, Bangalore 560037
#J-18808-Ljbffr
