Lead Data Engineer

AgileEngine is a top-ranking provider of software solutions to Fortune 500, Global 500, and Future 50 companies.Listed on Inc. 5000 among the fastest-growing US companies, we are always open to talented software, UX, and data experts in the Americas, Europe, and Asia.If you like a challenging environment where you're working with the best and are encouraged to learn and experiment daily, there's no better place — guaranteed!:) What you will do Migrate Indeed search index builders from old framework to Apache Spark and AWS; Providing Apache Spark best practices and mentoring; Producing high-quality code; Communication with Engineering & Product management; Alignment with existing development teams.Must haves  4+ years experience in software development in preferably Big Data Projects like Hadoop eco-systems; Should have experience with Kafka, Spark; Practical experience implementing Apache Spark jobs: knowing its features on partitioning, grouping, joins, importing the data into the cluster, exporting the data from the cluster; BS in Computer Science or equivalent practical experience; Practical experience with Java (ideally 2+ years); A plus if one has experience working with AWS, specifically AWS EMR (or ability to pick it up fast), although not mandatory; Upper-intermediate English level.Nice to haves Apache Hive; Apache Airflow.The benefits of joining us Professional growth Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps.Competitive compensation We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities.A selection of exciting projects Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands.Flextime Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.