Data Engineer (Cloud Migration & Big Data), Hybrid

Description

Job Summary: We are seeking Data Engineers with a solid technical foundation and an agile mindset to drive the transformation of Hadoop infrastructures to cloud environments in the banking sector. Key Highlights: 1. Transformation of Hadoop infrastructures to cutting-edge cloud environments 2. Participation in a large-scale project in the banking sector 3. Focus on continuous improvement and collaboration within cross-functional teams Data Engineer (Cloud Migration \& Big Data) We are looking for Data Engineers with strong technical expertise and an agile mindset to join a large-scale project in the banking sector. You will participate in transforming Hadoop infrastructures into leading-edge cloud environments. **Required Profile:** We seek professionals with business acumen (preferably with experience in the banking sector), analytical capability, and a results-oriented approach. If you are passionate about continuous improvement and working in cross-functional teams, this is the right opportunity for you. **Role Details:** **Work Model:** Hybrid (Madrid), 1/4 (on-site/remote). **Key Requirement:** Genuine B2 English (fluency for international environments). **Bonus:** Availability for occasional travel to Italy. **Your Key Challenges:** **Strategic Migration:** Migrate Hadoop infrastructure to Kubernetes Engine, COS, Spark, and Airflow as a Service. **Data Engineering:** Implement complex data transformations and ensure data quality using Scala, SQL, and Spark. **DevOps Automation:** Configure CI/CD pipelines and advanced orchestration with Airflow. **Gen AI Innovation:** Design effective prompts to optimize the software development lifecycle (SDLC). **Required Technical Stack:** **Core:** Spark (Scala), Hadoop, HDFS, Apache Airflow. **Cloud Infrastructure:** Kubernetes, S3/COS, Parquet/ORC formats. **Databases:** Strong proficiency in SQL and NoSQL. **Tools:** CI/CD (GitLab, Jenkins), Shell Scripting, Oozie. **Nice-to-Have:** Dremio, Elasticsearch, Kibana, Kafka (Streaming), and HashiCorp Vault. Spark, hadoop, apache, airflow, HDFS, kubernetes, SQL, noSQL, CI/CD, GitLab, Jenkins