PySpark Developer

Dellent is a consulting company focused in System Information and Telecommunications. Our goal is to help our candidates and consultants to take a step forward in their careers through projects that meet their needs and expectations. This project is part of the business investment banking and financial services area, that currently has its technological hub in Portugal and is working on the development of innovative solutions and support for all operations related to the business. We are looking for a PySpark Developer to work in this project in Porto (Hybrid). Must have: - More than 9 years of professional experience; - Minimum of 5 years of professional experience in software development or data engineering, with a strong, proven specialization in PySpark / Apache Spark; - Advanced expertise in Python programming (and the PySpark ecosystem); - Deep understanding of Apache Spark's internal architecture (Driver/Workers, DAG, Lazy Evaluation, caching mechanisms); - Strong proficiency in SQL and complex query optimization; - Hands-on experience with modern storage formats (Parquet, Delta Lake, Avro); - Experience working in a Cloud environment (AWS, Azure, or GCP) and with platforms like Databricks or EMR; - Proficiency with Git and CI/CD pipelines; - Experience with a workflow orchestrator (Apache Airflow, Prefect, etc.); - Strong commitment to software quality (unit testing, Spark data mocking); - Fluent in English. What can you expect from us? - Health insurance; - A personalized training plan, with a budget to spend on the training you find necessary; - Constant feedback so you can grow professionally; - Remote onboarding process; - Team events every semester so you can live new adventures; - Culture of proximity and transparency. Your ideas and needs are heard and valued by us! Regime de trabalho híbrido.