As a Data Engineer to join our dynamic, global team. This role requires strong expertise in Spark, databases, and cloud technologies to design, implement, and optimize scalable data solutions that support business requirements. The ideal candidate will have experience with data pipelines, database management, and testing methodologies in a distributed, fast-paced environment.
Responsibilities:Design, develop, and optimize scalable data pipelines and architectures using Apache Spark and databases.
Work with cloud platforms (IBM Cloud) to build robust data solutions and ensure seamless integration.
Collaborate closely with cross-functional teams to gather and understand business requirements and translate them into efficient data models and workflows.
Implement and maintain data structures, ensuring the integrity, availability, and scalability of the data.
Develop and implement testing strategies for data validation, ensuring high data quality and system performance.
Support the team in data analysis and insights to enable data-driven decision-making.
Contribute to documentation and best practices to facilitate knowledge sharing within the global team.
Participate in regular syncs with the worldwide team, ensuring clear communication and alignment across time zones.
High level job responsibilities:Develop elegant, flexible, maintainable, and scalable solutions to complex problems by delivering data and analytics.
Demonstrate experience leading development efforts and coordinating work for junior developers.
Define and consolidate development and modelling practices within a team.
Leverage strong collaborative skills to define requirements, develop plans, and deliver iteratively to support our organization's mission.
Leverage strong analytical skills to understand business processes and propose effective data solutions.
Communicate and manage relationships with business and development teams - provide guidance, mentorship, and direction, as required.
Translate business needs into technical requirements.
Learn new tools, technologies, and processes for continuous improvement.
As part of a self-directed team, take ownership of activities and deliver them on-time and with quality.