• Strong knowledge of database systems, data modelling, and data warehousing principles.
• Experience in data pipeline orchestration tools like Apache Airflow and Step Functions.
• Familiarity with data governance, security, and compliance practices.
• Excellent problem-solving and analytical skills, with a keen attention to detail.
• Strong communication skills and ability to effectively convey technical concepts to non-technical stakeholders.
• Leadership and mentoring abilities, with a track record of successfully leading and managing a team of data engineers.
• Ability to work in a fast-paced, dynamic environment and adapt to changing priorities.

Responsibilities

1. Architect and Design Data Solutions: Lead the design and architecture of scalable, efficient, and robust data pipelines and systems to handle large volumes of data from various sources.
2. Team Leadership: Manage and mentor a team of data engineers, ensuring successful project execution, fostering a collaborative environment, and providing technical guidance.
3. Data Integration: Oversee the integration of data from multiple sources, both internal and external, to ensure data consistency and accuracy.
4. Data Transformation: Develop and implement data transformation processes using Python and PySpark, ensuring data quality and proper data governance.
5. Data Warehouse Management: Design and maintain data warehouses on AWS, ensuring data availability, reliability, and security.
6. Performance Optimization: Identify performance bottlenecks in data processing and implement optimizations to improve overall data pipeline efficiency.
7. Data Governance and Security: Implement and enforce data governance policies and best practices to ensure data security, privacy, and compliance with relevant regulations.
8. Continuous Improvement: Stay up to date with the latest technologies, tools, and best practices in the data engineering space. Propose and implement process improvements to enhance productivity and data quality.
9. Collaboration: Collaborate with cross-functional teams, including data scientists, software engineers, and business stakeholders, to understand data requirements and deliver data solutions that meet business needs.
   Join the team and contribute to shaping the future of our data infrastructure as we harness the power of data to drive insights and innovation. If you are a passionate and driven Lead Data Engineer with expertise in Python, PySpark, and AWS, we'd love to hear from you!