As a Data Team Lead, you will oversee the design, development, and maintenance of the company's data systems, encompassing both real-time and batch processing. Your leadership will be pivotal in ensuring data accuracy, scalability, and efficiency, especially given the company's extensive transaction volume and user base.
Responsibilities
- Lead and mentor a team of data engineers to build and maintain scalable ETL pipelines, ensuring data integrity and security.
- Drive infrastructure planning and design for data platforms to ensure performance, scalability, and reliability.
- Collaborate with business and engineering teams to define data requirements, design data models, and develop data products.
- Optimize data systems for performance, scalability, and cost-efficiency across multiple environments, including cloud (e.g., GCP) and on-premises data centers.
- Oversee the entire lifecycle of data projects, from conception to deployment, ensuring timely and high-quality delivery.
- Continuously improve data infrastructure and engineering practices to meet the needs of a rapidly growing business.
- Provide technical guidance on adopting new technologies and data platforms to drive team innovation.
Requirements
- 5+ years of relevant experience in data engineering, including at least 2 years in team leadership or mentorship roles.
- Proven track record in designing, building, and maintaining complex ETL pipelines with a focus on scalability, reliability, and security.
- Strong proficiency in backend development, including shell scripting, SQL, and at least one programming language (e.g., Python, Scala).
- Expertise in distributed computing engines and platforms (e.g., ClickHouse, PySpark, Kafka, BigQuery, Airflow).
- Experience in planning and implementing infrastructure for large-scale data processing and storage solutions.
- Familiarity with data exploration and visualization tools (e.g., Superset, Tableau) and experience implementing data lake and data warehouse architectures.
- Practical experience with cloud-based data processing solutions (e.g., AWS Data Pipelines, GCP Dataflow).
- Experience with Change Data Capture (CDC) tools and methodologies, or other streaming solutions.
- Ability to manage stakeholders at various levels, with excellent interpersonal and communication skills.
- Experience in setting up or utilizing Data Catalogs is a plus.
- Passion for team development, fostering a collaborative and high-performance culture.
What Sets This Company Apart
The company processes over 3 billion transactions monthly for more than 6 million users, offering a dynamic and fast-paced environment. This role provides the opportunity to shape the future of the company's data infrastructure and support the growth of both the team and the organization. You'll collaborate closely with cross-functional teams, contributing to innovative solutions that drive business success.
Call to Action
If you meet the above qualifications and are eager to play a pivotal role in a rapidly growing environment, please submit your resume and cover letter. We look forward to welcoming you to our team and advancing our data engineering practices together.
