[Search & Recommendation System] Data Engineer/ Data Scientist

I am working with a leading large-scale tech company to hire a Senior Data Engineer / AI Platform Engineer who will shape their next-generation data and AI infrastructure, supporting high-traffic products and search/recommendation systems. We’re seeking an engineer strong in distributed systems, scalable data pipelines, and real-time ML applications.

Key Responsibilities

Data Engineering (Primary Focus)

• Build large-scale batch and streaming pipelines using Dataflow, Pub/Sub, BigQuery, and Airflow/Composer.

• Implement data governance practices including validation, quality monitoring, and anomaly detection.

• Optimize data storage, indexing, and retrieval performance to support search and recommendation workloads.

Search / Recommendation System Enablement

• Support data workflows and feature engineering for retrieval, ranking, and vector-based search systems.

• Work with Feature Store, Vector DB, and embedding pipelines to boost relevance and performance.

• Contribute to online serving strategies, A/B testing, and experiment design.

MLOps & AI Infrastructure

• Develop end-to-end ML pipelines using Vertex AI, Kubeflow, and CI/CD automation.

• Implement model versioning, A/B tests, data/model drift detection, and automated rollback mechanisms.

• Enhance real-time and batch feature computation efficiency.

Cloud Architecture & Backend Engineering

• Build scalable cloud architecture using GKE, Managed Instance Groups, and Cloud Run to handle burst traffic.

• Develop low-latency, high-throughput APIs using Python (FastAPI).

• Integrate multiple storage systems (BigQuery, Redis, Vector DB) and establish observability with Prometheus/Grafana.

Required Qualifications

• 3–5 years of experience in Data Engineering / Backend / Platform development.

• Strong proficiency in Python and experience with FastAPI.

• Hands-on expertise with GCP services: GKE, MIG, Cloud Run, BigQuery, Pub/Sub, Dataflow.

• Solid experience in distributed systems, high-throughput data processing, or burst-traffic scenarios.

• Strong skills in Docker, Kubernetes, Helm, HPA, and CI/CD automation.

Preferred Qualifications (Nice-to-Have)

• Experience building or supporting search or recommendation systems.

• Exposure to Feature Store, Vector DB, embeddings, RAG, or LLM-based systems.

• Experience operating systems with 10M+ daily requests or extreme peak loads.

• Familiarity with IaC tools such as Terraform.

Why This Role

• Work on high-impact systems powering millions of daily users.

• Collaborate with teams building search, recommendation, and AI-driven product experiences.

• Play a leading role in designing the next generation of data & ML infrastructure.

• Join a company with a strong data culture, complex engineering challenges, and long-term scalability goals.