Overview

Technical Challenge

Responsibilities

• Build and maintain scalable batch and streaming data pipelines that ingest, transform, and serve data for analytics and downstream applications.
• Develop and operate backend data services and APIs to enable secure, reliable access to curated datasets and metrics.
• Translate analytics and business intelligence needs into trusted data models, transformations, and reusable datasets.
• Implement Python-based solutions for data processing and analysis.
• Manage and maintain highly efficient data architectures, ensuring scalability and performance.
• Develop and maintain APIs (REST, gRPC) to serve data to internal or external systems.
• Build and refine CI/CD processes to improve data workflows and ensure seamless deployments.
• Implement and maintain cloud and DevOps foundations (IaC, CI/CD, containers, orchestration) to ensure secure, repeatable, and scalable delivery of AI services.
• Collaborate with teams across engineering, data science, and product to deliver robust data solutions.
• Apply deep knowledge of data engineering best practices and frameworks, ensuring data integrity and security.
• Work across multiple cloud environments such as GCP, Azure and AWS.

Required Skills

• Strong data engineering experience delivering production-grade pipelines and data platforms, with an emphasis on reliability and maintainability.
• Backend development capability, including writing clean, testable services and pipeline code in Python.
• Advanced SQL skills, including building complex transformations and optimizing query performance.
• Experience working with relational databases such as PostgreSQL, including schema design and performance tuning.
• Working knowledge of Databricks for data processing and platform usage in support of data engineering workloads.
• Experience using dbt to build, version, and manage analytics transformations and models.
• Work across multiple cloud environments such as GCP, Azure and AWS.
• Ability to apply foundational DevOps and cloud infrastructure practices—monitoring, CI/CD, environment management, and reliability—consistent with DevOps & Cloud Infrastructure expectations.

Preferred Skills

• Experience working with Snowflake for cloud data warehousing, modeling, and performance optimization.
• Familiarity applying GenAI to data workflows, such as data enrichment, quality checks, or analytics copilots.

UPLabs Summary