Data Engineer - Databricks

Responsibilities

As a Data Engineer, your main tasks will involve:

• Design, build, and maintain scalable data pipelines using Databricks and cloud-based data platforms.
• Develop, optimize, and automate ETL/ELT processes to support analytics, reporting, and machine learning initiatives.
• Work closely with Data Scientists, Analysts, ML Engineers, and business stakeholders to understand data requirements and deliver reliable datasets.
• Implement data ingestion frameworks from multiple structured and unstructured data sources.
• Build and maintain data models, data lakes, and data warehouses that support enterprise analytics needs.
• Ensure data quality, consistency, governance, and security across platforms.
• Optimize Spark workloads and Databricks environments for performance and cost efficiency.
• Support deployment and operationalization of machine learning solutions by providing production-ready datasets and feature pipelines.
• Collaborate with Azure, IT, and DevOps teams to implement CI/CD and DataOps best practices.
• Monitor, troubleshoot, and continuously improve data platform performance and reliability.

Your Skills and Experience

As a Data Engineer, your skills and experience will ideally include:

Required

• Bachelor's Degree in Computer Science, Engineering, Information Systems, Mathematics, or a related field.
• 3+ years of experience in Data Engineering, Data Warehousing, or Big Data environments.
• Strong programming skills in Python and SQL.
• Hands-on experience with Databricks, Apache Spark, and distributed data processing.
• Experience designing and developing ETL/ELT data pipelines.
• Experience working with Data Lakes and modern data architectures.
• Understanding of data modeling concepts including Star Schema and Dimensional Modeling.
• Experience working with Git and version control systems.
• Strong analytical and problem-solving skills.
• English level B2 or higher.

Preferred Qualifications

• Experience with Azure Data Platform services, including:
• • Azure Databricks
  • Azure Data Factory
  • Azure Data Lake Storage
  • Azure Synapse Analytics
  • Azure DevOps
• Experience working with Delta Lake and Lakehouse architecture.
• Knowledge of orchestration tools such as Airflow, Databricks Workflows, or Azure Data Factory.
• Experience supporting Machine Learning use cases and MLOps practices.
• Familiarity with MLflow and model deployment processes.
• Knowledge of CI/CD, Infrastructure as Code, and DataOps practices.
• Experience with streaming technologies such as Kafka or Event Hubs.
• Experience working in Agile environments.
• Insurance industry experience is a plus.

Nice to Have

• Experience collaborating with Data Science teams and supporting AI/ML initiatives.
• Knowledge of Generative AI, LLM data pipelines, vector databases, and embedding workflows.
• Experience with Unity Catalog, data governance, and metadata management.
• Exposure to cloud-native architectures and enterprise-scale data platforms.

If you are interested, please apply directly through this link or send your resume to nathalia.huerta@mx.zurich.com.

Position available only for candidates located in Mexico City, State of Mexico, or Guadalajara. Please apply only if you reside in one of these areas.