➡️ Apply here: Lead Databricks Engineer
🔔 Monitor #java #python #data_engineer jobs
👩💼 Want to stand out? Improve your resume to appeal to recruiters, hiring managers, and Applicant Tracking Systems. ➡️ Improve your resume
We are looking for a **Lead Databricks Engineer** with expertise in cloud infrastructure, big data processing, data integration, and analytics, particularly with Azure.
**Responsibilities:**
* Data integration, transformation, and modeling.
* Serving large amounts of structured & semi-structured financial data.
* Analyzing existing process logic.
* Implementing best practices in data architecture, including security, governance, compliance, and performance optimization.
* Optimizing data storage across Delta Lake for data management.
* Developing and maintaining scalable data pipelines and building new API integrations to support continuing increases in data volume and complexity.
* Collaborating with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision-making across the organization.
**Requirements:**
* At least 5+ years of professional experience in the data engineering field.
* Proficiency in Azure Databricks, Delta Lake, Python, and PySpark for data manipulation.
* Proficiency in designing and optimizing pipelines using Azure Data Factory, including performance tuning and optimizing data pipelines with Databricks Notebooks.
* Experience with Azure data platforms and related services.
* Good understanding of SQL and experience with relational database systems (e.g., MSSQL, PostgreSQL, Oracle DB).
* Understanding data structures, data modeling, and data architecture patterns (Data Lake, DWH, Data mesh, etc.).
* Ability to work in an agile environment and eagerness to learn and develop new skills.
* Good problem-solving skills and attention to detail.
* Strong written and spoken English.
**Nice to have:**
* Certifications in Azure data or Cloud Services (e.g., Azure Data Engineer Associate).
* Business understanding of the insurance or financial domain.
* Experience with BI Tools (Power BI).
**We offer:**
* Private healthcare insurance.
* Global travel medical and accident insurance.
* Regular performance assessments.
* Referral bonuses.
* Family-friendly initiatives.
* Learning and development opportunities including in-house training and coaching, professional certifications, over 22,000 courses on LinkedIn Learning Solutions, and much more.
* All benefits and perks are subject to certain eligibility requirements.