Data Engineer (Databricks) (f/m/d)

We are HCLTech, one of the fastest-growing large tech companies in the world and home to 225,000+ people across 60 countries, supercharging progress through industry-leading capabilities centered around Digital, Engineering and Cloud. The driving force behind that work, our people, are diverse, creative, and passionate, raising the bar for excellence on a regular basis. We, in turn, work hard to bring out the best in them as we strive to help them find their spark and become the best version of themselves that they can be. If all this sounds like an environment you’ll thrive in, then you’re in the right place. Join us on our journey in advancing the technological world through innovation and creativity.

We are looking for an experienced Data Engineer (Databricks) to join our team and drive the development of scalable, high-performance data solutions in a cloud environment.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure Databricks, PySpark, and SQL.
• Build and maintain data solutions leveraging Spark, Delta Lake, Databricks Notebooks, and Workflows.
• Develop robust ETL/ELT processes to ingest, transform, and deliver high-quality data.
• Collaborate with business and technical stakeholders to understand data requirements and implement efficient solutions.
• Participate in data modeling, warehousing, and architecture discussions.
• Implement CI/CD practices for data engineering projects using Git and Azure DevOps.
• Ensure data quality, governance, security, and compliance standards are met.
• Support performance tuning, troubleshooting, and optimization of data workloads.

Mandatory Skills

• PySpark
• Python
• SQL

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or a related field.
• Proven hands-on experience with Azure Databricks and large-scale data processing.
• Strong proficiency in Python, PySpark, and SQL.
• Experience working with cloud platforms, preferably Microsoft Azure (AWS/GCP experience also considered).
• Strong understanding of:
• Data Modeling
• Data Warehousing
• Experience with version control systems and CI/CD tools such as Git and Azure DevOps.
• Knowledge of data quality, governance, and best practices for enterprise data platforms.
• Ability to work independently while effectively collaborating within Agile teams.

Preferred Skills

• Experience with financial services, asset management, or investment data
• Understanding of pricing, positions, transactions, and market data domains
• Exposure to data governance and metadata management frameworks
• Experience in performance optimization of Spark workloads

We promote equal opportunities for all employees, regardless of their cultural and social background, gender, disability, age, religion, beliefs, and sexual identity. We give priority consideration to severely disabled applicants and those of equal status in the case of equal suitability.