Required Qualifications

Essential Technical Skills

• Databricks Platform: Proven hands-on experience with Databricks platform including workspace administration, cluster management, and workflow orchestration
• Unity Catalog: Demonstrated experience implementing Unity Catalog for unified governance, including metastore configuration, catalog/schema design, and access control policies
• Data Modelling: Strong expertise in designing and implementing data models, including dimensional modeling, data vault, and lakehouse architectures
• Delta Lake: Deep understanding of Delta Lake features including ACID transactions, time travel, and optimization techniques
• Apache Spark: Proficiency in Spark SQL, DataFrames, and performance tuning
• Programming: Strong skills in Python, SQL, and/or Scala for data processing
• Cloud Platforms: Experience with Azure, AWS, or GCP cloud services

Additional Technical Skills

• Data pipeline development and ETL/ELT processes
• Data governance frameworks and metadata management
• Performance optimization and troubleshooting
• CI/CD practices for data platforms
• Data quality and validation frameworks
• Understanding of data security and compliance requirements

Professional Experience

• Minimum 8-10 years in data architecture, engineering, or analytics roles
• At least 3-5 years of hands-on experience with Databricks platform
• Proven track record implementing Unity Catalog in enterprise environments
• Experience designing and implementing complex data models for large-scale systems
• Background working with Databricks Professional Services or similar partnerships preferred
• Experience across multiple industry sectors (Public Service, Financial Services, Healthcare, etc.) is advantageous

Certifications (Preferred)

• Data Engineer Professional
• Databricks Certified Associate Developer for Apache Spark
• Cloud platform certifications (Azure Data Engineer, AWS Data Analytics, or GCP Data Engineer)
• Relevant data management or analytics certifications

Soft Skills

• Strong analytical and problem-solving capabilities
• Ability to work collaboratively with Partners and cross-functional teams
• Client-focused mindset with relationship-building skills
• Self-motivated with ability to manage multiple priorities
• Experience working in agile environments

Key Responsibilities

End-to-End Data Architecture

• Collaborate with Databricks PS and customer 
• on comprehensive end-to-end data architecture on Databricks platform
• Design and implement data ingestion strategies from various source systems
• Architect scalable storage solutions leveraging Delta Lake and lakehouse architecture
• Develop efficient data processing frameworks using Apache Spark and Databricks workflows
• Create data consumption layers for analytics, reporting, and machine learning use cases
• Ensure seamless data flow across the entire data lifecycle

Governance, Security & Compliance

• Collaborate with Databricks PS on implementing robust governance frameworks
• Design and enforce security policies and access controls within Databricks environments
• Ensure compliance with regulatory requirements and industry standards
• Implement data lineage, audit trails, and monitoring capabilities
• Optimize performance through best practices in cluster configuration and query optimization
• Establish data quality standards and validation processes

Data Modelling & Design

• Define business-aligned data models that support organizational objectives
• Design Delta table structures optimized for performance and scalability
• Create dimensional and normalized data models based on business requirements
• Implement medallion architecture (Bronze, Silver, Gold layers) for data refinement
• Develop data schemas that support both analytical and operational workloads
• Document data models and maintain data dictionaries

Technical Leadership & Collaboration

• Lead technical discussions with clients and stakeholders to understand business requirements
• Provide technical guidance to development teams on Databricks best practices
• Work closely with Infrastructure (Infra), Applications (Apps), and Cyber teams for integrated delivery
• Mentor junior architects and engineers on Databricks technologies
• Present solution architectures and technical recommendations to client leadership

Solution Implementation

• Oversee implementation of Databricks solutions from design through deployment
• Ensure solutions meet performance, scalability, and reliability requirements
• Support proof-of-concepts and pilot implementations
• Conduct architecture reviews and ensure alignment with enterprise standards