Summary
Design and operate large-scale, governed data infrastructure across batch and streaming workloads. Stand up enterprise-grade ingestion, storage, orchestration, governance, and observability — and grow high-performance teams to own them end-to-end. Background spans distributed systems at AWS, big-data processing with Spark and Kafka, applied ML and GenAI in production, and security & compliance modernisation (APRA CPS 234).
Experience
Lead Engineer — Enterprise Data Platform
Present
Commonwealth Bank of Australia (CBA) · Sydney, Australia
- Lead architecture and technical design of CBA's enterprise data platform (IDAP) — ingestion, transformation, orchestration, governance, observability, and CI/CD on AWS, Snowflake, dbt, Airflow, Terraform.
- Own the platform roadmap with architects, product owners, security, and engineering leaders; define standards, reusable frameworks, and production readiness gates adopted across multiple business domains.
- Designed and implemented enterprise data sharing on AWS DataZone and Snowflake — governed self-service access with strengthened metadata, lineage, access controls, and audit.
- Delivered critical security and modernisation programs — APRA CPS 234 remediation, Airflow upgrades, container platform migration, certificate automation (retired 11 manual certs).
- Drove platform adoption through tenant onboarding, golden paths, and developer enablement — reducing time-to-first-data-product for new teams.
- Hands-on contributor across Python, Terraform, dbt, GitHub Actions, AWS — delivered 1,100+ production PRs and reusable platform components adopted across multiple teams.
Data & AI Architect
2021 — Recent
ZIP · Sydney, Australia
- Architected real-time event and analytics infrastructure on Snowplow, Kafka, and Amplitude — high-volume ingestion, schema management, and downstream activation across product surfaces.
- Designed and operated data warehouses and lakes on AWS Redshift, S3, and DynamoDB; standardised batch and streaming ingestion patterns across teams.
- Built ML deployment pipelines on Databricks Asset Bundles, Terraform, and GitLab; containerised models on EKS and Databricks Serverless with alerting and anomaly detection.
- Shipped production GenAI features using OpenAI APIs and Bedrock Agents, including a knowledge-base REST API on Kubernetes.
- Drove low-/no-code ingress & egress with Fivetran, Airbyte, Census, Braze; integrated OpsGenie, Atlan, Jira for incident management and lineage.
Big Data Engineer
2016 — 2021
Amazon Web Services (AWS) · Australia
- Built and operated distributed large-scale data processing solutions on Apache Spark for AWS customers.
- Implemented search and deep-learning training/inference platforms using SageMaker and OpenSearch.
- Partnered with customer data scientists to deploy and scale ML models in production — focus on reliability and cost.
- Drove customer code reviews and engineering excellence across Java, Python, and Kotlin services.
Earlier Experience
-
Startup Technical Consultant · AI Hello — USA
2015
-
Senior Java Consultant · Northern Trust — USA
2014 — 2016
-
Big Data Developer · Dictionary.com — Mexico
2013 — 2014
-
Software Developer · Ask.com — India
2009 — 2013
Core Skills
- Languages
- Python, Java, Scala, Go, SQL, dbt
- Cloud & Platforms
- AWS, Snowflake, Databricks, Kubernetes, EKS
- Streaming & Batch
- Kafka, Schema Registry, Snowplow, Kinesis · Apache Spark, AWS Glue, EMR
- Storage & Tables
- S3, Snowflake, Redshift, Delta Lake, Iceberg, DynamoDB, RDS
- Orchestration & CI/CD
- Airflow, GitHub Actions, GitLab, Terraform, CloudFormation, Databricks Asset Bundles
- Governance & Security
- AWS DataZone, Atlan, Access control & audit, IAM & encryption, APRA CPS 234
- ML & GenAI
- SageMaker, Bedrock Agents, OpenAI APIs, Databricks ML
- Observability
- Elementary, Soda, Dynatrace, OpenSearch
- Leadership
- Roadmap ownership · Hiring & team building · Design reviews · Mentoring · Stakeholder partnership
Certifications
- AWS Cloud — 5+ certifications
- Databricks — Apache Spark Developer
- Databricks — Generative AI
- Oracle Certified Java Programmer (1.7+)
- AppSec Champions Academy (16-hr, defensive coding)
Education
- Diploma — Data Science
MIT, Melbourne, Australia
- Bachelor's Degree — Computer Science
Achievements
- Deployed and scaled multiple ML models on Databricks & Spark
- Reduced defects by 30% via CI/CD pipelines
- Mentored juniors — lifted team productivity by 25%
Recognition
"You have brought with you a wealth of technical expertise and have gone over and above in delivery speed. You have helped us get a long way in adopting dbt and analytics-platform thinking. You have shown yourself to be an engineering leader within our squad, regularly guiding junior members and pushing them to grow."
— Clement Wang, COE · CBA
"Siva displayed expertise and an ability to communicate complex ideas and build alignment amongst leaders. When you spoke, people listened. You're a valued asset to IB&M Technology and CBA."
— Alistair MacArthur, Crew Lead · CBA