Summary
Overview
Work History
Education
Skills
Websites
Certifications
Awards
Timeline
Generic

Shrikant Agrawal

Cloud Data Engineer | Azure & AWS | Databricks Certified | DataOps & MLOps Specialist
Almere

Summary

Results-driven Cloud Data Engineer with 13 years of progressive experience in Azure and AWS solution architecture, currently delivering high-impact data solutions at Capgemini Netherlands. Architecting and operationalizing end-to-end scalable data platforms that drive business value and innovation.

Expert in Azure DevOps, ETL frameworks, and DataOps best practices, with a strong focus on automation, performance tuning, and cost-efficient cloud-native solutions. Proven track record in leading complex data initiatives, optimizing development workflows, and enabling seamless data integration across diverse enterprise systems.

Highly skilled in real-time monitoring, incident resolution, and maintaining high availability of mission-critical pipelines. Recognized for bridging the gap between data engineering, DevOps, and MLOps — ensuring faster delivery, better reliability, and measurable outcomes

Overview

13
13
years of professional experience

Work History

Senior Data Platform Engineer

ABN AMRO Bank N.V
12.2021 - Current
  • Developed real-time and batch data processing pipelines using Azure Databricks, Apache Spark (PySpark), and Delta Lake for credit reporting, data segmentation, and regulatory use cases.
  • Created a live Databricks monitoring dashboard with integrated alert mechanisms using Azure Monitor and PySpark, enabling proactive monitoring of critical pipelines.
  • Automated data quality checks, validation routines, and schema enforcement within Databricks notebooks, ensuring data accuracy and compliance.
  • Built and managed CI/CD pipelines using Azure DevOps, Git, and Bicep to automate deployment and infrastructure provisioning.
  • Designed scalable ETL/ELT frameworks for ingesting and transforming large volumes of data into Azure Data Lake and Delta Lake.
  • Implemented log monitoring and incident detection workflows, reducing issue resolution time and improving system reliability.
  • Applied DataOps practices including automation, testing, and version control to improve pipeline stability and reduce data latency.
  • Contributed to the deployment of secure, scalable cloud infrastructure on Azure following enterprise architecture standards.

Senior Data & DataOps Engineer

Standard Chartered Bank
06.2018 - 10.2021
  • Architected and deployed complex ETL pipelines using AWS Glue, AWS DMS, Lambda, S3, and CloudWatch for automated, scalable data ingestion and integration.
  • Designed and optimized data processing workflows using Azure Databricks, Data Factory, Azure Storage, and PySpark, improving pipeline efficiency and processing speed.
  • Developed and implemented DataOps frameworks using Docker for containerized data pipeline execution, ensuring reliability, reproducibility, and streamlined development workflows.
  • Built end-to-end monitoring and incident management solutions using Azure Monitor, Log Analytics, and custom alerting mechanisms, enabling high availability and rapid issue resolution for critical data workloads.

Senior ETL Engineer

JP Morgan Chase
06.2012 - 06.2018
  • Designed and developed ETL workflows using SQL and Oracle, enabling efficient and reliable data extraction, transformation, and loading across enterprise systems.
  • Optimized complex SQL queries and Oracle database performance, significantly improving data processing speed and reducing query latency.
  • Conducted ETL performance and load testing to ensure system scalability and stability under large-volume data processing scenarios.
  • Automated routine ETL operations using Shell scripting, reducing manual intervention and minimizing data processing errors.
  • Provided end-to-end ETL support, including issue resolution, root cause analysis, and performance tuning in collaboration with cross-functional teams.

Education

Bachelor of Computer Science And Engineering

Diploma in Computer Science And Engineering

Skills

Cloud Platforms:

Azure (Azure Data Factory, Azure Synapse, Azure Data Lake, Azure Databricks), AWS (S3, Redshift, Glue)

Big Data & Processing:

Databricks (Certified Data Engineer Professional), Apache Spark (PySpark), Delta Lake, Hadoop

Data Engineering & Pipelines:

ETL/ELT, DataOps, MLOps, Batch & Streaming Pipelines, Orchestration (Airflow, ADF), CI/CD for Data Pipelines

Programming & Scripting:

Python, PySpark, Scala, SQL, Shell Scripting, JSON, YAML

DevOps & Automation:

Azure DevOps, Git, Terraform, Docker, Jenkins, CI/CD Automation

Databases & Storage:

SQL Server, PostgreSQL, MySQL, Snowflake, Redshift, Cosmos DB, Azure Blob Storage, S3

Monitoring & Incident Management:

Azure Monitor, Log Analytics, Data Pipeline Monitoring, Alerting

Tools & Frameworks:

Databricks CLI, MLflow, Jupyter Notebooks, GitHub Actions, Visual Studio Code

Soft Skills:

Solution Architecture, Agile/Scrum, Stakeholder Communication, Technical Leadership, Documentation

Certifications

  • Certifed Databricks Data Engineer professional
  • Certifed Azure Solution Architect
  • Certifed AWS Architect
  • Certifed Azure DevOps

Awards

Custom Championship Award @ ABN AMRO Amsterdam April-2024


  • Utilized Databricks and PySpark to develop a comprehensive data pipeline, addressing critical pain points for the ABN specific team.
  • Engineered efficient data processing workflows for both daily and historical data, ensuring timely and reliable data refreshes in production.
  • Collaborated with integrated data set users to streamline notifications and issue resolution, resulting in a stable, incident-free production environment and high user satisfaction.

Timeline

Senior Data Platform Engineer

ABN AMRO Bank N.V
12.2021 - Current

Senior Data & DataOps Engineer

Standard Chartered Bank
06.2018 - 10.2021

Senior ETL Engineer

JP Morgan Chase
06.2012 - 06.2018

Bachelor of Computer Science And Engineering

Diploma in Computer Science And Engineering

Similar Profiles

CREATE PROFILE
Shrikant AgrawalCloud Data Engineer | Azure & AWS | Databricks Certified | DataOps & MLOps Specialist