Akshay Dwivedi
Data Engineer @ William Blair
About
Data Engineer at William Blair, where I design and build production data platforms that power investment decisions across four teams managing global equities, US value, US growth, and emerging market debt portfolios. I specialize in end-to-end data infrastructure — from ingestion and transformation to orchestration and AI-powered analytics — on Azure and AWS. Core Data Engineering Expertise: → Data Platforms & Warehousing: Databricks, Medallion Architecture (Bronze/Silver/Gold), Delta Lake, Unity Catalog, Apache Spark, PySpark, Delta Sharing → Pipeline Orchestration & ETL/ELT: Apache Airflow, Argo Workflows, DBT, Kafka, Debezium (CDC), batch and streaming pipelines → Cloud Infrastructure: Azure (AKS, Data Factory, Document Intelligence, OpenAI), AWS, Terraform (IaC), Docker, Kubernetes, Helm, ArgoCD, GitOps → Data Stores & Search: PostgreSQL, Elasticsearch (BM25 + HNSW hybrid search), Redis, vector databases → AI/ML Engineering: Production RAG systems, Azure OpenAI, Cohere multimodal embeddings, LLM-powered document understanding, MCP protocol Selected Impact: Optimized a Kubernetes ETL pipeline from 30 minutes to 90 seconds (20x improvement) by replacing Azure SDK calls with direct filesystem I/OArchitected a multimodal RAG platform processing financial documents end-to-end — PDF ingestion, semantic chunking, hybrid retrieval, and streaming LLM responses across 14 analytical sectionsDesigned Medallion-architecture data lakes for insurance claims at TCS and built CDC pipelines with Kafka + Debezium at Hosta AIImplemented GitOps-driven Kubernetes deployments with Terraform, Helm, and Azure Key Vault for secure, scalable production workloads Background: MS in Data Analytics Engineering, Northeastern University. 5+ years of professional experience across financial services, computer vision, and IT consulting. Previously contributed to conservation analytics at WWF.
United States
Chicago
Information Services
Data Architects, Data Infrastructure, Python + FastAPI , Databricks + DBT + Medallion Architecture , Kubernetes & Cloud Infrastructure (Azure + AWS) , RAG / LLM Systems , Retrieval-Augmented Generation (RAG), Kubernetes, Financial Analysis, DevOps, Azure Data Lake, Scala, Generative AI, Vector Databases, RAGs, Artificial Intelligence (AI), Microsoft Azure, R (Programming Language), Dremio, Azure SQL
Experience

Data Engineer
Chicago, IL
Engineered a Kubernetes-based ETL pipeline using Argo Workflows, Python, and direct filesystem I/O, reducing processing time from 30 minutes to 90 seconds (20x performance gain) while replacing a costly vendor-managed workflow Developed a production RAG system for investment recommendation reports using Azure OpenAI, Cohere multimodal embeddings, and Elasticsearch hybrid search (BM25 + HNSW) — processing financial PDFs end-to-end with semantic chunking, session-based multi-tenancy, and page-level citation tracking Architected and maintained a production Medallion Architecture data lake (Bronze/Silver/Gold layers) on Azure, delivering clean, validated, and governed datasets to analysts and portfolio managers across 4 investment teams Designed prompt engineering frameworks with team-specific templates, structured variable injection, and guardrail systems — enabling grounded, hallucination-resistant financial Q&A across global equities, US value, US growth, and emerging market debt portfolios Built real-time streaming SSE endpoints and citation-tracking systems returning source-attributed LLM responses with page-level references, improving analyst trust and auditability of AI-generated financial insights

Graduate Research And Teaching Assistant
Boston, Massachusetts, United States
Graduate Teaching Assistant for Course MSIM-6210 Information Visuals and Dashboards for Business • Assist in the review of course materials and content • Assist learners with identifying connections between academic learning and work experiences • Help in live sessions with materials, announcements, exhibits, and demonstrations • Provide support to students, faculty, and staff during live online class sessions • Interact in content-related forums as assigned • Engage learners through TA office hours and periodic engagement activities through the community platform • Provide individualized feedback to 50 learners on data visualization dashboards in PowerBI and Tableau, business analysis, SQL queries, and code reviews

Intramural Official
Boston, Massachusetts, United States
• Ensuring the safety of games by conducting them in an orderly manner and enforcing the laws of the game • Providing training and mentorship to newer officials while presenting them with constructive feedback

Graduate Teaching Assistant
Boston, Massachusetts, United States
Graduate Teaching Assistant for MISM-6205 Data Wrangling in Business. • Led group and individual tutoring sessions to support students in their coursework and exam preparation. • Graded projects for students and shared best practices for the topics along with assisting them to solve complex problems.

Data Engineer
Boston, MA
• Automated AWS database provisioning and configuration using Terraform modules, cutting deployment time by 30% and ensuring uniform, error-free database setups. • Executed Python-based data analytics projects with Postgres, Looker, Docker, and ECS. Achieved a 20% reduction in response times, enhancing customer satisfaction. • Developed a serverless Slack bot with AWS Lambda functions to pinpoint build stage and codepipeline errors during deployment resulting in accelerated error resolution time by 20% through precise location identification. • Established CDC pipelines with Kafka and Debezium for real-time change tracking across production databases, integrated with Databricks and DBT for multi-layer data modeling • Orchestrated unified batch and streaming data processing on AWS Cloud using technologies such as Amazon EMR, Kinesis, and AWS Glue, ensuring a comprehensive and scalable approach to data analysis and insights.

Senior Data Analyst
Pune Division
• Designed and implemented a Medallion Architecture (bronze/silver/gold) for a US-headquartered insurance company, integrating customer databases and reducing downstream data quality issues" • Engineered and crafted 7+ impactful dashboards in Power BI and Tableau,effectively translated complex data into actionable visualizations, enhancing data-driven decision-making processes for multiple stakeholders. • Spearheaded detailed analysis, optimizing data integration workflows, identifying bottlenecks, and driving NoSQL(AWS Dynamo DB) enhancements for a remarkable 25% reduction in data processing time. • Simplified ETL tasks using SSIS, Apache Airflow, Kubernetes, S3 Object Storage, and Python, creating a modern data setup. Cut data processing time from 7 to 3 hours for over 50,000 daily records. • Implemented effective data validation and integrity processes, ensuring accurate data mining outcomes. Achieved a 95% data accuracy rate, improving the reliability of insights and recommendations. • Created a robust monitoring and alerting system using AWS CloudWatch and custom scripts, reducing unplanned downtime by 40% and improving incident response times. • Collaborated with 20+ member of cross-functional teams to create and implement data management policies and best practices, leading to improved data security and compliance.

Data Analyst Intern
Bhopal, Madhya Pradesh, India
• Applied comprehensive data analysis to combine multiple data sources and highlight key findings, resulting in a 10% improvement in KPIs. • Executed A/B testing on various visualization formats, resulting in a 15% performance improvement uplift in user engagement and awareness campaigns.
Akshay Dwivedi's Contact Information
Phone
Find the Right Leads
Find Verified Contact Data
What LeadContact does well
Find verified emails, phone numbers, and decision-makers with 98% accuracy.
Find Leads
Find the right people by company, role, industry, location, and more.
925M+ professional profiles

Find Emails
Access verified email addresses for your target contacts.
657M+ emails

Find Phone Numbers
Get cross-validated phone data from multiple top sources.
239M+ phone numbers

More Accurate. Lower Cost.
Find contact data in 1 tool with 98% accuracy
LeadContact integrates leading enrichment tools to deliver more accurate contact data—without paying for each one.
Great conversations start with the right contact.
It’s time to find yours.




