Akshay Dwivedi

Akshay Dwivedi

Data Engineer @ William Blair

About

Data Engineer at William Blair, where I design and build production data platforms that power investment decisions across four teams managing global equities, US value, US growth, and emerging market debt portfolios. I specialize in end-to-end data infrastructure — from ingestion and transformation to orchestration and AI-powered analytics — on Azure and AWS. Core Data Engineering Expertise: → Data Platforms & Warehousing: Databricks, Medallion Architecture (Bronze/Silver/Gold), Delta Lake, Unity Catalog, Apache Spark, PySpark, Delta Sharing → Pipeline Orchestration & ETL/ELT: Apache Airflow, Argo Workflows, DBT, Kafka, Debezium (CDC), batch and streaming pipelines → Cloud Infrastructure: Azure (AKS, Data Factory, Document Intelligence, OpenAI), AWS, Terraform (IaC), Docker, Kubernetes, Helm, ArgoCD, GitOps → Data Stores & Search: PostgreSQL, Elasticsearch (BM25 + HNSW hybrid search), Redis, vector databases → AI/ML Engineering: Production RAG systems, Azure OpenAI, Cohere multimodal embeddings, LLM-powered document understanding, MCP protocol Selected Impact: Optimized a Kubernetes ETL pipeline from 30 minutes to 90 seconds (20x improvement) by replacing Azure SDK calls with direct filesystem I/OArchitected a multimodal RAG platform processing financial documents end-to-end — PDF ingestion, semantic chunking, hybrid retrieval, and streaming LLM responses across 14 analytical sectionsDesigned Medallion-architecture data lakes for insurance claims at TCS and built CDC pipelines with Kafka + Debezium at Hosta AIImplemented GitOps-driven Kubernetes deployments with Terraform, Helm, and Azure Key Vault for secure, scalable production workloads Background: MS in Data Analytics Engineering, Northeastern University. 5+ years of professional experience across financial services, computer vision, and IT consulting. Previously contributed to conservation analytics at WWF.

Country

United States

City

Chicago

Industry

Information Services

Skill

Data Architects, Data Infrastructure, Python + FastAPI , Databricks + DBT + Medallion Architecture , Kubernetes & Cloud Infrastructure (Azure + AWS) , RAG / LLM Systems , Retrieval-Augmented Generation (RAG), Kubernetes, Financial Analysis, DevOps, Azure Data Lake, Scala, Generative AI, Vector Databases, RAGs, Artificial Intelligence (AI), Microsoft Azure, R (Programming Language), Dremio, Azure SQL

Experience

William Blair

Data Engineer

William Blair

LinkedIn
2024-2 - Present · 2 yrs 8 mos

Chicago, IL

Engineered a Kubernetes-based ETL pipeline using Argo Workflows, Python, and direct filesystem I/O, reducing processing time from 30 minutes to 90 seconds (20x performance gain) while replacing a costly vendor-managed workflow Developed a production RAG system for investment recommendation reports using Azure OpenAI, Cohere multimodal embeddings, and Elasticsearch hybrid search (BM25 + HNSW) — processing financial PDFs end-to-end with semantic chunking, session-based multi-tenancy, and page-level citation tracking Architected and maintained a production Medallion Architecture data lake (Bronze/Silver/Gold layers) on Azure, delivering clean, validated, and governed datasets to analysts and portfolio managers across 4 investment teams Designed prompt engineering frameworks with team-specific templates, structured variable injection, and guardrail systems — enabling grounded, hallucination-resistant financial Q&A across global equities, US value, US growth, and emerging market debt portfolios Built real-time streaming SSE endpoints and citation-tracking systems returning source-attributed LLM responses with page-level references, improving analyst trust and auditability of AI-generated financial insights

Northeastern University

Graduate Research And Teaching Assistant

Northeastern University

LinkedIn
2023-5 - 2023-8 · 4 mos

Boston, Massachusetts, United States

Graduate Teaching Assistant for Course MSIM-6210 Information Visuals and Dashboards for Business • Assist in the review of course materials and content • Assist learners with identifying connections between academic learning and work experiences • Help in live sessions with materials, announcements, exhibits, and demonstrations • Provide support to students, faculty, and staff during live online class sessions • Interact in content-related forums as assigned • Engage learners through TA office hours and periodic engagement activities through the community platform • Provide individualized feedback to 50 learners on data visualization dashboards in PowerBI and Tableau, business analysis, SQL queries, and code reviews

Northeastern University

Intramural Official

Northeastern University

LinkedIn
2021-9 - 2023-8 · 2 yrs

Boston, Massachusetts, United States

• Ensuring the safety of games by conducting them in an orderly manner and enforcing the laws of the game • Providing training and mentorship to newer officials while presenting them with constructive feedback

Northeastern University

Graduate Teaching Assistant

Northeastern University

LinkedIn
2023-1 - 2023-4 · 4 mos

Boston, Massachusetts, United States

Graduate Teaching Assistant for MISM-6205 Data Wrangling in Business. • Led group and individual tutoring sessions to support students in their coursework and exam preparation. • Graded projects for students and shared best practices for the topics along with assisting them to solve complex problems.

Hosta AI

Data Engineer

Hosta AI

LinkedIn
2022-7 - 2023-1 · 7 mos

Boston, MA

• Automated AWS database provisioning and configuration using Terraform modules, cutting deployment time by 30% and ensuring uniform, error-free database setups. • Executed Python-based data analytics projects with Postgres, Looker, Docker, and ECS. Achieved a 20% reduction in response times, enhancing customer satisfaction. • Developed a serverless Slack bot with AWS Lambda functions to pinpoint build stage and codepipeline errors during deployment resulting in accelerated error resolution time by 20% through precise location identification. • Established CDC pipelines with Kafka and Debezium for real-time change tracking across production databases, integrated with Databricks and DBT for multi-layer data modeling • Orchestrated unified batch and streaming data processing on AWS Cloud using technologies such as Amazon EMR, Kinesis, and AWS Glue, ensuring a comprehensive and scalable approach to data analysis and insights.

Tata Consultancy Services

Senior Data Analyst

Tata Consultancy Services

LinkedIn
2019-6 - 2021-8 · 2 yrs 3 mos

Pune Division

• Designed and implemented a Medallion Architecture (bronze/silver/gold) for a US-headquartered insurance company, integrating customer databases and reducing downstream data quality issues" • Engineered and crafted 7+ impactful dashboards in Power BI and Tableau,effectively translated complex data into actionable visualizations, enhancing data-driven decision-making processes for multiple stakeholders. • Spearheaded detailed analysis, optimizing data integration workflows, identifying bottlenecks, and driving NoSQL(AWS Dynamo DB) enhancements for a remarkable 25% reduction in data processing time. • Simplified ETL tasks using SSIS, Apache Airflow, Kubernetes, S3 Object Storage, and Python, creating a modern data setup. Cut data processing time from 7 to 3 hours for over 50,000 daily records. • Implemented effective data validation and integrity processes, ensuring accurate data mining outcomes. Achieved a 95% data accuracy rate, improving the reliability of insights and recommendations. • Created a robust monitoring and alerting system using AWS CloudWatch and custom scripts, reducing unplanned downtime by 40% and improving incident response times. • Collaborated with 20+ member of cross-functional teams to create and implement data management policies and best practices, leading to improved data security and compliance.

World Wildlife Fund

Data Analyst Intern

World Wildlife Fund

LinkedIn
2018-10 - 2019-4 · 7 mos

Bhopal, Madhya Pradesh, India

• Applied comprehensive data analysis to combine multiple data sources and highlight key findings, resulting in a 10% improvement in KPIs. • Executed A/B testing on various visualization formats, resulting in a 15% performance improvement uplift in user engagement and awareness campaigns.

Muskaan Dreams

Data Analyst

Muskaan Dreams

LinkedIn
2016-1 - 2018-9 · 2 yrs 9 mos

India

Education

Northeastern University

Northeastern University

LinkedIn

Data Analytics Engineering

2021-8 - 2023-8 · 2 yrs 1 mo
Rajiv Gandhi Prodyogiki Vishwavidyalaya

Rajiv Gandhi Prodyogiki Vishwavidyalaya

LinkedIn

Information Technology

2015 - 2019 · 4 yrs

Akshay Dwivedi's Contact Information

Email

******@***.com

Phone

(**) *** ****

Find the Right Leads
Find Verified Contact Data

Try with: Jensen Huang @ nvidia.com Click to autofill
LeadContact awards, five-star ratings, and GDPR compliance badges

What LeadContact does well

Find verified emails, phone numbers, and decision-makers with 98% accuracy.

Find Leads

Find Leads

Find the right people by company, role, industry, location, and more.

925M+ professional profiles

Find Leads
Find Emails

Find Emails

Access verified email addresses for your target contacts.

657M+ emails

Find Emails
Find Phone Numbers

Find Phone Numbers

Get cross-validated phone data from multiple top sources.

239M+ phone numbers

Find Phone Numbers

More Accurate. Lower Cost.

Find contact data in 1 tool with 98% accuracy

LeadContact integrates leading enrichment tools to deliver more accurate contact data—without paying for each one.

LeadContact Logo
Competitor Tools

All these = $289 per month

Great conversations start with the right contact.

It’s time to find yours.