Shweta Verma

As a dedicated Data Engineer in the dynamic field of Consulting, My expertise lies in leveraging the power of Databricks PySpark to streamline data workflows, enabling organizations to unlock valuable insights and drive informed decision-making. In my role, I actively engage with clients to understand their unique data requirements and challenges. I employ strong analytical and problem-solving skills to design and implement robust data pipelines on the Databricks platform. By utilizing PySpark, I harness the immense processing capabilities of Apache Spark to efficiently handle large-scale data processing and analysis tasks. Throughout my tenure, I have successfully collaborated with diverse clients from various industries, ranging from healthcare to retail. Whether it involves data ingestion, transformation, or modeling, I continuously strive to optimize workflows, enhance data quality, and improve overall performance. Additionally, I am proficient in designing and implementing scalable data architectures, ensuring the seamless integration of different data sources and systems. I have hands-on experience with data lake technologies, data warehousing, and cloud platforms, allowing me to create robust and flexible solutions that adapt to evolving business needs.

Roles and Responsibilities: • Responsible for Hadoop Development and Implementation including loading from disparate data sets. • Responsible for gathering client's requirement to prioritize business questions for the client for using the data effectively. • Worked on pre-processing data using Hive and Pig. • Developed framework to move the data from HDFS to Hive. • Orchestrated the flow of data from Hive to SQL Server using shell scripts. • Recommended suitable statergy for the most suitable data source to derive insights.

Roles and Responsibilities: • Architecting and Delivering projects for large Customers on Big Data Platforms. • Design and build Hadoop solutions for big data problems. • Developed MapReduce application using Hadoop, MapReduce programming and HBase. • Involved in developing the Pig scripts • Involved in developing the Hive Reports. • Developed the SQOOP scripts in order to make the interaction between Pig and MySQL Database.

Worked on : Content Based Image Retrieval (CBIR) The project involves CBIR technique is used to search for the images from large scale databases through the visual contents of the images such as color, shape, texture and spatial layout to represent and index the image. The visual contents are extracted and the similar images are searched in the feature database using image retrieval system.