D A T A E N G I N E E R
I am a passionate Data Engineer, having architected solutions using distributed computing frameworks like Spark, across cloud platforms such as AWS and GCP utilizing orchestration tools such as Kubernetes, Docker and Airflow. I have used Scala, Spark, and Python to create advanced components, optimize performance, and streamline processes within the realm of big data. I am on a journey to become a data steward by trying to understand the business aspect of data-driven decision-making by pursuing my master's in Data Science and Artificial Intelligence. I have achieved notable recognitions, including winning the Economic Times Campus Star competition and securing top positions in nationwide hackathons. I'm on the lookout for an exciting team and projects where can I make impactful contributions.
Data Engineer @ Accenture
Ahmedabad, Gujarat · Jan 2022 - Present
- Led a digital transformation project by migrating 39 AWS-backed Tableau Dashboards to GCP, replicating dashboards using complex SQL queries.
- Designed and built an encryption component for the platform using Scala and Spark, providing encryption and hashing functionalities.
- Developed a user-friendly web application's backend that automates the generation of Python Airflow DAGs, eliminating the need for manual code writing.
- Seamlessly integrated the Airflow REST API into the application, facilitating efficient communication and interaction with Airflow.
- Implemented comprehensive support for performing CRUD operations on DAGs across multiple environments, including local setups, Amazon MWAA, and Google Cloud Composer.
- Conducted extensive research and created multiple Python scripts for comparing and migrating RDS and Redshift databases, generating SQL statements for database creation, updates, and deletions.
- Demonstrated meticulousness in handling database transformations by ensuring coverage of all possible cases for ALTER queries.
- Developed and automated ETL pipelines for clients, meeting predefined KPIs.
- Skills: AWS Identity and Access Management (AWS IAM) · Amazon Elastic MapReduce (EMR) · Amazon S3 · Azure Data Factory · Java · Apache Kafka · Python · Apache Spark · SQL · Google Cloud Platform (GCP) · Amazon Web Services (AWS) · Scala · Apache Airflow · Data Engineering · Data Analytics
Software Engineer Intern @ HOPS Healthcare
Ahmedabad, Gujarat · Mar 2021 - Jun 2021
- Guided by Mr Vivek Patel, to develop a robust pipeline for extracting critical information from informal conversations.
- Led the creation of a user-friendly web application that empowered healthcare professionals to analyse and securely store reports within the healthcare system.
- Contributed to the initial design phase as well as the database design, and played a key role in developing foundational models.
- Designed and implemented a parsing bot utilising BioBERT and Regex techniques to automate the extraction of key elements from reports, streamlining the analysis process.
- Skills: Python (Programming Language) · SQL · Natural Language Processing (NLP) · Leadership · Django · BERT (Language Model) · Regular Expressions