'Data Engineering using Microsoft Azure' workshop hosted by SMCS and VentureDive

July 2-3, 2024: The School of Mathematics and Computer Science (SMCS) at the Institute of Business Administration (IBA), in collaboration with VentureDive, organized a two-day workshop titled ‘Data Engineering using Microsoft Azure’. Numerous IBA students from both the School of Business Studies (SBS) and SMCS participated.

The workshop featured esteemed experts from VentureDive, including M. Salman Ahsan, Lead Data Engineer; Sehrish Muhammad, Lead Data Engineer; Karar Ali, Senior Data Engineer; and M. Ibrahim Irshad, Business Intelligence Lead.

The workshop commenced with a dynamic introduction to the world of data engineering, cloud computing and core data concepts highlighting its crucial role in modern businesses that thrive on data-driven decision-making. Key topics included the role of a Data Engineer, Azure Data Services, and Microsoft Azure Overview.

Hands-on sessions enabled participants to apply their knowledge. They engaged in creating Azure Storage Accounts, configuring Blob Storage and Data Lake Storage, managing data, and implementing security features to protect data integrity and privacy. Another hands-on session focused on data ingestion and processing, where participants created data pipelines, ingested data from various sources into Azure, and processed data using Azure Databricks and Apache Spark.

Day-two of the workshop focused on comprehensive exploration of data warehousing on Azure. Participants learned about Azure Synapse Analytics, designing data warehouses, and query optimization. Through a hands-on session, participants engaged in practical exercises to reinforce their learning.

An activity on data set analysis and business insights challenged participants with real world scenarios, allowing them to enhance their analytical skills. This was followed by a session on data analysis and visualization basics, where participants learned fundamental concepts, explored tools like Azure Synapse Studio and Power BI, and received practical tips for designing effective charts and graphs. The workshop concluded with an overview of Large Language Models (LLMs), introducing participants to innovative AI technologies and showcasing real-world applications of LLMs across various industries.

The two-day workshop was a comprehensive journey through the landscape of data engineering and Microsoft Azure. Through a blend of theoretical sessions and hands-on activities, participants gained a solid foundation in data storage, ingestion, processing, warehousing, and analysis.