DI04000071 · Semester 4 · 5G scheme
Introduction to Data Analysis
Notes and solutions following the Gujarat Technological University (GTU) syllabus for DI04000071. Independent material — not issued by the university.
Computer EngineeringComputer Science & Engineering
Materials
Complete Book
The whole syllabus, unit by unit, written for the exam.
Complete Book
106 pages
Premium Preview · 11 pages
PYQ Solutions
Past university papers, fully solved.
Solved past papers — detailed answers
31 pages · English + ગુજરાતી
Premium Preview · 3 pages
Syllabus units
Unit 1: Introduction to Data Analysis
- Sources and nature of data, classification of data (structured, semi-structured, unstructured)
- Characteristics of data
- Introduction to Big Data platform
- Need of data Analysis
- Evolution of analytic scalability
- Analytical process
- Analysis vs. reporting
- Modern data analysis tools
- Applications of data analysis.
- Key roles for successful analysis
- Various phases of data analytics lifecycle - discovery, data preparation, model planning, model building, communicating results, and operationalization
Unit 2: Python libraries for Data Analysis and Data extraction
- NumPy - Difference between array and list, N dimension array - 1D array, 2D array, 3D array, Zeros matrix, Ones matrix, Identity matrix, Reshape, Working with random number, Stacking - Vertical stacking, horizontal stacking, Working with RGB Image, image as a numpy array.
- Pandas - Working with Dataframes Read csv and xlsx file, Analyze the basic dataset characteristics, Perform different merge and sort operations with multiple dataframes. Handle missing values in Dataframe, Analyze the DataFrame with loc and iloc, nlargest(), nsmallest(), add or remove an attribute from the DataFrame.
- Reading Files
- Scraping the Web - Purpose, Legality and Ethical Considerations, Overview of popular libraries (Beautiful Soup, Requests, Selenium)
- Beautiful Soup (Purpose and Use Cases, Parsing HTML, Working with HTML tags, Accessing tag attributes, Simple HTML Parsing Example, Extracting Data from Web Pages)
- Requests (Basics of sending GET/POST requests, accessing response content, and authenticating with Requests)
- Cleaning and Munging
- Rescaling
- Data Normalization and Transformation
- Dimensionality Reduction
Unit 3: Statistical Analysis
- Regression modeling
- Multivariate analysis
- Apply basics of descriptive statistics including measures of central tendency such as mean, median, and mode
- Different correlation techniques: Pearson's Correlation Coefficient
- Methods of Least Squares
- scatterplots and other graphical techniques to identify the correlation between variables
- Different probability distributions such as Normal, Poisson, Exponential, Bernoulli Definition and Importance
Unit 4: Data Visualization
- Introduction to Data Visualization
- Importance of Data Visualization
- Basic Data Visualization with matplotlib
- Customizing matplotlib plots
- Data visualization with Seaborn
- Interactive visualization with Plotly
- Time series data visualization
- Advance plots: Yiolin plots, Box plots
Unit 5: Generative AI and Big Data Visualization with Power BI
- Recent trends in data analysis, predictive vs generative AI
- Generative AI in Analytics: synthetic data, automated dashboards, natural language queries, storytelling with data, AI copilots in BI
- Big Data: Visualizing Big Data, Pre-attentive Attributes, Challenges of Big Data Visualization
- Power BI: Data transformation & summarization, Dashboards: bar, line, pie charts, AI visuals: Q&A, decomposition tree, key influencers, Visualizing large datasets & big data considerations
- Generative AI Challenges & Solutions: bias, fairness, privacy, security
About this course
Study materials, question papers, and solutions for Introduction to Data Analysis (DI04000071) - Diploma Engineering, Semester 4