Data preparation, analysis and visualisation

Explain the fundamental concepts of data analytics, including its definition and how it differs from related fields such as business intelligence, and data science. (AC 1.1):

• Data analytics is the process of examining data sets to uncover hidden patterns, correlations, trends, and insights.
• It involves digging into the data to reveal insights that may not be immediately apparent.
• The purpose of data analytics is to transform raw data into actionable insights that inform strategic decision-making and drive business success.
• Business Intelligence differences: BI's focus on converting data into actionable intelligence for real-time decision-making. They develop dashboards and reports to display key metrics and performance indicators, using tools like Power BI and SQL to provide executives with a real-time view of the organisation's performance, enabling them to make informed decisions quickly.
• Data Science distinctions: It involves using advanced techniques such as machine learning, statistical modelling, and predictive analytics to uncover patterns and forecast future trends.

Evaluate its applications across various business sectors. (AC1.2):

• Retail and E-commerce: sales forecasting, managing customer relationships, and setting pricing strategies. By analysing historical sales data, customer behaviour, and market trends, companies can predict future sales patterns, manage inventory, tailor sales strategies to different customer segments, and dynamically adjust pricing to maximise revenue.
• Healthcare and Life Sciences: patient readmission risk prediction, treatment pathway optimisation, resource allocation and capacity planning, drug discovery patterns, population health management, medical imaging analysis
• Financial Services: credit scoring and loan default prediction, anti-money laundering and fraud detection, customer churn prediction, portfolio risk analysis, regulatory compliance monitoring, algorithmic trading strategies
• Manufacturing: predictive maintenance using IoT data, quality control and defect prediction, supply chain optimisation, production scheduling, energy consumption analysis, supplier performance analytics 

Summarise the key characteristics of Big Data. (AC 1.3):

• Big Data is characterised by its large volume, high velocity, and diverse variety.
• It’s used to effectively manage and analyse large, complex datasets by understanding their characteristics.

Explain the importance of each stage in the Data Analytics Life Cycle. (AC 2.1):

Problem Statement Formulation:

• What is the central problem?
• Who experiences the impact of this issue?
• What is the intended resolution?
• Are there any quantifiable objectives that need to be achieved?
• What steps are required for implementation?

Data Collection

• Primary Methods:

1. Transactional Data Collection
2. Surveys
3. Questionnaires
4. Interviews
5. Observations

• Secondary Sources:

1. Databases
2. APIs
3. Public Datasets
4. Industry Reports
5. Censuses
6. Social Media and Online Content

Data Cleaning

1. Handling Missing Values
2. Correcting Inconsistencies
3. Removing Duplicates
4. Identifying and Removing Outliers
5. Integrating Data

Exploratory Data Analysis (EDA):

• Summarises and visualises data to understand its structure and patterns using descriptive statistics, charts, and graphs to identify basic trends and relationships.
• Trend Analysis:

1. Identifies trends over time (e.g., sales, customer traffic, product popularity) to pinpoint periods of decline and assess the impact of various factors.

• Correlation Analysis:

1. Examines relationships between variables (e.g., marketing campaigns and sales, demographics and purchasing) to understand mutual influence.

• Segmentation Analysis:

1. Group customers by characteristics (e.g., purchase behaviour, visit frequency, demographics) to reveal insights into sales drivers and hindrances.

• Predictive Analysis:

1. Uses statistical models or machine learning to forecast future sales trends based on historical data and predict the impact of changes.

• Data Visualisation:

1. chart selection based on message, dashboard design principles, colour theory and accessibility, interactive elements, storytelling techniques, and annotation strategies

Evaluate how each stage of the Life Cycle contributes to deriving actionable insights and making informed decisions. (AC 2.2):

• Insight Generation:

1. pattern recognition (trends, cycles, anomalies), comparative analysis and benchmarking, predictive indicators' identification, root cause analysis techniques, correlation vs causation understanding

• Decision Support:

1. decision trees and scenario planning, risk-reward analysis, what-if analysis capabilities, recommendation engines, performance monitoring systems

• Value Realisation:

1. ROI calculation methodologies, impact measurement, continuous improvement cycles (PDCA), success story documentation, stakeholder satisfaction measurement

Compare and evaluate quantitative and qualitative data using an evaluation framework (AC 3.1:)

• Quantitative data refers to information that can be measured and expressed in numerical terms. It’s the objective side of data, used for statistical analysis. It enables precise comparisons and identifies trends. Examples include height, temperature, and revenue. It can be discrete data, representing values that can take on only specific, distinct values. These are typically whole numbers. For instance, the number of employees in a company, the number of products sold, or the number of cars on the road. Continuous Data is more fluid. It can take any value within a range and be infinitely divisible. Continuous data flows smoothly, enabling precise analysis.
• Qualitative data: Describes and categorises rather than measures. For example, attributes and characteristics. Social media posts or survey response data help us understand the qualitative aspects of our subject.
• Nominal versus Ordinal. Nominal Data is categorisation without any order. E.g. gender, types of fruit, or nationality. Ordinal Data has a clear order or ranking. For example, education levels or customer satisfaction ratings. 

Classify data as structured, unstructured, or semi-structured. (AC 3.2):

• Structured Data: Data neatly organised in a fixed schema, often displayed in tabular form. Easily searchable and analysable, for quick and precise data retrieval. For example, databases or spreadsheets.
• Unstructured Data: No predefined structure and comes in formats like text, images, and videos. Social media posts, emails, and videos are all examples.
• Semi-structured Data: This format doesn’t adhere to a fixed schema but uses tags or markers to organise elements. XML files and JSON documents are examples that provide some structure while allowing for varied data representation.

Assess the significance of each data type for effective data analysis and interpretation. (AC 3.3):

• Analysis Method Impact:

1. Structured data databases and spreadsheets enable efficient data management and streamlined analysis.
2. Unstructured data captures rich, detailed information, offering deep insights that structured formats might miss.
3. Semi-structured data strikes a balance between flexibility and organisation.

• Interpretation Challenges: Understanding the structure of data is crucial for managing, storing, and analysing it, as well as for selecting the right tools and methods for decision-making.

Explain and assess key data analytics and visualisation tools. (AC 4.1):

• Tools

1. Excel, Power BI and Tableau

• Techniques

1. Describe a dataset with statistics.
2. Use DAX to aggregate, apply conditional logic, and filter data for analysis and reporting.
3. Use BI tools to visualise data.

• Significance

1. Understanding business requirements is crucial for aligning our work with organisational goals and delivering actionable insights.
2. Interactive, DAX-powered dashboards enable stakeholders to visually explore data, uncover trends, and make informed decisions. DAX facilitates dynamic metric calculations (e.g., Total Sales, Total Quantity, Average Order Value), ensuring accurate and flexible analysis.
3. Building adaptive dashboards provides tailored insights, empowering decision-makers to act confidently on the data.

Demonstrate understanding of setup and installation processes for essential software, equipping you to effectively use these tools for data analysis and visualisation. (AC 4.2):

• Emphasise step-by-step installation, including:

1. Checking system requirements
2. Downloading installers from official sources
3. Completing installation wizards
4. Understanding directory paths and where files are stored
5. Configuring first-time setup options
6. Organise project directories
7. Use consistent naming conventions
8. Manage data files securely and ethically (link to data governance)
9. Ensure local settings don’t violate security policies 

Classify key types of data visualisations and consider their appropriate use cases. (AC 5.1):

Comparison Visualisations:

• Used to compare categories or metrics.
• Examples:

1. Bar chart
2. Column chart
3. Stacked bar
4. Dot plots

• Use case examples:

1. Comparing sales performance by product
2. Staffing levels by department

Trend / Time Series Visualisations

• Used to show change over time.
• Examples:

1. Line chart
2. Area chart

• Use case examples:

1. Monthly revenue trends
2. Daily website traffic

Composition Visualisations

• Used to show how parts contribute to a whole.
• Examples:

1. Pie chart (with caution)
2. Donut chart
3. Treemap
4. Stacked area chart

• Use case examples:

1. Market share by region
2. Distribution of booking sources

• Encourage learners to critically evaluate when NOT to use pie charts.

Demonstrate the use of appropriate software tools to create basic visualisations from given datasets. (AC 5.2):

• Tools

1. Power BI Desktop and Tableau Public

• Techniques

1. Apply the three essential elements of data storytelling: Data, Visuals, and Narrative.
2. These elements provide a structured approach for verifying trends, analysing key data points, and communicating insights clearly.

• Match the visualisation type with the data type and the message to be conveyed.

Evaluate the advantages and limitations of advanced visualisation techniques compared to basic ones, and draw conclusions about how they enhance data storytelling. (AC 5.3):

• Storytelling is the "why" behind the data, empowering users to make informed and impactful decisions.
• Data provides factual insights and forms the foundation of our story. Visuals, such as charts and dashboards, simplify complex information, making trends and patterns easier to interpret.
• Narrative connects data points into a logical, engaging storyline, adding relevance and meaning to the insights.
• Identify trends by analysing historical data to verify patterns, such as increases in product returns. After confirming the trend, analyse key data points, such as customer feedback or demographic data, to uncover root causes.
• Construct a narrative that clearly presents insights and smoothly transitions from problem identification to actionable recommendations.

Explain how effective data handling contributes to accurate analysis and model building. (AC 6.1):

• Standardisation ensures consistency across our dataset.
• Cleaning offers a straightforward approach to organising and formatting data.
• Duplicate removal ensures each entry is counted only once.
• Removing outliers is a critical step for maintaining reliability, which enhances efficiency and clarity in our analysis.
• Feature engineering is transforming our dataset into a robust source for analysis.

Demonstrate proficiency in applying data cleaning, feature engineering, and data transformation techniques. (AC 6.2):

• Power Query, Tableau Online and Excel.
• Excel provides an accessible platform ideal for basic cleaning tasks and initial DAX data exploration.
• Power Query and Tableau Online enable a seamless transition from data cleaning to visualisation, allowing us to analyse and report directly within the same platform.

Classify different data preprocessing methods according to their purpose and function. (AC 6.3):

• Standardise data formats, names, and categories.
• Handle missing values by deleting, imputing, or replacing values to fill gaps accurately.
• Remove duplicate entries to maintain data accuracy.
• Manage outliers to prevent any skewed analysis.
• Feature engineering to generate new insights by creating additional data fields. 

Use advanced visualisation methods to analyse complex data (AC 7.1):

Distribution Visualisations

• Used to show frequency patterns and variability.
• Examples:

1. Histogram
2. Boxplot
3. Density plot

• Use case examples:

1. Customer age distribution
2. Variation in delivery times

Relationship Visualisations

• Used to show correlations and interactions between variables.
• Examples:

1. Scatter plot
2. Bubble plot
3. Heatmap

• Use case examples:

1. Relationship between price and demand
2. Correlation between marketing spend and leads

Assess the effectiveness of visualisation techniques (AC 7.2):

• Visualisations simplify complex data, making it more accessible and actionable.
• A well-structured dashboard ensures that our visualisations are not only professional but also easy to navigate and interpret, delivering maximum value to users. 

Create interactive visuals that communicate insights to targeted audiences (AC 7.3):

• Slicers and filters are interactive tools that help us focus on specific details and customise our analysis.
• Dashboards are insightful, interactive, and aligned with organisational objectives.

• Business requirements are a critical first step in any analysis.
• Build interactive dashboards that bring analysis to life.
• Knowledge application in the workplace.
• DAX enables the dynamic calculation of metrics such as Total Sales, Total Quantity, and Average Order Value. 

Clarify and communicate complex technical information in a way that is accessible to both technical and non-technical stakeholders. (AC 8.2):

• Building interactive dashboards allows you to provide tailored insights to stakeholders.
• Dashboards that adapt to user inputs make data exploration intuitive, empowering decision-makers to uncover trends and act confidently on the insights you provide.

Demonstrate the use of meaningful visualisations and generate insights from data using appropriate statistical concepts. (AC 8.3):

• DAX calculations ensure your analysis is both accurate and flexible.
• Dashboards, powered by DAX, enable stakeholders to visually explore data.

Draw conclusions from their analysis and justify an informed business recommendation based on the evidence. (AC 8.4):

• Aligning work with organisational goals ensures that key challenges are addressed and actionable insights delivered.
• Dashboards, powered by DAX, enable stakeholders to uncover trends and make informed, data-driven decisions.