Data analytics and Python for data preparation

Overview

Framework:
RQF
Level:
Level 4
Unit No:
F/651/7442
Credits:
26
Guided learning hours:
236 hours

Aim

This unit introduces learners to the essential principles of data analytics and the practical use of Python for preparing data for analysis and visualisation. Learners will explore how data is collected, structured, and understood through key analytical concepts, and will develop hands-on skills using Python to clean, transform and prepare data sets for further insight. By the end of the unit, learners will be able to apply Python tools to carry out basic data preparation tasks and demonstrate a foundational understanding of how analytics supports evidence-based decision-making in real-world contexts.

Unit Learning Outcomes

1

Be able to analyse data analytics applications.

Assessment Criteria

  • 1.1

    Analyse relevant applications of data analytics in the chosen data domain, demonstrating a breadth of understanding.   

    Applications of data analytics in chosen domains (AC 1.1):

    • Business domain: customer segmentation, churn prediction, sales forecasting, market basket analysis, A/B testing, pricing optimisation
    • Healthcare domain: patient outcome analysis, disease prediction models, treatment effectiveness analysis, resource allocation optimisation
    • Financial domain: fraud detection, credit risk assessment, portfolio optimisation, algorithmic trading strategies
    • Retail domain: inventory management, demand forecasting, recommendation systems, customer lifetime value analysis
    • Manufacturing domain: predictive maintenance, quality control analysis, supply chain optimisation, production efficiency monitoring
    • Marketing domain: campaign effectiveness measurement, attribution modelling, customer journey analytics, sentiment analysis

  • 1.2

    Explain how data analytics and AI can address specific challenges or opportunities.

    Challenges and opportunities addressed by data analytics and AI (AC 1.2):

    Challenges addressed: data quality issues (missing values, outliers, inconsistencies), data silos and integration problems, scalability issues with large datasets, real-time processing requirements, interpretability of complex models, privacy, and security concerns

    Opportunities enabled: automated decision-making, predictive insights for proactive strategies, personalisation at scale, operational efficiency improvements, new revenue streams through data monetisation, competitive advantage through data-driven innovation, cost reduction through process optimisation

2

Be able to use Python and data science tools.

Assessment Criteria

  • 2.1

    Use Python and data science tools to manipulate and analyse data.

    Using Python and data science tools to manipulate and analyse data (AC 2.1):

    • Data manipulation with pandas: DataFrame creation and indexing, merging/joining datasets, groupby operations and aggregations, pivot tables and melt, handling missing data (dropna, fillna), data type conversions and categorical data handling
    • Numerical computing with NumPy: array operations and built-in methods, statistical functions (mean, median, std), indexing and slicing, numpy image analysis
    • Data analysis workflows with Python DA packages: feature engineering, data imputation, categorical encoding, discretising, transformations and pipelines
    • Integration with other tools: reading/writing various file formats (txt, CSV, Excel, JSON, SQL), API data retrieval with requests library, and Kaggle API

  • 2.2

    Review and evaluate Python code and queries.

    Reviewing and evaluating Python code and queries (AC 2.2):

    • Code quality assessment: Linter compliance checking, code readability and documentation standards, variable naming conventions, function design and modularity, DRY (Don't Repeat Yourself) principles
    • Testing and validation: PEP8, clean code, try-except, pdb, debuggers, AI pair programming
    • Version control review: Git, GitHub and code reviews, pull request best practices, commit message standards, and branching strategies evaluation

  • 2.3

    Apply coding techniques and tools for effective data analysis.

    Applying coding techniques and tools for effective data analysis (AC 2.3):

    • Advanced pandas techniques: method chaining for readable pipelines, lambda functions, custom functions with apply, query and visualisation
    • Visualisation techniques: matplotlib for technical plots, seaborn for non-technical visualisations, and plotly for interactive dashboards.
    • Development environment tools: Jupyter Notebooks, VS Code with Python extensions, virtual environments (venv), package management with pip, reproducible environments with requirements.txt

3

Be able to use integrated development environments (IDE).

Assessment Criteria

  • 3.1

    Integrate AI tools into data analysis workflows

    Integrating AI tools into data analysis workflows (AC 3.1):

    • AI-powered code assistance: GitHub Copilot for code completion and generation, Generative AI for debugging assistance
    • Workflow integration strategies: prompt engineering for code generation, AI tool selection criteria, validation of AI-generated code, combining AI suggestions with domain expertise, and documentation of AI-assisted development

  • 3.2

    Apply generative AI for storytelling with data

    Applying generative AI for storytelling with data (AC 3.2):

    • Narrative generation: using gen AI tools to create data story outlines, generating executive summaries from analysis results, creating audience-appropriate explanations, and translating technical findings to business language
    • Visualisation enhancement: AI-assisted chart selection, automated insight highlighting, generating visualisation descriptions for accessibility, and creating infographic layouts with AI tools
    • Interactive storytelling: Streamlit apps with AI-powered explanations.
    • Communication strategies: tailoring complexity to audience expertise, generating multiple narrative versions, creating data-driven presentations, building interactive reports with AI explanations

  • 3.3

    Evaluate analysis outcomes, identify limitations, and consider alternatives

    Evaluating analysis outcomes, identifying limitations, and considering alternatives (AC 3.3):

    • Outcome evaluation: comparing results against business objectives, validating findings with domain experts, assessing practical importance, checking for consistency across different approaches
    • Limitation identification: data quality limitations and their impact, model assumptions and violations, generalisation boundaries, ethical considerations and biases
    • Alternative approaches: comparing different analytical methods, exploring various visualisation techniques, assessing trade-offs between complexity and interpretability, considering different tools and technologies, evaluating build vs. buy decisions
    • Critical reflection practices: documenting decision rationale, maintaining analysis logs, peer review processes, lessons learned documentation, and continuous improvement strategies

4

Be able to demonstrate effective data management practices.

Assessment Criteria

  • 4.1

    Apply and assess effective techniques for collecting, cleaning, storing, and processing data

    Applying and assessing effective techniques for collecting, cleaning, storing, and processing data (AC 4.1):

    Data collection techniques:


    • Primary data collection: Primary Methods: Transactional Data Collection, Surveys, Questionnaires, Interviews, Observations, Secondary Sources:
    • Secondary data sources: Databases, APIs, Public Datasets, Industry Reports, Censuses, Social Media and Online Content

    Data cleaning techniques:


    • Quality assessment: data profiling and quality metrics, missing data pattern analysis (is NAN), outlier detection methods (Winsorizer, ArbitraryOutlierCapper, OutlierTrimmer), duplicate identification and resolution, consistency checking across fields
    • Cleaning operations: imputation strategies (mean, median, arbitrary, categorical, drop), standardisation and normalisation techniques, one-hot, ordinal and rare label encoding of categorical variables, handling date/time formats, text cleaning and standardisation

    Data storage solutions:

    Database systems: relational databases (PostgreSQL, MySQL) for structured data, NoSQL databases for unstructured data

    Data processing techniques:

    Batch processing: ETL pipeline design and implementation

  • 4.2

    Justify best practices for data handling and processing

    Justifying best practices for data handling and processing (AC 4.2):

    Data governance best practices:

    • Documentation standards: Industrial data standards, data culture, GS1, HL7, HIPAA, ISO20022, EDIFACT, X12, ISO10303 or STEP.
    • Privacy and compliance: GDPR compliance implementation (consent, right to erasure, data minimisation), Privacy by Design principles, data anonymisation and pseudonymisation techniques, audit trail maintenance, data retention policies
    • Security practices: encryption at rest and in transit, access control and authentication, data masking for sensitive information, secure data sharing protocols, and incident response procedures

    Quality assurance frameworks:

    • Data quality dimensions: completeness (missing value thresholds), consistency (cross-field validation rules), accuracy (source verification methods), timeliness (freshness requirements), uniqueness (deduplication strategies), validity (format and range checks)
    • Monitoring and alerting: automated quality checks in pipelines, SLA definition and tracking, anomaly detection in data flows, dashboard creation for quality metrics, escalation procedures for quality issues