Training & Certifications

Data Warehouse with Pentaho Data Integration

This program presents an overview of data warehouse, multi-dimensional, ETL concept, the extensive use of Pentaho Data Integration to build a powerful data warehouse solution. The practical approach of this program involved many cases we face before, so that participants will get a highly valuable skills from attending the class.

Program Objectives

At the end of the program, the participants will be able to :

  1. Understand the concepts and topics of Data Warehouse, Dimensional Modeling, OLAP and ETL
  2. Use Pentaho Data Integration to build simple jobs / transformations
  3. Consume data from several data sources
  4. Build and Populating Fact and Dimensional tables
  5. Apply troubleshooting techniques
  6. Schedule job / transformation

Target Audience

This program is designed for those new to Data Warehouse and ETL or need to understand the basics of Pentaho Data Integration.

Duration

This program is 5 days of intensive training class.

Requirement

Basic understanding one of several popular Database Management System (Oracle, SQL Server, MySQL, etc.) and of Structured Query Language (SQL)

Program Modules

Module 1: Introduction to Data Warehouse
  1. Data Warehouse
    2. Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP)
  2. Data Warehouse and OLAP
  3. Delivering Solution with ETL (Extract, Transform, Load) Tool

Module 2: Installation and Configuration
  1. Java Runtime Environment / Java Development Kit
  2. Pentaho Data Integration
  3. XAMPP package (Apache HTTP Server and MySQL)
  4. SQLYog – a GUI based mysql client
  5. Data and Script samples

Module 3: Short Introduction to MySQL
  1. MySQL Storage Engines
  2. Administering MySQL via PHPMyAdmin
  3. PHI-Minimart sample database installation

Module 4: Pentaho Data Integration (Kettle)
  1. Introducing Kettle as Pentaho’s ETL Suite
  2. Architecture
  3. Components
    • Spoon : Graphical UI Designer for job / transformation steps
    • Pan : Command line batch script for transformation execution
    • Kitchen : Command line batch script for transformation execution
    • Carte : Cluster server
  4. Job / Transformation
    • Step and Hop
    • Row and Meta Data
    • Relation between job and transformation

Module 5: Getting Started with Spoon
  1. File system and RDBMS based Repository
  2. Spoon Development Environment
  3. Database Connections
  4. Job and Transformation
    • Creating job
    • Creating transformation
    • Calling transformation from job
  5. Configuring Log

Module 6: Multi Dimensional Modelling
  1. Normalized versus Multi Dimensional Model
  2. Fact and Dimension Tables
  3. Star Schema and Snowflake Schema
  4. Tasks:
    • Create a Kettle transformation to map PHI-Minimart transactional database sample to dimensional modeling database
    • Create logs for each steps

Module 7: Change Data Capture (CDC)
  1. What is CDC?
  2. Why CDC is so hard that heavily relied on data source?
  3. SQL Server 2008’s CDC feature demonstratin
  4. Tasks:
    • Create a Kettle transformation to map PHI-Minimart transactional database sample to dimensional modeling database
    • Create logs for each steps

Module 8: Slowly Changing Dimension (SCD)
  1. Slowly Changing Dimension to solve master data historical problems
  2. SCD Types
  3. Use of Kettle’s step to solve several SCD types with several schema:
    • Insert / Update
    • Punch Through

Module 9: Orphan / Late Arrival
  1. What is Late Arrival Dimension?
  2. Typical Situations where Late Arrival occurs
  3. Best practice of Late Arrival’s handling

Module 10: OLAP View of Multidimensional Data (Mondrian / JPivot)
  1. Mondrian Installation
  2. Creating scheme based on our fact and dimension tables
  3. View and navigate our Cube using Web Browser

Module 11: Data Staging
  1. What is Data Staging?
  2. Background : Physical I/O versus In-Memory Processing
  3. Task:
    • Create a transformation to join from 3 data sources : text file, Excel spreadsheet, and RDBMS
    • Create a currency staging table to solve sequential dependence problem

Module 12: Advance Controls
  1. Environment Variables
  2. Shared Objects
  3. Error Handling
  4. Email job results
  5. Task:
    • Create a dynamic tables dump using variable and looping control
    • Refining existing transformations to use email alert

Module 13: Automation
  1. Using Windows Task Scheduler to schedule ETL running job and transformation
Scroll to Top