Atricle Dump
#1 in Business Subscribe Email Print

You are here: Home > Business > Management > Components of a Data Warehouse Architecture - Part 1, ETL and the Staging Area

Tags

  • crucial
  • getting
  • quality assessment
  • mentioned above
  • international statistics

  • Links

  • 6 Reasons Why Phentramin Will Make You Lose Weight
  • How To Start a Lawn
  • How and Why Skin Ages
  • Atricle Dump - Components of a Data Warehouse Architecture - Part 1, ETL and the Staging Area

    The Evolution of Magazine
    Magazine has come a long way since it started being published in the US less than three centuries ago. Even Benjamin Franklin published one himself.But published magazines then only lasted for a few months. This is because publishers are limited geographically and the cost of magazine printing could be quite expensive. A few people could afford getting a subscription. And the target of the publishers were the “educated class” because most magazines before were abo
    stimated volume
  • standardisation and transformation rules applied (if any)
  • DW database field to which data will be loaded
  • In many cases, data quality assessment and data cleansing steps also take place in the staging area. Design and implementation of the automated ETL process, often represents a major part of the man effort to develop a DW (international statistics estimate that it exceeds
    Metrics to Measure and Control the Performance of Sales Department
    The salespeople and company sales force are those who actually generate your income dollars. Sure, it's important to have a good product, support which can respond fast, pretty design and make people write about your company in magazines, but still the most important part is to get the resulted leads converted into actual dollars.It's quite easy to organize the sales department. For instance, you can hire someone who have experience in managing this kind of depart
    Decision support systems are usually based on the development of Data Warehouse infrastructures. A data warehouse (hereafter DW) architecture has two major areas: the staging area and the presentation area. In this article we present the staging area. The sources from which data shall be systematically extracted, in order to be loaded in the DW, are determined. The database schema documentation of these sources, is reviewed in order to design the data extraction logic. Documentation quality of the data structures of these sources, influences the degree of difficulty in designing the data extraction logic. Data extracted are loaded in the staging area, either as simple files or as updates in database tables. The staging area may have various stages. Extraction of data from sources, transformation of data into new structures and data loading in the DW, a process known as ETL, takes places in the staging area.

    The extraction process requires the determination of source relational tables – fields, from which data shall be extracted (as mentioned above, documentation of these structures is crucial for design). The design of the extraction process determines:

    • the frequency of data extraction
    • the extraction method (e.g. changes only) and technology (database partial replication)
    • the database instance or the file in which data are initially loaded, in the staging area
    Moreover, the volume of data to be extracted is estimated, in order to plan for computational & storage capacity. Estimation sheets known as ‘volumetric sheets’ are developed with the following information per source field:

      • extraction frequency
      • estimated volume
      • standardisation and transformation rules applied (if any)
      • DW database field to which data will be loaded
    In many cases, data quality assessment and data cleansing steps also take place in the staging area. Design and implementation of the automated ETL process, often represents a major part of the man effort to develop a DW (international statistics estimate that it exceeds
    Host a Successful Fundraising Event with Inflatable Rentals
    If your organization or school is looking for a new and exciting way to raise funds, try hosting a carnival with inflatable rentals. With inflatable obstacle courses, inflatable sumo wrestling, inflatable bungee basketball and many other fun inflatable games, setting up and taking down a carnival doesn’t need to be a big production. It’s well within your organization’s reach to net several thousand dollars over a night or a weekend with inflatable rentals and a carniva
    ewed in order to design the data extraction logic. Documentation quality of the data structures of these sources, influences the degree of difficulty in designing the data extraction logic. Data extracted are loaded in the staging area, either as simple files or as updates in database tables. The staging area may have various stages. Extraction of data from sources, transformation of data into new structures and data loading in the DW, a process known as ETL, takes places in the staging area.

    The extraction process requires the determination of source relational tables – fields, from which data shall be extracted (as mentioned above, documentation of these structures is crucial for design). The design of the extraction process determines:

    • the frequency of data extraction
    • the extraction method (e.g. changes only) and technology (database partial replication)
    • the database instance or the file in which data are initially loaded, in the staging area
    Moreover, the volume of data to be extracted is estimated, in order to plan for computational & storage capacity. Estimation sheets known as ‘volumetric sheets’ are developed with the following information per source field:

      • extraction frequency
      • estimated volume
      • standardisation and transformation rules applied (if any)
      • DW database field to which data will be loaded
    In many cases, data quality assessment and data cleansing steps also take place in the staging area. Design and implementation of the automated ETL process, often represents a major part of the man effort to develop a DW (international statistics estimate that it exceeds
    4 Ways To Make This Year Your Most Profitable Ever
    1. Establish Key Performance IndicatorsThese Key Performance Indicators should measure a variety of financial areas in your business e.g. value of an average transaction, cost per sale, profit margins, cost per inquiry, the lifetime value of a client etc. In this way you will have some very clear yardsticks or benchmarks on which you can base any future profit enhancement initiatives.Write down 4 initiatives against each Key Performance Indicator that you
    ing in the DW, a process known as ETL, takes places in the staging area.

    The extraction process requires the determination of source relational tables – fields, from which data shall be extracted (as mentioned above, documentation of these structures is crucial for design). The design of the extraction process determines:

    • the frequency of data extraction
    • the extraction method (e.g. changes only) and technology (database partial replication)
    • the database instance or the file in which data are initially loaded, in the staging area
    Moreover, the volume of data to be extracted is estimated, in order to plan for computational & storage capacity. Estimation sheets known as ‘volumetric sheets’ are developed with the following information per source field:

      • extraction frequency
      • estimated volume
      • standardisation and transformation rules applied (if any)
      • DW database field to which data will be loaded
    In many cases, data quality assessment and data cleansing steps also take place in the staging area. Design and implementation of the automated ETL process, often represents a major part of the man effort to develop a DW (international statistics estimate that it exceeds
    December's Marketing Magic for New Year Success
    Can you believe it’s almost Christmas? The holidays are here and then the start of a new year will be upon us. Now is a perfect time to reflect on your business and see if it is all that it can be. Is it running at full speed and do you have all the clients that you need?If not, it’s time to get those engines moving and start into action before the New Year begins. I know, it’s the holidays and there’s a lot’s going on. But also it’s the perfect opportunity to
    technology (database partial replication)
  • the database instance or the file in which data are initially loaded, in the staging area
  • Moreover, the volume of data to be extracted is estimated, in order to plan for computational & storage capacity. Estimation sheets known as ‘volumetric sheets’ are developed with the following information per source field:

      • extraction frequency
      • estimated volume
      • standardisation and transformation rules applied (if any)
      • DW database field to which data will be loaded
    In many cases, data quality assessment and data cleansing steps also take place in the staging area. Design and implementation of the automated ETL process, often represents a major part of the man effort to develop a DW (international statistics estimate that it exceeds
    Getting Your Business To Fly The Flourishing Way
    Is your business still sitting on the runway? We call a business that can fly a Flourishing Business. If you follow the Flourishing Business Methodology you can get your business to really take off. There are tried and true methods for creating a successful business. These include structured sales and marketing, efficient employees, loyal customers, focusing your business plan, systematic processes, positive cash flow and cultivating value in your marketplace.
    stimated volume
  • standardisation and transformation rules applied (if any)
  • DW database field to which data will be loaded
  • In many cases, data quality assessment and data cleansing steps also take place in the staging area. Design and implementation of the automated ETL process, often represents a major part of the man effort to develop a DW (international statistics estimate that it exceeds 70% of total effort). The DW staging area, is often implemented in a separate physical server (staging server), thus adding complexity and cost. However, this approach has certain advantages like:

    • isolation of raw data which are extracted from sources, from processed data which are accessible by business analysts
    • additional security and process quality, given that DW users have no access in this area
    • load sharing, given that ‘data preparation’ tasks and DW querying tasks are handled by separate systems.
    • development of a central metadata repository which maintains documentation for all involved systems: operational systems (data sources), ETL process, data warehouse, BI tools and predefined reports
    Various types of raw data processing, take place at the staging area:
    • Data standardization: data transformation to a standard format, if needed
    • Sorting of records
    • Matching and merging records of the same entity, which are derived from different sources (e.g. order records of the same Customer from different order handling systems), after standardization
    • Processing of calculated facts (facts derived from detailed data e.g. total monetary value of an order)
    • Management of surrogate keys, which replace operational systems keys
    • Enrichment of records with default values, if required
    • Production of aggregate data, if needed
    • Data conversion according to the technological platform used by the DW (DBMS, operating system)
    The ETL process is automated by software and executed periodically to update the DW. Copyright 2006 – Kostis Panayotakis

    HTTP = HTML link (for blogs, profiles,phorums):
    <a href="http://www.articledump.net/article/21654/articledump-Components-of-a-Data-Warehouse-Architecture--Part-1-ETL-and-the-Staging-Area.html">Components of a Data Warehouse Architecture - Part 1, ETL and the Staging Area</a>

    BB link (for phorums):
    [url=http://www.articledump.net/article/21654/articledump-Components-of-a-Data-Warehouse-Architecture--Part-1-ETL-and-the-Staging-Area.html]Components of a Data Warehouse Architecture - Part 1, ETL and the Staging Area[/url]

    Related Articles:

    Where to Advertise Your Small Business Online…Without Paying a Penny

    Ten Employability Skills For 2010

    Three Myths About Resume Writing

    Bookmark it: del.icio.us digg.com reddit.com netvouz.com google.com yahoo.com technorati.com furl.net bloglines.com socialdust.com ma.gnolia.com newsvine.com slashdot.org simpy.com shadows.com blinklist.com