CDW110 EXAM 2023-2024 ACTUAL EXAM 200 QUESTIONS AND CORRECT DETAILED ANSWERS|AGRADE
CHAPTER 1-Caboodle Console
The Caboodle Console is a web application housed on the Caboodle server. It includes the following:
Dictionary
Dictionary Editor
Executions
Work Queue
Configuration
CHAPTER 1-Data Warehouse
In a data warehouse, multiple sources may load data pertaining to a single entity. This means that more than one package may populate a given row in a Caboodle table. As a result, there may be multiple business key values associated with a single entity in a Caboodle table.
CHAPTER 1-ETL
Extract, Transform, Load
CHAPTER 1-SSIS Package
The architecture of Caboodle includes a staging database and a reporting database. Data is extracted from source systems (like Clarity), transformed in the staging database, and presented for users in the reporting database. This movement of data is realized via a set of SQL Server Integration Services (SSIS) packages.
CHAPTER 1-Data Lineage
Generally, data lineage refers to the process of identifying the source of a specific piece of information. In Caboodle, data lineage is defined at the package level.
CHAPTER 1-Star Schema
The standard schema for a dimensional data model. The name refers to the image of a fact table surrounded by many linked dimension tables, which loosely resembles a star.
The Caboodle data model structure is based on a “star schema” ‐ where one central fact table will join to many associated lookup or dimension tables. This structure provides the foundation of the Caboodle data model.
CHAPTER 1-DMC
DATA MODEL COMPONENT
No table in Caboodle “stands alone.” Each is considered part of a Data Model Component, which refers to the collection of metadata tables that support the ETL process and reporting views stored in the FullAccess schema.
Each DMC gets a type. Strict table naming conventions are followed in Caboodle, so that a table’s suffix provides information about its structure and purpose.
These suffixes are:
· Dim for dimensions (e.g. PatientDim)
· Fact for facts (e.g. EncounterFact)
· Bridge for bridges (e.g. DiagnosisBridge)
· DataMart for data marts (e.g. HospitalReadmissionDataMart)
· AttributeValueDim for EAV tables (e.g. PatientAttributeValueDim)
· X for custom tables (e.g. CustomFactX)
CHAPTER 1-Staging Database
The Caboodle database into which records are loaded by SSIS packages and stored procedures.
CHAPTER 1-Reporting Database
The architecture of Caboodle includes a staging database and a reporting database. Data is extracted from source systems (like Clarity), transformed in the staging database, and presented for users in the reporting database. This movement of data is realized via a set of SQL Server Integration Services (SSIS)
packages.
CHAPTER 1-Dbo Schema
STAGING DATABASE
Import tables and Mapping tables live here. This is
primarily used by administrators for moving data into Caboodle.
REPORTING DATABASE
The dbo schema stores reporting data and acts as the
data source for SlicerDicer. The Caboodle Dictionary reflects the contents of the dbo schema.
CHAPTER 1-FullAccess Schema
STAGING DATABASE
The FullAccess schema does not exist on the Staging database.
REPORTING DATABASE
The FullAccess schema houses views that simplify reporting. FullAccess should be your default schema when reporting.
CHAPTER 1-Identify key characteristics of the dimensional data model
MADE for report writers.
· Simpler and more intuitive.
· Easily extensible.
· More performant..
CHAPTER 1-Identify the benefits of the dimensional data model?
It centralizes its structure around more flexible fact and dimension tables, and the joins between these are more intuitive.
CHAPTER 1-Identify documentation resources for reporting out of Caboodle
Caboodle Dictionary
Reporting with Caboodle document
Caboodle ER diagram
CHAPTER 1-Identify reporting needs that best fit Caboodle
Custom data packages can be written by Caboodle developers to accommodate your organization’s reporting needs.
CHAPTER 1-How does Epic data flow into Caboodle
Epic data moves between several databases before it gets to Caboodle.
CHRONICLES flows into CLARITY via ETL. After transformation, the data is stored in a relational database on a separate server. Even though the structure of the Chronicles and Clarity databases differ significantly, the ETL process preserves the relationships mapped in Chronicles.
CLARITY flows into Caboodle data is extracted
from Clarity, transformed in the staging database, and presented for users in the reporting database. This movement of data is realized via a set of SQL Server Integration Services (SSIS) packages.
CHAPTER 1-How does Non-Epic data flow into Caboodle
The Caboodle developer designs custom DMCs (Data Model Components) and writes SSIS
packages to bring additional data into the warehouse. This may be additional Epic data from Clarity or non‐Epic data from 3rd party sources.
CHAPTER 2-Which of the following would you expect to be stored in a dimension table?
A. Users
B. ED Visits
C. Diagnosis definitions
A. Users
B. ED Visits
Users and diagnosis definitions would be in their own dimension tables. ED Visits are considered reportable, measureable events and would be in a fact table.
CHAPTER 2-Which of the following are possible in Caboodle? For each of the answers you select, use the Caboodle Dictionary to find an example.
A. Joining a Fact table to another Fact table
B. Joining a Fact table to a Dimension table
C. Joining a Dimension table to a Dimension table
D. Joining two different Fact tables to the same Profile Dimension table
(Examples may vary)
a. EncounterFact can join to HospitalAdmissionFact
b. MedicationOrderFact can join to PatientDim
c. PatientDim can join to ProviderDim
CHAPTER 2-Which of the following can be populated by more than one package? Select ALL that apply.
A. A single table
B. A single row in a table
A & B
CHAPTER 2-TRUE or FALSE: Caboodle and Clarity can be used interchangeably.
FALSE
CHAPTER 2-Fact Table
The core table types in a dimensional data model are facts and dimensions. A row in a fact table corresponds to the occurrence of some significant, measurable event, such as the ordering of a medication or the posting of a charge transaction.
A fact table typically contains lookup columns to several dimension tables. This arrangement somewhat resembles a star and so is referred to as a “star schema
Fact tables typically join to dimension tables, but may also join to other types of DMCs, such as other fact tables and bridge tables.
All fact tables have a Count column, which stores 0 for the negative rows, 0 for any row that represents deleted data, and 1 otherwise.
The names of fact tables end with “Fact.”
All Fact Tables in Caboodle use a surrogate key as their primary key.
CHAPTER 2-Dimension Table
The core table types in a dimensional data model are facts and dimensions.
A row in a dimension table represents a concrete entity which provides context for an event, such as the patient for whom a medication was ordered. A fact table typically contains lookup columns to several dimension tables. This arrangement somewhat resembles a star and so is referred to as a “star schema.”
Fact tables typically join to dimension tables, but may also join to other types of DMCs, such as other fact
tables and bridge tables. It is possible, but not common, for a dimension table to contain lookup columns to other DMCs.
All Dimension Tables in Caboodle use a surrogate key as their primary key.
CHAPTER 2-Surrogate Key
A system‐generated identifier in Caboodle. Not inherently meaningful to report consumers.
All fact and dimension tables in Caboodle use a surrogate key as their primary key. Surrogate key column names match the name of the table. Surrogate key values are created during the ETL process. Each
surrogate key value uniquely identifies one row of the table.
Surrogate key values are not the same as source identifiers.
CHAPTER 2-Source Identifier
Unique identifier in the source data. Consists of two columns in Caboodle: IdType and Id. Also known as a business key.
During the ETL process, a load package must provide an IdType and Id, collectively known as the business key. The IdType is a label for the type of identifier loaded by the package while the Id stores the unique identifier of the entity in the source.
CHAPTER 2-Foreign Key
FOREIGN KEYS in Caboodle end in Key. They are called LOOKUP Columns. It is a column in a table that joins to another table.
A FOREIGN KEY is a key used to link two tables together. A FOREIGN KEY is a field (or collection of fields) in one table that refers to the PRIMARY KEY in another table. The table containing the foreign key is called the child table, and the table containing the candidate key is called the referenced or parent table.
CHAPTER 2-Lookup Column
The columns of a fact table contain relevant metrics and foreign keys, or lookup columns, to link out to entities related to the event.
Lookup columns in Caboodle never contain null or
unmatched values.
ForeignKeys in Caboodle end in Key; called lookup
columns. Column in a table that joins to another table. Points to another table.
CHAPTER 3-TRUE or FALSE: Any change to patient data in Clarity will generate a new row in PatientDim after the next extract.
False. Assuming the data is extracted to Caboodle, only changes to Type 2 data would generate a new row in PatientDim.
CHAPTER 3-What is a durable key?
Type 2 tables have a durable key column which stores the same value for all rows in that table that
correspond to the same entity. This allows for grouping or summarizing data by the entity.
CHAPTER 3-Change Type
There are two change types: Type 1 and Type 2.
Caboodle tables and columns are given a change type, which determines whether or not that table or column tracks historical data in the source.
How a table tracks data over time.
Type 1 = Current
Type 2 = Storing data over time
CHAPTER 3-Consider the following query:
SELECT DurableKey FROM
Which of the following statements are true?
A. This query will only run if is Type 2
B. This query will run regardless of the Change Type of
C. The value in the DurableKey column could repeat in multiple rows of the results
D. There is no table in Caboodle for which this query will run
a. This query will only run if is Type 2
c. The value in the DurableKey column could repeat in multiple rows of the results
CHAPTER 3-Type 1 Data
Data that is overwritten in Caboodle each time it changes (so no historical values are preserved). For example, a patient’s birth date is Type 1 Data since it is assumed that this would never change (even if it does, the change is likely a correction rather than a genuine change).
CHAPTER 3-StartDate Column
The StartDate column contains the starting date for which the Type 2 data in the row became valid in
Caboodle.
CHAPTER 3-EndDate Column
The EndDate column contains the ending date for the range in which the data in the row was valid. TYPE 2
CHAPTER 3-Durable Key
A durable key is a system‐generated identifier that facilitates grouping the rows in a Type 2 table by the entity they represent. For example, each time an employee changes jobs, a new row is created in
EmployeeDim. Every employee would be assigned a single durable key value which is then populated on each of their rows in EmployeeDim.
Durable Keys store same value for multiple rows that
refer to the same entity in a table
CHAPTER 3-Durable Key Lookup
Ends in DurableKey: joins to DurableKey column
CHAPTER 3-Type 2 Entity
Many tables in Caboodle reference patients, providers, and employees. PatientDim, ProviderDim, and EmployeeDim are Type 2 tables in Caboodle.
Patients, providers, and employees are examples of Type 2 entities in Caboodle. When a fact table references a patient, it is referencing a Type 2 entity, and it has two ways to do so: by the primary key or by the durable key of PatientDim.
CHAPTER 1-What are the differences between a normalized and dimensional data model?
In a normalized data model, the focus is on not repeating data, which reduces the size of the
database. In a dimensional data model, the focus is on ease of reporting and uses the star schema, which focuses on a central fact table pertaining to a reportable event and surrounding dimension tables providing context for the event.
CONTEXT…It gives CONTEXT
CHAPTER 1-Briefly define the roles of the Caboodle report writer, administrator, and developer.
· The Caboodle report writer queries data that already exists in the database. They use their knowledge of the tools and the source database(s) to conduct research into the necessary data points required for a given report.
· The Caboodle administrator uses the Caboodle Console to manage and monitor the ETL process. They troubleshoot ETL errors and handle configuration steps for the database.
· The Caboodle developer designs custom DMCs (Data Model Components) and writes SSIS packages to bring additional data into the warehouse. This may be additional Epic data from Clarity or non‐Epic data from 3rd party sources.
CHAPTER 1-TRUE or FALSE: Naming conventions are enforced in Caboodle.
True. Naming conventions, such as Fact, are enforced in Caboodle.
CHAPTER 1-What is the relationship between SlicerDicer and Caboodle?
SlicerDicer is Epic’s self-service reporting tool in Hyperspace that dynamically queries Caboodle data.
CHAPTER 4-Suppose you’re creating a report using EncounterFact and ProviderDim that displays the name of each provider for each encounter. If an encounter’s data loads into EncounterFact, but that provider’s data hasn’t been extracted to ProviderDim, what value would appear in ProviderDim.Name for this encounter?
There would be an inferred row in ProviderDim, therefore ProviderDim.Name would show “*Unknown” for the encounter.
CHAPTER 4-A foreign key column in Caboodle contains the value ‐3. What could this mean?
There are two possibilities: Either the data that once populated this row in Caboodle has been deleted from the source or this is the row with ‐3 as its primary key, for which all foreign key columns are also ‐3. To know which is true, check the primary key value for the given row: a non‐negative value indicates that this row represents deleted data in the source.
CHAPTER 4-If a foreign key is null in Clarity, what will appear in the corresponding column in Caboodle?
If the foreign key was null in the source, then a ‐1 will appear in Caboodle to represent an unspecified value. If the foreign key’s data lineage was not defined by the source package that loaded data for a particular row, then a ‐2 will appear in Caboodle to represent a value that is not applicable.
CHAPTER 4-A row in a Caboodle table has a primary key value of ‐2. What value will be stored in non‐foreign key columns with a data type of “Date” for this row?
A. ‐2
B. NULL
C. 12/31/1840
D. *Not Applicable
b. NULL
For the row with a primary key of ‐2, foreign key columns will store a value of ‐2, string columns will
store ‘*Not Applicable’, and all other data type columns will store NULL.
CHAPTER 4-TRUE or FALSE: All inferred rows in Caboodle have a primary key of ‐1.
False
Inferred rows in Caboodle have non‐negative primary key values because they represent an entity that Caboodle has inferred the existence of. The foreign key columns for such rows will be set to ‐1 until the Caboodle ETL process updates the data.
CHAPTER 4-Referential Integrity
Referential integrity is enforced in Caboodle, meaning that lookup columns will never contain null values. Furthermore, every value in a lookup column has a match in the destination table.
Caboodle enforces referential integrity by:
- Placing “extra” rows in every table
- These rows have surrogate key values of -1, -2, and -3
- Applying default foreign key values (-1, -2, -3)
- Creating inferred rows
CHAPTER 4-Inferred Row
When a foreign key value is loaded into Caboodle, the ETL process uses its Id and IdType to determine the correct surrogate key value to store. If the referenced row does not exist yet, the ETL process creates a new row in the target table and assigns a new surrogate key value as its primary key. Such a row is called an inferred row.
Inferred rows are created when one table has more up-to-date information than another.
Two common scenarios:
A. Unmatched references in the source
B. Testing or development
Every table in Caboodle has three “extra” rows.
These rows provide a place for foreign keys to point.
CHAPTER 4-The -1 Row
AS PART OF REFERENTIAL INTEGRITY
When a foreign key value is NULL in the source system, the foreign key is said to be UNSPECIFIED.
(Unspecified Foreign Key)
In this case, the corresponding foreign key in Caboodle is set to ‐1.
Since many date columns in Caboodle are
foreign keys to the DateDim table, NULL dates from Clarity are often represented with a ‐1 value DateKey
columns in Caboodle.
CHAPTER 4-The -2 Row
AS PART OF REFERENTIAL INTEGRITY
When a foreign key is not applicable (N/A) as defined by the SSIS package, Caboodle sets the value to -2. This is handled by the ETL process.
CHAPTER 4-The -3 Row
AS PART OF REFERENTIAL INTEGRITY
If an account was deleted in the source, all foreign key values are set to ‐3.
When a record is hard‐deleted in the source system or no longer passes the filters required for extraction,
Caboodle tracks the deletion in one of two ways.
· For Type 1 DMCs, the existing row is updated with default values representing the deletion. The primary key remains unchanged.
· For Type 2 DMCs, a new row is added that represents the deletion. Existing Type 2 data remains, while Type 1 data is overwritten with the default values. The primary key﴾s﴿ and durable key will remain unchanged.
CHAPTER 5-What two columns do all bridge tables have?
Every bridge table has a ComboKey to which other tables join and a Foreign Key used to join to its associated dimension table.
CHAPTER 5-TRUE of FALSE: AttributeValueDim tables are more granular than their associated fact table.
True
The AttributeValueDim tables can store multiple attribute‐value combinations on separate rows
for the same entity, making them more granular than their associated fact table
CHAPTER 5-Bridge Table
Bridge tables exist to capture many‐to‐many relationships, such as the one between patients and the diagnoses on their problem list.
Every bridge table has two columns. They are called ComboKey and Key, where is the name of the table to which a bridge has been formed. The
CHAPTER 5-Entity‐Attribute‐Value data structure
The entity-attribute-value (EAV) framework allows you to essentially add data to existing Caboodle tables without changing the structure of those tables.
Values in an EAV model can be any data type, but every value within a column must be the same data type. The typical way to handle this is to make the column a varchar column and store all of the values as a string.
CHAPTER 5-What is the difference between a MappingFact table and a Bridge table?
Both Bridge tables and MappingFact tables are used to map many‐to‐many relationships. However, Bridge tables use a combo key as part of their definition and always link to a single dimension table while MappingFact tables may link to one or more fact or dimension tables.
CHAPTER 5-AttributeValueDim Table
One form of EAV data in Caboodle is the AttributeValueDim DMCs. These tables contain attribute‐value pairs and can be created for different entities. For instance, there is the PatientAttributeValueDim table to go along with PatientDim, or the DepartmentAttributeValueDim table to go along with DepartmentDim.
AttributeValueDim tables are typically Type 2 tables.
CHAPTER 5-DataMart Table
In the context of Caboodle, data marts are, broadly, collections of Caboodle data on a topic. Some data marts serve as comprehensive data sources for a particular reporting need, making it easier for business intelligence developers to write reports. For example, the HospitalReadmissionAnalyticsDataMart contains all the data relevant for predicting the likelihood of a future hospital readmission.
Data marts are very similar to fact tables because they both store data about significant, measurable
events and refer to data in dimension tables.
Data Mart tables typically have a source of “Caboodle”.
- Data Marts
- Similar to Fact tables
- Group together data relevant to a specific reporting topic
CHAPTER 5-SetDim Tables
STORES GROUPER INFORMATION
In Chronicles, the VCG master file stores groupers, which are lists of Chronicles records (most commonly), ICD codes, or other concepts used together frequently in Hyperspace. VCG groupers are maintained by application analysts in Hyperspace, but can be useful for writing Caboodle reports as well.
PRACTICE EXAM-Epic recommends using which schema as the default schema for report writing.
CHOOSE ONLY ONE ANSWER
A. dbo schema on staging database
B. dbo schema on the reporting database
C. Epic schema on either database
D. FullAccess schema on the reporting database
D. FullAccess schema on the reporting database
The FullAccess schema provides a logical layer for report writers in Caboodle. It contains formatting for Boolean columns and performs some simple joins to make reporting easier.
PRACTICE EXAM-TRUE or FALSE: If a data point exists in Clarity, but not in Caboodle, your organization can create a custom SSIS package to extract it to Caboodle.
TRUE
Caboodle is customizable, meaning your organization can bring in additional data from Clarity as well as third party sources.
PRACTICE EXAM-PharmacyDim is a Type 1 dimension table in Caboodle. Each pharmacy in Epic has a unique record in the PHR master file. Which of the following best describes the difference between the PharmacyKey and PharmacyEpicId columns in PharmacyDim?
CHOOSE ONLY ONE ANSWER
A. When no PHR record ID exits, PharmacyKey stores -1 and PharmacyEpicId is null
B. When no PHR record ID exits, PharmacyKey is null and PharmacyEpicId is -1
C. PharmacyKey uniquely identifies a row in PharmacyDim; PharmacyEpicId stores the PHR record ID
D. PharmacyKey stores the PHR record ID; PharmacyEpicId uniquely identifies a row in PharmacyDim
C. PharmacyKey uniquely identifies a row in PharmacyDim; PharmacyEpicId stores the PHR record ID.
A user doesn’t have access to an RW Report unless both the user and the report share a report group in common.
PRACTICE EXAM-How can you tell if a column is a foreign key in Caboodle?
CHOOSE ALL THAT APPLY
A. The column is marked as Type 2
B. The ER diagram lists it as a foreign key
C. The column is contained in a fact table
D. The destination table is listed to the right of the column description
B. The ER diagram lists it as a foreign key
D. The destination table is listed to the right of the column description
Type 1 tables do not track changes, so do not need a DurableKey or IsCurrent column.
PRACTICE EXAM-Tables in Caboodle are categorized as either Type 1 or Type 2. This best describes which of the following?
CHOOSE ONLY ONE ANSWER
A. Caboodle enforces referential integrity
B. Caboodle can track changes in the source
C. Caboodle uses the dimensional data model
D. Caboodle uses the entity-attribute-value data model
B. Caboodle can track changes in the source.
All Type 2 tables have special columns to help track these changes; DurableKey, StartDate, EndDate, and IsCurrent.
PRACTICE EXAM-Consider the following query: Which of the following logical expressions will return all hospital admissions where the admitting provider was the same as the discharging provider?
SELECT EncounterEpicCsn, AdmittingProviderKey, DischargingProviderKey
FROM HospitalAdmissionFact
WHERE and AdmittingProviderKey >0 and DischargingProviderKey >0
CHOOSE ONLY ONE ANSWER
A. AdmittingProviderKey = DischaringProviderKey
B. AdmittingProviderDurableKey = DischargingProviderKey
C. AdmittingProviderDurableKey = DischargingProviderDurableKey D. All of the above are equally valid
C. AdmittingProviderDurableKey = DischargingProviderDurableKey
When comparing type 2 entities, like providers, it most accurage to compare the durable key values, sincy there may be more than one surrogate key value associated with a single entity.
PRACTICE EXAM-Table A has a foreign key to Table B. An execution runs, which includes a package that loads data to Table A, but does not contain any packages which will populate Table B. Which of the following tables could have an inferred row added to it as a result of this execution?
CHOOSE ONLY ONE ANSWER
A. Only Table A
B. Only Table B
C. Both Table A and Table B
D. Neither Table A nor Table B
B. Only Table B
Inferred rows are created in the table that is referenced by a foreign key’s value. When data is loaded into table A, it’s foreign key is populated. If Table B does not yet have that foreign key value, and inferred row is created.
PRACTICE EXAM-If the primary key of a row in a fact table is -1, then
CHOOSE ALL THAT APPLY
A. The foreign keys will store -1
B. The foreign keys will store “Unspecified”
C. The foreign keys will store “Unknown”
D. Columns with a data type of varchar will store “Unspecified”
A. The foreign keys will store -1
D. Columns with a data type of varchar will store “Unspecified”
The -1 row of every Caboodle table behaves in the same way: it’s foreign keys will store -1 “Unspecified”, and all other data type columns will be NULL.
PRACTICE EXAM-A ComboKey is used as a foregin key to which of the following types of DMCs?
CHOOSE ONLY ONE ANSWER
Fact
Dimension
SetDim
DataMart
AttributeValueDim
Bridge
Bridge
Although it is the destination of the join, the combo key column of a bridge table is not the primary key. Rather, multiple rows may have the same combo key value.
OVERALL-Granularity
The level of detail for one row in the table.
OVERALL-Primary Key
The Primary Key is the column(s) that can uniquely ID a row in a table.
OVERALL-Surrogate Key
Surrogate Key is a key where the values don’t exist in the source data. These values are NOT in Clarity. They are generated in the ETL process…They are created when moved.
OVERALL-NAME COVENTIONS IN CABOODLE
Each DMC gets a type. Strict table naming conventions are followed in Caboodle, so that a table’s suffix provides information about its structure and purpose. These suffixes are:
· Dim for dimensions (e.g. PatientDim)
· Fact for facts (e.g. EncounterFact)
· Bridge for bridges (e.g. DiagnosisBridge)
· DataMart for data marts (e.g.HospitalReadmissionDataMart)
· AttributeValueDim for EAV tables (e.g. PatientAttributeValueDim)
· X for custom tables .e.g. CustomFactX
OVERALL-ER Diagram
An entity‐relationship diagram, or ER diagram, is a visual tool that displays information about how database objects relevant to a particular reporting area join to one another. Caboodle ER diagrams are automatically generated for fact tables and data mart tables in the dictionary.
DEFINITION-SURROGATE KEY
A surrogate key is any column or set of columns that can be declared as the primary key instead of a “real” or natural key. Sometimes there can be several natural keys that could be declared as the primary key, and these are all called candidate keys. So a surrogate is a candidate key.
OVERALL-CSN
Contact Serial Number
OVERALL-Type 1
Type 1 tables and columns in Caboodle do not track historical data. When a Type 1 value is updated in the source, all instances of that value in Caboodle are updated the next time the data gets extracted.
If a table is Type 1—THEN all of the columns in that table are Type 1.
OVERALL-Type 2
Caboodle retains historical values by creating multiple rows for each entity, with each row containing the values for a particular time period.
Typically, Type 2 tables contain a mix of Type 1 and Type 2 columns. A Type 2 table must contain at least one Type 2 column. KNOW THIS!!
Type 2 tables typically have a mix of Type 1 and Type 2 columns. Only changes to Type 2 data in the source will result in a new row being added to a Type 2 table for a given entity. When Type 1 data changes, Caboodle stores only the current value in all existing rows for the entity.
OVERALL-IsCurrent
The IsCurrent column is a flag that stores the value of 1 if the row holds the most current information and 0 otherwise.
OVERALL-Enforcing Referential Integrity
Caboodle uses three mechanisms for enforcing referential integrity:
· Every table in Caboodle contains three rows with surrogate keys of ‐1, ‐2, and ‐3. These rows serve as the destination for lookup columns in other tables.
· When source data is unavailable to populate a lookup column, Caboodle supplies a default value of ‐1, ‐2, or ‐3. Which value is used depends on the circumstance behind the missing information.
· When data loaded in one Caboodle table is more up‐to‐date than data in another table, the Caboodle ETL process generates an inferred row in the delayed table. This row represents the entity being referenced by the more up‐to‐date table. Inferred rows act as place holders until the relevant information is loaded into Caboodle.
OVERALL-Inferred Row
When a foreign key value is loaded into Caboodle, the ETL process uses its Id and IdType to determine the correct surrogate key value to store. If the referenced row does not exist yet, the ETL process creates a new row in the target table and assigns a new surrogate key value as its primary key. Such a row is called an inferred row.
KNOW THIS
When an inferred row is created, all its string values are set to *Unknown. Additionally, all of the inferred row’s foreign key columns are assigned a value of ‐1 and any remaining data columns are set to NULL.
The inferred row is a placeholder, so a later ETL will update these default values when that information is populated in the source database, regardless of the table’s change type
OVERALL-EAV
Entity Attribute Value
The three major components of the EAV data model are entities, attributes, and values.
Let’s define these terms by looking at a particular example: suppose your organization is tracking patient hair color. In this case, an ENTITY is a patient and the ATTRIBUTE is hair color. A VALUE would be a particular patient’s hair color, such as red.
(General Reporting Tips) If a query refers to more than one table, all columns should be prefixed by a descriptor (table name or alias)
Using descriptors ensures you have unambiguous column references, preventing issues that can occur when two tables contain columns with the same name.
Chapter 1. (Study Checklist) Caboodle Console
The Caboodle Console is a web application housed on the Caboodle server. It includes the following:
Dictionary
Dictionary Editor
Executions
Work Queue
Configuration
Chapter 1. (Study Checklist) Data Warehouse
In a data warehouse, multiple sources may load data pertaining to a single entity. This means that more than one package may populate a given row in a Caboodle table. As a result, there may be multiple business key values associated with a single entity in a Caboodle table.
Chapter 1. (Study Checklist) ETL
Extract, Transform, Load
Chapter 1. (Study Checklist) SSIS Package
The architecture of Caboodle includes a staging database and a reporting database. Data is extracted from source systems (like Clarity), transformed in the staging database, and presented for users in the reporting database. This movement of data is realized via a set of SQL Server Integration Services (SSIS) packages.
Chapter 1. (Study Checklist) Data Lineage
Generally, data lineage refers to the process of identifying the source of a specific piece of information. In Caboodle, data lineage is defined at the package level.
Chapter 1. (Study Checklist) Star Schema
The standard schema for a dimensional data model. The name refers to the image of a fact table surrounded by many linked dimension tables, which loosely resembles a star.
The Caboodle data model structure is based on a “star schema” ‐ where one central fact table will join to many associated lookup or dimension tables. This structure provides the foundation of the Caboodle data model.
Chapter 1. (Study Checklist) DMC
DATA MODEL COMPONENT
No table in Caboodle “stands alone.” Each is considered part of a Data Model Component, which refers to the collection of metadata tables that support the ETL process and reporting views stored in the FullAccess schema.
Each DMC gets a type. Strict table naming conventions are followed in Caboodle, so that a table’s suffix provides information about its structure and purpose.
These suffixes are:
· Dim for dimensions (e.g. PatientDim)
· Fact for facts (e.g. EncounterFact)
· Bridge for bridges (e.g. DiagnosisBridge)
· DataMart for data marts (e.g. HospitalReadmissionDataMart)
· AttributeValueDim for EAV tables (e.g. PatientAttributeValueDim)
· X for custom tables (e.g. CustomFactX)
Chapter 1. (Study Checklist) Staging Database
The Caboodle database into which records are loaded by SSIS packages and stored procedures.
Chapter 1. (Study Checklist) Reporting Database
The architecture of Caboodle includes a staging database and a reporting database. Data is extracted from source systems (like Clarity), transformed in the staging database, and presented for users in the reporting database. This movement of data is realized via a set of SQL Server Integration Services (SSIS)
packages.
Chapter 1. (Study Checklist) Dbo Schema
STAGING DATABASE
Import tables and Mapping tables live here. This is
primarily used by administrators for moving data into Caboodle.
REPORTING DATABASE
The dbo schema stores reporting data and acts as the
data source for SlicerDicer. The Caboodle Dictionary reflects the contents of the dbo schema.
Chapter 1. (Study Checklist) FullAccess Schema
STAGING DATABASE
The FullAccess schema does not exist on the Staging database.
REPORTING DATABASE
The FullAccess schema houses views that simplify reporting. FullAccess should be your default schema when reporting.
(ETL Terms) Execution
An execution is the process that extracts data from a source system using packages, transforms the data in the staging database, and loads it to Caboodle for reporting. You create and run executions in the Caboodle Console.
(ETL Terms) Extract
Extracts to Caboodle from Clarity can be either backfill or incremental. Backfill extracts load or reload every row in a table from Clarity, whereas incremental extracts load only changed rows. Existing data is available while extracts are in progress.
(ETL Terms)package
A package is a definition of an extract of data from one specific source to a specific import table. For example, a fact might have packages for Epic inpatient data, Epic outpatient data, and several non-Epic data sources. Packages are defined in SSIS .dtsx files.
Chapter 1. (Study Checklist) Identify key characteristics of the dimensional data model.
MADE for report writers.
· Simpler and more intuitive.
· Easily extensible.
· More performant..
Chapter 1. (Study Checklist) Identify documentation resources for reporting out of Caboodle
Caboodle Dictionary
Reporting with Caboodle document
Caboodle ER diagram
Chapter 1. (Study Checklist) Identify reporting needs that best fit Caboodle
Custom data packages can be written by Caboodle developers to accommodate your organization’s reporting needs.
(General Reporting Tips) Add a filter to most queries to exclude Caboodle’s special rows for unspecified, not applicable, and deleted records, which have surrogate keys of -1, -2, and -3
Include only rows where the key is greater than 0.
(General Reporting Tips) Caboodle has a numbers table, NumbersDim, that you can use as needed in your reports
NumbersDim contains the integers from 1 to 1,000,000, which you can reference to help manipulate strings and complete other processes. If you need more than 1,000,000 rows to accomplish a task, you can refer to NumbersDim multiple times in your query.
Chapter 1. (Study Checklist) How does Epic data flow into Caboodle
Epic data moves between several databases before it gets to Caboodle.
CHRONICLES flows into CLARITY via ETL. After transformation, the data is stored in a relational database on a separate server. Even though the structure of the Chronicles and Clarity databases differ significantly, the ETL process preserves the relationships mapped in Chronicles.
CLARITY flows into Caboodle data is extracted
from Clarity, transformed in the staging database, and presented for users in the reporting database. This movement of data is realized via a set of SQL Server Integration Services (SSIS) packages.
Chapter 1. (Study Checklist) How does Non-Epic data flow into Caboodle
The Caboodle developer designs custom DMCs (Data Model Components) and writes SSIS
packages to bring additional data into the warehouse. This may be additional Epic data from Clarity or non‐Epic data from 3rd party sources.
CHAPTER 1. (Reviewing the Chapter) What are the differences between a normalized and dimensional data model?
In a normalized data model, the focus is on not repeating data, which reduces the size of the
database.
In a dimensional data model, the focus is on ease of reporting and uses the star schema, which focuses on a central fact table pertaining to a reportable event and surrounding dimension tables providing context for the event.
CONTEXT…It gives CONTEXT
CHAPTER 1. (Reviewing the Chapter) Briefly define the roles of the Caboodle report writer, administrator, and developer.
· The Caboodle report writer queries data that already exists in the database. They use their knowledge of the tools and the source database(s) to conduct research into the necessary data points required for a given report.
· The Caboodle administrator uses the Caboodle Console to manage and monitor the ETL process. They troubleshoot ETL errors and handle configuration steps for the database.
· The Caboodle developer designs custom DMCs (Data Model Components) and writes SSIS packages to bring additional data into the warehouse. This may be additional Epic data from Clarity or non‐Epic data from 3rd party sources.
CHAPTER 1. (Reviewing the Chapter) TRUE or FALSE: Naming conventions are enforced in Caboodle.
True. Naming conventions, such as Fact, are enforced in Caboodle.
CHAPTER 1. (Reviewing the Chapter) What is the relationship between SlicerDicer and Caboodle?
SlicerDicer is Epic’s self-service reporting tool in Hyperspace that dynamically queries Caboodle data.
Chapter 1. (After-Class Exercise) Where does Epic data in Caboodle come from?
From Clarity, which gets its data from Chronicles
Chapter 1. (After-Class Exercise) True or False: Only Epic can Define SSIS packages to move data from Clarity to Caboodle.
False.
Chapter 1. (After-Class Exercise) Caboodle is designed to… (Choose all that apply):
A. Store Epic data
B. Store non-Rpic Data
C. Make reporting easy
D. Interoperate with SlicerDicer
E. Store all of the data from Clarity.
Answer: A, B, C, D
Chapter 1. (After-Class Exercise) True or False: Caboodle is considered a normalized database.
False. Clarity is a normalized database. Caboodle has been denormalized in order to simplify reporting.
Chapter 1. (After-Class Exercise) Name three documentation resources for Caboodle Reporting.
Caboodle Dictionary
Reporting with Caboodle document
Caboodle ER Diagram
Chapter 1. (After-Class Exercise) When writing reports, why is it better to use your organization’s Caboodle console rather than the console used in training?
The Console used in training will not reflect your organization’s custom development. For that reason, it is best to use your Caboodle Console rather than Epic’s training tools for writing reports.
Chapter 1. (After-Class Exercise) Use the Caboodle Dictionary to identify the Caboodle tables and columns that extract data from the following items. If you encounter multiple results, write down 2 of them.
A. CVG 410 (termination date): CoverageDim.CoverageTerminationDate
B. FSD 1230 (whether a flowsheet value reading is abnormal) FlowsheetValueFact.Abnormal
c. HSB 89210 (anesthesia start time): AnesthesiaRecordFact.AnethesiaStartInstant
D. LPL 1164 (present on admisiion diadnoses): DiagnosisEventFact.PresentOnAdmission, ProblemListFact.PresentOnAdmission
Chapter 2. (Study Checklist) Fact Table
The core table types in a dimensional data model are facts and dimensions. A row in a fact table corresponds to the occurrence of some significant, measurable event, such as the ordering of a medication or the posting of a charge transaction.
A fact table typically contains lookup columns to several dimension tables. This arrangement somewhat resembles a star and so is referred to as a “star schema
Fact tables typically join to dimension tables, but may also join to other types of DMCs, such as other fact tables and bridge tables.
All fact tables have a Count column, which stores 0 for the negative rows, 0 for any row that represents deleted data, and 1 otherwise.
The names of fact tables end with “Fact.”
All Fact Tables in Caboodle use a surrogate key as their primary key.
Chapter 2. (Study Checklist) Dimension Table
The core table types in a dimensional data model are facts and dimensions.
A row in a dimension table represents a concrete entity which provides context for an event, such as the patient for whom a medication was ordered. A fact table typically contains lookup columns to several dimension tables. This arrangement somewhat resembles a star and so is referred to as a “star schema.”
Fact tables typically join to dimension tables, but may also join to other types of DMCs, such as other fact
tables and bridge tables. It is possible, but not common, for a dimension table to contain lookup columns to other DMCs.
All Dimension Tables in Caboodle use a surrogate key as their primary key.
Chapter 2. (Study Checklist) Surrogate Key
A system‐generated identifier in Caboodle. Not inherently meaningful to report consumers.
All fact and dimension tables in Caboodle use a surrogate key as their primary key. Surrogate key column names match the name of the table. Surrogate key values are created during the ETL process. Each
surrogate key value uniquely identifies one row of the table.
Surrogate key values are not the same as source identifiers.
Chapter 2. (Study Checklist) Source Identifier
Unique identifier in the source data. Consists of two columns in Caboodle: IdType and Id. Also known as a business key.
During the ETL process, a load package must provide an IdType and Id, collectively known as the business key. The IdType is a label for the type of identifier loaded by the package while the Id stores the unique identifier of the entity in the source.
For Epic data, columns end in EpicId or EpicCsn
Chapter 2. (Study Checklist) Foreign Key
FOREIGN KEYS in Caboodle end in KEY. They are called LOOKUP Columns. It is a column in a table that joins to another table.
A FOREIGN KEY is a key used to link two tables together. A FOREIGN KEY is a field (or collection of fields) in one table that refers to the PRIMARY KEY in another table. The table containing the foreign key is called the child table, and the table containing the candidate key is called the referenced or parent table.
Chapter 2. (Study Checklist) Lookup Column
The columns of a fact table contain relevant metrics and foreign keys, or lookup columns, to link out to entities related to the event.
Lookup columns in Caboodle never contain null or
unmatched values.
ForeignKeys in Caboodle end in Key; called lookup
columns. Column in a table that joins to another table. Points to another table.
Chapter 2. (Study Checklist) Where do you look up the data lineage of a table?
Table-level data lineage can be found in the Sources section of the Caboodle Dictionary for a specific table
Chapter 2. (Study Checklist) Where do you look up the data lineage of a column?
Column-level data lineage for each package is found by expanding that section under the description of each column.
The columns value may reflect the same value as the source, or maybe not and modified by SSIS
Chapter 2. (Study Checklist) How would you identify Epic data in Caboodle?
Typically entered in Hyperspace and stored in Chronicle. Data moves from Chronicles to Clarity to Caboodle, so Clarity is a source database for Caboodle.
Chapter 2. (Study Checklist) How would you identify non-Epic data in Caboodle?
Instead of seeing “Clarity” as the source data, non-Epic data sources will show the name of the non-Epic database
For Epic data, columns end in EpicId or EpicCsn
Chapter 2. (Study Checklist) The difference between the terms ‘lookup column’ and ‘foreign key’
All lookup columns in Caboodle ending the word KEY. Some lookup columns are foreign keys that they join to the primary key of the destination table.
CHAPTER 2. (Reviewing the Chapter) Which of the following would you expect to be stored in a dimension table?
A. Users
B. ED Visits
C. Diagnosis definitions
A. Users
C. Diagnosis Definitions
Users and diagnosis definitions would be in their own dimension tables.
ED Visits are considered reportable, measurable events and would be in a fact table.
CHAPTER 2. (Reviewing the Chapter) Which of the following are possible in Caboodle? For each of the answers you select, use the Caboodle Dictionary to find an example.
A. Joining a Fact table to another Fact table
B. Joining a Fact table to a Dimension table
C. Joining a Dimension table to a Dimension table
D. Joining two different Fact tables to the same Profile Dimension table
(Examples may vary)
a. EncounterFact can join to HospitalAdmissionFact
b. MedicationOrderFact can join to PatientDim
c. PatientDim can join to ProviderDim
Chapter 2. (Reviewing the Chapter) Which of the following can be populated by more than one package? Select ALL that apply.
A. A single table
B. A single row in a table
A & B
Chapter 2. (Reviewing the Chapter) TRUE or FALSE: Caboodle and Clarity can be used interchangeably.
FALSE
Chapter 2. (Reviewing the Chapter) True or False: Fact tables can only be joined to dimension tables.
False. The Caboodle Dictonary contains several lookup columns between two fact tables and between two dimension tables. For example, PatientDim can be joined to ProviderDim and HospitalAdmissionFact can be joined to EncounterFact.
Chapter 2. (In-Class Exercises) What is the primary key of MedicationOrderFact?
MedicationOrderKey
Chapter 2. (In-Class Exercises) How can lookup columns, or foreign keys, be identified in the Dictionary?
Tables that link to or from a given table are listed under Properties, and hyperlinks to the destination table are provided next to lookup column descriptions.
Chapter 2. (In-Class Exercises) In Caboodle, can a fact table have a lookup column to another fact table?
Yes. For Example, MedicationOrderFact has a lookup column to EncounterFact.
Chapter 2. (In-Class Exercises) What is the difference between MedicationOrderKey and MedicationOrderEpicId?
MedicationOrderKey stores a surrogate key value generated by Caboodle to identify one row in MedicationOrderFact. MedicationOrderEpicId stores the ID of an order record in Epic, or the ORD.1.
Chapter 2. (In-Class Exercises) Navigate to MedicationDim. What is the difference between MedicationKey and MedicationEpicId?
MedicationKey Stores a surrogate key value generated by Caboodle to identify one row in MedicationDim. MedicationEpicId stores the ID of a medication record in Epic, or the ERX.1.
Chapter 2. (In-Class Exercises) True or False: A maximum of one foreign key can exist between two tables in Caboodle?
False
CHAPTER 3. (Reviewing the Chapter) TRUE or FALSE: Any change to patient data in Clarity will generate a new row in PatientDim after the next extract.
False. Assuming the data is extracted to Caboodle, only changes to Type 2 data would generate a new row in PatientDim.
CHAPTER 3. (Reviewing the Chapter) What is a durable key?
Type 2 tables have a durable key column which stores the same value for all rows in that table that
correspond to the same entity. This allows for grouping or summarizing data by the entity.
CHAPTER 3. (Reviewing the Chapter) Consider the following query:
SELECT DurableKey FROM
Which of the following statements are true?
A. This query will only run if is Type 2
B. This query will run regardless of the Change Type of
C. The value in the DurableKey column could repeat in multiple rows of the results
D. There is no table in Caboodle for which this query will run
a. This query will only run if is Type 2
c. The value in the DurableKey column could repeat in multiple rows of the results
CHAPTER 3. (Study Checklist) Change Type
There are two change types: Type 1 and Type 2.
Caboodle tables and columns are given a change type, which determines whether or not that table or column tracks historical data in the source.
How a table tracks data over time.
Type 1 = Current
Type 2 = Storing data over time
CHAPTER 3. (Study Checklist) Type 1 Data
Type 1 data reflects the current content of the source as of the last Caboodle extract. In other words, Type 1 tables and columns in Caboodle do not track historical data. When a Type 1 value is updated in the source, all instances of that value in Caboodle are updated the next time the data gets extracted.
CHAPTER 3. (Study Checklist) Type 2 Data
In some cases, Caboodle will keep track of precious values after those values are no longer stored in the original source. Caboodle retains historical values by creating multiple rows for each entity, with each row containing the values for a particular time period. For example, an individual patient may appear in several rows of PatientDim, each representing what Caboodle knew about the patient for a different time period.
CHAPTER 3. (Study Checklist) StartDate Column
The StartDate column contains the starting date for which the Type 2 data in the row became valid in
Caboodle. For a row that holds the earliest version of the data recorded in Caboodle the StartDate is set to the value of EarliestAllowableDate in the configuration table Config.ConfigurationValues.
CHAPTER 3. (Study Checklist) EndDate Column
The EndColumn contains the ending date for the range in which the data in the row was valid in Caboodle. For a row that holds the most recent version of the data occurring in Caboodle the EndDate is set to the value of the LatestAllowableDate in the configuration table Config.ConfigurationValues.
CHAPTER 3. (Study Checklist) Durable Key
A durable key is a system‐generated identifier that facilitates grouping the rows in a Type 2 table by the entity they represent. For example, each time an employee changes jobs, a new row is created in
EmployeeDim. Every employee would be assigned a single durable key value which is then populated on each of their rows in EmployeeDim.
Durable Keys store same value for multiple rows that
refer to the same entity in a table
CHAPTER 3. (Study Checklist) Durable Key Lookup
Ends in DurableKey: joins to DurableKey column
CHAPTER 3. (Study Checklist) Type 2 Entity
Many tables in Caboodle reference patients, providers, and employees. PatientDim, ProviderDim, and EmployeeDim are Type 2 tables in Caboodle.
Patients, providers, and employees are examples of Type 2 entities in Caboodle. When a fact table references a patient, it is referencing a Type 2 entity, and it has two ways to do so: by the primary key or by the durable key of PatientDim.
CHAPTER 3. (Study Checklist) identify the change type of a table
This is documented in the table Properties in the Dictionary.
CHAPTER 3. (Study Checklist) Identify the change type of a column?
In the Caboodle dictionary it lists if the column is type 2
CHAPTER 3. (Study Checklist) Identify whether a change in the source data will trigger an update to an existing value or create a new row
If the data is type 1, all values overwrite old values. If the data is type 2, new data will be created
CHAPTER 3. (Study Checklist) Use the durable key lookup column to aggregate data
The durable key lookup columns can also be used to aggregate rows for the same entity without joining to the dimension at all. For example, to count the number of encounters for each patient in EncounterFact, apply a GROUP BY using PatientDurableKey.
CHAPTER 3. (Study Checklist) Join to current data in a Type 2 table
- Join the durable key lookup column in the starting table to the durable key of the target table.
- Filter to the current row of the Type 2 target table.
CHAPTER 3. (Study Checklist) How Caboodle tracks changes in the source data
Foreign keys reference the row that was current at the time of extract
CHAPTER 3. (Study Checklist) How the granularity of a Type 2 table differs from the granularity of a Type 1 table
Type 2 column values track historical data, Type 1 column values reflect the current source data. Type 1 data is overwritten in Type 2 tables to reflect current source data. Only changes to Type 2 are tracked.
CHAPTER 3. (Study Checklist) The difference between the primary key and the durable key of a table
CHAPTER 3. (Study Checklist) The difference between a surrogate key lookup column and a durable key lookup column
CHAPTER 3. (Study Checklist) The difference between a durable key lookup column and a durable key column
CHAPTER 4. (Reviewing the Chapter) Suppose you’re creating a report using EncounterFact and ProviderDim that displays the name of each provider for each encounter. If an encounter’s data loads into EncounterFact, but that provider’s data hasn’t been extracted to ProviderDim, what value would appear in ProviderDim.Name for this encounter?
There would be an inferred row in ProviderDim, therefore ProviderDim.Name would show “*Unknown” for the encounter.
CHAPTER 4-A foreign key column in Caboodle contains the value ‐3. What could this mean?
There are two possibilities: Either the data that once populated this row in Caboodle has been deleted from the source or this is the row with ‐3 as its primary key, for which all foreign key columns are also ‐3. To know which is true, check the primary key value for the given row: a non‐negative value indicates that this row represents deleted data in the source.
CHAPTER 4-If a foreign key is null in Clarity, what will appear in the corresponding column in Caboodle?
If the foreign key was null in the source, then a ‐1 will appear in Caboodle to represent an unspecified value. If the foreign key’s data lineage was not defined by the source package that loaded data for a particular row, then a ‐2 will appear in Caboodle to represent a value that is not applicable.
CHAPTER 4-A row in a Caboodle table has a primary key value of ‐2. What value will be stored in non‐foreign key columns with a data type of “Date” for this row?
A. ‐2
B. NULL
C. 12/31/1840
D. *Not Applicable
b. NULL
For the row with a primary key of ‐2, foreign key columns will store a value of ‐2, string columns will
store ‘*Not Applicable’, and all other data type columns will store NULL.
CHAPTER 4-TRUE or FALSE: All inferred rows in Caboodle have a primary key of ‐1.
False
Inferred rows in Caboodle have non‐negative primary key values because they represent an entity that Caboodle has inferred the existence of. The foreign key columns for such rows will be set to ‐1 until the Caboodle ETL process updates the data.
CHAPTER 4-Referential Integrity
Referential integrity is enforced in Caboodle, meaning that lookup columns will never contain null values. Furthermore, every value in a lookup column has a match in the destination table.
Caboodle enforces referential integrity by:
- Placing “extra” rows in every table
- These rows have surrogate key values of -1, -2, and -3
- Applying default foreign key values (-1, -2, -3)
- Creating inferred rows
CHAPTER 4-Inferred Row
When a foreign key value is loaded into Caboodle, the ETL process uses its Id and IdType to determine the correct surrogate key value to store. If the referenced row does not exist yet, the ETL process creates a new row in the target table and assigns a new surrogate key value as its primary key. Such a row is called an inferred row.
Inferred rows are created when one table has more up-to-date information than another.
Two common scenarios:
A. Unmatched references in the source
B. Testing or development
Every table in Caboodle has three “extra” rows.
These rows provide a place for foreign keys to point.
CHAPTER 4-The -1 Row
AS PART OF REFERENTIAL INTEGRITY
When a foreign key value is NULL in the source system, the foreign key is said to be UNSPECIFIED.
(Unspecified Foreign Key)
In this case, the corresponding foreign key in Caboodle is set to ‐1.
Since many date columns in Caboodle are
foreign keys to the DateDim table, NULL dates from Clarity are often represented with a ‐1 value DateKey
columns in Caboodle.
CHAPTER 4-The -2 Row
AS PART OF REFERENTIAL INTEGRITY
When a foreign key is not applicable (N/A) as defined by the SSIS package, Caboodle sets the value to -2. This is handled by the ETL process.
CHAPTER 4-The -3 Row
AS PART OF REFERENTIAL INTEGRITY
If an account was deleted in the source, all foreign key values are set to ‐3.
When a record is hard‐deleted in the source system or no longer passes the filters required for extraction,
Caboodle tracks the deletion in one of two ways.
· For Type 1 DMCs, the existing row is updated with default values representing the deletion. The primary key remains unchanged.
· For Type 2 DMCs, a new row is added that represents the deletion. Existing Type 2 data remains, while Type 1 data is overwritten with the default values. The primary key﴾s﴿ and durable key will remain unchanged.
CHAPTER 5-What two columns do all bridge tables have?
Every bridge table has a ComboKey to which other tables join and a Foreign Key used to join to its associated dimension table.
CHAPTER 5-TRUE or FALSE: AttributeValueDim tables are more granular than their associated fact table.
True
The AttributeValueDim tables can store multiple attribute‐value combinations on separate rows
for the same entity, making them more granular than their associated fact table
CHAPTER 5-Bridge Table
Bridge tables exist to capture many‐to‐many relationships, such as the one between patients and the diagnoses on their problem list.
Every bridge table has two columns. They are called ComboKey and Key, where is the name of the table to which a bridge has been formed. The
CHAPTER 5-Entity‐Attribute‐Value data structure
The entity-attribute-value (EAV) framework allows you to essentially add data to existing Caboodle tables without changing the structure of those tables.
Values in an EAV model can be any data type, but every value within a column must be the same data type. The typical way to handle this is to make the column a varchar column and store all of the values as a string.
CHAPTER 5-What is the difference between a MappingFact table and a Bridge table?
Both Bridge tables and MappingFact tables are used to map many‐to‐many relationships. However, Bridge tables use a combo key as part of their definition and always link to a single dimension table while MappingFact tables may link to one or more fact or dimension tables.
CHAPTER 5-AttributeValueDim Table
One form of EAV data in Caboodle is the AttributeValueDim DMCs. These tables contain attribute‐value pairs and can be created for different entities. For instance, there is the PatientAttributeValueDim table to go along with PatientDim, or the DepartmentAttributeValueDim table to go along with DepartmentDim.
AttributeValueDim tables are typically Type 2 tables.
CHAPTER 5-DataMart Table
In the context of Caboodle, data marts are, broadly, collections of Caboodle data on a topic. Some data marts serve as comprehensive data sources for a particular reporting need, making it easier for business intelligence developers to write reports. For example, the HospitalReadmissionAnalyticsDataMart contains all the data relevant for predicting the likelihood of a future hospital readmission.
Data marts are very similar to fact tables because they both store data about significant, measurable
events and refer to data in dimension tables.
Data Mart tables typically have a source of “Caboodle”.
- Data Marts
- Similar to Fact tables
- Group together data relevant to a specific reporting topic
CHAPTER 5-SetDim Tables
STORES GROUPER INFORMATION
In Chronicles, the VCG master file stores groupers, which are lists of Chronicles records (most commonly), ICD codes, or other concepts used together frequently in Hyperspace. VCG groupers are maintained by application analysts in Hyperspace, but can be useful for writing Caboodle reports as well.
PRACTICE EXAM-A Epic recommends using which schema as the default schema for report writing.
CHOOSE ONLY ONE ANSWER
A. dbo schema on staging database
B. dbo schema on the reporting database
C. Epic schema on either database
D. FullAccess schema on the reporting database
D. FullAccess schema on the reporting database
The FullAccess schema provides a logical layer for report writers in Caboodle. It contains formatting for Boolean columns and performs some simple joins to make reporting easier.
PRACTICE EXAM-A TRUE or FALSE: If a data point exists in Clarity, but not in Caboodle, your organization can create a custom SSIS package to extract it to Caboodle.
TRUE
Caboodle is customizable, meaning your organization can bring in additional data from Clarity as well as third party sources.
PRACTICE EXAM-A PharmacyDim is a Type 1 dimension table in Caboodle. Each pharmacy in Epic has a unique record in the PHR master file. Which of the following best describes the difference between the PharmacyKey and PharmacyEpicId columns in PharmacyDim?
CHOOSE ONLY ONE ANSWER
A. When no PHR record ID exits, PharmacyKey stores -1 and PharmacyEpicId is null
B. When no PHR record ID exits, PharmacyKey is null and PharmacyEpicId is -1
C. PharmacyKey uniquely identifies a row in PharmacyDim; PharmacyEpicId stores the PHR record ID
D. PharmacyKey stores the PHR record ID; PharmacyEpicId uniquely identifies a row in PharmacyDim
C. PharmacyKey uniquely identifies a row in PharmacyDim; PharmacyEpicId stores the PHR record ID.
A user doesn’t have access to an RW Report unless both the user and the report share a report group in common.
PRACTICE EXAM-A How can you tell if a column is a foreign key in Caboodle?
CHOOSE ALL THAT APPLY
A. The column is marked as Type 2
B. The ER diagram lists it as a foreign key
C. The column is contained in a fact table
D. The destination table is listed to the right of the column description
B. The ER diagram lists it as a foreign key
D. The destination table is listed to the right of the column description
Type 1 tables do not track changes, so do not need a DurableKey or IsCurrent column.
PRACTICE EXAM-A Table A has a foreign key to Table B. An execution runs, which includes a package that loads data to Table A, but does not contain any packages which will populate Table B. Which of the following tables could have an inferred row added to it as a result of this execution?
CHOOSE ONLY ONE ANSWER
A. Only Table A
B. Only Table B
C. Both Table A and Table B
D. Neither Table A nor Table B
B. Only Table B
Inferred rows are created in the table that is referenced by a foreign key’s value. When data is loaded into table A, it’s foreign key is populated. If Table B does not yet have that foreign key value, and inferred row is created.
PRACTICE EXAM-A If the primary key of a row in a fact table is -1, then
CHOOSE ALL THAT APPLY
A. The foreign keys will store -1
B. The foreign keys will store “Unspecified”
C. The foreign keys will store “Unknown”
D. Columns with a data type of varchar will store “Unspecified”
A. The foreign keys will store -1
D. Columns with a data type of varchar will store “Unspecified”
The -1 row of every Caboodle table behaves in the same way: it’s foreign keys will store -1 “Unspecified”, and all other data type columns will be NULL.
PRACTICE EXAM-A ComboKey is used as a foregin key to which of the following types of DMCs?
CHOOSE ONLY ONE ANSWER
A. Fact
B. Dimension
C. SetDim
D. DataMart
E. AttributeValueDim
F. Bridge
Bridge
Although it is the destination of the join, the combo key column of a bridge table is not the primary key. Rather, multiple rows may have the same combo key value.
OVERALL-Granularity
The level of detail for one row in the table.
OVERALL-Primary Key
The Primary Key is the column(s) that can uniquely ID a row in a table.
OVERALL-Surrogate Key
Surrogate Key is a key where the values don’t exist in the source data. These values are NOT in Clarity. They are generated in the ETL process…They are created when moved.
OVERALL-NAME COVENTIONS IN CABOODLE
Each DMC gets a type. Strict table naming conventions are followed in Caboodle, so that a table’s suffix provides information about its structure and purpose. These suffixes are:
· Dim for dimensions (e.g. PatientDim)
· Fact for facts (e.g. EncounterFact)
· Bridge for bridges (e.g. DiagnosisBridge)
· DataMart for data marts (e.g.HospitalReadmissionDataMart)
· AttributeValueDim for EAV tables (e.g. PatientAttributeValueDim)
· X for custom tables .e.g. CustomFactX
OVERALL-ER Diagram
An entity‐relationship diagram, or ER diagram, is a visual tool that displays information about how database objects relevant to a particular reporting area join to one another. Caboodle ER diagrams are automatically generated for fact tables and data mart tables in the dictionary.
DEFINITION-SURROGATE KEY
A surrogate key is any column or set of columns that can be declared as the primary key instead of a “real” or natural key. Sometimes there can be several natural keys that could be declared as the primary key, and these are all called candidate keys. So a surrogate is a candidate key.
OVERALL-CSN
Contact Serial Number
OVERALL-Type 1
Type 1 tables and columns in Caboodle do not track historical data. When a Type 1 value is updated in the source, all instances of that value in Caboodle are updated the next time the data gets extracted.
If a table is Type 1—THEN all of the columns in that table are Type 1.
OVERALL-Type 2
Caboodle retains historical values by creating multiple rows for each entity, with each row containing the values for a particular time period.
Typically, Type 2 tables contain a mix of Type 1 and Type 2 columns. A Type 2 table must contain at least one Type 2 column. KNOW THIS!!
Type 2 tables typically have a mix of Type 1 and Type 2 columns. Only changes to Type 2 data in the source will result in a new row being added to a Type 2 table for a given entity. When Type 1 data changes, Caboodle stores only the current value in all existing rows for the entity.
PRACTICE EXAM-A Which of the following types of coumns exist in all fact and dimension tables in Caboodle? Chaoose ALL answers that apply.
A. A surrogate key that acts as the primary key of the table.
B. Count column
C. Zero or more foreign keys
D. A DurableKey column
A: A surrogate key that acts as the primary key of the table
C: Zero or more foreign keys
All fact and dimension tables have a surrogate key primary key column. Fact tables have a Count column, but dimension tables may not. Only snapshot tables have a DurableKey column.
PRACTICE EXAM-A Some tables in Caboodle take snapshots of source data. This best describes which of the following? Choose only ONE answer.
A. Caboodle enforces refrential integrity
B. Some Caboodle tables use change tracking.
C. Caboodle uses the dimensional data models
D. Caboodle uses the entity-attribute-value data model
B: Some Caboodle tables use change tracking
Snapshot tables have special columns to help track these changes: DurableKey, StartDate, EndDate and IsCurrent
OVERALL-IsCurrent
The IsCurrent column is a flag that stores the value of 1 if the row holds the most current information and 0 otherwise.
PRACTICE EXAM-A Consider the following query: Which of the following logical expressions will return all hospital admissions where the admitting provider was the same as the discharging provider?
SELECT EncounterEpicCsn, AdmittingProviderKey, DischargingProviderKey
FROM HospitalAdmissionFact
WHERE and AdmittingProviderKey >0 and DischargingProviderKey >0
CHOOSE ONLY ONE ANSWER
A. AdmittingProviderKey = DischaringProviderKey
B. AdmittingProviderDurableKey = DischargingProviderKey
C. AdmittingProviderDurableKey = DischargingProviderDurableKey D. All of the above are equally valid
C. AdmittingProviderDurableKey = DischargingProviderDurableKey
When comparing snapshot entities, like providers, it is most important to compare the durable key values, since there may be more than one surrogate key value associated with a single entity.
OVERALL-Enforcing Referential Integrity
Caboodle uses three mechanisms for enforcing referential integrity:
· Every table in Caboodle contains three rows with surrogate keys of ‐1, ‐2, and ‐3. These rows serve as the destination for lookup columns in other tables.
· When source data is unavailable to populate a lookup column, Caboodle supplies a default value of ‐1, ‐2, or ‐3. Which value is used depends on the circumstance behind the missing information.
· When data loaded in one Caboodle table is more up‐to‐date than data in another table, the Caboodle ETL process generates an inferred row in the delayed table. This row represents the entity being referenced by the more up‐to‐date table. Inferred rows act as place holders until the relevant information is loaded into Caboodle.
OVERALL-Inferred Row
When a foreign key value is loaded into Caboodle, the ETL process uses its Id and IdType to determine the correct surrogate key value to store. If the referenced row does not exist yet, the ETL process creates a new row in the target table and assigns a new surrogate key value as its primary key. Such a row is called an inferred row.
KNOW THIS
When an inferred row is created, all its string values are set to *Unknown. Additionally, all of the inferred row’s foreign key columns are assigned a value of ‐1 and any remaining data columns are set to NULL.
The inferred row is a placeholder, so a later ETL will update these default values when that information is populated in the source database, regardless of the table’s change type
OVERALL-EAV
Entity Attribute Value
The three major components of the EAV data model are entities, attributes, and values.
Let’s define these terms by looking at a particular example: suppose your organization is tracking patient hair color. In this case, an ENTITY is a patient and the ATTRIBUTE is hair color. A VALUE would be a particular patient’s hair color, such as red.
(General Modeling Terms) Business Key
A business key is an identifier for a record in Caboodle. The business key is a combination of column identifies a record, and is based on source system identifiers. For example, a business key could consist of a patient’s ID number and an ID type column identifying the number as an MRN. In other words, the business key is tied to the actual extracted data, unlike a surrogate key. For reporting efficiency, durable and surrogate keys should be used in reports instead of business keys.
(General Modeling Terms) Data Lineage
The data lineage is a precise technical explanation of where data in a package is extracted from in a source database. This explanation can be as simple as a source table and column in Clarity or much more logically complex as in the following ex:
(see doc @ https://galaxy.epic.com/?#Browse/page=1!68!50!3517622&from=Galaxy-Redirect)
(General Modeling Terms) Durable Keys
A durable key is an identifier for a record in Caboodle, shared across rows for the same record. Unlike a business key, a durable key consists of a single column and isn’t derived or extracted from the source data. For example, because patient data is tracked over time, a single patient record might have multiple rows associated with it in PatientDim. Each row has the same durable key to make it easy to find all rows with a particular patient. Durable keys are used only in tables that store snapshot (type 2) data.
(General Modeling Terms) Foreign Keys
A foreign key is the primary key for a row in a different table in Caboodle, used to link two tables together. For example, ProcedureOrderFact contains a PatientKey column, which corresponds to the surrogate key in PatientDim. Business intelligence developers use foreign keys to write reports using data from multiple tables.
For ease and efficiency of report writing, foreign key columns in Caboodle have refrential integrity. In other words, foreign key columns never contain null or unmatched values.
(General Modeling Terms) Caboodle contains rows with -1,-2,-3 as the primary keys, to indicate data that is unspecified, not appllicable, or deleted.
When a foreign key value is expected but its value is null in the source system, the foreign key is said to be unspecified. In this case, the corresponding foreign key in Caboodle is set to -1. For example, suppose a patient hasn’t been assigned a primary care physician. When the patient’s record is loaded into Caboodle, the PrimaryCareProviderKey column is set to -1. That column would then join to the -1 row in the ProviderDim table.
Foreign keys also refer to the -1 row for inferred data. For example, suppose your organization hires a new physician, and that physician is assigned as a patient’s primary care provider. If the patient’s record is loaded into Caboodle before the new provider’s record, it contains a foreign key reference to a row that doesn’t yet exist, but is expected to be loaded eventually. In this case, Caboodle makes a new inferred row in ProviderDim for the expected provider. When the new provider’s information is eventually loaded into Caboodle, its values overwrite the default values that were assigned to the inferred row.
When a foreign key doesn’t apply for a certain row, the foreign key is said to be not applicable. In this case, the corresponding foreign key in Caboodle is set to -2. For example, BillingAccountFact contains information related to both hospital billing accounts and professional billing accounts. Only hospital billing accounts have admitting providers for the hospital encounter, so rows for professional billing accounts have AdmittingProviderKey set to -2.
When a row in Caboodle is marked as deleted, the surrogate key isn’t hard-deleted from your database. Instead, all strings are replaced with *Deleted, foreign keys are set to -3, and other columns have null values.
(General Modeling Terms) Surrogate key
A surrogate key is a unique identifier for a row in Caboodle, and is also a table’s primary key. Unlike a business key, a surrogate key isn’t derived or extracted from the source data. Instead, it’s applied in Caboodle. Unlike a durable key, each row has a different surrogate key.
(General Modeling Terms) Snapshot data
A snapshot column tracks changes to data in the column by creating a new instance of the row each time data in the column changes. A snapshot DMC contains one or more snapshot columns. For example, if you use snapshot data for columns related to a patient’s address, a new row for the patient is created each time the patient’s address changes.
If a column or DMC isn’t marked as snapshot, it doesn’t track changes over time and stores only the most recent information for a record. If the data changes, it overwrites the previous value. For example, if you don’t track changes to a column for a patient’s Social Security number, any changes to a patient’s SSN overwrite the previous value in existing rows for the patient.
Snapshot data represents information at a particular point in time. The dates associated with this snapshot aren’t clinically relevant because the dates correspond with when the data was loaded into Caboodle, not when an event actually occurred or was changed in an upstream system like Chronicles. When you write reports, never treat snapshot data as historical clinical data.
Prior to August 2020, snapshot data is called type 2 data, and other data is called type 1 data.
(Dimensional Modeling Terms) Data model component (DMC)
A DMC is a Caboodle-specific concept that refers to a table in the Caboodle reporting database and its supporting infrastructure. DMCs can be facts, dimensions, bridges, data marts, or informational tables.
(Dimensional Modeling Terms) Fact
Fact tables hold all the measures in Caboodle. These are the primary tables that contain the many rows of data created in the source system, such as individual encounters, orders, and transactions. These facts are typically joined to multiple dimensions to add extra information that helps define the facts. These clusters of facts with multiple dimensions make up the star schema on which a dimensional data warehouse is built. In general, fact tables in Caboodle contain numeric data and link to dimensions and profile dimensions for other information.
(Dimensional Modeling Terms) Dimension
A dimension is a table in Caboodle that contains attributes describing one or more facts. Dimensions are joined to facts to provide descriptive information about them. For example, PatientDim contains patient information that applies to many DMCs.
(Dimensional Modeling Terms) Bridge
A bridge is a table that stores information where many-to-many relationships are common. For example, consider diagnoses and patient records. Each type of record might be related to many of the other type: a patient can have multiple diagnoses, and a diagnosis can appear on many patients. In a bridge table, each key references a unique combination of records that exists in your organization’s extracted data. This method of storage helps improve performance because if multiple patient records have the same associated diagnoses, the combination of diagnoses appears only once instead of many times.
(Dimensional Modeling Terms) Data Mart
A data mart is a collection of Caboodle data on a topic. Some data marts serve as comprehensive data sources for a particular reporting need, making it easier for business intelligence developers to write reports. For example, the HospitalReadmissionAnalyticsDataMart contains all the data relevant for predicting the likelihood of a future hospital readmission. Data marts are very similar to fact tables because they both store data about significant, measurable events and refer to data in dimension tables. However, data marts can do direct key lookup to retrieve data, which means they can link to other tables using a single Key row instead of two rows for the ID and ID type.
(General Reporting Tips) Starting in August 2019, add the following SQL command to the top of every report:SET TRANSACTION ISOLATION LEVEL SNAPSHOT;
Running your reports with snapshot isolation level ensures the reports and Caboodle executions don’t interfere with each other.
(General Reporting Tips) Use the views on the FullAccess schema.
This schema simplifies your reports by aggregating the information you find in facts, profile dimensions, and bridge tables. It also includes only the data that passes Caboodle’s data quality checks, which are available starting in August 2018. If you use the dbo schema, reports can include data that’s considered invalid.
Materials to Review Before the CDW110v Exam
https://galaxy.epic.com/?#Browse/page=8400!68!280!4022506,4022507,4022508