Configuring Sources

One or more Sources can be added to a Project.
Hover over the menu icon in the top left corner and go to the Sources menu.

 

MetaData that will be used as Input for VaultSpeed can be located either in Files or in a Source DataBase. To allow the VaultSpeed Tool to read MetaData from a Database, a Database Link has to be specified. 

An overview of the database links can be found by clicking on the “DBlinks” Button in the Right Upper Corner of the Source Page.

Adding a DataBase Link can be done by clicking on the “+DB link” Button in the Right Upper Corner of the DB Links Page, of by clicking on the  Button while Adding a Source (see Add a new Source).

Database Information (including User Login and Password !) can be stored either in the VaultSpeed Cloud Application(not recommended), or in the Agent configuration (locally setup by the Client) :

  • Choosing the Agent Properties File Option will use the DataBase Information as stored in the Local Agent Setup. For each DataBase Link, an entry <dblinkname>.url = <url> should exist in the "connections.properties" file, where the dblinkname is the same as the Database Link Name in the VaultSpeed Cloud Application. Details on adding a DataBase Link to the VaultSpeed Agent can be found in the Installation instructions for the VaultSpeed Agent, 2. Edit the Agent Configuration to match your Paths, Open connections.properties. The (Source Type dependent) Format for the URLs in the Properties File can be found at JDBC URL formats :

  • Choosing Settings Form will store the DataBase Information in the Cloud Application :

 

After Adding the Links, the DB links Screen will show all Links, with an URL if the Settings Form was used.
The URL can be updated by double-clicking the URL, and clicking on the eye icon to make the URL visible.

By clicking the orange button at the right of a DataBase Link, a job will be started to Test the DataBase Connection. Pressing the red task button at the Bottom right will show the Result of the Test :

 


Add a New Source

To create a new Source, click the button :

 

  • Choose a Name for the Source

  • Select the Project to which the Source should be added.

  • Choose a Short Name. This Name will be part of the Satellite Names that VaultSpeed will generate, in order to indicate what the Source System for a Satellite is, for e.g. SAT_DEMO_PRODUCTS (where DEMO is the Short Name of the Source System).

  • The Record Source Name is an optionally informative field in the HUB. Here you can add a value to uniquely identify a source.

    • Three parameters impact this field/value (all on Data Vault level):

      • ADD_RECORD_SOURCE_ATTRIBUTE: Enabling this will add an extra record source attribute to the hub object.

      • RECORD_SOURCE: The name of the column that stores the record source in the HUB.

      • RECORD_SOURCE_VALUE_TEMPLATE: The template how to store the record source, this can be done by a template or by hardcoding a string. The templates should always be lower case like seen in the next examples:

        • <record_src_name> (the value of the field Record Source Name in the source creation)

        • <table_name> or <table_short_name> or <table_abbreviated_name>
          --> the value between the <> will be replaced with the real values during code creation

  • The Source System Business Key is the Value that will be inserted into the Source Business Key Column for Multi Master Hubs (see Multi Master Hubs and Sat Splitting).

  • Change Data Capture describes how this Source tracks changes : 

    • Change Date Capture (The Source System records INSERTs, UPDATEs and DELETEs applied to Source Tables and makes a Record available of what changed, where, and when, in simple Relational 'Change Tables' )

    • Modification Date (Source Tables contain a Column which contains a Date & Time on which the Source Record was Created and/or Updated)

    • No Change Data Capture (The Source System contains no indication of when a Record was Created and/or Updated; Vaultspeed will need to detect changes).

For Options Change Date Capture and Modification Date a number of Parameters will have to be set on Source Level by clicking on the Tab after Adding a Source.

When using Modification Date, the data will be filtered using a loading window, this means that no deletes can be detected in this case.  The name of the field that is used in this source must be entered under the parameter CDC_TIMESTAMP or be set per object when configuring the source objects.

If No Change Data Capture is used, then a full dump of the data is expected for every incremental load, keys that are no longer in the data set will be marked as deleted.

If a source has a mix of these settings, set it to the one with the highest number of tables.  Afterwards this can be modified on the table level.

  • The Source Type contains the MetaData Collector Location :

    • Agent (Vaultspeed will parse the MetaData from a Data Model in a DataBase).  A DataBase Link has to be Selected or Created (see DataBase Links) and a Physical Schema has to be defined. In case multiple Physical Schema's are used in the Source DataBase, a Source will have to be created for each Schema.

    • File (Vaultspeed will parse the MetaData from one or more Files using the browser, no data will be uploaded to the site, only the parsed metadata).

In both cases, VaultSpeed will only gather MetaData, not the data itself.

 

  • Build Flag : turning off this Flag means that this Source System will no longer be included in the Data Vault(s) of this project.

Press “Save” to Add the Source, “cancel” to Stop without Creating a Source or “clear” to Clear all Values entered.

Review the Parameters on Source Level by clicking the “Parameters” button at the right of a source. Make sure to check Change Data Capture Parameters (Categories "CDC parameters" and "Attribute names"). The "Schema names" Parameters will be set based on the source name, this can be tweaked by changing the parameters on a higher level (see parameter description).


Source Metadata

 

Import Source Metadata

When you click on a Source, additional Tabs will appear in the Top Bar :

Click on the "Select Objects" Tab :

Depending on the Source Type, 2 or 3 Buttons will be shown in the Top Right Corner.

For Source Type File:

For Source Type Agent:

For Source Type "File" one or more files have to be uploaded by using the “upload files” Button.
By pressing this Button, the Upload Window opens. Click Choose Files in order to select the file(s) to be uploaded, and click on the Upload Button. File format information can be found at https://vaultspeed.atlassian.net/wiki/spaces/AF/pages/72777775.


When the File Structure is successfully Loaded, the following Message will be shown :

The Table Names will NOT be shown in the Object List yet, this will only happen after the next Step.

For both Source Types, click the “Get Objects” Button. This will submit a few Tasks in order to retrieve the Source MetaData. Track the Progress of these Tasks by pressing the red tasks Button in the Bottom Right Corner; 3 Jobs should be Completed Successfully (the first 2 will be executed by the agent): 

When the Task are Completed, Refresh the Object List by Clicking the Refresh Button . The Imported DataBase Objects will appear in the List.


Select Objects 

By Default, all Imported Objects will be considered as deselected (Select Flag is turned Off).

Objects can be included by Turning on the Selected Flag in 2 ways :

  • To easily include all the Objects that were found, click the Tick Mark next to the Selected Flag in the Header Row. This option will turn on the selected flag for all objects. This can be combined with the search option to quickly enable or disable a set of similarly named objects.

  • Turn on the selected flag for each object individually.

Another possibility is to use a CSV file with a selection of the tables you want to include. This CSV needs to be put in the source_metadata folder in the local agent. After adding the file, click “Get Objects” and then “Upload csv”, the object selection should now be updated.
The CSV file should have the format: 

  • Filename: include_table_list.csv 

  • Columns: owner (= name of the schema where the table is available ),table_name (= name of the table)

Example file: include_table_list.csv

Exists in Source Flag 

When an object is no longer available in the source metadata, it will be indicated using the Exists in source flag and a red text color.

 


Exclude Objects 

The Excludes tab can be used to define Filters to Exclude Objects and/or Attributes.

Wildcards can be used in this Filter.

Example : when Temporary Tables (Tables with Prefix "temp") have to be omitted, click “Add exclude object type” in the Objects Tab to Add a Filter and use Value:

After Adding the Exclusion Rule, an Impact Check can be done. The Number of Tables that will be Excluded will be shown :

Exclude Rules can be Changed and/or Deleted by double-clicking the Exclude Line.


Exclude Attributes 

In a similar way Data Columns can be Excluded.  

For example: Add "update_timestamp" as Column Type. Ignore Reason is a Field which contains a Description why these Columns are excluded :

CDC metadata attributes such as transaction timestamp and cdc_flag should be excluded here when reading from cdc tables. Vaultspeed will add them later as metadata attributes.


Remove Object Name Patterns 

The Remove Object Name Patterns Tab allows you to remove a Pattern from Object Names.

In the Remove Object Name Patterns screen, use “Add remove pattern” to Add a Pattern which will be removed from the names of all objects in this source. the remove Reason is a Field which contains a Description of why these Object Name Patterns are excluded:

After Adding the Remove Rule, an Impact Check can be done. The Number of Tables that will be involved will be shown:

 

As a result, the Table Name without the Pattern will be used throughout the DataVault generation. For e.g. a Source Table tws_clients will treated as Table clients.

Be aware that the Pattern will be removed everywhere in a Table Name, and not only as Prefix such as the above example.

 


Create a Release 

 

Create a Source Release 

Navigate to the Releases & Builds tab :

Click “+Release” to Add a new Release :

 

The "Keep from previous Release" Options allow you to 'reset' the Model and regenerate MetaData based on only the latest Source Model. If there is a production release, then it will reset the source to the status of that release instead of completely resetting it.

  • Everything : All Tables, Columns and Constraints that were imported in Previous Release are retained, new Tables, Columns and/or Constraints will be Added. 

  • Tables and Columns (remove Constraints) : All Tables and Columns that were imported in Previous Release are retained, new Tables and Columns will be Added and all Constraints will be Dropped and Added (again). 

  • Tables (remove Columns and Constraints) : All Tables that were imported in Previous Release are retained, new Tables will be Added and all Columns/Constraints will be Dropped and Added (again). 

  • Nothing : All Tables, Columns and Constraints will be Dropped and Added (again). 

 

Disabling the Re-import flag will create a new release without reading from the source again. This is useful when you have to make a small change to the previous release and do not want to read the source metadata again, or because the source has changed in the meantime but you do not want to include those changes yet.

Creation of a Prototype Build

After Clicking the “Add” Button, 3 Tasks are started in order to create a Release :

  • Read the Source MetaData (Agent).

  • Send the MetaData from the Agent to the Cloud Application (Agent).

  • Create the Release on the Cloud Application.

These Tasks may take a while to complete. 

Track the Progress of these Tasks by opening the task center

Refresh the Status of the Tasks by clicking the following Icon.

Refresh the Release Overview by pressing the “refresh” Icon on the releases screen. The New Release will be shown :
 

The Release status of a New Release is "WORK IN PROGRESS", meaning this Source is not yet ready to be included in a Data Vault. And the lock at the right is open which means that the source model has not been locked yet and can still be edited.


Copy a source

To copy a source, click the button :
Remark: if the latest release of this source to be copied is from before the end of 2020, make sure to create a new release with a new source metadata import. Otherwise, the copy source function could fail. Copy Source will be disabled if the selected source does not have a locked release.

Only the project is not automatically chosen. So make sure to select a project.

Please make sure the new source has a unique name, Short Name, Record Source Name and Source System Business Key within a project and a unique name across projects.

Next to the normal fields, three extra toggles are shown:

Copy Source parameters: Copy the source parameter values to the new source (except schemas).

Copy Table parameters: Copy the table parameter values to the new source.

Copy Source Schema: Copy the values of the source schemas to the new source.

 

Click Copy:

Copying a source will insert two tasks:

Copy a source release: The source metadata + source editor metadata will be copied.

Save source version: The preparation for the hub: Single Master/ Multi master screen will be executed.

Important: The single master multi master properties + sat splitting will not be copied and need to be done by the end user. Thats why the status of the release of the newly copied source will state: work in progress.

So if there is no need for changing the multi-master properties or adding a sat splitting, just select the latest release and click Lock. Otherwise, make the needed changes and then click lock. The source can now be used in the Data Vault.


Managing Objects, Relations, and Attributes

When you click on a Release, an additional Tab will appear in the Top Bar.


Source Editor

When using the Editor, a set of objects needs to be added to the canvas, where they are visualized with their dependencies and where they can be edited. Objects can be added by using the Selection pane which shows all available objects from the selected source. In order to add objects, simply click and drag an object to the editor pane. To load all objects from the source, drag “ALL OBJECTS” item to the editor. Note that a maximum of 30 objects can be shown at one time, this is to avoid performance issues. When you attempt to add more than 30 objects, then only the first 30 will actually be shown.

The “ALL OBJECTS” option will load all Tables and Relations (up to 30). This is mainly useful for smaller sources as a way to get an overview of the source model.

A Single Object can be added by clicking and dragging on a single item in the selection pane with a depth value of 0. Depth is used to get adjacent connected objects. Depth 0 means only a single object is loaded whereas depth > 0 would load additional connected objects in the relevant layers (i.e. depth 1 one layer deep, depth 2 - two layers deep, and so on).

This option will add the Selected Table to the canvas.

Multiple Objects can be added by selecting and dragging objects from the selection pane with a depth value larger than 0. This option will only show the selected object and the objects to which a relationship(s) exists. The depth value defines the number of levels of relationships that will be shown. 

Depth: 1 → Only the Selected Table (COUNTRIES) and the Tables which have a Relationship to this Table (REGIONS, LOCATIONS) will be shown :

Depth: 2 → Only the Selected Table (COUNTRIES), the Tables which have a Relationship (Depth=1) to this Table (REGIONS, LOCATIONS), and the Tables which have a Relationship (Depth=2) to the Depth=1 Tables (REGIONS has no related Tables except COUNTRIES, JOB_LOCATIONS and DEPARTMENTS  are related to LOCATIONS) will be shown :

The type of name shown in the selection screen and the canvas can be modified in the extras menu.
The available options are:

  • full name(default): This means that the name of the object from the source, minus the removed patterns, is used.

  • abbreviated name: Use the abbreviated names of the objects.

  • short name: Use the short names of the objects.

This can help for sources where the source object names are not intelligible, and the abbreviated names are used to give meaningful names to the objects for use in the Data Vault.

 

The editor can be cleared by clicking on the clean page item found in the toolbar as shown below. The toolbar also contains some key actions for zooming in and out, for deleting a selected item, for cantering objects in the window, and for changing the layout of objects from vertical to horizontal and vice versa.

Some objects may appear red in the selection pane as well as on the canvas. This indicates that there is a problem with the object which needs to be fixed before you can proceed.

Add object with an error to the canvas. Selecting an object will show its details with an error message in the properties pane as shown below.

In the above example, the error message states that the object has no Primary Key defined. In this case, the object only contains information about a relationship between other objects. This kind of object should be defined as a Many-to-Many Link which requires at least two Foreign Keys.

In order to define this table as a Many-to-Many Link, double-click in the properties pane and a number of fields will become editable. Change the object type to "Many-to-Many Link" and press the “Save” button to commit the changes: you can also change the type of an object by right-clicking on it in the canvas and using the context menu.

 

Use the KEYS tab to verify that all necessary relations exist. On the canvas, the table will no longer be highlighted in red, and is shown as a "Many to Many Link":

 


Manage Objects

In order to change object properties, click on the to show properties pane. Double-click on the proeprties panes and a number of fields will become editable. Press “Save” button to commit the changes you made.

The following object properties can be changed :

Short Name : The Short Name for a Table. This is used for creating LNK table names.

Abbreviated Name : The Abbreviated Name for a Table. This is used for SAT and HUB names, as well as HKEY attribute names.

Object Type : Select one of the following :

  • HUB & Satellite : For this Table a Hub and a Satellite will be generated in the Data Vault Model.

  • Reference Table : This Table is a Reference Table.

  • Satellite : For this Table only a Satellite Object will be generated in the Data Vault Model. This object needs a 1 to 1 relation to a Hub object. 

  • Many to Many Link : the Table only contains information about a Relationship between other Tables.
    A Many to Many Link needs at least 2 relationships to HUB & Satellite objects.

  • Non-Historized Link: (also known as Transactional link) - type of relation based link, where data can not be modified in the source system.
    A Non-historized Link needs at least 2 relationships to HUB & Satellite objects.

  • Same as Link:  A type of link that indicates that the related objects have the same content.
    A Same as Link needs at least 2 relationships to HUB & Satellite objects.

  • Hierarchical Link: A type of Link that defines a parent child relationship.
    A Hierarchical Link needs at least 2 relationships to HUB & Satellite objects.

More types will be added in the future.

CDC : Select one of the following :

  • Change Date Capture : the Source System records INSERTs, UPDATEs and DELETEs applied to Source Tables and makes a Record available of what changed, where, and when, in simple Relational 'Change Tables'.

  • Modification Date : Source Tables contain a Column which contains Date & Time on which the Source Record was Created and/or Updated.

  • No Change Data Capture : the Source System contains no indication of when a Record was Created and/or Updated; VaultSpeed will need to track all changed Records).

Comments : Add a Comment for this Table.

Multi Active : Can more than one valid entry in the Satellite be assigned to a Business Key in the Hub for a certain Point-in-Time? (In the Data Vault Standard this is called a Multi-Active Satellite).

Objects can be ignored. In that Case they will not be Used during Data Vault Builds. In order to ignore an object, toggle “Ignored” button and enter ignore reason below it and save the changes. To un-ignore object, simple disable this toggle.

Another option is to open ignored objects view using the “Extra” menu and selecting the “See all Ignored Objects” menu item:

 

This will open a table view of all ignored objects in the selected source release:

Click on the Unignore Table on the right :

 


Manage Parameters

A number of Parameters can be changed on Object Level (See 3. Parameters for more details on Parameters) . In order to do this, Click on the Table. A properties pane will appear. Parameters can be edited on in the Parameters section. Double click on the parameter to change it.

Press “Save” button to commit the changes you made.


Manage Attributes

In order to change Attribute properties, click on the attribute in the editor and the properties pane will show its information which can be edited. Attribute properties have two sections: General and Attribute Types. Double click on the Attribute property to go into edit mode. After changes are made make sure to save them before continuing.

The following Attributes properties can be changed within the Properties pane:

Short Name: the Short Name (automatically generated based on the short name length parameter). When there are multiple LNK’s between two tables, the short name of the attribute which uniquely defines the link will be added to the LNK & LKS names (this can be changed later in the DV name management).

Abbreviated Name: the Abbreviated Name(automatically generated based on the abbreviated name length parameter), this is used when creating attributes in the DV with a prefix or suffix, such as business keys and subsequence attributes.

Data Type : When the CAST_SOURCE_ATTRIBUTES parameter is enabled, you can change the Data Type of the attribute.

Data length, precision and scale : Same as data type.

Comments : Add a comment to this attribute.

Format Mask : Define the Data format, if nothing is defined, the general format mask will be used (DV parameters).
Note that this format must be valid for the target database type, meaning that ids must be entered when the source is used in a SQL Server or Azure Synapse Data Vault.

Numeric Decimal : Select either ',' or '.' as decimal Separator.

Mass Storage : Large Source Fields can be marked as "Mass Storage Field".  The Target Data type will be CLOB, VARCHAR(MAX), ... (depending on Technology).  Marking a Field as "Mass Storage Field" will also turn off the History Attribute for this Field. 

Nullable : Specifies if this Field contain Null Values.

Ignored : Attribute is excluded from further data vault modeling.

Attribute Type : Select one of the following :

  • Universal Identifier : The Field is a Universal Identifier (this is the same as setting it to Business key, and is mainly added for documentation purposes).

  • Business Key : The Field is a Business Key.

  • Primary Key : The Field is part of the Primary Key.

  • Unique Key : The field is part of a unique key, you can enter a name or let us generate one. For multi column UK’s you have to enter the same name, or use ctr-click to select multiple attributes on the canvas and then right click and set unique key for all of them.

  • Non Historic : This option will turn the History Attribute of this Field off, and will consider the Field as "Attribute". Attribute will not be used in hash diff calculation.

  • Attribute : The Field is a descriptive Attribute.

  • Modification Date : This field is the Modification Date, this will ignore this attribute and set the CDC_TIMESTAMP parameter for the table to the name of this attribute.

  • Subsequence Attribute : For Multi Active Satellites, this attribute together with the business key should be unique. This is also sometimes referred to as the Dependent Child attribute.
    For example: a source table contains for the same business key all descriptions in all 3 different languages and so then the language attribute is the Subsequence Attribute.

  • Unique Identifier: An attribute types used with Non-Historized Link object

The attribute type can also be changed by right clicking on an attribute in the canvas.

 

Short and Abbreviated Names can be downloaded and uploaded from a CSV File. These actions can be found inside the Extra dropdown menu as shown in the picture below:

 

Using the Download Button, 2 CSV Files will be downloaded on your Computer, a Short and Abbreviated Names overview containing Short Names and Abbreviated Names for all Tables in the Source ...

... and a Short and Abbreviated Column Names overview containing Short Names and Abbreviated Names for all Columns in the Source ...

After changing one or more files, press on Upload Short and Abbreviated names button and follow the instructions.

The Mass Update menu item allows to Update Multiple Objects and Attributes for Multiple Objects using a Pattern/Expression.

In order to Perform this Update, Fill the Pattern to select a set of attributes and Select the Attribute Field to change.

For e.g. when you want all Columns that contain _ID in their name to Not Nullable, fill _ID as Pattern, choose Attribute Field "Nullable" and switch Nullable Flag off :

Click on the preview button to see the Impact before executing the Mass Update. A list with all Columns that are impacted will be shown, including the current Nullable Value :

On this Screen you can still decide to omit some of the proposed Updates by deselecting the Line.

Click on update to execute the Update.


Manage Constraints/Relations

Relationships between objects are shown with a single line. In order to view constraints (relations), click on the relationship line and fields used in the relationship will be shown (see video below). Clicking away on the canvas will close the relations view and return to the relationship view. In addition, clicking on the relationship or relation line will show its properties inside the properties pane.

In order to draw a relationship, add two objects to the editor. Then simply hover on the attribute from which a relationship needs to be drawn. Click on the blue arrow and start drawing a line. Drag it onto the target attribute. When the line has been drawn click to save the relationship. Leaving Key name field empty will auto generate the constraint name based on the attribute names. Relationship may contain multiple relations/constraints, so it is possible to draw multiple lines before saving the realtionship.

Similarly, a self-relationship can also be added between attributes of the same object:


In order to select an object from another source, first the source needs to be changed in the selection pane:

 

The orange colour identifies the current source in the dropdown while purple is used for other available sources within the project. Greyed out selection items are disabled since they have no locked releases.  After selecting another source, there is an option to select one of its locked releases after which an object from that source can be loaded into the editor. Similarly to the basic functionality of the graphical editor, the objects from another source can be dragged to the editor. After an object from another source is added to the canvas, it can be used to create links across sources by drawing a relationship. Drawing a relationship is only possible from current source objects to objects of the other source, not the other way around. Objects from another source are identified by the purple colour and contained the schema name in front of the object name. This twice needs to be a Hub and Satellite:

Details of the objects and their attributes from another source cannot be edited - only relationships can be drawn or deleted.


 

Graphical Overview

 

The Graphical Overview allows you to quickly create a Visual Representation of a Source. This is especially useful for checking whether all Relationships are defined, this will not show anything when you have only one object.

Click “Generate” to see a visualisation of the complete Source :

Table Attributes can be Added to the Canvas by Clicking on Attribute and then generate:

The Shortest Pad Option shows the shortest RelationShip between 2 selected Objects, for e.g. 

The Inspect Single Object Option shows selected Object, and its RelationShips to a certain Level (Depth). for e.g. Depth 1 means the RelationShips to the requested Object itself : 

The Compare Option will show the differences between the previous Source Release and the current one. New and re-Enabled RelationShips will be shown in Green, Disabled and Removed RelationShips in Red.

The Legend will show the different Colours and Icons used on the Visualization, turning “Disable physics” on will prevent Objects from 'moving around' on the Canvas ...

 


 

Save the source model

 

When a Source has been set up completely (make sure no more Errors exist !) the State of the Source has to be 'Saved'. Click on the New Build Tab :


“Save source model” will lock the Release (meaning no further Changes can be made to the source structure without creating a new Release). A "Create Final Version" Task and a “Save source version” task will be queued.

When the tasks are finished, go back to the Releases & Builds Tab. The lock icon is now closed to indicate that this Release is Locked, the Release Status remains "WORK IN PROGRESS".

 



Single/Multi Master Hubs and Satellite Splitting

 

After Locking a Release by creating a Final Build, the Tab HUBS & SATS is revealed. The goal of this Tab is to set some Properties for Hubs and Satellites as preparation for a Data Vault Build.

3 Tabs can be found on this Page : 

  • HUBS in order to define whether a Table is considered as Single or Multi Master Table

  • SAT Splitting in order split Satellites into more Satellites

  • Graphical Sat Splitting Editor in order split Satellites into more Satellites in a more graphical way.

Single/Multi Master Hubs 

Defining a Hub as Multi Master will add a Source Key Column to the Hubs and include this Source Key in the Hash Key Calculation. The Content of this Source Key Column is defined when Setting Up a Source (Source Name in BK). Defining a Hub as Multi Master is necessary when a Hub is used by different Sources and there is no guarantee that the same Business Key has the same meaning in the different Sources, or when there is no Source which is the Master i.e. contains all keys.

For e.g. HUB_EMPLOYEES is used by different Sources, but EMPLOYEE_ID 1234 in Source X is not the same Person as EMPLOYEE_ID 1234 in Source Y. By defining the Hub as Multi Master 2 Hub Records will be created :

1 2 3 SRC_BK    EMPLOYEE_BK X         1234 Y         1234

By Default, all Hubs are defined as Multi Master.

The Hub type can be set one by one, or by using the Convert all buttons in combination with a search.

The following button will show a List of all Hubs defined as Multi Master

 


 

Satellite Splitting

In some case is it useful to split the content of a Satellite, either by Rate Of Change or by Content. Another Reason to Split a Satellite could be GDPR Compliance. If certain Attributes have to be 'physically' Deleted in a Satellite, Separate them with a Satellite Split and Enable 'Hard Delete' during Data Vault Creation.

2 Types of Satellites will be generated : Hub Satellites and Link Satellites. The button allows you to switch between both:

 

Means there are not enough Attributes in the Satellite, so it can't be Split.

Splitting a Satellite can be done in 2 ways. In the Example below, we will Split Satellite Employees. This Satellite (SAT_MM_CONTACTS) has the following Attributes :

 

We will Split it up in a CONTACT_DESC Part :

 

... and a CONTACT Part :

 

Click on an Object in order to Split it. A new Screen will open :

 

Click on the Add button in order to Add A Split Destination, and give the Destination a Name and a Description (Reason to Split). Do this for both CONTACT_DESC and CONTACT. 

 

Both Destinations are shown on the Canvas now.

 

You can Drag and Drop Attributes to a Destination(multiple attributes can be selected at a time):

 

The Attributes are now Split into 2 Destinations/Satellites:

 

Attributes can be removed from a Split Destination (again) by Drag & Drop it to either another Destination or the Attributes List.

An Attribute can be used in multiple Split Destinations :

 

Use the following Button in order to Change the Name of a Destination, the Name will become editable :

Use the following Button in order to Delete a Destination. This Button is only available when the Destination no longer contains Attributes.

 

Another Way to Split is by using the attribute based split, which can be found on both Overview and Detail Canvas for SAT Splitting.

Please Note that this option is not restricted to a certain Table, but to the whole Source! 

In order to Split, fill a Split Name, the Attribute(s) that should be moved to that Split, and a Split Name for the 'remaining' Attributes. The 'Split only sats with all the attributes from the list, or split sats with any of the columns'  Option is by default turned on. Click Add to Split :

When there are already splits, it will ask for confirmation if it is ok to skip those:

Clicking the

will show an overview of all splits.

The splitting can also be done in a more graphical way

In this example we can split sat_mm_motorcycles by dragging and dropping this satellite on the canvas.

Add a Split Destination by right clicking on the main object

In this case we add two splits, name and update.

 

To add attributes to the splits, just drag and drop them from the main object.

They will turn green once used. To remove an object just drag it back to the main object.


Complete the Source Release

 

The Last Step for a Source is to Lock the Source Release in a Final Build. In this way the Source is Ready to be used in a Data Vault Model.

Click lock/save in the Upper Right Corner in order to lock the Source Version.

Clicking Lock will start the Task to Lock the Source Version. Check the Status in the Task Overview.

In the Releases & Builds Tab, The release status is "COMPLETED".

A Completed Source can no longer be edited. If you want to Edit a Source, or the Source itself has changed, Create a New Release.

A Data Vault Model will always be built from all Sources in a Project, unless the Source Build Flag is turned off.