Introduction to the Data Model
The data model screen allows you to visually design the data model of your application.
Views can also be included in a data model along with physical tables. A view gives a runtime presentation of data from one or more tables in one concentrated overview and is therefore always up-to-date. There are no rows stored in a view.
It is also possible to create a snapshot . In a snapshot, just as in a view, data from one or more tables can be combined, though in a snapshot this data is actually saved. This can considerably improve performance. The RDBMS has provisions to easily refresh the data in a snapshot (sometimes even automatically).
A function-typed table is read-only and uses a table-valued function generated by the Software Factory as a data source. It can solve performance problems if data in a query is filtered based on parameters instead of retrieving all data and then filtering.
A reference is a relation between tables. The integrity between these tables is automatically guaranteed on the basis of a column comparison. A reference consists of a target and source table. The source table is the lookup table.
The Data Model screen
Since a data model can consist of many tables with columns, an overview can quickly become very unclear. To avoid this, it is possible to produce several diagrams each describing a subset of the complete model. A task is available to produce one's own diagram. This diagram is always at the top of the list of diagrams and can only be used by the user who is logged in. This prevents two developers getting in each other's way when designing.
It is possible to create a personal diagram. This diagram is not visible for other developers and is not bound by any rules. It is also possible to import existing diagrams into a personal diagram. When a personal diagram already exists, the choice is provided whether the personal diagram should be replaced or should be merged with the diagram that must be imported.
Diagrams, and therefore the data model, can be designed graphically via the Design tab
Tables and views in the Design tab page
This screen has a number of functions that help when designing the data model. A list is visible in the left hand column with all the tables and views. Besides this, there is a similar list with all the references. These can be checked and unchecked to use them within the diagram. In both lists it is possible to find objects using the find-option in the context menu (or by using the shortcut CTRL + F).
Under this list, there is an overview of the diagram and there are a number of standard parameters that can be modified.
Tasks are available above the tables to automatically add extra levels or link tables to the diagram.
The One level deeper and One level higher tasks select all linked tables above or below the selected table. This allows fast development of a diagram, which displays linked tables.
The Link table task automatically creates a link table between two tables. It can be indicated in the pop-up which table is the main table and which is the subordinate.
To create a link table between two tables, both tables have to have a primary key
The context menu (right-hand mouse button in the work sheet) offers the following options:
Refresh - re-load the data model, for instance, to see modifications from other designers.
Diagram - has a sub-menu to save, load and rename diagrams.
Arranging - produces a graphical display by manually positioning the tables yourself. The distance is determined through an option in Settings.
Arrange references - leave the tables alone, but position the references so that they do not run through the tables.
Fit to screen - remove the white edges around the data model.
Align - has a sub-menu to align selected objects in various directions.
Print - the data model is printed on the standard printer.
Print preview - displays a print preview.
Export - has a sub-menu to export the graphical data model to various formats such as GIF, PNG, PDF, etc.
Normal/Reference - switch between Normal and References. If References is selected, it is possible to draw references between columns.
Add - a table or reference can be directly added to the model.
Delete - delete a table or reference from the diagram and/or model. With delete table, the table is not deleted but only removed from the data model. With delete reference, there is a choice between removing the reference just from the diagram or also from the model.
Adding new tables, views, table-valued functions, and snapshots can be done using the tab Tables next to Design but can also be accessed easily via the buttons in the diagram.
An overview of all tables is accessible via the Tables tab page.
Overview of the 'Tables' tab
It is possible to specify if the database management system should keep a table into memory to optimize performance. Available options are:
- Yes (durable)
- Yes (transient) - only available for SQL server projects
More information about memory optimized tables in SQL server can be found see here.
For DB2 projects, the property KEEPINMEM will be used. More information about this property can be found here.
For Oracle project, the option CACHE will be used. More information about this option can be found here.
SQL Server 2016 introduced support for temporal tables (also known as system-versioned temporal tables) as a database feature. That brings built-in support for providing information about data stored in the table at any point in time rather than only the data that is correct at the current moment in time. Temporal tables is a database feature that was introduced in ANSI SQL 2011. System versioned tables are tables whose data is maintained in the history tables. This history is maintained by SQL Server itself.
In the Software Factory you can activate system versioning by switching on the field System versioning in Tables. This is the only setting that needs to be done by the developer. Since this would cause a datamodel change, the Software Factory will generate code in the CREATE and UPGRADE scripts. The two date fields (tsf_valid_from and tsf_valid_to) are generated in the script automatically, as hidden fields. However, the Software Factory will NOT create these columns in the Column list in the Software Factory itself. When enabled, the Software Factory will create a history table with the name [table_id]_history. It will appear in the CREATE and UPGRADE script too, and again, you will not find it in the list of tables in the Software Factory. Because of the absence of this table, the settings in Data conversion for the original table will also apply to the history table.
When you want to query data from a table for a certain point in time, you can use for system_time as of. For example:
select * from customer for system_time as of '2019-09-01 T10:00:00.0000000'
You will notice that in the result of this example columns tsf_valid_from and tsf_valid_to are not shown. That's because they are hidden. If you need to see the values of these columns, you have to make these columns explicit in the select list. For example:
select customer_name, tsf_valid_from, tsf_valid_to from customer for system_time as of '2019-09-01 T10:00:00.0000000'
In the Software Factory this is only implemented for SQL Server. More information about system versioning can be found here.
Warning: This is an advanced feature that should not be used without a proper understanding of SQL Server's implementation of partitioning.
Partitioning allows data in a table to be stored in separate chunks on the file system. For very large tables this could lead to a performance increase.
To get started with partitioning for your end application, a partition scheme needs to be created:
menu Data > Data model > tab Tables > tab Form
Open the pop-up next to the Partition scheme field.
The following information is required:
- Partition Scheme: A name for the partition scheme.
- Domain: A domain to determine which data type will be used to partition the data. This domain needs to be of the same data type as the column that is going to be used as the partition column.
- Range: The alignment for the boundaries of the partition ranges. The boundaries can be aligned either left or right.
Enter a partition scheme
Open the Partition boundaries detail tab. Boundaries determine where the data is going to be split. In the example below, every year on January 1st.
Set the partition boundaries.
Set the partition boundaries
Return to the underlying table.
To apply the partitioning to a table:
Select one of the created partitions in the field Partition scheme, and select a column in the field Partition column. The partition column is the column that contains the data that will be used to partition the table.
- A database upgrade is necessary to apply or remove data partitioning, since this concerns a change in the data model.
The columns are defined within a table under the columns tab. A column has the following properties:
Columns tab page
- Project ID - of the current project.
- Version ID - of the current project version.
- Table ID - the table within which the column is created.
Column - the column name in the database (ID).
Domain - determines the data type and control of the field (ID).
Column alias - a column alias can be optionally added here (ID).
Column description - a description of the column can be optionally placed here.
- Sequence number - determines the physical sequence in the database.
- Case type - determines whether this field is only filled with upper case letters, lower case letters or a combination of both.
- Identity - the database will automatically issue a number for this field.
- Seed - begin value of the number.
- Increment - steps with which the number is increased.
- Primary key - indicates whether the field is part of the primary key.
- Column type - determines how the column is displayed (Editable / Read only / Hidden). Combined with the Access type the column display can also be Unauthorized (see Hidden, read-only, full rights or unauthorized).
- Mandatory - determines whether a field is mandatory or optional.
- Foreign key - indicates whether a field is a foreign key from another table.
- Default value type - allows to choose between a constant value as default value or an expression to use as default value.
- Default constant value - fixed value that is filled in when adding a record, with a change this value is not filled in.
- Default query - the expression used to determine the default value. The query cannot use any of the column values of the new record.
Default values for columns will be placed in the database. Default queries may be placed in the database to be leveraged by insert statements in application logic. This depends on whether or not the database platform supports this. SQL Server databases are currently the only databases that support this.
Type of calculation - indicates with which calculation the field is filled. A choice can be made from:
None - the user enters a value himself
Expression - the GUI executes the calculations. This is not stored in the database. The column itself is also not stored in the database.
Calculated column - the database executes the calculation and stores this in the table.
Calculated column (function) - the database executes the calculation with the help of a function.
Calculated field query - in which a query can be specified to have a value in this field calculated by the GUI or database, for instance t1.price * t1.quantity. This value is not stored in the database.
It is possible in the Software Factory to define virtual fields that are not stored in the database but are evaluated by the user interfaces. This type of field is called an expression field. For these expression fields, that are sometimes also called look-up info fields, no upgrade of the end product is necessary.
A person can be displayed as first name + prefix + surname. This name is composed and therefore does not have to be stored in the database but is often used in the GUI.
With calculated fields, a query has to be provided with which the value is retrieved by the user interface. For instance, this can be a composition of several fields, a (range) function or a subquery. Reference can be made in the query to other fields from the table by using the alias t1.
The query that is provided with expression fields will be added to the select clause of the query. For example:
select t1.[customer_id], t1.[customer_naam], ( select t1.street + ' ' + t1.number_number + ', ' + c.name from country c where c.country_id = t1.country_id ) as t1.[address] from customer t1
The columns on which the expression is dependent are shown on the Expression dependencies tab. If one of these columns changes in the end application, then the expression field will also be updated. If, for instance, street changes, then the composite address has to use the new street.
Calculated field with a Query
Columns for a table or view can be copied from all existing tables and views using the Copy task in the Copy columns tab page.
A view with the option to copy columns
When executing the Copy columns task, a pop-up screen is displayed to optionally enter a prefix for the copied columns. This prefix is placed before the id of the copied column.
For example, the name column can be copied from the employee table using the employee prefix, resulting in column id employee_name. When the column id already starts with the specified prefix, then the prefix is ignored.
After copying columns to a table or view, all columns in the table or view are renumbered automatically.
Pop-up that appears when tables are copied
After all columns are defined and/or copied and Automatic or Modifiable is chosen for the view option, then the from clause of the view can be constructed by executing the Generate view from clause task.
A view is a logical table that is composed from fields from other tables. The data in a view is not stored in the database but composed at runtime on the basis of a query.
- Automatic: use this method when only data from the linked (source) tables need to be used in the view. The from clause is created automatically. This is generated, but it can also be forced by executing the Generate view from clause task.
- Modifiable: use this method to be able to modify the from clause, where clause, group by clause or having clause of the view. The select clause is built up on the basis of the view table and view column fields in the columns.
- Template: the last option provides the possibility to create a template in which the entire select query is written, which then needs to be woven into the view. The Template option provides the freedom to use the most complex selection structures or to construct them dynamically. Creating this template is explained here.
Table-valued function as a table
It is possible to use a table-valued function as a table. A function-typed table is read-only and uses a table-valued function generated by the Software Factory as a data source.
A function-typed table can solve performance problems, for example, when retrieving the history in a query within a specific timeframe. The performance improves if data in a query is filtered based on parameters instead of retrieving all data and then filtering. Another example is the selection of a specific project or company throughout the entire application.
This feature is only available for SQL Server (and not yet for DB2 or Oracle) and only supports inline statements.
To create a table-valued function:
menu Data > Data model > tab Tables > tab Form
Select Table type 'Function'.
Table type 'Function'
In tab Tables, open tab Columns > tab Form.
Now set the input parameters for the function table:
For the relevant columns, check the Function input parameter box.
Function input parameter
Now add code to the function with a control procedure:
menu Business Logic > Functionality > tab Control procedure
Add the code. A return value has to contain the table as declared in the data model.
To fill the function input parameters:
menu User interface > Subjects > tab Default > tab Data > tab Prefilters
- Add a prefilter to the table with "equal to" or "is null" conditions.
Generated code for a function table
The numbers , , etc refer to the steps as described above.
/* Drop function my_own_project first. */ if exists (select 1 from sysobjects where name = 'my_own_project' and (type = 'FN' or type = 'TF' or type = 'IF')) drop function my_own_project go create function my_own_project -- ( @project_id project_id --[2, 3] ) returns table -- This function returns an inline table with the following definition: --( -- project_description description primary key, -- -- amount_of_hours amount null -- --) --control_proc_id: function_my_own_project -- --template_id: function_my_own_project --prog_object_item_id: function_my_own_project --template_description: Select my project from projects. return ( -- select project_description, amount_of_hours from project where project_id = @project_id ); go grant select on my_own_project to public go
The Software Factory automatically creates indexes for foreign keys and sort sequences. To optimize performance, it might be necessary to add additional indexes to a table. Indexes can be created in the Indexes tab page.
The Thinkwise platform supports clustered and non-clustered indexes, full-text indexes (SQL Server) and Encoded Vector indexes (DB2). It can also be indicated whether an index has to be unique.
Unique indexes can be marked to exclude null-values. When marked to exclude null-values, if any of the indexed columns of the row contains a null-value, this row will not be checked by the unique index.
A reference is a relation between two tables. The target table has one or more fields (the foreign key) of which the combined values have to appear as the primary key in the source table. This column comparison is specified in the reference columns.
Tables and views
References can be created by means of drag & drop in the Reference status. In the Normal status, the Reference status can be temporarily activated by holding down the Ctrl key. This makes it possible to create a reference by dragging with the mouse from the source to the target table. A pop-up then appears in which the column comparison is presented.
The column comparison is based on the primary key of the source table, which therefore has to be present. If a column from the source table does not appear in the target table, then a checkmark will appear in front of the column to indicate that this column will be created. This checkmark can be unchecked whereupon an existing column from the target table has to be chosen. Only columns that have the same domain as the column in the source table can be linked.
When adding or modifying a reference, it can also be indicated whether the reference has to be used as detail and/or as look-up.
Pop-up for creating references
It is possible to switch off the integrity check of the reference by the RDBMS by unchecking the checkmark next to Check integrity. The reference then only serves to display details or look-ups in the user interfaces.
Incidentally, just the integrity between tables can be checked. When the source or target table is a view, the Check integrity option is turned off as a default.
Validating the data model is possible in the Validation tab. Through this, errors can already be identified at an early stage. The Validation component is explained in more detail here.
Example of the validation screen
Starting the application
When the data model is completed, the database can be created or upgraded. This is described in Creation. When there is no user interface set up, there is nothing visible in the end product. Tables are accessible after a menu has been set up.
Starting the user interface is possible once the database has been created or by using the Mock Database Provider to display a preview without an end product database.