Transformation Blocks

Transformation Blocks #

The transformation mapping is at the heart of ADH. This is the part that links the input pattern to the output pattern, via a series of links and functional blocks. These links and blocks are graphical representations of the mapping process. A link directly between the input and output will copy the incoming value without modification to the output. Adding a block provides scope for transforming the incoming value in various ways. Blocks can be strung together to create complex mapping processes.

When editing a mapping, transformation blocks can be selected from the palette and placed on the mapping canvas. Most blocks have a number of properties that can be set. Double-click on a block to open its properties, first tab will contain block specific properties to configure block for transformation and the second tab will be a notes tab. The mapping creator can add comments to the notes to explain what the block is doing for future reference.

Data Generation Blocks #

Data Generation blocks are used to provide data from a source other than the input pattern. They can be regarded as ‘hard coded’ items. Unlike most other blocks, there is no ‘in’ path to the block, only a single ‘out’ path.

Boolean Constant #

Allows the user to specify a Boolean value to be used in the transformation.

Constant #

Allows the user to specify a constant value to be used in the transformation.

Currency Constant #

Allows the user to use attributes of a SICS currency as constants in a mapping. There are three outputs - code, description (full name) and object.

Example:

If the outgoing document needs the name of the currency, this can be provided using the description output from this block.

Date/Time #

Allows the user to generate the current system date.

Document Property #

Allows the user to use one property of the document being transformed.

Entry Code Constant #

Allows the user to use attributes of a SICS entry code as constants in a mapping. There are three outputs - code, name and object.

Example:

If the outgoing document needs the name of an entry code, this can be provided using the name output from this block.

Legal Area Constant #

Allows the user to use attributes of a SICS legal area as constants in a mapping. There are four outputs - composite code, code, name and object.

There is a choice of four alternatives; only one may be selected. In each case, select the Legal area required.

Example:

If the outgoing document needs the name of the country this can be provided by selecting the Country option, and the specific country, and the using the name output from this block.

Reporting Unit Constant #

Allows the user to use attributes of a SICS reporting unit as constants in a mapping. There are three outputs - code, name) and object.

Example:

If the outgoing document needs the name of the reporting unit, this can be provided using the name output from this block.

System Property #

Allows the user to use one property from the system.

UUID #

Generates a new, unique UUid each time the block is processed. The output is a 36-character string with format ‘xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx’, where each ‘x’ is one of these characters: 0-9, a-f. For example: ‘123e4567-e89b-12d3-a456-426614174000’.

There are no properties for this block.

Domain Data #

Retrieves domain data (business, claim, accounting, etc.) from the SICS database.

General rules:

• The incoming values of all selected Input Fields will be used for lookup (all must match)
• An output field of type ‘Xxx Object’ means that this type of SICS domain object is output. For example, ‘Section Object’ means the matching section domain object (a ‘SicsScopeOfCoverImpl’ instance, if found) is output. Input/output fields of type ‘Xxx Object’ can be connected to the same kind of field on another ‘Domain Data’ block.
• It is also possible to use SICS domain objects with other ADH transformation blocks’ inputs/outputs that do no more than, block or allow the input value to be propagated to an output or, test if the input is null/empty.
  Such blocks are: Empty, Not Empty, boolean-Gate (not the ‘Switch’ input), data-Gate, Default, boolean-Branch (not the ‘Switch’ input), matches-Branch (not the ‘Switch’ input), If (not the ‘Test’ input).
  A common scenario would be to pass a SICS object output from one Domain Data block, through a boolean Gate block and into another Domain Data block. The Gate block would be configured to block the pass-through of the SICS object, and also other data inputs on the Gate block, if certain conditions are not satisfied. Thus avoiding that the second Domain Data block executes a database query with too few predicates (which could be very time-consuming).

Section type:

• For input Insured Period From-Date, the value must be formatted as mentioned below
• For input U/W Year, the value must be formatted as ‘yyyy’
• For input Unique Market Reference, it will be considered a match if either INSURED_PERIOD.EMINF_UMR or INSURED_PERIOD.EDIINF_UMR matches
• For outputs Cover 100% and Excess 100%: For non-prop business, this will be the limit fields for which refOptionalField().isCover()/isExcess() is true. For prop business, the value will be null/empty
• There is no input to select Life Cycle Phase/Status. Only the latest Phase/Status will be considered.

Claim Section type:

• For input Claim Reference, if a section has at least one reference that matches the given Claim Reference, then this constitutes a match (for this section)
• For input Layer, the value will be tested against the ‘Layer No.’ on the Life Cycle tab of each section linked with the claim
• For input Claim Reference Type, the value will be referenced from section-specific Claim References -> ‘Claim Reference Type’. The value has to be formatted as Reference Type Code.
• For input Loss name,the value refers to the claim ‘Name’.
• For input U/W year, the value will be tested against the ‘Underwriting Year’ on the Insured Period tab of the Business linked with the claim. The value must be formatted as ‘yyyy’.
• For input Date of Loss From Date and Date of Loss To Date, the value will be taken from Date of Loss From and To Properties of the claim.
• For input Partner Role, the value will be considered from Business Partner type linked to the claim. The value has to be formatted as Relationship Type Code.
• For output Claim Reference, the value will be empty if there are more than one reference on the matching claim section
• For output Claim Main Currency, Cover and Excess, values will be taken from the section-specific ‘Claim Figures’ tab
• For output Last Booked Currency: If there are multiple currencies on the last booked date (ACC_AS_OF_DATE) for the matching claim, then a transformation problem will be created if the block property ‘Fail on missing’ was selected, else the output will be empty.
• For output hasFGU, the value will be referenced from properties of claim section FGU checkbox. If the FGU checkbox is ticked, the hasFGU will output the value as true. If the FGU checkbox is not ticked, the hasFGU will output the value as false in the mapping.

Partner type:

• For input Partner Identifier, if a partner has at least one identifier that matches the given Partner Identifier, then this constitutes a match (for this partner)
• For output Partner Identifier, the value will be empty if there are more than one identifier on the matching partner.

Claim Ledger Figures type:

• Retrieves Ledger Figures from the latest entry.
• If the system parameter ‘Claim - Claim Accounting - Claim Ledger - Hide Zero Amounts on Claim Ledger’ is selected, all ledger entries with a zero amount will be skipped. As a consequence, if all entries are zero, the Domain Data block will output nothing.
• For input Claim Reference, if a section has at least one reference that matches the given Claim Reference, then this constitutes a match (for this section)
• For input Layer, the value will be tested against the ‘Layer No.’ on the Life Cycle tab of each section linked with the claim
• Input field ‘Currency’ will select a specific latest Ledger Figures currency.
• Input field ‘Accounting Classification’ will select a specific latest Ledger Figures accounting classification.
• Input field ‘Entry Code’ will select a specific latest Ledger Figures entry code. The value must be coded as e.g. ‘30’ for Paid Loss.
• If - after all inputs have been applied - the claim has latest Ledger Figures in multiple Sections, Currencies, Accounting Classifications and/or Entry Codes, indexing (see below) can be used to iterate over all latest Ledger Figures per section/currency/accounting classification/entry code.
• For output Entry Code, the value will be code + description e.g. ‘30 Paid Loss’
• For output Balance Amount, the value will always be an accumulated amount.

FGU Advisory Figures type:

• Retrieves FGU Advisory Figures sorted based on as-of-date (most current first)
• For input Claim Reference, if a section has at least one reference that matches the given Claim Reference, then this constitutes a match (for this section)
• For input Layer, the value will be tested against the ‘Layer No.’ on the Life Cycle tab of each section linked with the claim
• Input field ‘As of Date’ will select a specific FGU Advisory Figures entry. The value must be formatted as mentioned below. Only the date part will be used to find a match, the time part will be ignored.
• For all output Entry Code fields, the value will be code + description e.g. ‘30 Paid Loss’
• If - after all inputs have been applied - the claim has FGU Advisory Figures in multiple Sections and/or As of Dates, indexing (see below) can be used to iterate over all FGU Advisory Figures per section/asOfDate.

Date format: Dates must be formatted as follow:

YYYYMMDD YYYY.MM.DD YYYY-MM-DD

where is:

HHMMSS HH:MM:SS HH.MM.SS

The year MUST be represented with 4 digits. Allowed values range from ‘0000’ to ‘9999’. The month MUST be represented with 2 digits. Allowed values range from ‘01’ to ‘12’. The day MUST be represented with 2 digits. Allowed values range from ‘01’ to ‘31’, although ‘29’ and ‘30’ may be invalid depending on the year and month. The normal leap year rule is applied. A date string representing a date outside the allowed min-max interval is considered invalid. The min date is 17530101. The max date is 99991231.

The time part is optional. If present, a single space MUST separate the date and time part.

The hour MUST be represented with 2 digits. Allowed values range from ‘00’ to ‘23’. The minute MUST be represented with 2 digits. Allowed values range from ‘00’ to ‘59’. The second MUST be represented with 2 digits. Allowed values range from ‘00’ to ‘59’. Leap seconds are not supported. A time string representing a time outside the allowed min-max interval is considered invalid. The min time is 00:00:00. The max time is 23:59:59.

Indexing:

There may be cases when ADH data input is not sufficient to uniquely determine a domain object (using connections to Input Fields). It is possible to configure a mapping to iterate (loop) over the set of domain objects looked up, as follows.

• Add an Input Pattern repeating group (e.g. ‘iterator’, though the name is not significant except the xPath must be equal to the group name) within what you would otherwise have as the lowest level repeating group (sometimes ‘page’, sometimes ‘page/row’, …)
• Add one Input Pattern field within that group (typically named ‘index’, though the name is not significant except the xPath must be equal to the field name)
• Connect the ‘index’ Input Pattern field directly to the ‘Index’ input of the Domain Data block.

If/when the ‘Index’ input is connected in this way from an Input Pattern field, the ‘iterator’ input data group will be replicated as many times as there are multiple domain object results in the Domain Data block. This is called ‘augmenting’ the input data.

Each replication of the ‘iterator’ input data group will contain the ‘index’ input field with a value of 1, 2, 3, … During transformation, the connection from ‘iterator/index’ to the Domain Data block input ‘Index’ will select the 1st, 2nd, 3rd, … domain object from the result set, and the output fields from the Domain Data block will get values according to the selected domain object.

This way, it will be possible to generate one output message for each matching domain object. Also, additional transformation blocks can be used to block (using Gate (master)) processing of certain matches.

A Domain Data block configured in this way is called ‘indexed & augmenting’. A mapping may contain only one such block.

It is also possible to connect the ‘Index’ input of a Domain Data block from the output of some other block. This will not augment the input data, even if an Input Pattern field happens to be indirectly connected to the ‘Index’ input.

Index Limit:

The Domain Data blocks currently have a performance-related limit to fetch 10 items/records which is not sufficient in some cases. To overcome this challenge, input field Index Limit can be used in all the Domain Data blocks. Any numeric value provided to this input field will set the limit of the database query, and thus override the default of 10.

Index Size: Indexed domain data blocks have an optional output field named ‘Index size’, to help identify the last object returned from an indexed domain data block.

• All domain data blocks have this optional output field.
• If domain data block is indexed, then ‘Index size’ will be equal to the total number of domain objects found.
• If ‘Index limit’ is explicitly set, then ‘Index size’ will be the lowest of ‘Index limit’ and number of domain objects found.
• If domain data block is NOT indexed, then ‘Index size’ will be empty/null.

Execution Information:

A Domain Data block may provide output as expected, or fail in various ways. If the block executes successfully, the output field Execution Code will output 0, and Failure Reason will be empty. If the block execution fails, however, there are two main scenarios:

1. The block has Fail On Missing = true.

In this case, a Problem will be created, describing the issue. The output fields Execution Code and Failure Reason will not provide any information.

2. The block has Fail On Missing = false.

In this case, no Problem will be created, and detailed information about the failure will be provided in the output fields Execution Code and Failure Reason.

The Execution Code can take on these values:

• 1 = no domain object found
• 2 = more than one domain object found
• 3 = there was an exception (invalid data) when retrieving output field value(s) from the domain object

The Failure Reason will provide a textual description of the failure.

Random String #

Generates a new, almost unique random string each time the block is processed. The output is a 10- or 8-character string where the characters are taken from the basic Base64 (RFC 4648) character set: a-z, A-Z, 0-9, + and /. For example: ‘DX+kJ/qz’. Although the string is not guaranteed to be globally unique, it is much shorter than the 36-character string produced by the UUID block, and may serve its purpose in several situations.

Example:

If the transformation needs a random string which is unique with a high probability (though not 100%), this can be provided using the output from this block.

Data Transformation Blocks #

These blocks create one or more outputs based on the inputs, where the output is transformed from the input. The output data is not intended to be a direct copy or reformatting of the input. (See also Data Format Blocks below).

Bool to Str #

This block takes a single Boolean (true or false) input, and propagates a string output depending on the value.

Concat #

Joins two or more inputs into a single output.

Example:

This is useful for dates, adding ‘-’ between year, month and day - e.g. 2016-07-29. The input count is set at 3, where the inputs are year, month and day. In this example there are no prefix or suffixes. The separator is set to ‘-’, and the result is 2016-07-29.

Pattern #

The pattern block will create a single output string from the pattern string, substituting the numbered input markers with the corresponding input values. It is one way of concatenating inputs together.

Each input is identified by %n, where ’n’ is the number of the input. Thus %1 is the first input, %2 is the second, etc. The pattern defines the output from the block.

Example:

Input count = 2. First input value is the text of the input field ‘Sender Percentage’, and carries the value ‘2.0’. Second input value is the text of the input field ‘Sender Name’ and carries the value ‘Pargetter& Lloyd’. The pattern is defined as ‘Partner %2 has a share of %1%%’.

The output would be ‘Partner Pargetter& Lloyd has a share of 2.0%’.

Replace #

This block will replace any instance of the pattern in the input value with contents of the replacement. The pattern and replacement do not need to be the same length.

It is useful for removing whitespace from a string - the pattern is set as a single space, and the replacement as null. Every instance of space in the value is replaced by null, thus removing all the spaces.

Special characters can be specified in both pattern and replacement to remove or include tab and newline formatting.

Replace (RegEx) #

This block will replace each substring of the input string that matches the given regular expression pattern with the given replacement. The pattern and replacement do not need to be the same length. The first 9 capture groups in the pattern may be referred to in the replacement as $1 - $9. Note that $ and \ have special meanings in the replacement string. In order to treat them as literal characters, escape them as \$ and \\.

Example:

The input value is A1234 05/09/16 - where there are several spaces between the beginning and end of the string. The desired output replaces the space with a single -. Set the pattern to \s+ (which detects multiple whitespace), and the replacement to -. The output is A1234-05/09/16.

Split #

Used to break an input into several parts. The split is made on each occurrence of the separator string in the input. If the separator occurs more times than the number of output counts, the remainder of the input is discarded. The separator is not transferred to the output.

Special characters can be specified in the separator to break on tab and newline formatting or on a backslash.

Example:

Could be used for breaking a date in the form 2016-09-22 into year, month and day parts, by setting the separator to ‘-’ and the number of output counts to 3. The outputs would be 2016, 09 and 22. If the number of outputs was only 2, the day part (22) would be discarded.

Split (Pos) #

This block will split the input based on the physical positions (the index), of the input string and will pass on data after the first index position, up to the next index position. The number of outputs will depend on the number of index positions defined in the format property. If the input string is shorter than the first index position defined in the format, all outputs are empty.

The first position in the string is index zero.

Compare with Substr, which returns a single output only.

Example:

This block is useful when you need to extract part of the input, and when the input is always in the same format and the length. Other block types may be more helpful if the format or length changes from document to document.

Assume an input value of ‘ABCEDF’.

Split (RegEx) #

This will split a single input into several outputs, based on a regular expression. Each output corresponds to one capture group of the RegEx in order. Only data matching the RegEx is passed to the output.

Example:

This block is good for extracting part of a string when the general format is known, but the length of each component can vary. For instance, a sender transaction reference includes an identifier to a section. The string is ABCDE1205363/ 024/ 003/ 1/14, and the section identifier is 003. SICS requires a two-digit identifier in this case, so the leading zero is not required. The regular expression \w+/\s?\w+/\s?\d(\d\d)/.* will extract the ‘03’ from this string. If only one output is required, the output count may be set to 1. In this case, if the output count is set greater than 1, the remaining outputs will all be empty, as only one RegEx group is specified in the format (the (\d\d) part).

Split (Time) #

The Split (time) block parses a single date/time input into one or more outputs. The outputs default to Out1, Out2 etc., but can be renamed.

Example:

Good for extracting the year from the date - for instance if the inception date is given as 1 juin, 16, and the input format is specified as ddMMM,yy, with a locale of French (Switzerland), and the output is defined as Year: yyyy, the block will output Year = 2016.

Substr #

This block returns one output from a single input, based on position. Compare with Split(pos), which can return multiple outputs. Substr will pass on data from the first index position, up to but not including the end index position. The first position in the string is index zero. No error is set if the end index position is greater than the length of the input string.

Both start and end are optional. If start is omitted, the output commences from the beginning of the input. If end is omitted, the block will return from the start index to the end of the input string. (If both are omitted, it follows that the input string is output unchanged).

Example:

Assume an input value of ‘ABCEDF’.

Translate #

The Translate block takes one or more inputs, and looks up a set of corresponding outputs from the table specified in the Translation table property. The number of inputs is dependent on the chosen table definition, and the number of outputs is dependent on the output type defined in the table. For further information on translation table definition see Reference Items -Translation Tables.

If the input values do not find a matching entry in the table, the block will either set an error, or pass on null values.

Example:

This block is typically used to translate the name of the message sender into the agency/id pair required in an ACORD RLC message. If the chosen table is ‘ACORD party id’ the input will be ’name’ and the outputs will be ID and agency. Where the incoming document may have various renderings of the input name (including variations in spaces and punctuation), a new line must be added to the table for each one. The output for each line may be the same. Fail on missing is useful for notifying you that a new line should be added to the table.

Translate can also be used to substitute currency codes, or to generate period dates (e.g. the incoming document has Q2, and required output is 01/03 - 30/06).

Trim #

Trim will remove spaces, tabs and newlines from the input field. When the mode is Right, any spaces, tabs or newlines beyond the last character of the input string will be removed. When the mode is Left, any such symbols before the first character are removed, and when the mode is Both, they are removed from before and after.

Example:

This block is useful for removing leading and trailing whitespace from a field before it is passed for output. If the input string is Pargetter& Lloyd, and the mode is Right, the output is Pargetter& Lloyd (with the trailing spaces/tabs removed).

Data Format Blocks #

These change the format of the input data to output date, but do not change the contents. All data in the single output is present in the single input.

Lower #

Transforms the string content on input to all lower case on output.

There are no properties for this block.

Example:

An input string of PARGETTER & Lloyd will become pargetter&lloyd on output.

Number Format #

This block normalizes an input to a number and changes the decimal separator. The block will remove leading, trailing and embedded spaces in addition to thousand separators from the input.

Example:

The ACORD standard is to use ‘.’ as the decimal separator. Numbers in the European format of 123,45 would be reformatted as 123.45 using the default settings

Time Format #

This is a powerful block that can change one date and time format to another. Any data item (e.g. day, month, year) required in the output must be present in the input, but for example names of months can be converted to numbers. The provided input formats (see below) will be applied in the order they were added to the block properties. The first input format that does not raise an error/problem, will be used.

Example:

Again, this block is helpful for changing incoming dates to the ACORD standard (yyyy-MM-dd). If the input date is 8 juin, 16, setting the input format to ddMMM,yy (and input locale to French), and output format to ‘yyyy-MM-dd’ (and output locale to English) will result in 2016-06-08 being output.

Upper #

Transforms the string content on input to all UPPER CASE on output.

There are no properties for this block.

Example:

An input string of PARGETTER & Lloyd will become PARGETTER & LLOYD on output.

Data Flow Blocks #

These blocks allow choices to be made, based on interpreting the input data. The output is either one or more input values, or empty.

Branch (boolean) #

The block has a single input switch and an arbitrary set of inputs. If the switch value is true, the block will propagate all inputs to the True outputs. If the switch value is false, the block will propagate all inputs to the False outputs.

Example:

This block is useful if you want to differentiate the mapping based on a condition. The example below will map amount to either balance_due_to_sender or balance_due_by_sender based on the condition that amount is less than zero.

Figure 26 Branch (Boolean) example

Branch (matches) #

This block has a single input switch and an arbitrary set of inputs. For the first regular expression Match that matches the (complete) switch value, the block will propagate all inputs to the Match outputs.

If the switch value doesn’t match any of the regular expressions, the block will propagate all inputs to the Default outputs. Note that it is allowed to use an empty as regular expression, but this can only occur once. An empty regular expression will match the switch value if and only if the value is empty/missing.

Example:

The example below will propagate the value of field value to Output: Match1 1, Match2 1, Match3 1, Match4 1 or Default 1 based on the value of field RowName, tested against 4 regular expressions defined in the block’s properties.

Figure 27 Branch (matches) example

Default #

This block allows two or more inputs, and copies to the single output the first that is not empty/null.

Example:

This block can be used to select between incoming fields, when we ideally want to take the first one, but use the second if the first is missing. The second might be a constant to be used when the message has an empty value. It can also be used to check that at least one in a range of input fields contains a value, and set an error if none does.

Error (fixed) #

This block is used primarily in validations to set a fixed message when an error is detected. The fixed message is shown in the description column of the problem tab in View document.

Example:

Typically, the validation will include a test that the document to be transformed is identified a document that can be transformed by the mapping. This can be done using an ‘In’ block - is the document type of ‘account’ in the list of document types? If the answer is false, an error such as ‘Invalid document type’ can be displayed when the document is validated.

Error (variable) #

This block is used primarily in validations to set a variable message when an error is detected. The error message to be used is an input to the block. This allows content from the document to be included in the message. The constructed message is shown in the description column of the problem tab in View document.

Example:

Typically, the validation will include a test that the document to be transformed is identified a document that can be transformed by the mapping. This can be done using an ‘In’ block - is the document type of ‘account’ in the list of document types? If the answer is false, an error such as ‘Invalid document type = account’ can be displayed when the document is validated. This would be achieved by adding a Concat block to map the input field value after the fixed text (‘Invalid document type =’), and feed this into the error block.

Filter #

This block either accepts (passes to output) or blocks (does not pass to output), depending on ‘mode’, a single input value if it matches the regular expression. If the filter test fails, it passes on an empty value. Note that this is not a substring test, the complete input value must match the regular expression.

Example:

Filter can be used with a default block to help select a value to be mapped to output. Say the incoming document has two possible sources for an output item, and we need the non-zero one. Default by itself won’t make the choice (it is checking for empty, but neither input is empty), but if we filter the first input and set the mode to block matching values when the value is zero, the filter will pass on empty value to the default block, which will then use the second input.

Gate (bool) #

The block has a single input switch and one or more input-output pairs. All the inputs are copied to the corresponding outputs if the true or false switch is ’true’. If the switch is false, nothing is copied to the corresponding outputs.

Example:

This feature is useful for determining if a page on the input side contains enough relevant data to create an output message. In this situation, the switch can test the presence of several input items (compare with Gate (data) below, which only tests one). When the conditions are met (the switch is true) all of the inputs are passed through the gate, which typically has a high input count (one for each output item to be mapped).

Gate (data) #

The block has a single input switch and one or more input-output pairs. All the inputs are copied to the corresponding outputs if the input to the switch is not empty or null. If the switch input is either empty or null, nothing is copied to the corresponding outputs.

Example:

One option is if you want to map a partner, but only if the id of the party is present in the input. The input may include the name and address, but these may not be much use without a formal identifier. The gate maps id, agency, name, address and contact details, but is switched on the presence of ‘id’. If id is empty, the output from the gate is empty.

Gate (master) #

This block is used to control the current Leaf Repeating Group occurrence. There is a single Boolean input, and no output. Only one gate (master) block should be connected to the Leaf Repeating Group, and any blocks in the data path prior to the gate (master) block should not be used to push data to the output pattern.

Example:

A table will frequently include row and column headings that are not required in the output. Using a greater than test (is row number greater than 1?) and a gate (master) block, we can suppress the first row.

If #

The block consists of one input test, and one or more outputs. For each output, there are true and false inputs. When the input test is true, all the True inputs are copied to the corresponding outputs. When the input test is false, all the False inputs are copied to the corresponding outputs.

Example:

Can be used in many ways, but referring back to Gate(data), if a party is present in the input it can be mapped, but if not, a set of hard-coded party details can be used instead. Set the value count to 5 (one each for id, agency, name, address, and contact details. Map id to True1, agency to True2, etc., and map constant values for id, agency etc. to False1, False2 and so on. The test is if the input id is not empty. When empty (false) the False inputs are passed to the corresponding outputs, when there is a value (true) the True inputs are passed on.

Sort #

This block is used to sort the input data. The occurrences of one of more repeating Input Pattern groups can be sorted according to the specified Sort Criteria blocks.

There are a number of Sort inputs, and as many Criteria inputs. There is no output.

The connections to the Sort inputs can come directly from Input Pattern fields in the group to be sorted, or via other transformation blocks. All connection paths that lead into the Sort Block’s Sort inputs must use transformation blocks (if any) that are used solely for this purpose. Such blocks must not be used to push data to the Output Pattern.

The sort block cannot be directly connected to the (repeating) group to be sorted. Rather, the group is determined by the Input Pattern fields that are connected (directly or indirectly) to the ‘Sort’ inputs of the sort block. At least one of these Input Pattern fields must be a first-level child of the group to be sorted. (In the example below, firstName and lastName are first-level children of user because the name group is non-repeatable. So is orgUnit. Whereas code and amount are not, because the entry group is repeatable.) Connected Input Pattern fields that are not first-level children of the group to be sorted, must be reachable in the ancestor hierarchy of the group to be sorted. (In the example below, Sender is such an ancestor field.)

The connections to the Criteria inputs must come directly from Sort Criteria blocks. Sort and Criteria inputs with the same index belong together.

At most one ‘Sort’ block should be connected to a repeating group. Connecting several blocks may lead to unpredictable behaviour. Sorting takes place using the Sort input values in index order. That is, Sort 1 is the major sort level, Sort 2 is the next, and so on.

Missing (empty) sort values areconsidered smaller than any non-missing value.

Example:

Figure 28 Sorting example

Sort Criteria #

This block is used to control the sorting operation performed by the Sort block to which the Criteria block is connected. The Sort Criteria block hasno inputs, one Criteria output, and a number of properties.

Comparison Blocks #

These blocks all return true or false in the output, based on comparing input values with each other. The output from these blocks is always a single Boolean.

Equals #

This block compares two or more inputs, and if they are all the same generates true as the result. If anyone is different, the result is false. The result is emitted on the Out output. The negated value of the result is emitted on the Negated output. As the method of comparing strings, numbers and date/time varies, the data type can be specified in the properties. Depending on the data type, additional properties are available, as described below.

Example:

Can be used in validations to compare values - one of which could be a constant. An error is set (in the validation) if the values are different.

Greater Than #

The Greater Than block compares two inputs. If the first is greater than the second, the output is true.

Example:

Can be used to determine if a value is positive or negative. Set the second input as a zero constant. If the first input is greater, then the value is positive.

In #

A single input is checked against a table of allowed values defined in the block properties. If the input matches a value, the output is set to true.

Example:

Useful in validations, if the document itself carries a template or mapping identifier. If the identifier value is not in the block table, an error is set.

Less Than #

The Less Than block compares two inputs. If the first is less than the second, the output is true.

Example:

Can be used to determine if a value is positive or negative. Set the second input as a zero constant. If the first input is less, then the value is negative.

Repetition test #

Generates true or false on the Same output depending on whether the input(s) has the same value for the current Leaf Repeating Group occurrence as for the preceding one. The Same output will be false for the very first occurrence. The Not Same output value is the negation of the Same output value.

Example:

Typically used in conjunction with the @retainData attribute of an Output pattern (see above). Sometimes a document may have data split between pages - for example page one showing general details such as currency and broker, whilst page two has transaction details. So long as there is one common value on both pages, they can be linked using the repetition test to create one output message. Without the Repetition test, it is likely neither page would provide sufficient data to create a valid output message.

Retain (boolean) #

Used to retain a boolean value across occurrences of a Leaf Repeating Group.

There are two inputs - In and Reset. Both accept Boolean values only.

There are three Boolean outputs - Out, Empty and Changed.

First, if the Reset input is true, the retained value is reset (set to false).

Then, if the current retained value is false, sets the retained value to true if the In input value if true.

Finally, propagates the retained value to the Out output.

The retain block retains its value from one Leaf Repeating Group occurrence to another. The initial retained value is false (reset).

The Empty output provides the opposite boolean value as Out.

The Changed output provides true if the retained value changed from false to true.

There are no properties for this block.

Retain (data) #

Used to retain a data value across occurrences of a Leaf Repeating Group. There are two inputs - In and Reset. In accepts any data, Reset accepts Boolean values only. There are three outputs - Out, Empty and Changed.

First, if the Reset input is true, the retained value is reset (set to null/empty).

Then, if the current retained value is null, or the retain block property is Retain Last Value, sets the retained value to the In input value if this value is not empty and not null.

Finally, propagates the retained value to the Out output. The retain block retains its value from one Leaf Repeating Group occurrence to another. The initial retained value is null (reset).

The Empty output provides true if Out is null.

The Changed output provides true if the retained value changed from null to not null, or - if not reset - changed from the preceding Leaf Repeating Group occurrence transformation.

Example:

Process table by columns. Each column repetition has 3 cells (Leaf Repeating Group occurrences), for premium, commission and claim, respectively (in random order), that would go into the same output pattern occurrence. Assume it is a business requirement that premium must be present. A ‘Retain’ block for premium could be used to trigger an error should the premium be missing when processing the last cell within the column. The block would be reset when the column number changes.

Arithmetical Operation Blocks #

The arithmetic blocks provide some basic arithmetic transformations. More complex operations can be constructed by linking blocks together.

General observation on how numeric blocks work #

The following notes apply to any block that handles numeric inputs - these include the NumberFormat block and the Equals, Greater Than and Less Than blocks if they are defined as receiving numeric input. More strictly:

• The block has one or more string inputs that will be interpreted as number(s).
• The block has a ‘decimal separator’ property.

When executed, the block will, for each input, create an intermediate string by scanning the input string characters and,

• if the character is a digit (‘0’ - ‘9’), ‘+’ or ’ - ‘, keep it,
• else, if the character is the ‘decimal separator’, replace it with ‘.’,
• else, if the character is ‘.’, ‘,’, space, single quote, tab or space, ignore it,
• else, raise a ’not a digit’ error.

If no error was raised, convert the intermediate string to a decimal number, using standard Java conversion (with ‘.’ as decimal separator); this may raise a conversion error. If there is no error, the intermediate ‘.’ is replaced by the decimal separator of the block.

If in the end an error was raised and there is an applicable ‘Keep Only Last Decimal Separator’ correction (automatic or manual - see Automatic corrections below), then the original string will be replaced by a string where all occurrences of the ‘decimal separator’ except the last one have been removed. Then the block transformation will be re-executed.

Accumulator #

The accumulator block is used to sum values across occurrences of a Leaf Repeating Group. There are two inputs - In expects a numeric value, and Reset a boolean. There is a single output ‘Out’, which outputs the current accumulated value. The initial accumulated value is ’empty’.

The connection to the ‘Reset’ input is processed prior to the connection to the ‘In’ input. When the ‘Reset’ input is true, the accumulator is reset to empty. When the ‘Reset’ input is false, the accumulator keeps its accumulated value from the previous LRG occurrence processing. The numeric value (if any) on the ‘In’ input is then added to the current accumulated numeric value (treated as 0 if empty).

Example:

Process a table by columns. Each column repetition has three cells, (say for premium, commission and claim), that create a single output pattern. If you want the total value for the column, use an accumulator block to sum the total of these three amounts. The accumulator should be reset when the column number changes (using a repetition test block).

Add #

Sums the inputs. Input strings are parsed into numbers (using the decimal separator defined in properties). Empty and null inputs are ignored. If all inputs are empty or null, an empty value is created on output.

Example:

Used to add numbers together, for example when creating a total of monetary items.

Add(Time) #

Adds a set length of time, defined in the properties, to the input value.

Example:

Can be used to create a period end date from a start date, for example by adding three months (to create a quarter) or one year (to create an annual period).

Add(Variable Time) #

Adds the input days/months/years to the input date to give a new output date. There is a single output Date, and an input date. The other inputs vary according to the input option specified in the block properties (Years, months, days).

Example:

Can be used to add an elapsed time to a date.

Divide #

The divide block accepts two inputs. The output is the result of dividing the first input by the second. If either of the inputs is empty or null, the output will be empty. An error is set if either input cannot be parsed into a number.

Example:

Used to divide two numbers, for example when converting a percentage into a decimal fraction.

Multiply #

The multiply block accepts two inputs. The output is the result of multiplying the first input by the second. If either of the inputs is empty or null, the output will be empty. An error is set if either input cannot be parsed into a number.

Example:

Used to multiply two numbers - for example when calculating our share from 100% amounts.

Negate #

Multiples each input by -1, thus reversing the sign. The block can accept multiple inputs. If any input does not contain a number, all outputs are set to empty value. Can only be used with numeric inputs - to reverse a Boolean, use ‘Not’.

Example:

Used to reverse the sign of an input item.

Round #

Rounds a single input according to the round option and length specified in the properties. An attempt to round a non-numeric input will result in an error being set.

Example:

The result from divide or multiply blocks may include many decimal places. The output pattern may have a set precision for numbers - typically two decimal places for amounts, and seven decimal places for percentages. The round block allows you to create values that conform to the output standard.

Round (Time) #

Rounds a single date or time to the beginning or end of a period. It can also round to the beginning or end of the previous or next period. The method and period are set in the block properties. An error results if the input format does not match the specified time format.

Example:

If you need to create a period (start and end dates) from a single date occurring within the period, this is a useful block. Create one block to round to beginning of same month to give start date, and another to round to end of same month to give the end date.

Subtract #

Subtracts the sum of input 2 and greater from input 1. In its simplest form there are just two inputs, and the second is deducted from the first to give the output value. Additional inputs are summed before deducting from the first. Empty and null values are treated as zero. Other non-numeric inputs will result in an error being set.

Example:

Incoming documents may have individual unsigned fields for credit and debit versions of the same technical item. Subtracting the debit value from the credit value will give the correct signed total of the two.

Subtract (Dates) #

Accepts two input dates, and subtracts the second from the first to give an elapsed time between the two. The elapsed time can be measured in whole years, whole months or days, and the number of outputs vary according to the output specified in the block properties.

Example:

Can be used to create the elapsed time between two dates.

Subtract (Time) #

Subtracts a set length of time, defined in the properties, from the input value.

Example:

Can be used to create a period start date from an end date, for example by subtracting three months (to create a quarter) or one year (to create an annual period).

Subtract (Variable Time) #

Subtracts the input days/months/years from the input date to give a new output date. There is a single output Date, and an input date. The other inputs vary according to the input option specified in the block properties (Years, months, days).

Example:

Can be used to subtract an elapsed time from a date.

Truncate #

Truncates a single input according to the length specified in the properties. An attempt to truncate a non-numeric input will result in an error being set.

Logical Blocks #

The logical blocks all generate a single Boolean output, than can then be used in subsequent logical blocks, or in blocks that require a Boolean input - e.g. Gate (bool).

And #

Generates an output of true if all inputs are true. If any input is false, the output is false. The inputs must all be Booleans.

Example:

AGate(bool) can be used to determine if the output should be generated. The gate can be switched on the presence of certain key fields - such as underwriting year AND receiver share. Use the AND block to test that both underwriting year and receiver share are not empty, and if true, populate the output from the gate.

Contains #

Generates an output of true if the input value contains one of the strings defined in the properties. The defined strings can appear anywhere in the input value. If none of the defined strings is present, the output is false.

Example:

A party name may be spelled in various ways, but the ‘Contains’ block can check if it includes a key word, such as ‘Swiss’. The output can then be used to control an ‘If’ block to populate the output pattern accordingly. When true, one set of values can be set, when false a different set can be used.

Empty #

Generates an output of true if the input is empty. Spaces are not generally regarded as empty, unless the trim property is set on.

Example:

Can be used to test whether an input field has been provided in the document or not. The output is then used in further logic blocks or data flow blocks to determine what to do with missing data. For example, if transaction type is missing, an Empty block feeding an If block can control the output to be produced.

Ends With #

Generates an output of true if the input value ends with the string defined in the block properties. If the defined string is not present at the end, the output is false. The trim option allows spaces at the end of the input string to be ignored. Compare with ‘Contains’, and ‘Ends With’.

Example:

Can be used in a similar way to ‘Contains’, but checking only the end of the input string.

Matches #

Generates a result of true if the (complete) input string matches the regular expression defined in the block properties. If the string does not match, the result is false. The result is emitted on the Out output. The negated value of the result is emitted on the Negated output.

Example:

One use is in validations, to check if an input is zero or not. If the balance amount is zero, but there are non-zero liquid items, it suggests a problem with the input document. Use the ‘Matches’ block with regular expression 0*\.?0* (which tests for 0.0, and variants thereof) to determine if the balance is zero.

Not #

Generates an output of true if the input is false, and vice-versa. The input must be a Boolean.

There are no properties for this block.

Example:

Used to reverse a Boolean output. If you want to check if a string does not contain a value, use the ‘Contains’ block, followed by a ‘Not’ block to reverse the output.

Not Empty #

Generates an output of true if the input is not empty. Spaces and tabs are generally regarded as not empty, unless the trim property is set on.

Example:

Can be used to test whether an input field has been provided in the document or not. The output is then used in further logic blocks or data flow blocks to determine what to do with missing data. For example, if transaction type is present, a Not Empty block feeding an If block can control the output to be produced.

Or #

Generates an output of true if at least one input is true. If no inputs are true, the output is false. The inputs must all be Booleans.

Example: In a validation, you might want to test if the balance is empty or zero. Use an ‘Empty’ block to test the first case, and a ‘Matches’ block for the second, then an Or block to combine the result.

Starts With #

Generates an output of true if the input value begins with the string defined in the properties. If the defined string is not present at the start, the output is false. The trim option allows spaces at the beginning of the input string to be ignored. Compare with ‘Contains’, and ‘Ends With’.

Example:

Can be used in a similar way to ‘Contains’, but checking only the start of the input string.

XOR #

Generates an output of true if one and only one input is true. If all inputs are false, or more than one input is true, the output is false. The inputs must all be Booleans.

Example:

Another block useful in validations. We might want to check there are no liquid items in an account and the balance is zero. We can use a default block and NotEmpty block to check if there are liquid items, and a ‘Matches’ block to test for zero. We then use XOR to combine the results, setting an error if either there are liquid items and balance is zero (true and true), of if there are no liquid items and the balance is not zero (false and false). If there are liquid items and the balance is not zero (true and false), or no liquid items and the balance is zero (false and true) no error is set.