Transformer Reference

Transformers are used for extracting and converting values which then can be used by matchers. For a minimal working configuration you need at least one extract transformer and one output.

Convert List

The convert list transformation accepts a list of values as an input, applies the selected operation and returns a modified list.

This transformer supports a dynamic amount of inputs. For each input exactly one output will be available. Each input is independent from the other inputs.

The following operations are available:

Extract

The extract operation selects one or multiple sub values from each entry in the list using the provided path(s). If multiple values were selected, then they will be joined as a single text, optionally separated by the provided separator.

Empty values such as null and empty texts will be automatically removed. If the result is an empty array it will instead return null.

For a detailed description about the path syntax, please refer to the extract transformer description.

Example with One Path

• Path: aValue
• Separator: ignored

[
  {
    "aValue": "A1",
    "bValue": "B1"
  },
  {
    "aValue": "A2"
  },
  {
    "bValue": "B3"
  },
  {
    "aValue": "A4",
    "bValue": "B4"
  }
]

[
  "A1",
  "A2",
  "A4"
]

Example with Two Paths

• Paths: aValue and bValue
• Separator: ,
• Must Exist: disabled

[
  {
    "aValue": "A1",
    "bValue": "B1"
  },
  {
    "aValue": "A2"
  },
  {
    "bValue": "B3"
  },
  {
    "aValue": "A4",
    "bValue": "B4"
  }
]

[
  "A1,B1",
  "A2,",
  "B3,",
  "A4,B4"
]

Make Unique

Make unique removes duplicate entries from the list. Two entries are considered duplicates if their type and value are equal.

Example:

[
  "A",
  "A",
  1234,
  "B",
  1234,
  "1234"
]

[
  "A",
  1234,
  "B",
  "1234"
]

Slice

Slice returns a part of the provided list. If the offset is larger than the length of the provided list, then an empty list will be returned. If offset plus limit is larger than the provided list, then only the remaining part will be returned. The first element in the list is at offset 0.

Example:

• Offset: 1
• Limit: 2

[
  "A",
  "B",
  "C",
  "D",
  "E"
]

[
  "B",
  "C"
]

Sort

Sort orders the lists entries in ascending or descending order.

When ordering ascending with different data types for the entries, then numeric values will be sorted first, then texts, other values and null values. Sorting descending will reverse this order. The order for non-numeric and non-text values is undefined.

Example

• Ascending: enabled

[
  "A",
  2,
  1.5,
  "C",
  "B",
  "1"
]

[
  1.5,
  2,
  "1",
  "A",
  "B",
  "C"
]

Verify Size

Verify size will ensure that the provided lists size is within the boundaries. The maximum or minimum size can be disabled by providing -1. If the size requirements were not met, then an empty value is returned otherwise the original list is returned.

Example Outside Bounds

• Minimum Size: 3
• Maximum Size: 10

[
  "A",
  "B"
]

null

Example Inside Bounds

• Minimum Size: 3
• Maximum Size: 10

[
  "A",
  "B",
  "C"
]

[
  "A",
  "B",
  "C"
]

Convert Text

The convert text transformation accepts a text, applies the selected operation and returns the modified text.

This transformer supports a dynamic amount of inputs. For each input exactly one output will be available. Each input is independent from the other inputs.

The following operations are available:

Hash Value

Hash value applies one of the provided hash functions on the text input and returns the hash in hex format.

Example

• Hash Function: MD5

"Tilores"

"ef8394d0e4896d07e70e4106df2bf560"

Keep Only Numbers

Keep only numbers removes all non-number characters.

Example

"T1L0R35"

"1035"

Normalize Diacritical Characters

Normalize diacritical characters replaces characters such as German Umlaut with their character base.

Example

"än éᶍample"

"an example"

Normalize White-Spaces

Normalize white-spaces fixes duplicate horizontal white-spaces such as the space character or tabs and replaces them with a single white space. White-spaces at the beginning or the end of the text will be removed completely.

Example

"  an       Examp le  "

"an Examp le"

Remove All Numbers

Remove all numbers removes all number characters.

Example

"T1L0R35"

"TLR"

Remove Common Names

Remove common names replaces the text with an empty value if it is in the selected preset name list.

Example with Common Name

• Preset: First Names (US)
• Number of Top Most Common Names to Ignore: 20

"michael"

null

Example with Uncommon Name

• Preset: First Names (US)
• Number of Top Most Common Names to Ignore: 20

"tilo"

"tilo"

Remove Spaces

Remove spaces removes all spaces from the provided text.

Example

"  an       Examp le  "

"anExample"

Replace Text

Replace text searches for the provided text and replaces all occurrences with the new value. If the search value was not found, then the original text is returned.

Example with Simple Replacement

• Search: old
• Replace With: new
• Use Regular Expression: disabled

"old town with old church"

"new town with new church"

Example with Regular Expression

• Search: ^(.*), (.*)$
• Replace With: new: $2 $1
• Use Regular Expression: enabled

"Smith, John"

"new: John Smith"

Example with Non-Matching Regular Expression

• Search: ^(.*), (.*)$
• Replace With: new: $2 $1
• Use Regular Expression: enabled

"John Smith"

"John Smith"

Use Substring

Use substring returns the first characters of the provided text.

Example

• Length: 4

"Tilores"

"Tilo"

Map Value

Map value looks up the input text in a reference list and returns the corresponding mapped value. This is useful for converting values like country names to country codes.

• Reference List: the ID of the reference list to use for mapping
• Passthrough Unmapped: if enabled, values not found in the reference list are returned unchanged; if disabled (default), unmapped values return empty

Example with Found Value

• Reference List: countries (with rows like ["united states", "us"])
• Passthrough Unmapped: disabled

"united states"

"us"

Example with Unmapped Value (Passthrough Disabled)

• Reference List: countries
• Passthrough Unmapped: disabled

"unknown country"

null

Example with Unmapped Value (Passthrough Enabled)

• Reference List: countries
• Passthrough Unmapped: enabled

"unknown country"

"unknown country"

Extract

The extract transformation selects a value from the input using the provided path.

The transformer supports exactly one input, but provides a dynamic amount of outputs (each containing the same value).

The extract can work in either the simple path mode or in a jq-like path syntax.

For the jq-like path syntax, please refer to the official jq documentation. Please note, that not all jq features might be available. Furthermore, please note, that using the jq-like syntax might have a negative impact on the performance and should be avoided if possible.

In simple path mode, the field names and list indexes are separated by a single dot ..

Simple Path Mode Examples

• Path: firstName
• Case Sensitive: disabled

{
  "firstName": "John",
  "lastName": "Smith"
}

"john"

• Path: name.first
• Case Sensitive: disabled

{
  "name": {
    "first": "John",
    "last": "Smith"
  }
}

"john"

• Path: names.1.first
• Case Sensitive: disabled

{
  "names": [
    {
      "first": "Jane",
      "last": "Doe",
    },
    {
      "first": "John",
      "last": "Smith"
    }
  ]
}

"john"

• Path: name
• Case Sensitive: disabled

{
  "name": {
    "first": "John",
    "last": "Smith"
  }
}

{
  "first": "john",
  "last": "smith"
}

• Path: (empty)
• Case Sensitive: disabled

"John"

"john"

• Path: firstName
• Case Sensitive: disabled

{
  "firstName": "",
  "lastName": "Smith"
}

null

• Path: firstName
• Case Sensitive: disabled

{
  "lastName": "Smith"
}

null

• Path: firstName
• Case Sensitive: enabled

{
  "firstName": "John",
  "lastName": "Smith"
}

"John"

Filter In/Out

The filter in/out transformer removes or keeps only certain values.

This transformer supports exactly one input and output.

Examples

• Filter Out: disabled
• List of Values to Filter: ["A", "B"]

"A"

"A"

• Filter Out: disabled
• List of Values to Filter: ["A", "B"]

"C"

null

• Filter Out: enabled
• List of Values to Filter: ["A", "B"]

"A"

null

• Filter Out: enabled
• List of Values to Filter: ["A", "B"]

"C"

"C"

Using a Reference List

Instead of providing an inline list of values, you can reference a reference list containing the values to filter. The reference list must have a token column that contains the values to match against.

Example

• Filter Out: disabled
• Values Reference List: allowed-values (with rows like ["A"], ["B"])

"A"

"A"

This is particularly useful when you have a large list of values or want to reuse the same filter values across multiple transformers.

Flip Values

The flip values transformation accepts two inputs and flips them during indexing. This is an easy way to compare values from input a with values from input b during matching or searching.

This transformer supports exactly two inputs and two outputs.

Example

• Phase: Indexing

"John"

"Jane"

"Jane"

"John"

• Phase: Linking or Searching

"John"

"Jane"

"John"

"Jane"

Fork

The fork transformer can be used for simple and complex branching, including IF and SWITCH statements or cloning values for different results.

The transformer supports a dynamic number of inputs and will create exactly the same number of outputs for each configured condition, e.g. two inputs and three conditions will result in six outputs.

The fork strategy will define whether all conditions will be checked or if the check stops after the first satisfied condition (treating all other conditions as if they were not satisfied).

Conditions

The output for each condition will contain a value if the condition is satisfied, otherwise its outputs will all be null.

Each condition must have one of the following condition types.

Equal Values

Equal values will be satisfied if the value input equals the provided static value or another input. Whether two inputs or one input and a static value will be compared, can be toggled using the "compare with other input" checkbox.

All equality checks are case sensitive. If the data was extracted originally without the case sensitive option in the extract transformer, then you must provide a lower case variant for the static value.

Always True

This condition will always be satisfied. Use this for cloning values or creating a else or default branch when setting up an IF or SWITCH statement.

Match Regular Expression

This will be satisfied if the input matches the provided regular expression.

All equality checks are case sensitive. If the data was extracted originally without the case sensitive option in the extract transformer, then ensure that your regular expression matches with any lower case variant of the input.

Rule Set Type

This will be satisfied if the current phase is one of the selected phases. Use with caution as this might lead to unexpected, but correct results.

Example for Cloning Values

When working with the same data and applying two or more different transformations afterwards an easy way would be to use multiple outputs from one extract transformer. While this might require you to duplicate some common transformations, the alternative might be to split a single branch using the fork into multiple branches that contain the same data.

• 1 Input
• Fork Strategy: All Satisfied Conditions
• Condition 1:
  • Condition Type: Always True
• Condition 2:
  • Condition Type: Always True

"John"

"John"

"John"

• 2 Inputs
• Fork Strategy: All Satisfied Conditions
• Condition 1:
  • Condition Type: Always True
• Condition 2:
  • Condition Type: Always True

"John"

"Smith"

"John"

"Smith"

"John"

"Smith"

Example for IF Statement

A simple IF statement (allowing a value to pass if a condition is satisfied) is also possible.

• 2 Inputs
• Fork Strategy: First Statisfied Condition
• Condition 1:
  • Condition Type: Equal Values
  • Value From Input: 1
  • Compare With Other Input: disabled
  • Equals Value: John

With matching condition:

"John"

"Smith"

"John"

"Smith"

Without matching condition:

"Jim"

"Smith"

null

null

Example for IF-ELSE Statement

This is an extension of a IF statement with an else branch.

• 2 Inputs
• Fork Strategy: First Statisfied Condition
• Condition 1:
  • Condition Type: Equal Values
  • Value From Input: 1
  • Compare With Other Input: disabled
  • Equals Value: John
• Condition 2:
  • Condition Type: Always True

With matching condition:

"John"

"Smith"

"John"

"Smith"

null

null

Without matching condition:

"Jim"

"Smith"

null

null

"Jim"

"Smith"

Example for Comparing Inputs

This is similar like the simple IF statement, with the difference that two inputs were compared.

• 2 Inputs
• Fork Strategy: First Statisfied Condition
• Condition 1:
  • Condition Type: Equal Values
  • Value From Input: 1
  • Compare With Other Input: enabled
  • Equals Value from Input: 2

With matching condition:

"John"

"John"

"John"

"John"

Without matching condition:

"Jim"

"John"

null

null

Example for SWITCH Statement

This is an example for a SWITCH statement where multiple conditions can be satisfied. For simplicity this example only uses one input.

• 1 Input
• Fork Strategy: All Statisfied Conditions
• Condition #1:
  • Condition Type: Equal Values
  • Value From Input: 1
  • Compare With Other Input: disabled
  • Equals Value: John
• Condition #2:
  • Condition Type: Equal Values
  • Value From Input: 1
  • Compare With Other Input: disabled
  • Equals Value: Jim
• Condition #3:
  • Condition Type: Match Regular Expression
  • Value From Input: 1
  • Matches With Regular Expression: J.* (meaning: must start with J)

Conditions #1 and #3 satisfied:

"John"

"John"

null

"John"

Conditions #2 and #3 satisfied:

"Jim"

null

"Jim"

"Jim"

Condition #3 satisfied:

"Jane"

null

null

"Jane"

Generate Value

The generate value transformation creates new values that can be used within the whole transformation process. This may be useful for fallback/default values.

This transformer supports exactly one input and output.

The following generation strategy are available:

Static Text

This strategy returns the configured value at the output.

Example

• Value: John Smith

(anything)

"John Smith"

Random Number

This strategy returns a random number. The lowest and highest number are also possible returned values.

Example

• Lowest Number: 0
• Highest Number: 10

(anything)

3

Number Range

This strategy returns a list with all the numbers between the lowest and highest number (including both values).

Example

• Lowest Number: 0
• Highest Number: 10

(anything)

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Join

The join transformer can be used for merging multiple branches into a single branch. This is e.g. helpful after a fork or when the same data is present in different fields. Furthermore the join is also helpful when working with text lists to create a single, text out of it.

The join supports a dynamic number of outputs and by default one input for each output. It is possible to change how many inputs are expected by modifying the input per outputs configuration.

The join strategy will define how the data is processed.

First Non-Empty Value

This will return the first non-empty value from the inputs with the same name and return it as the output.

Example

• 2 Outputs
• Inputs Per Output: 2

"John"

null

"J."

"Smith"

"John"

"Smith"

Concatenate Texts

This will concatenate the texts with the same input name into the corresponding output, optionally separated by the provided separator. Empty values will be ignored.

If an input contains a string list, then each element will be concatenated.

Example with String Inputs

• 2 Outputs
• Inputs Per Output: 2

"John"

null

"J."

"Smith"

"John,J."

"Smith"

Example with String Lists

• 1 Output
• Inputs Per Output: 1

["John", "Jim", null, "Jane"]

"John,Jim,Jane"

Merge Into Array

This will merge the values with the same input name into a list for the corresponding output. Empty values (null) will be ignored.

• 2 Outputs
• Inputs Per Output: 2

"John"

null

"J."

"Smith"

["John", "J."]

["Smith"]

Make Date

The make date transformer will construct a date from different inputs. This is an easy way to combine multiple date related fields into a single text representation.

Examples of valid values for the full input, where 2023 represents the year, 4 represents the month and 5 represents the day:

• 2023-04-05
• 2023-4-5
• 2023-04-5
• 2023-04
• 2023

The other inputs must be either a number or a text that can be interpreted as a number.

The output will combine different inputs, but prioritize the value from the full input.

The full output will always be in the 2023-04-05 format. Missing values will be filled with 0, unless the validate option is enabled. The other outputs will be strings representing the corresponding part from the full output, e.g. 04

If the validate option is enabled and the year, month or day is empty, then an empty value will be returned on all outputs.

This transformer supports the following inputs and outputs: full, year, month and day.

Example with Full Input Priority

• Validate: enabled

"2023-04"

null

"12"

5

"2023-04-05"

"2023"

"04"

"05"

Examples with Partial Date

• Validate: enabled

"2023-04"

null

null

null

null

null

null

null

• Validate: disabled

"2023-04"

null

null

null

"2023-04-00"

"2023"

"04"

"00"

Examples with Invalid Date

• Validate: enabled

null

2023

2

31

null

null

null

null

• Validate: disabled

null

2023

2

31

"2023-02-31"

"2023"

"02"

"31"

Normalize Address

The normalize address transformer normalizes a free-form street / address line into a human-readable canonical form using country specific reference lists. As part of that it detects and extracts the house number, and it can independently normalize the city and the postal code.

This transformer supports up to five inputs (street, countryCode, houseNumber, city, postalCode) and four outputs (normalizedStreet, normalizedHouseNumber, normalizedCity, normalizedPostalCode). The countryCode input is required and must be an ISO 3166-1 Alpha-2 code; it governs all country specific rules.

The street is normalized in several steps: optional preprocessing (regex compound splitting, e.g. Hauptstr. → Haupt str.), house number extraction, tokenization, token mapping (e.g. str → Straße), particle aware capitalization (particles such as den / der are kept lowercase) and optional postprocessing. House numbers are recognized either by explicit markers (Nr., No., #, nº, civico, …) or by position (leading for anglophone / French formats, trailing for most European ones) and keep ranges and suffixes (12-14, 19c → 19C). The city is canonicalized via the city reference list, and the postal code is reformatted using built-in per-country rules (no reference list). All reference lists are optional; if a country has no matching rows the pipeline degrades to a tokenize + capitalize cleanup. Block-addressing countries (e.g. jp, kr, cn) skip house number extraction.

Configuration:

• Preprocessing Reference List (optional)
• Token Mapping Reference List (optional)
• Particles Reference List (optional)
• Postprocessing Reference List (optional)
• City Reference List (optional)
• Remove Structurally Invalid Postal Codes: disabled or enabled

Example

"Hauptstr. 19c"

"de"

"Hauptstraße"

"19C"

Example with City and Postal Code

"1234 Main Rd"

"us"

"NYC"

"221621010"

"Main Road"

"1234"

"New York"

"22162-1010"

Normalize Company

The normalize company transformer splits a company name into the name itself and its legal form. Version 2 also reports what else it recognized in the name.

Two versions exist. Version 2 is driven by country specific reference lists and is what new configurations should use. Version 1 is the legacy implementation: it is frozen, still supported, and is what a transformer runs when it does not declare a version. Nothing changes for an existing configuration unless you opt in.

Normalize Company (version 2)

Version 2 resolves the legal form from reference lists instead of compiled-in data, so the forms Tilores recognizes can be extended without waiting for a release. Detection is scoped per country: the country input decides which jurisdiction's forms apply, which is what keeps Est a legal form in the Gulf states and an ordinary word in France.

It supports two inputs (default, country) and six outputs:

The name is processed in several steps: optional preprocessing (regex rewrites that move a leading legal form to the end, or separate a form written without a space), legal form detection and removal, group function detection, branch and location detection, and optional postprocessing. The lists that shape the company name are all required; the ones feeding the groupFunction, location and elfCode outputs are optional, and leaving one out simply keeps that output empty.

Configuration. Every list that shapes the company name is required: wiring only some of them produces a name that is quietly worse than the same configuration wiring all of them, so the transformer refuses to load instead. The optional lists are exactly those feeding a reporting output; without one, that output stays empty and the company name is unaffected.

• Legal Form Detection Reference List (required)
• Legal Form Alias Reference List (required)
• Preprocessing Reference List (required)
• Postprocessing Reference List (required)
• Company Token Reference List (required)
• Generic Token Reference List (required)
• Root Separator Reference List (required)
• Root Glue Word Reference List (required)
• Group Function Detection Reference List (optional)
• Group Function Alias Reference List (optional)
• Branch Designator Reference List (optional)
• Location City Reference List (optional)
• Location Country Reference List (optional)
• Location City Token Reference List (optional)
• ISO 20275 ELF Code Reference List (optional)
• Advanced Legal Form Detection: enabled (default) or disabled
• Strip Group Function From Root: disabled (default) or enabled
• Strip Branch Location From Root: disabled (default) or enabled

Example

"Tilo Tech GmbH"

"DE"

"tilo tech"

"gmbh"

Example with Every Spelling of One Form

Each of these reaches the same legalForm, which is the point of the alias list: two records of one company written differently produce the same value.

"Tilo Tech Gesellschaft mit beschränkter Haftung"

"DE"

"tilo tech"

"gmbh"

Example with a Group Function

The group function is reported but left in the name by default: within one group it is often the only thing telling two companies apart, so removing it would merge them.

"Tilo Tech Holding GmbH"

"DE"

"tilo tech holding"

"gmbh"

"holding"

Example with a Language Without Word Spacing

Chinese, Japanese and Korean names do not separate words, so the legal form is written directly against the company name. Preprocessing inserts the boundary and the form is then detected as usual.

"ソニー株式会社"

"JP"

"ソニー"

"kk"

Normalize Company (version 1, legacy)

The normalize company transformer will split the provided text into a company name and its legal form if present.

This transformer supports exactly one input and two outputs.

Examples with Simple Legal Form Detection

• Advanced Legal Form Detection: disabled

"Tilo Tech GmbH"

"Tilo Tech"

"GmbH"

null

• Advanced Legal Form Detection: disabled

"Example LLC New York"

"Example LLC New York"

null

null

Examples with Advanced Legal Form Detection

• Advanced Legal Form Detection: enabled

"Tilo Tech GmbH"

"Tilo Tech"

"gmbh"

null

• Advanced Legal Form Detection: enabled

"Example LLC New York"

"Example"

"llc"

"New York"

If the country input will provide a valid ISO 3166 Alpha-2 country code and advanced legal form detection is enabled, Tilores will try to resolve country specific legal form aliases.

• Advanced Legal Form Detection: enabled

"Tilo Tech Gesellschaft mit beschränkter Haftung"

"DE"

"Tilo Tech"

"gmbh"

null

Normalize Country

The normalize country transformer normalizes a free-form country value into a lowercase ISO 3166-1 Alpha-2 country code (e.g. de, us). It resolves country names, ISO Alpha-2 / Alpha-3 codes, abbreviations and common misspellings against a user-provided reference list.

This transformer supports one required input (country) and two optional inputs (city, street), and produces a single output (countryCode).

Resolution follows a fallback cascade: the country value is looked up in the country name reference list first; if that fails and the city input is connected, the city reference list is consulted; if that also fails and the street input is connected, street specific tokens (e.g. strasse → de) are matched; finally the configured fallback country is used. If nothing matches and no fallback is set, the output is null. Lookups are accent- and case-insensitive.

Configuration:

• Fallback Country (ISO 3166-1 alpha-2)
• Country Name Reference List
• City Reference List (optional)
• Street Token Reference List (optional)

Example

"finnland"

"fi"

Example with City Fallback

• Fallback Country: "us"

"unknown"

"München"

"de"

"unknown"

"Little Vil"

"us"

Normalize Email

The normalize email transformer will ensure a common structure for email addresses and provides additional outputs for parts of an email address such as the local part.

This transformer supports exactly one input and two outputs.

Examples

• Remove Plus Addressing: disabled or enabled

foo @example.com

foo@example.com

foo

example.com

example

com

null

null

• Remove Plus Addressing: disabled

foo+123@example.com

foo+123@example.com

foo+123

example.com

example

com

null

null

• Remove Plus Addressing: enabled

foo+123@example.com

foo@example.com

foo

example.com

example

com

null

null

• Remove Plus Addressing: disabled or enabled

john.smith@example.com

john.smith@example.com

john.smith

example.com

example

com

john

smith

Note: name recognition in emails only works with the following patterns:

• <first>.<last>@<domain>
• <first>_<last>@<domain>
• <first>.<last>+<plus tag>@<domain>
• <first>_<last>+<plus tag>@<domain>

Normalize Name

The normalize name transformer splits a personal name into its parts: the titles, the given names, the family name and any suffix.

Two versions exist. Version 2 is driven by reference lists and is what new configurations should use. Version 1 is the legacy implementation: it is frozen, still supported, and is what a transformer runs when it does not declare a version. Nothing changes for an existing configuration unless you opt in.

Normalize Name (version 2)

Version 2 resolves titles, suffixes, surname particles, transliteration and surname structure from reference lists instead of compiled-in data, so what Tilores recognizes can be extended without waiting for a release.

Every list is looked up through the same three tiers, and the first tier that holds the value wins:

country  ->  language(s)  ->  *

The country input supplies the first tier. The languages come from the country-to-language list, so a configuration provides a country code and gets language-scoped behaviour for free. The last tier applies everywhere.

A country with several languages lists them in order (Switzerland: German, French, Italian, Romansh). The countryLanguageStrategy attribute decides which of them are used:

Dott. Marco Della Valle in a Swiss record gives the academic title dr, first name marco and last name della valle. With first it reads German only, does not know Dott. as a title, and gives the first name dott, middle names marco della and last name valle.

The optional language input names the record's own language, an ISO 639-3 code such as fra, and overrides the country's languages. Connect it when the source carries one, for example a correspondence language.

Inputs

The five structured inputs are optional. Connect them when the source data already splits the name: each one replaces the corresponding part of the parse while still being normalized, so a source that supplies a clean surname and a messy given name gets the surname respected and the given name repaired.

A provided part decides which part a token belongs to, not what the token is. A title in the first name field (Dr Hans) still goes to the title outputs, and so does an academic or social title in the last name field (Dr Huber); other title words are common surnames (Marin, Major, Graf) and stay. A second given name in the first name field (Thomas Johannes) is a middle name, and a particle among the given names stays there (María del Carmen). A suffix in the last name field (Smith Jr) goes to suffix, but a last name that is only a suffix word (Senior) is the family name. The family name itself stays whole (Lo Russo, Ud Din), and where a culture has two it is split into both (Garcia Lopez) without losing a word: De La Cruz Lopez gives de la cruz and lopez. A particle of any language stays with the word after it (Abu Awad, Le Coassin).

Two kinds of provided value are not treated as a split. A value that is not a person, such as an email address or a company name, gives no name, as it does in the full name. And a first or last name given alone that holds the whole full name, a common data-entry pattern, is parsed as the full name. With no default input connected, a first or last name given alone with two or more words is taken as the full name in the same way (Elia Ntaousani in the last name field gives first name elia, last name ntaousani); a single word stays the part it was provided as.

Outputs

firstName and lastName are always Latin. The native outputs carry the same two parts in the original script, from the same parse, and only where they differ from firstName and lastName: for a Latin name they are empty.

What happens to a name

In detail:

• Values that are not a person give no name: an email address, a placeholder such as unknown, or a company name with a legal form (GmbH, Ltd, & Co). A word that contains a digit is dropped: Hans Müller 2 gives hans / müller.

• Titles and suffixes are taken out of the name and returned in title, academicTitle, nobilityTitle and suffix, in the spelling of the record's language (Dott., Mme, Dr. med. univ.) and in other scripts.

• A comma means the family name comes first: everything before it is the family name.

• Particles stay with the family name (van, de la, bin), except after a comma, where they belong to the given names.

• Where a culture uses two surnames, lastName holds the one used to identify the person (the first in Spanish, the last in Portuguese) and secondaryLastName the other. A known given name after the first name stays a given name, including Marian names such as del Carmen.

• Chinese, Korean and Vietnamese names may be written family name first or last; the spelling of each part decides which is the family name.

• A name in another script is transliterated to Latin for firstName and lastName, with the letter table of the record's language (Ukrainian г is h, Russian г is g). firstNameNative and lastNameNative keep the original spelling.

• Punctuation is repaired: an apostrophe inside or at the end of a name is removed (O'Brien, Desire'), and a hyphen written with a space beside it is closed up. In a name of only two words such a hyphen separates them instead (Wolfgang -Traxler).

• A value with only one word fills neither name, because nothing tells a given name alone from a family name alone. To match such values, compare the input with the first and last name outputs in a rule.

The structured inputs get the same treatment: a title in the first name field is still a title, and a company in the name fields still gives no name.

Configuration

Title, suffix and particles are required; without them the transformer does not load. Each remaining list is optional, and leaving one out only turns off what it drives.

• Title Reference List (required)
• Suffix Reference List (required)
• Surname Particles Reference List (required)
• Country Code to Language (optional) -- supplies the language tier
• Letter Transliteration (optional) -- makes names in other scripts Latin
• Culture Surname Structure (optional) -- drives secondaryLastName
• Known Given Names (optional) -- Spanish given names, for three-part names
• Romanised Syllables (optional) -- decides the name order in Chinese, Korean and Vietnamese names
• Name Token Map (optional) -- the most frequent name tokens of the scripts no letter rule can read
• Preprocessing Rules (optional)
• Postprocessing Rules (optional)

Example

"Prof. Dr. John Michael Doe Jr."

"US"

"dr prof"

"john"

"michael"

"doe"

"jr"

Example with Transliteration

The title, the given names and the family name are all recognized in Arabic script. Every part comes back in Latin, so this record can match a record for the same person written in Latin.

"السيد محمد عبدالله ناصر سلمان"

"SA"

"mr"

"mohammed"

"abdullah nasser"

"salman"

Example with Two Family Names

Spanish names carry a paternal and a maternal surname. The culture surname structure list says how many surnames the culture uses and which one is the primary, so lastName stays the matching value and the second surname is reported separately.

"Jose Maria Garcia Lopez"

"ES"

"jose"

"maria"

"garcia"

"lopez"

Example with the Family Name First

A comma means the family name comes first, and everything before it is the family name.

"Garcia Lopez, Maria Isabel"

"ES"

"maria"

"isabel"

"garcia"

"lopez"

Example with a Surname Particle

The particles list attaches van, de la, bin and their equivalents to the family name, so the particle cannot be read as a middle name.

"Ludwig van Beethoven"

"DE"

"ludwig"

"van beethoven"

Example with a Post-Nominal Suffix

A suffix is matched with every dot removed, so M.D. and MD reach the same row and give the same value.

"Robert Kennedy M.D."

"US"

"robert"

"kennedy"

"md"

Folding a Diminutive

There is no output pin for this. Folding bill onto robert does not structure a name: it relabels a part the parse already got right, which is what a mapping does. A configuration that needs it applies a reference list of its own downstream of firstName, where it can see the fold and decide whether the result reaches a matching key.

Use Map Value to fold, with pass-through enabled so a name that is not a known variant flows on unchanged:

Use Filter In/Out with the same list, keeping only matching values, to get the given name only when it could be a diminutive:

Normalize Name (version 1, legacy)

The normalize name transformer structures and optionally transliterates the provided personal name based on the provided language.

This transformer supports one required input (default) and one optional input (language). It produces multiple outputs representing structured parts of the name.

Supported Outputs

• fullName
• title
• academicTitle
• nobilityTitle
• firstInitial
• firstName
• secondName
• thirdName
• lastName
• suffix

If language is provided and recognized, language-specific normalization and transliteration are applied. Otherwise, the transformer defaults to English.

Examples in English

• Input language: unknown

"Prof. Dr. John Michael Doe Jr."

"Prof. Dr. John Michael Doe Jr."

null

"dr prof"

null

"j"

"john"

"michael"

null

"doe"

"jr."

Examples with Transliteration

• Input language: ara (Arabic, ISO 639-2 code)

"السيد محمد عبدالله ناصر سلمان"

"السيد محمد عبدالله ناصر سلمان"

"mr"

"mohammad"

"abdullah"

"nasser"

"salman"

null

Partial Outputs

• Input language: ARA (case-insensitive ISO 639-2 code)

"السيد محمد سلمان"

"السيد محمد سلمان"

"mr"

"mohammad"

null

null

"salman"

null

Invalid or Unsupported Inputs

• Non-string inputs will result in all outputs being null.

123

null

null

null

• Numeric text is treated as a string but produces no structured parts.

"123"

"123"

null

null

Language Input and ISO 639-2 Codes

The optional language input allows the transformer to apply language-specific parsing and transliteration rules.

• The transformer accepts ISO 639-2 three-letter codes (e.g. eng for English, ara for Arabic, deu for German).
• Codes are case-insensitive (ara and ARA are treated the same).
• If the provided code is not recognized, the transformer falls back to English.
• When recognized, transliteration of non-Latin scripts is applied to enable consistent normalization (e.g., Arabic names → Latin characters).
• Supported languages:
  • Arabic ara
  • English eng

Normalize Phone Number

The normalize phone number transformer will normalize the provided phone number and split it into a local number and a country code. And also outputs the E.164 formatted version.

This transformer supports exactly two inputs and three outputs.

The default country is used to provide context information for better number recognition. You can either provide a record specific value using the extract transformer, or provide a static value using the generate value transformer. Different country name formats will be accepted, but it is recommended to provide a ISO 3166-1 Alpha-2 or Alpha-3 code.

The presence of a country code in the phone number (typically represented by a + sign, will overwrite the default country). If the country cannot be assumed and no default country was provided, then the phone number cannot be normalized.

Example

"+49650-253-0000"

"US"

"6502530000"

"49"

"+496502530000"

In this example, the default country is ignored, because the +49 in the input number clearly indicates a german phone number.

Output

The output must be the final step for each transformation branch. Only outputs can be used from matchers. Each output must have a unique label to identify it later in the matchers.

Persisted Output

Setting the option targetFieldPath to a non empty value (e.g. transformed.name) will cause the transformation output to be stored into the provided field path which must also be available in the schema.