# Transformer Reference

Transformers are used for extracting and converting values which then can be
used by [matchers](matcher.md). For a minimal working configuration you need at
least one [extract transformer](#extract-1) and [one output](#output).

## Convert List

The convert list transformation accepts a list of values as an input, applies
the selected operation and returns a modified list.

This transformer supports a dynamic amount of inputs. For each input exactly one
output will be available. Each input is independent from the other inputs.

The following operations are available:

### Extract

The extract operation selects one or multiple sub values from each entry in the
list using the provided path(s). If multiple values were selected, then they
will be joined as a single text, optionally separated by the provided separator.

Empty values such as `null` and empty texts will be automatically removed. If
the result is an empty array it will instead return `null`.

For a detailed description about the path syntax, please refer to the
[extract transformer description](#extract-1).

#### Example with One Path

* Path: `aValue`
* Separator: ignored

||| Input
```json
[
  {
    "aValue": "A1",
    "bValue": "B1"
  },
  {
    "aValue": "A2"
  },
  {
    "bValue": "B3"
  },
  {
    "aValue": "A4",
    "bValue": "B4"
  }
]
```
||| Output
```json
[
  "A1",
  "A2",
  "A4"
]
```
|||

#### Example with Two Paths

* Paths: `aValue` and `bValue`
* Separator: `,`
* Must Exist: disabled

||| Input
```json
[
  {
    "aValue": "A1",
    "bValue": "B1"
  },
  {
    "aValue": "A2"
  },
  {
    "bValue": "B3"
  },
  {
    "aValue": "A4",
    "bValue": "B4"
  }
]
```
||| Output
```json
[
  "A1,B1",
  "A2,",
  "B3,",
  "A4,B4"
]
```
|||

### Make Unique

Make unique removes duplicate entries from the list. Two entries are considered
duplicates if their type and value are equal.

#### Example:

||| Input
```json
[
  "A",
  "A",
  1234,
  "B",
  1234,
  "1234"
]
```
||| Output
```json
[
  "A",
  1234,
  "B",
  "1234"
]
```
|||

### Slice

Slice returns a part of the provided list. If the offset is larger than the
length of the provided list, then an empty list will be returned. If offset plus
limit is larger than the provided list, then only the remaining part will be
returned. The first element in the list is at offset 0.

#### Example:

* Offset: 1
* Limit: 2

||| Input
```json
[
  "A",
  "B",
  "C",
  "D",
  "E"
]
```
||| Output
```json
[
  "B",
  "C"
]
```
|||

### Sort

Sort orders the lists entries in ascending or descending order.

When ordering ascending with different data types for the entries, then numeric
values will be sorted first, then texts, other values and null values. Sorting
descending will reverse this order. The order for non-numeric and non-text
values is undefined.

#### Example

* Ascending: enabled

||| Input
```json
[
  "A",
  2,
  1.5,
  "C",
  "B",
  "1"
]
```
||| Output
```
[
  1.5,
  2,
  "1",
  "A",
  "B",
  "C"
]
```
|||

### Verify Size

Verify size will ensure that the provided lists size is within the boundaries.
The maximum or minimum size can be disabled by providing `-1`. If the size
requirements were not met, then an empty value is returned otherwise the
original list is returned.

#### Example Outside Bounds

* Minimum Size: 3
* Maximum Size: 10

||| Input
```json
[
  "A",
  "B"
]
```
||| Output
```
null
```
|||

#### Example Inside Bounds

* Minimum Size: 3
* Maximum Size: 10

||| Input
```json
[
  "A",
  "B",
  "C"
]
```
||| Output
```json
[
  "A",
  "B",
  "C"
]
```
|||

## Convert Text

The convert text transformation accepts a text, applies the selected operation
and returns the modified text.

This transformer supports a dynamic amount of inputs. For each input exactly one
output will be available. Each input is independent from the other inputs.

!!!
All operations support both individual text inputs as well as lists of texts.
Providing a list of texts will apply the operation on each text individually.
!!!

The following operations are available:

### Hash Value

Hash value applies one of the provided hash functions on the text input and
returns the hash in hex format.

#### Example

* Hash Function: MD5

||| Input
```json
"Tilores"
```
||| Output
``` json
"ef8394d0e4896d07e70e4106df2bf560"
```
|||

### Keep Only Numbers

Keep only numbers removes all non-number characters.

#### Example

||| Input
```json
"T1L0R35"
```
||| Output
```json
"1035"
```
|||

### Normalize Diacritical Characters

Normalize diacritical characters replaces characters such as German Umlaut with
their character base.

#### Example

||| Input
```json
"än éᶍample"
```
||| Output
```json
"an example"
```
|||

### Normalize White-Spaces

Normalize white-spaces fixes duplicate horizontal white-spaces such as the space
character or tabs and replaces them with a single white space. White-spaces at
the beginning or the end of the text will be removed completely.

#### Example
||| Input
```json
"  an       Examp le  "
```
||| Output
```json
"an Examp le"
```
|||

### Remove All Numbers

Remove all numbers removes all number characters.

#### Example
||| Input
```json
"T1L0R35"
```
||| Output
```json
"TLR"
```
|||

### Remove Common Names

Remove common names replaces the text with an empty value if it is in the
selected preset name list.

#### Example with Common Name

* Preset: First Names (US)
* Number of Top Most Common Names to Ignore: 20

||| Input
```json
"michael"
```
||| Output
```json
null
```
|||

#### Example with Uncommon Name

* Preset: First Names (US)
* Number of Top Most Common Names to Ignore: 20

||| Input
```json
"tilo"
```
||| Output
```json
"tilo"
```
|||

### Remove Spaces

Remove spaces removes all spaces from the provided text.

#### Example
||| Input
```json
"  an       Examp le  "
```
||| Output
```json
"anExample"
```
|||

### Replace Text

Replace text searches for the provided text and replaces all occurrences with
the new value. If the search value was not found, then the original text is
returned.

#### Example with Simple Replacement

* Search: `old`
* Replace With: `new`
* Use Regular Expression: disabled

||| Input
```json
"old town with old church"
```
||| Output
```json
"new town with new church"
```
|||

#### Example with Regular Expression

* Search: `^(.*), (.*)$`
* Replace With: `new: $2 $1`
* Use Regular Expression: enabled

||| Input
```json
"Smith, John"
```
||| Output
```json
"new: John Smith"
```
|||

#### Example with Non-Matching Regular Expression

* Search: `^(.*), (.*)$`
* Replace With: `new: $2 $1`
* Use Regular Expression: enabled

||| Input
```json
"John Smith"
```
||| Output
```json
"John Smith"
```
|||

### Use Substring

Use substring returns the first characters of the provided text.

#### Example

* Length: 4

||| Input
```json
"Tilores"
```
||| Output
```json
"Tilo"
```
|||

### Map Value

Map value looks up the input text in a [reference list](reference-lists.md) and returns
the corresponding mapped value. This is useful for converting values like country names
to country codes.

* Reference List: the ID of the reference list to use for mapping
* Passthrough Unmapped: if enabled, values not found in the reference list are returned
  unchanged; if disabled (default), unmapped values return empty

#### Example with Found Value

* Reference List: `countries` (with rows like `["united states", "us"]`)
* Passthrough Unmapped: disabled

||| Input
```json
"united states"
```
||| Output
```json
"us"
```
|||

#### Example with Unmapped Value (Passthrough Disabled)

* Reference List: `countries`
* Passthrough Unmapped: disabled

||| Input
```json
"unknown country"
```
||| Output
```json
null
```
|||

#### Example with Unmapped Value (Passthrough Enabled)

* Reference List: `countries`
* Passthrough Unmapped: enabled

||| Input
```json
"unknown country"
```
||| Output
```json
"unknown country"
```
|||

## Extract

The extract transformation selects a value from the input using the provided
path.

The transformer supports exactly one input, but provides a dynamic amount of
outputs (each containing the same value).

The extract can work in either the simple path mode or in a jq-like path syntax.

For the jq-like path syntax, please refer to the [official jq documentation](https://jqlang.github.io/jq/manual/).
Please note, that not all jq features might be available. Furthermore, please
note, that using the jq-like syntax might have a negative impact on the
performance and should be avoided if possible.

In simple path mode, the field names and list indexes are separated by a single
dot `.`.

#### Simple Path Mode Examples

* Path: `firstName`
* Case Sensitive: disabled

||| Input
```json
{
  "firstName": "John",
  "lastName": "Smith"
}
```
||| Output
```json
"john"
```
|||

* Path: `name.first`
* Case Sensitive: disabled

||| Input
```json
{
  "name": {
    "first": "John",
    "last": "Smith"
  }
}
```
||| Output
```json
"john"
```
|||

* Path: `names.1.first`
* Case Sensitive: disabled

||| Input
```json
{
  "names": [
    {
      "first": "Jane",
      "last": "Doe",
    },
    {
      "first": "John",
      "last": "Smith"
    }
  ]
}
```
||| Output
```json
"john"
```
|||

* Path: `name`
* Case Sensitive: disabled

||| Input
```json
{
  "name": {
    "first": "John",
    "last": "Smith"
  }
}
```
||| Output
```json
{
  "first": "john",
  "last": "smith"
}
```
|||

* Path: ` ` *(empty)*
* Case Sensitive: disabled

||| Input
```json
"John"
```
||| Output
```json
"john"
```
|||

* Path: `firstName`
* Case Sensitive: disabled

||| Input
```json
{
  "firstName": "",
  "lastName": "Smith"
}
```
||| Output
```json
null
```
|||

* Path: `firstName`
* Case Sensitive: disabled

||| Input
```json
{
  "lastName": "Smith"
}
```
||| Output
```json
null
```
|||

* Path: `firstName`
* Case Sensitive: enabled

||| Input
```json
{
  "firstName": "John",
  "lastName": "Smith"
}
```
||| Output
```json
"John"
```
|||

## Filter In/Out

The filter in/out transformer removes or keeps only certain values.

This transformer supports exactly one input and output.

#### Examples

* Filter Out: disabled
* List of Values to Filter: `["A", "B"]`

||| Input
```json
"A"
```
||| Output
```json
"A"
```
|||

* Filter Out: disabled
* List of Values to Filter: `["A", "B"]`

||| Input
```json
"C"
```
||| Output
```json
null
```
|||

* Filter Out: enabled
* List of Values to Filter: `["A", "B"]`

||| Input
```json
"A"
```
||| Output
```json
null
```
|||

* Filter Out: enabled
* List of Values to Filter: `["A", "B"]`

||| Input
```json
"C"
```
||| Output
```json
"C"
```
|||

### Using a Reference List

Instead of providing an inline list of values, you can reference a
[reference list](reference-lists.md) containing the values to filter. The reference
list must have a `token` column that contains the values to match against.

#### Example

* Filter Out: disabled
* Values Reference List: `allowed-values` (with rows like `["A"]`, `["B"]`)

||| Input
```json
"A"
```
||| Output
```json
"A"
```
|||

This is particularly useful when you have a large list of values or want to reuse
the same filter values across multiple transformers.

## Flip Values

The flip values transformation accepts two inputs and flips them during indexing.
This is an easy way to compare values from input `a` with values from input `b`
during matching or searching.

This transformer supports exactly two inputs and two outputs.

#### Example

* Phase: Indexing

||| Input (a)
```json
"John"
```
||| Input (b)
```json
"Jane"
```
||| Output (a)
```json
"Jane"
```
||| Output (b)
```json
"John"
```
|||

* Phase: Linking or Searching

||| Input (a)
```json
"John"
```
||| Input (b)
```json
"Jane"
```
||| Output (a)
```json
"John"
```
||| Output (b)
```json
"Jane"
```
|||

## Fork

The fork transformer can be used for simple and complex branching, including IF
and SWITCH statements or cloning values for different results.

The transformer supports a dynamic number of inputs and will create exactly the
same number of outputs for each configured condition, e.g. two inputs and three
conditions will result in six outputs.

The fork strategy will define whether all conditions will be checked or if the
check stops after the first satisfied condition (treating all other conditions
as if they were not satisfied).

#### Conditions

The output for each condition will contain a value if the condition is satisfied,
otherwise its outputs will all be `null`.

Each condition must have one of the following condition types.

##### Equal Values

Equal values will be satisfied if the value input equals the provided static
value or another input. Whether two inputs or one input and a static value will
be compared, can be toggled using the "compare with other input" checkbox.

All equality checks are case sensitive. If the data was extracted originally
without the case sensitive option in the [extract transformer](#extract-1), then
you must provide a lower case variant for the static value.

##### Always True

This condition will always be satisfied. Use this for cloning values or creating
a else or default branch when setting up an IF or SWITCH statement.

##### Match Regular Expression

This will be satisfied if the input matches the provided
[regular expression](https://en.wikipedia.org/wiki/Regular_expression).

All equality checks are case sensitive. If the data was extracted originally
without the case sensitive option in the [extract transformer](#extract-1), then
ensure that your regular expression matches with any lower case variant of the
input.

##### Rule Set Type

This will be satisfied if the current phase is one of the selected phases. Use
with caution as this might lead to unexpected, but correct results.

#### Example for Cloning Values

When working with the same data and applying two or more different transformations
afterwards an easy way would be to use multiple outputs from one
[extract transformer](#extract-1). While this might require you to duplicate
some common transformations, the alternative might be to split a single branch
using the fork into multiple branches that contain the same data.

* 1 Input
* Fork Strategy: All Satisfied Conditions
* Condition 1:
  * Condition Type: Always True
* Condition 2:
  * Condition Type: Always True

||| Input
```json
"John"
```
||| Output #1
```json
"John"
```
||| Output #2
```json
"John"
```
|||

* 2 Inputs
* Fork Strategy: All Satisfied Conditions
* Condition 1:
  * Condition Type: Always True
* Condition 2:
  * Condition Type: Always True

||| Input (1)
```json
"John"
```
||| Input (2)
```json
"Smith"
```
||| Output #1 (1)
```json
"John"
```
||| Output #1 (2)
```json
"Smith"
```
||| Output #2 (1)
```json
"John"
```
||| Output #2 (2)
```json
"Smith"
```
|||

#### Example for IF Statement

A simple IF statement (allowing a value to pass if a condition is satisfied) is
also possible.

* 2 Inputs
* Fork Strategy: First Statisfied Condition
* Condition 1:
  * Condition Type: Equal Values
  * Value From Input: 1
  * Compare With Other Input: disabled
  * Equals Value: `John`

With matching condition:

||| Input (1)
```json
"John"
```
||| Input (2)
```json
"Smith"
```
||| Output #1 (1)
```json
"John"
```
||| Output #1 (2)
```json
"Smith"
```
|||

Without matching condition:

||| Input (1)
```json
"Jim"
```
||| Input (2)
```json
"Smith"
```
||| Output #1 (1)
```json
null
```
||| Output #1 (2)
```json
null
```
|||

#### Example for IF-ELSE Statement

This is an extension of a IF statement with an else branch.

* 2 Inputs
* Fork Strategy: First Statisfied Condition
* Condition 1:
  * Condition Type: Equal Values
  * Value From Input: 1
  * Compare With Other Input: disabled
  * Equals Value: `John`
* Condition 2:
  * Condition Type: Always True

With matching condition:

||| Input (1)
```json
"John"
```
||| Input (2)
```json
"Smith"
```
||| Output #1 (1)
```json
"John"
```
||| Output #1 (2)
```json
"Smith"
```
||| Output #2 (1)
```json
null
```
||| Output #2 (2)
```json
null
```
|||

Without matching condition:

||| Input (1)
```json
"Jim"
```
||| Input (2)
```json
"Smith"
```
||| Output #1 (1)
```json
null
```
||| Output #1 (2)
```json
null
```
||| Output #2 (1)
```json
"Jim"
```
||| Output #2 (2)
```json
"Smith"
```
|||

#### Example for Comparing Inputs

This is similar like the simple IF statement, with the difference that two
inputs were compared.

* 2 Inputs
* Fork Strategy: First Statisfied Condition
* Condition 1:
  * Condition Type: Equal Values
  * Value From Input: 1
  * Compare With Other Input: enabled
  * Equals Value from Input: 2

With matching condition:

||| Input (1)
```json
"John"
```
||| Input (2)
```json
"John"
```
||| Output #1 (1)
```json
"John"
```
||| Output #1 (2)
```json
"John"
```
|||

Without matching condition:

||| Input (1)
```json
"Jim"
```
||| Input (2)
```json
"John"
```
||| Output #1 (1)
```json
null
```
||| Output #1 (2)
```json
null
```
|||

#### Example for SWITCH Statement

This is an example for a SWITCH statement where multiple conditions can be
satisfied. For simplicity this example only uses one input.

* 1 Input
* Fork Strategy: All Statisfied Conditions
* Condition #1:
  * Condition Type: Equal Values
  * Value From Input: 1
  * Compare With Other Input: disabled
  * Equals Value: `John`
* Condition #2:
  * Condition Type: Equal Values
  * Value From Input: 1
  * Compare With Other Input: disabled
  * Equals Value: `Jim`
* Condition #3:
  * Condition Type: Match Regular Expression
  * Value From Input: 1
  * Matches With Regular Expression: `J.*` (meaning: must start with `J`)

Conditions #1 and #3 satisfied:

||| Input (1)
```json
"John"
```
||| Output #1 (1):
```json
"John"
```
||| Output #2 (1):
```json
null
```
||| Output #3 (1):
```json
"John"
```
|||

Conditions #2 and #3 satisfied:

||| Input (1)
```json
"Jim"
```
||| Output #1 (1):
```json
null
```
||| Output #2 (1):
```json
"Jim"
```
||| Output #3 (1):
```json
"Jim"
```
|||

Condition #3 satisfied:

||| Input (1)
```json
"Jane"
```
||| Output #1 (1):
```json
null
```
||| Output #2 (1):
```json
null
```
||| Output #3 (1):
```json
"Jane"
```
|||

## Generate Value

The generate value transformation creates new values that can be used within
the whole transformation process. This may be useful for fallback/default values.

This transformer supports exactly one input and output.

The following generation strategy are available:

### Static Text

This strategy returns the configured value at the output.

#### Example

* Value: `John Smith`

||| Input
```
(anything)
```
||| Output
```json
"John Smith"
```
|||

### Random Number

This strategy returns a random number. The lowest and highest number are also
possible returned values.

#### Example

* Lowest Number: 0
* Highest Number: 10

||| Input
```
(anything)
```
||| Output
```json
3
```
|||

### Number Range

This strategy returns a list with all the numbers between the lowest and highest
number (including both values).

#### Example

* Lowest Number: 0
* Highest Number: 10

||| Input
```
(anything)
```
||| Output
```json
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
```
|||

## Join

The join transformer can be used for merging multiple branches into a single
branch. This is e.g. helpful after a [fork](#fork) or when the same data is
present in different fields. Furthermore the join is also helpful when working
with text lists to create a single, text out of it.

The join supports a dynamic number of outputs and by default one input for each
output. It is possible to change how many inputs are expected by modifying the
input per outputs configuration.

The join strategy will define how the data is processed.

### First Non-Empty Value

This will return the first non-empty value from the inputs with the same name
and return it as the output.

#### Example

* 2 Outputs
* Inputs Per Output: 2

||| Input #1 (1)
```json
"John"
```
||| Input #1 (2)
```json
null
```
||| Input #2 (1)
```json
"J."
```
||| Input #2 (2)
```json
"Smith"
```
||| Output (1)
```json
"John"
```
||| Output (2)
```json
"Smith"
```
|||

### Concatenate Texts

This will concatenate the texts with the same input name into the corresponding
output, optionally separated by the provided separator. Empty values will be
ignored.

If an input contains a string list, then each element will be concatenated.

#### Example with String Inputs

* 2 Outputs
* Inputs Per Output: 2

||| Input #1 (1)
```json
"John"
```
||| Input #1 (2)
```json
null
```
||| Input #2 (1)
```json
"J."
```
||| Input #2 (2)
```json
"Smith"
```
||| Output (1)
```json
"John,J."
```
||| Output (2)
```json
"Smith"
```
|||

#### Example with String Lists

* 1 Output
* Inputs Per Output: 1

||| Input
```json
["John", "Jim", null, "Jane"]
```
||| Output
```json
"John,Jim,Jane"
```
|||

### Merge Into Array

This will merge the values with the same input name into a list for the
corresponding output. Empty values (`null`) will be ignored.

* 2 Outputs
* Inputs Per Output: 2

||| Input #1 (1)
```json
"John"
```
||| Input #1 (2)
```json
null
```
||| Input #2 (1)
```json
"J."
```
||| Input #2 (2)
```json
"Smith"
```
||| Output (1)
```json
["John", "J."]
```
||| Output (2)
```json
["Smith"]
```
|||

## Make Date

The make date transformer will construct a date from different inputs. This
is an easy way to combine multiple date related fields into a single
text representation.

Examples of valid values for the full input, where `2023` represents the year,
`4` represents the month and `5` represents the day:

* 2023-04-05
* 2023-4-5
* 2023-04-5
* 2023-04
* 2023

The other inputs must be either a number or a text that can be interpreted as a
number.

The output will combine different inputs, but prioritize the value from the full
input.

The full output will always be in the `2023-04-05` format. Missing values will
be filled with 0, unless the validate option is enabled. The other outputs will
be strings representing the corresponding part from the full output, e.g. `04`

If the validate option is enabled and the year, month or day is empty, then an
empty value will be returned on all outputs.

This transformer supports the following inputs and outputs: full, year, month
and day.

#### Example with Full Input Priority

* Validate: enabled

||| Input (full)
```json
"2023-04"
```
||| Input (year)
```json
null
```
||| Input (month)
```json
"12"
```
||| Input (day)
```json
5
```
||| Output (full)
```json
"2023-04-05"
```
||| Output (year)
```json
"2023"
```
||| Output (month)
```json
"04"
```
||| Output (day)
```json
"05"
```
|||

#### Examples with Partial Date

* Validate: enabled

||| Input (full)
```json
"2023-04"
```
||| Input (year)
```json
null
```
||| Input (month)
```json
null
```
||| Input (day)
```json
null
```
||| Output (full)
```json
null
```
||| Output (year)
```json
null
```
||| Output (month)
```json
null
```
||| Output (day)
```json
null
```
|||

* Validate: disabled

||| Input (full)
```json
"2023-04"
```
||| Input (year)
```json
null
```
||| Input (month)
```json
null
```
||| Input (day)
```json
null
```
||| Output (full)
```json
"2023-04-00"
```
||| Output (year)
```json
"2023"
```
||| Output (month)
```json
"04"
```
||| Output (day)
```json
"00"
```
|||

#### Examples with Invalid Date

* Validate: enabled

||| Input (full)
```json
null
```
||| Input (year)
```json
2023
```
||| Input (month)
```json
2
```
||| Input (day)
```json
31
```
||| Output (full)
```json
null
```
||| Output (year)
```json
null
```
||| Output (month)
```json
null
```
||| Output (day)
```json
null
```
|||

* Validate: disabled

||| Input (full)
```json
null
```
||| Input (year)
```json
2023
```
||| Input (month)
```json
2
```
||| Input (day)
```json
31
```
||| Output (full)
```json
"2023-02-31"
```
||| Output (year)
```json
"2023"
```
||| Output (month)
```json
"02"
```
||| Output (day)
```json
"31"
```
|||

## Normalize Address

The normalize address transformer normalizes a free-form street / address line
into a human-readable canonical form using country specific
[reference lists](reference-lists.md). As part of that it detects and extracts
the house number, and it can independently normalize the city and the postal
code.

This transformer supports up to five inputs (`street`, `countryCode`,
`houseNumber`, `city`, `postalCode`) and four outputs (`normalizedStreet`,
`normalizedHouseNumber`, `normalizedCity`, `normalizedPostalCode`). The
`countryCode` input is required and must be an ISO 3166-1 Alpha-2 code; it
governs all country specific rules.

The street is normalized in several steps: optional preprocessing (regex
compound splitting, e.g. `Hauptstr.` → `Haupt str.`), house number extraction,
tokenization, token mapping (e.g. `str` → `Straße`), particle aware
capitalization (particles such as `den` / `der` are kept lowercase) and optional
postprocessing. House numbers are recognized either by explicit markers (`Nr.`,
`No.`, `#`, `nº`, `civico`, …) or by position (leading for anglophone / French
formats, trailing for most European ones) and keep ranges and suffixes
(`12-14`, `19c` → `19C`). The city is canonicalized via the city reference list,
and the postal code is reformatted using built-in per-country rules (no
reference list). All reference lists are optional; if a country has no matching
rows the pipeline degrades to a tokenize + capitalize cleanup. Block-addressing
countries (e.g. `jp`, `kr`, `cn`) skip house number extraction.

Configuration:

* Preprocessing Reference List (optional)
* Token Mapping Reference List (optional)
* Particles Reference List (optional)
* Postprocessing Reference List (optional)
* City Reference List (optional)
* Remove Structurally Invalid Postal Codes: disabled or enabled

#### Example

||| Input (street)
```json
"Hauptstr. 19c"
```
||| Input (countryCode)
```json
"de"
```
||| Output (normalizedStreet)
```json
"Hauptstraße"
```
||| Output (normalizedHouseNumber)
```json
"19C"
```
|||

#### Example with City and Postal Code

||| Input (street)
```json
"1234 Main Rd"
```
||| Input (countryCode)
```json
"us"
```
||| Input (city)
```json
"NYC"
```
||| Input (postalCode)
```json
"221621010"
```
||| Output (normalizedStreet)
```json
"Main Road"
```
||| Output (normalizedHouseNumber)
```json
"1234"
```
||| Output (normalizedCity)
```json
"New York"
```
||| Output (normalizedPostalCode)
```json
"22162-1010"
```
|||

## Normalize Company

The normalize company transformer splits a company name into the name itself
and its legal form. Version 2 also reports what else it recognized in the name.

Two versions exist. **Version 2** is driven by country specific
[reference lists](reference-lists.md) and is what new configurations should
use. **Version 1** is the legacy implementation: it is frozen, still supported,
and is what a transformer runs when it does not declare a version. Nothing
changes for an existing configuration unless you opt in.

### Normalize Company (version 2)

Version 2 resolves the legal form from reference lists instead of compiled-in
data, so the forms Tilores recognizes can be extended without waiting for a
release. Detection is scoped per country: the `country` input decides which
jurisdiction's forms apply, which is what keeps `Est` a legal form in the Gulf
states and an ordinary word in France.

It supports two inputs (`default`, `country`) and six outputs:

| Output | Contains |
|--------|----------|
| `companyName` | the company name with the legal form removed. This is the value to match on |
| `legalForm` | the canonical legal form (`gmbh`, `ltd`, `kk`), the same value for every spelling of it |
| `other` | text removed from the name that is not the legal form |
| `groupFunction` | the word marking the company's role inside a group (holding, management, property, …) |
| `location` | a location recognized inside the name (`Siemens AG Zweigniederlassung Leipzig` → `leipzig`) |
| `elfCode` | the ISO 20275 Entity Legal Form code, for delivery. Never use it as a matching value |

The name is processed in several steps: optional preprocessing (regex rewrites
that move a leading legal form to the end, or separate a form written without a
space), legal form detection and removal, group function detection, branch and
location detection, and optional postprocessing. The lists that shape
the company name are all required; the ones feeding the `groupFunction`,
`location` and `elfCode` outputs are optional, and leaving one out simply keeps
that output empty.

Configuration. Every list that shapes the company name is required: wiring only
some of them produces a name that is quietly worse than the same configuration
wiring all of them, so the transformer refuses to load instead. The optional
lists are exactly those feeding a reporting output; without one, that output
stays empty and the company name is unaffected.

* Legal Form Detection Reference List (required)
* Legal Form Alias Reference List (required)
* Preprocessing Reference List (required)
* Postprocessing Reference List (required)
* Company Token Reference List (required)
* Generic Token Reference List (required)
* Root Separator Reference List (required)
* Root Glue Word Reference List (required)
* Group Function Detection Reference List (optional)
* Group Function Alias Reference List (optional)
* Branch Designator Reference List (optional)
* Location City Reference List (optional)
* Location Country Reference List (optional)
* Location City Token Reference List (optional)
* ISO 20275 ELF Code Reference List (optional)
* Advanced Legal Form Detection: enabled (default) or disabled
* Strip Group Function From Root: disabled (default) or enabled
* Strip Branch Location From Root: disabled (default) or enabled

!!!warning Advanced Legal Form Detection defaults differently in the two versions
It is **disabled** by default in version 1 and **enabled** by default in
version 2. A configuration that never set it explicitly will therefore start
removing legal forms found in the middle of a name when it moves to version 2,
which changes the resulting company name. Set it explicitly when migrating if
that is not wanted.
!!!

#### Example

||| Input
```json
"Tilo Tech GmbH"
```
||| Input (country)
```json
"DE"
```
||| Output (companyName)
```json
"tilo tech"
```
||| Output (legalForm)
```json
"gmbh"
```
|||

#### Example with Every Spelling of One Form

Each of these reaches the same `legalForm`, which is the point of the alias
list: two records of one company written differently produce the same value.

||| Input
```json
"Tilo Tech Gesellschaft mit beschränkter Haftung"
```
||| Input (country)
```json
"DE"
```
||| Output (companyName)
```json
"tilo tech"
```
||| Output (legalForm)
```json
"gmbh"
```
|||

#### Example with a Group Function

The group function is reported but left in the name by default: within one
group it is often the only thing telling two companies apart, so removing it
would merge them.

||| Input
```json
"Tilo Tech Holding GmbH"
```
||| Input (country)
```json
"DE"
```
||| Output (companyName)
```json
"tilo tech holding"
```
||| Output (legalForm)
```json
"gmbh"
```
||| Output (groupFunction)
```json
"holding"
```
|||

#### Example with a Language Without Word Spacing

Chinese, Japanese and Korean names do not separate words, so the legal form is
written directly against the company name. Preprocessing inserts the boundary
and the form is then detected as usual.

||| Input
```json
"ソニー株式会社"
```
||| Input (country)
```json
"JP"
```
||| Output (companyName)
```json
"ソニー"
```
||| Output (legalForm)
```json
"kk"
```
|||

### Normalize Company (version 1, legacy)

!!!
Version 1 is frozen: it receives no new behaviour. It remains the default for
any transformer that does not declare a version, so existing configurations
keep working unchanged. Use version 2 for new configurations.
!!!

The normalize company transformer will split the provided text into a company
name and its legal form if present.

This transformer supports exactly one input and two outputs.

#### Examples with Simple Legal Form Detection

* Advanced Legal Form Detection: disabled

||| Input
```json
"Tilo Tech GmbH"
```
||| Output (companyName)
```json
"Tilo Tech"
```
||| Output (legalForm)
```json
"GmbH"
```
||| Output (other)
```json
null
```
|||

* Advanced Legal Form Detection: disabled

||| Input
```json
"Example LLC New York"
```
||| Output (companyName)
```json
"Example LLC New York"
```
||| Output (legalForm)
```json
null
```
||| Output (other)
```json
null
```
|||

#### Examples with Advanced Legal Form Detection

* Advanced Legal Form Detection: enabled

||| Input
```json
"Tilo Tech GmbH"
```
||| Output (companyName)
```json
"Tilo Tech"
```
||| Output (legalForm)
```json
"gmbh"
```
||| Output (other)
```json
null
```
|||

* Advanced Legal Form Detection: enabled

||| Input
```json
"Example LLC New York"
```
||| Output (companyName)
```json
"Example"
```
||| Output (legalForm)
```json
"llc"
```
||| Output (other)
```json
"New York"
```
|||

If the `country` input will provide a valid ISO 3166 Alpha-2 country code and
advanced legal form detection is enabled, Tilores will try to resolve country
specific legal form aliases.

* Advanced Legal Form Detection: enabled

||| Input 
```json
"Tilo Tech Gesellschaft mit beschränkter Haftung"
```
||| Input (country)
```json
"DE"
```
||| Output (companyName)
```json
"Tilo Tech"
```
||| Output (legalForm)
```json
"gmbh"
```
||| Output (other)
```json
null
```
|||

## Normalize Country

The normalize country transformer normalizes a free-form country value into a
lowercase ISO 3166-1 Alpha-2 country code (e.g. `de`, `us`). It resolves country
names, ISO Alpha-2 / Alpha-3 codes, abbreviations and common misspellings
against a user-provided [reference list](reference-lists.md).

This transformer supports one required input (`country`) and two optional inputs
(`city`, `street`), and produces a single output (`countryCode`).

Resolution follows a fallback cascade: the `country` value is looked up in the
country name reference list first; if that fails and the `city` input is
connected, the city reference list is consulted; if that also fails and the
`street` input is connected, street specific tokens (e.g. `strasse` → `de`) are
matched; finally the configured fallback country is used. If nothing matches and
no fallback is set, the output is `null`. Lookups are accent- and
case-insensitive.

Configuration:

* Fallback Country (ISO 3166-1 alpha-2)
* Country Name Reference List
* City Reference List (optional)
* Street Token Reference List (optional)

#### Example

||| Input (country)
```json
"finnland"
```
||| Output (countryCode)
```json
"fi"
```
|||

#### Example with City Fallback

* Fallback Country: "us"

||| Input (country)
```json
"unknown"
```
||| Input (city)
```json
"München"
```
||| Output (countryCode)
```json
"de"
```
|||

||| Input (country)
```json
"unknown"
```
||| Input (city)
```json
"Little Vil"
```
||| Output (countryCode)
```json
"us"
```
|||

## Normalize Email

The normalize email transformer will ensure a common structure for email
addresses and provides additional outputs for parts of an email address such as
the local part.

This transformer supports exactly one input and two outputs.

#### Examples

* Remove Plus Addressing: disabled or enabled

||| Input
```json
foo @example.com
```
||| Output (email)
```json
foo@example.com
```
||| Output (local)
```json
foo
```
||| Output (domain)
```json
example.com
```
||| Output (sld)
```json
example
```
||| Output (tld)
```json
com
```
||| Output (first)
```json
null
```
||| Output (last)
```json
null
```
|||

* Remove Plus Addressing: disabled

||| Input
```json
foo+123@example.com
```
||| Output (email)
```json
foo+123@example.com
```
||| Output (local)
```json
foo+123
```
||| Output (domain)
```json
example.com
```
||| Output (sld)
```json
example
```
||| Output (tld)
```json
com
```
||| Output (first)
```json
null
```
||| Output (last)
```json
null
```
|||

* Remove Plus Addressing: enabled

||| Input
```json
foo+123@example.com
```
||| Output (email)
```json
foo@example.com
```
||| Output (local)
```json
foo
```
||| Output (domain)
```json
example.com
```
||| Output (sld)
```json
example
```
||| Output (tld)
```json
com
```
||| Output (first)
```json
null
```
||| Output (last)
```json
null
```
|||

* Remove Plus Addressing: disabled or enabled

||| Input
```json
john.smith@example.com
```
||| Output (email)
```json
john.smith@example.com
```
||| Output (local)
```json
john.smith
```
||| Output (domain)
```json
example.com
```
||| Output (sld)
```json
example
```
||| Output (tld)
```json
com
```
||| Output (first)
```json
john
```
||| Output (last)
```json
smith
```
|||

Note: name recognition in emails only works with the following patterns:

* `<first>.<last>@<domain>`
* `<first>_<last>@<domain>`
* `<first>.<last>+<plus tag>@<domain>`
* `<first>_<last>+<plus tag>@<domain>`

## Normalize Name

The normalize name transformer structures and optionally transliterates the
provided personal name based on the provided language.

This transformer supports one required input (`default`) and one optional input
(`language`). It produces multiple outputs representing structured parts of the
name.

#### Supported Outputs

* `fullName`
* `title`
* `academicTitle`
* `nobilityTitle`
* `firstInitial`
* `firstName`
* `secondName`
* `thirdName`
* `lastName`
* `suffix`

If `language` is provided and recognized, language-specific normalization and
transliteration are applied. Otherwise, the transformer defaults to English.

---

#### Examples in English

* Input language: unknown

||| Input
```json
"Prof. Dr. John Michael Doe Jr."
```
||| Output (fullName)
```json
"Prof. Dr. John Michael Doe Jr."
```
||| Output (title)
```json
null
```
||| Output (academicTitle)
```json
"dr prof"
```
||| Output (nobilityTitle)
```json
null
```
||| Output (firstInitial)
```json
"j"
```
||| Output (firstName)
```json
"john"
```
||| Output (secondName)
```json
"michael"
```
||| Output (thirdName)
```json
null
```
||| Output (lastName)
```json
"doe"
```
||| Output (suffix)
```json
"jr."
```
|||

---

#### Examples with Transliteration

* Input language: `ara` (Arabic, ISO 639-2 code)

||| Input
```json
"السيد محمد عبدالله ناصر سلمان"
```
||| Output (fullName)
```json
"السيد محمد عبدالله ناصر سلمان"
```
||| Output (title)
```json
"mr"
```
||| Output (firstName)
```json
"mohammad"
```
||| Output (secondName)
```json
"abdullah"
```
||| Output (thirdName)
```json
"nasser"
```
||| Output (lastName)
```json
"salman"
```
||| Output (suffix)
```json
null
```
|||

---

#### Partial Outputs

* Input language: `ARA` (case-insensitive ISO 639-2 code)

||| Input
```json
"السيد محمد سلمان"
```
||| Output (fullName)
```json
"السيد محمد سلمان"
```
||| Output (title)
```json
"mr"
```
||| Output (firstName)
```json
"mohammad"
```
||| Output (secondName)
```json
null
```
||| Output (thirdName)
```json
null
```
||| Output (lastName)
```json
"salman"
```
||| Output (suffix)
```json
null
```
|||

---

#### Invalid or Unsupported Inputs

* Non-string inputs will result in all outputs being `null`.

||| Input
```json
123
```
||| Output (fullName)
```json
null
```
||| Output (firstName)
```json
null
```
||| Output (lastName)
```json
null
```
|||

* Numeric text is treated as a string but produces no structured parts.

||| Input
```json
"123"
```
||| Output (fullName)
```json
"123"
```
||| Output (firstName)
```json
null
```
||| Output (lastName)
```json
null
```
|||

---

#### Language Input and ISO 639-2 Codes

The optional `language` input allows the transformer to apply
language-specific parsing and transliteration rules.

* The transformer accepts **ISO 639-2 three-letter codes** (e.g. `eng` for English, `ara` for Arabic, `deu` for German).
* Codes are case-insensitive (`ara` and `ARA` are treated the same).
* If the provided code is not recognized, the transformer falls back to English.
* When recognized, transliteration of non-Latin scripts is applied to enable
  consistent normalization (e.g., Arabic names → Latin characters).
* Supported languages:
  * Arabic `ara`
  * English `eng`

## Normalize Phone Number

The normalize phone number transformer will normalize the provided phone number
and split it into a local number and a country code. And also outputs
the E.164 formatted version.

This transformer supports exactly two inputs and three outputs.

The default country is used to provide context information for better number
recognition. You can either provide a record specific value using the
[extract transformer](#extract-1), or provide a static value using the
[generate value transformer](#generate-value). Different country name formats
will be accepted, but it is recommended to provide a
[ISO 3166-1](https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes) 
Alpha-2 or Alpha-3 code.

The presence of a country code in the phone number (typically represented by a
`+` sign, will overwrite the default country). If the country cannot be assumed
and no default country was provided, then the phone number cannot be normalized.

#### Example

||| Input (number)
```json
"+49650-253-0000"
```
||| Input (defaultCountry)
```json
"US"
```
||| Output (nationalNumber)
```json
"6502530000"
```
||| Output (countryCode)
```json
"49"
```
||| Output (e164)
```json
"+496502530000"
```
|||

In this example, the default country is ignored, because the `+49` in the input
number clearly indicates a german phone number.

## Output

The output must be the final step for each transformation branch. Only outputs
can be used from [matchers](matcher.md). Each output must have a unique label to
identify it later in the matchers.

### Persisted Output
Setting the option `targetFieldPath` to a non empty value (e.g. `transformed.name`) will cause the transformation
output to be stored into the provided field path which must also be available in the schema.