# Release Notes

These are the latest changes for Tilores.

# Version 0

# v0.37.0

released on 2023-05-25

NEW Introduced a first version of edge insights, that provide statistics and aggregations on an entities edges and duplicates.

Affected Components:

tilores-cli v0.24.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
tilores-insights v0.3.0 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)

# v0.36.0

released on 2023-05-10

NEW Added the What-IF Machine, that enables you to see what would happen if certain records were not present in an entity.

Affected Components:

tilores-cli v0.23.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
tilores-core v0.24.0 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)
tilores-plugin-api v0.10.0 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)

# v0.35.0

released on 2023-04-27

NEW Added an experimental feature to compress the entity files in S3 in order to improve the performance for large entities. It can be turned on any time by setting the Lambda environment variable ENTITY_FILE_COMPRESSION to gzip for the assembly and disassemble functions. Activating this feature will compress the entity file on the next entity change.

Affected Components:

tilores-core v0.23.0 (upgrade by running tilores-cli deploy)

# v0.34.1

released on 2023-04-27

FIXED Fixed a potential memory race condition when using the jq-like syntax introduced in release v0.33.0.

FIXED Fixed an issue that was introduced due to the fix in release v0.34.0, causing execution plans not to be executed when an entity did not exist at all.

Affected Components:

tilores-core v0.22.1 (upgrade by running tilores-cli deploy)

# v0.34.0

released on 2023-04-26

NEW Added possibility to define the log level via the Lambda environment variable LOG_LEVEL, which can be one of the following values:

trace (currently unused)
debug
info (default)
warn
error
fatal (currently unused)
panic (currently unused)

NEW Added debug log messages to the assembly process.

FIXED Fixed an issue where an already processed execution plan was reprocessed, potentially leading to data loss if another record was added to the same entity in the meantime. This only occurred extremely seldom under very specific conditions (very high parallel load of lots of very similar data).

Affected Components:

tilores-core v0.22.0 (upgrade by running tilores-cli deploy)

# v0.33.0

released on 2023-04-20

NEW Added option to extract data from a record using a jq-like syntax. When enabling this feature in the extract transformer, the path .myfield.sub is equal to the default myfield.sub. Additionally it allows you to write more complex extractions like e.g. .myfield.sub | map(.prop). Please refer to the jq manual for more details.

CHANGED When the assemble recognizes a locked entity, it will now retry it internally instead of returning with an error and let the lambda service retry it at a later point in time. This will make it easier to differentiate actual errors from locked entities in the metrics.

FIXED Fixed some rare occurring errors in the tilores-cli while erasing the instance data.

Affected Components:

tilores-cli v0.22.1 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
tilores-core v0.21.0 (upgrade by running tilores-cli deploy)

# v0.32.0

released on 2023-04-14

NEW Added a new transformer for generating values for various use cases. The transformer allows to generate a new static text value, a random number or a sequential list of numbers.

CHANGED Upgraded internal dependencies.

Affected Components:

tilores-core v0.20.0 (upgrade by running tilores-cli deploy)

# v0.31.0

released on 2023-04-12

NEW Added a new option to the token matcher to include multiple tokens into the same index. This is helpful for rules where a single token does not have enough variants, resulting in relatively few often reused indexes, eventually leading to bad performance.

CHANGED Upgraded internal dependencies.

Affected Components:

tilores-core v0.19.0 (upgrade by running tilores-cli deploy)

# v0.30.0

released on 2023-04-11

NEW The custom scalar Time now accepts ISO 8601 format as an input, in addition to the previously supported RFC3339Nano format.

The output format when a field of the type Time is queried is always RFC3339Nano, regardless of the format it was originally added.

Affected Components:

tilores-cli v0.22.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
tilores-insights v0.2.0 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)

# v0.29.0

released on 2023-03-27

NEW It is now possible to query an entity by any of its known record IDs via the GraphQL API.

BREAKING Removed support for the fake commands (deploy, destroy, run) as announced in release v0.17.1.

You need at least go 1.20 or later to be able to upgrade to tilores-cli v0.20.0

Affected Components:

tilores-cli v0.20.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
go-plugin v0.1.1 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)
tilores-plugin-api v0.9.0 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)
tilores-core v0.18.0 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)

# v0.28.0

released on 2023-03-24

NEW It is now possible to disable indexing of some matchers of a rule to receive better performance and cost efficiency. This is especially helpful in situations where many similarity or geo matchers are used in a single rule, leading to a high amount of index entries.

Affected Components:

tilores-core v0.17.0 (upgrade by running tilores-cli deploy after upgrading the tilores-cli)

# v0.27.0

released on 2023-03-17

NEW Added the following meta data to a record, that can be queried using Athena:

submitTimestamp - timestamp for when a record was submitted via the API
assembleTimestamp - timestamp for when a record was assembled

Both timestamps are stored in the format RFC 3339.

Affected Components:

tilores-core v0.16.0 (upgrade by running tilores-cli deploy)
tilores-plugin-api v0.8.0 (upgrade by running tilores-cli deploy)

# v0.26.3

released on 2023-03-16

CHANGED Changed logging to include log levels and format them using JSON for easier usage with CloudWatch Log Insights.

Affected Components:

tilores-core v0.15.4 (upgrade by running tilores-cli deploy)

# v0.26.2

released on 2023-03-15

FIXED The transformer for extracting values now correctly returns null instead of an empty string when the extracted values was an empty string. This could have caused subsequent transformers not to behave correctly.

Affected Components:

tilores-core v0.15.3 (upgrade by running tilores-cli deploy)

# v0.26.1

released on 2023-03-14

FIXED The transformer for extracting values from a previous transformation step now correctly handles empty paths. That means for an empty path it provides the incoming data to the next transformer or resolver, but evaluates the "must exist" and "case sensitive" options.

Affected Components:

tilores-core v0.15.2 (upgrade by running tilores-cli deploy)

# v0.26.0

released on 2023-03-13

NEW Introduced record insights, that provide filtering, statistics and aggregations on entity records.

BREAKING The custom scalars Time, Map and Any are now always provided by Tilores. Customers who have them defined in their GraphQL schema files need to remove them.

Affected Components:

tilores-cli v0.20.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
tilores-insights v0.1.0 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)

# v0.25.1

released on 2023-02-28

CHANGED Upgraded internal dependencies.

Affected Components:

tilores-core v0.15.1 (upgrade by running tilores-cli deploy)

# v0.25.0

released on 2023-02-21

CHANGED Switched standard AWS Lambda execution policy to a more restrict custom policy. This prevents rare cases of CloudWatch log group (re)creation during destroying a deployment while still ingesting data. This could have lead to situations where a later deployment might fail as the soon-to-be created log groups are already present.

CHANGED Added additional outputs to the terraform module for easier customization. Please refer to the module documentation for the whole list of outputs.

Affected Components:

terraform-aws-tilores-core v0.12.0 (upgrade by running tilores-cli deploy)

# v0.24.0

released on 2023-02-02

CHANGED Upgraded internal dependencies to support golang v1.20.0.

Affected Components:

tilores-cli v0.19.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
internal dependencies (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)

# v0.23.1

released on 2022-12-02

FIXED Fixed an issue that was blocking the assemble process when submitting records in the same request with the same ID. These cases will now be detected and written into the dead-letter-queue.

Affected Components:

tilores-core v0.14.2 (upgrade by running tilores-cli deploy)

# v0.23.0

released on 2022-11-10

BREAKING Changed the way how the rules can be configured. This introduces some advanced configuration possibilities and enables us to provide more flexible matching and preprocessing in the future as well as better UI support for rule configuration.

Please be aware, that the documentation has not yet been updated everywhere. For a good example on how to configure the new rules we advise you to look at the example for the public walkthrough or simply contact us.

This is a breaking change. Using the old configuration format in combination with the new implementation will not work. However, the old configuration can be converted into the new schema and existing instances will continue to work in most cases. For those small subtle differences where the data is stored differently we provided a migration plan for affected customers.

Affected Components:

tilores-cli v0.18.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
tilores-core v0.13.0 (upgrade by running tilores-cli deploy)

# v0.22.1

released on 2022-08-26

FIXED Fixed an issue which caused the erase command to fail when using an external backend configuration and not having terraform initialized.

Affected Components:

tilores-cli v0.17.1 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.22.0

released on 2022-08-18

NEW The phonetic noDiacritOrSpecials now detects and decodes unicodes, such as \u00f5 and \\u00f5.

Affected Components:

tilores-core v0.12.0 (upgrade by running tilores-cli deploy)

# v0.21.0

released on 2022-08-01

NEW It is now possible to configure CORS for the API.

To add a CORS configuration edit your deployment/tilores/main.tf and add the following (or parts of it) to your tilores module:

cors_configuration = {
  allow_credentials = true
  allow_headers     = ["Content-Type"]
  allow_methods     = ["OPTIONS", "POST"]
  allow_origins     = ["https://example.com"]
  expose_headers    = ["X-Custom-Header"]
  max_age           = 300
}

Affected Components:

terraform-aws-tilores-core v0.10.0 (upgrade by running tilores-cli deploy)

# v0.20.0

released on 2022-07-29

BREAKING It is now possible to use the lambda authorizer introduced in v0.19.0 without creating custom scope resolvers.

Update 2022-08-01: This release was previously marked as NEW . However, due to using a Terraform feature that was introduced in Terraform version v1.2, this is unfortunately an unplanned breaking change and requires you to update Terraform accordingly. We would like to apologize for any inconvenience.

Affected Components:

tilores-cli v0.17.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
tilores-directive-oauth-scope v0.2.0 (upgrade by running tilores-cli upgrade and tilores-cli deploy after upgrading the tilores-cli)

# v0.19.0

released on 2022-07-28

NEW It is now possible to configure the API authorizer in Terraform core module and use a lambda authorizer.

NEW It is now possible to configure the default API stage in Terraform core module.

Affected Components:

terraform-aws-tilores-core v0.9.0 (upgrade by running tilores-cli deploy)

# v0.18.1

released on 2022-06-17

FIXED Fixed an issue which caused the destroy command to fail when using an external backend configuration in combination with a fresh copy from a SCM.

Affected Components:

tilores-cli v0.16.1 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.18.0

released on 2022-06-13

BREAKING Due to API changes the commands rules-simulate and rules-test will only work for v0.16.0 and later versions of tilores-cli.

You need at least go 1.18 or later to be able to upgrade to tilores-cli v0.16.0

Affected Components:

tilores-cli v0.16.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.17.1

released on 2022-06-02

CHANGED Marked the tilores-cli deploy fake-api and tilores-cli destroy fake-api commands as deprecated. Both commands will be removed in a future release.

Affected Components:

tilores-cli v0.15.1 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.17.0

released on 2022-05-25

NEW Introduced the tilores-cli plan command, which helps to check which AWS resources would be affected during a deployment.

Affected Components:

tilores-cli v0.15.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.16.1

released on 2022-05-20

FIXED Fixed that the CLI command erase unexpectedly exited under certain conditions.

Affected Components:

tilores-cli v0.14.1 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.16.0

released on 2022-05-17

NEW Introduced a new matcher token overlap, for recognizing whether the two compared fields have common text tokens.

NEW Introduced the company name and company legal form modifiers.

Affected Components:

tilores-core v0.11.0 (upgrade by running tilores-cli deploy)

# v0.15.0

released on 2022-04-04

NEW It is now possible to configure the Terraform Cognito module for Web-UIs using an OAuth code flow.

NEW It is now possible to retrieve the relevant values from the Terraform Cognito module to create a custom domain for the hosted Cognito UI.

Both changes only apply to users who use the default Terraform Cognito module during their deployment. The full list of parameters and return values can be found in the Terraform Registry.

Affected Components:

terraform-aws-tilores-cognito v0.4.0 (upgrade by running tilores-cli deploy)

# v0.14.0

released on 2022-04-01

BREAKING We wish it would be some kind of an april fools' joke, but it is not: we have renamed the *-entites table into the correct spelling *-entities.

CHANGED We upgraded the underlying terraform AWS provider to the latest version.

This release contains breaking changes. Affected customers were supported with a migration plan to prevent data loss. Non-production customers can easily upgrade by running:

tilores-cli destroy
tilores-cli deploy

Affected Components:

terraform-aws-tilores-core v0.8.0
terraform-aws-tilores-cognito v0.3.0
terraform-aws-tilores-fake-api v0.5.0

# v0.13.0

released on 2022-03-30

NEW It is now possible to get the API ID and the API stage during the deployment. This enables customers to create a base path mapping for their custom domains using terraform.

By default the attributes can be referenced as module.tilores.api_id and module.tilores.api_default_stage_id.

Affected Components:

terraform-aws-tilores-core v0.7.0 (upgrade by running tilores-cli deploy)

# v0.12.1

released on 2022-03-29

FIXED Fixed the versions of terraform submodules in the core module to prevent a possibly breaking change.

Affected Components:

terraform-aws-tilores-core v0.6.1 (upgrade by running tilores-cli deploy)

# v0.12.0

released on 2022-03-21

NEW Introduced a CloudWatch Tilores dashboard which shows the most relevant Tilores metrics.

Affected Components:

func-customer-metrics v0.2.0 (upgrade by running tilores-cli deploy)
terraform-aws-tilores-core v0.6.0 (upgrade by running tilores-cli deploy)

# v0.11.0

released on 2022-03-18

NEW Introduced the tilores-cli erase command, which helps mainly during development by erasing all data within Tilores without the need to redeploy.

Affected Components:

tilores-cli v0.14.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.10.0

released on 2022-03-11

BREAKING Fixed a bug that under certain conditions caused the API to no longer respond, resulting in temporary or permanent timeouts.

BREAKING Removed the flag --dispatcher-version from the tilores-cli init command. The version is now always fixed to prevent breaking the Tilores deployment in the future.

NEW Introduced the tilores-cli upgrade command, which helps upgrading Tilores to the latest version and resolves breaking changes automatically.

This release contains breaking changes. To apply these changes you must follow these steps.

Upgrade to the latest Tilores CLI:

go install github.com/tilotech/tilores-cli@latest

Run the upgrade command:

tilores-cli upgrade

Update your deployment:

tilores-cli deploy

Affected Components:

tilores-cli v0.13.0
tilores-core v0.10.0
tilores-plugin-api v0.7.0
tilores-plugin-fake-dispatcher v0.6.0
go-plugin v0.1.0

# v0.9.2

released on 2022-03-03

NEW It is now possible to change access token validity duration.

Affected Components:

terraform-aws-tilores-cognito v0.2.0 (upgrade by running tilores-cli deploy)

# v0.9.1

released on 2022-02-14

FIXED Avoid possibility of sending the same usage metrics more than once.

Affected Components:

terraform-aws-tilores-core v0.5.1 (upgrade by running tilores-cli deploy)

# v0.9.0

released on 2022-02-11

NEW Introduced a modifier for normalizing spaces.

NEW Introduced a modifier for removing spaces.

CHANGED Added modifier support for the following matchers: complete, exactXorEmpty, fieldEquality, similarity and token

Affected Components:

tilores-core v0.9.0 (upgrade by running tilores-cli deploy)

# v0.8.3

released on 2022-02-11

NEW Usage metrics are now sent to api.tilotech.io. A GraphQL introspection query can be done to check which usage metrics are sent.

CHANGED Terraform AWS provider is now fixed to major version 3 to avoid breaking changes.

Affected Components:

func-customer-metrics v0.1.1 (upgrade by running tilores-cli deploy)
terraform-aws-tilores-core v0.5.0 (upgrade by running tilores-cli deploy)
terraform-aws-tilores-cognito v0.1.2 (upgrade by running tilores-cli deploy)
terraform-aws-tilores-fake-api v0.4.1 (only if the fake api is deployed, upgrade by running tilores-cli deploy fake-api)

# v0.8.2

released on 2022-02-08

NEW Added US East (Ohio) us-east-2 to List of supported regions.

Affected Components:

no components are affected by this release

# v0.8.1

released on 2022-02-04

FIXED Fixed issue that sometimes lambda fails accessing kinesis due to a wrong region setting.

Affected Components:

tilores-core v0.8.1 (upgrade by running tilores-cli deploy)

# v0.8.0

released on 2022-02-02

NEW Introduced a new matcher for text similarity, that provides many standard similarity algorithms.

NEW You can now use the split function in field expressions.

FIXED Fixed deployment to regions other than eu-west-1. List of supported regions.

Affected Components:

tilores-core v0.8.0 (upgrade by running tilores-cli deploy)
tilores-cli v0.12.2 (upgrade by running go install github.com/tilotech/tilores-cli@latest)
terraform-aws-tilores-core v0.4.1 (upgrade by running tilores-cli deploy)

# v0.7.0

released on 2022-02-01

NEW Improved the field expressions used in matchers. You can now use complex expressions.

Affected Components:

tilores-core v0.7.0 (upgrade by running tilores-cli deploy)

# v0.6.1

released on 2022-01-31

CHANGED Added curl examples for mutations and queries in the documentation.

CHANGED Added warning, that usage from Windows is currently unsupported.

CHANGED Added git to the required deploy dependencies.

FIXED Fixed a bug which caused the tilores-cli deploy command to sometimes exit with an error message about missing dependencies - despite them being available.

Affected Components:

tilores-cli v0.12.1 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.6.0

released on 2022-01-28

NEW It is now possible to deploy multiple instances from the same Tilores project. This can be either in the same AWS account or in different ones, e.g. for production and development environments. Check out how it's done.

CHANGED Improved the output when running tilores-cli rules test to indicate if all tests are passing or no tests were provided.

Affected Components:

tilores-cli v0.12.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.5.0

released on 2022-01-26

NEW Introduced a new tilores-cli command, rules test.

Affected Components:

tilores-cli v0.11.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.4.0

released on 2022-01-25

NEW Introduced a new tilores-cli command, rules simulate.

Affected Components:

tilores-cli v0.10.0 (upgrade by running go install github.com/tilotech/tilores-cli@latest)

# v0.3.0

released on 2022-01-25

NEW Introduced the metaphone phonetic.

NEW Introduced the soundex phonetic.

Affected Components:

tilores-core v0.6.0 (upgrade by running tilores-cli deploy)

# v0.2.0

released on 2022-01-25

NEW Introduced a new matcher for geographical coordinates and distance matching.

Affected Components:

tilores-core v0.5.0 (upgrade by running tilores-cli deploy)

# v0.1.0

released on 2022-01-12

NEW This is the first official beta release of Tilores. We recommend you to go through the getting started guide to get an idea what Tilores is all about.

Affected Components:

tilores-core v0.4.0
tilores-cli v0.9.0
tilores-plugin-api v0.6.0
terraform-aws-tilores-core v0.4.0
terraform-aws-tilores-cognito v0.1.1
tilores-directive-oauth-scope v0.1.0
func-scavenger v0.2.0