Engine settings

You can customize table engine settings to better match your needs.

The engine options are: engine_partition_key, engine_sorting_key, engine_primary_key, engine_sampling_key, engine_ttl and engine_settings.

If engine_partition_key is empty or not passed as a parameter, the underlying Data Source doesn't have any partition unless there's a Date column. In that case the Data Source is partitioned by year. If you want to create a Data Source with no partitions, send engine_partition_key=tuple().

Don't change the following settings unless you are familiar with them and understand their impact. If you're unsure, contact Tinybird at support@tinybird.co or in the Community Slack.

The supported engine settings are:

  • index_granularity
  • merge_with_ttl_timeout
  • ttl_only_drop_parts
  • min_bytes_for_wide_part
  • min_rows_for_wide_part
  • min_age_to_force_merge_seconds
  • min_age_to_force_merge_on_partition_only
  • enable_replacing_merge_with_cleanup_for_min_age_to_force_merge

index_granularity

Maximum number of data rows between the marks of an index.

Default value: 8192.

merge_with_ttl_timeout

Minimum delay in seconds before repeating a merge with delete TTL.

Default value: 14400 seconds (4 hours).

ttl_only_drop_parts

Controls whether data parts are fully dropped in MergeTree tables when all rows in that part have expired according to their TTL settings.

When ttl_only_drop_parts is disabled (by default), only the rows that have expired based on their TTL settings are removed.

When ttl_only_drop_parts is enabled, the entire part is dropped if all rows in that part have expired according to their TTL settings.

Default value: 0.

min_bytes_for_wide_part

Minimum number of bytes/rows in a data part that can be stored in Wide format. You can set one, both or none of these settings.

min_age_to_force_merge_seconds

Minimum age of a part in seconds before it becomes eligible for forced merging. Useful for ReplacingMergeTree tables where you want to ensure old parts are merged and deduplicated within a predictable time window.

Default value: 0 (disabled).

min_age_to_force_merge_on_partition_only

When enabled, min_age_to_force_merge_seconds only applies to whole partitions rather than individual parts.

Default value: 0 (disabled).

enable_replacing_merge_with_cleanup_for_min_age_to_force_merge

When enabled alongside min_age_to_force_merge_seconds on a ReplacingMergeTree, forced merges also clean up obsolete rows (duplicates that have been replaced). Without this setting, forced merges compact parts but don't remove old versions.

Default value: 0 (disabled).

Example combining these settings in a .datasource file:

ENGINE "ReplacingMergeTree"
ENGINE_SORTING_KEY "id"
ENGINE_SETTINGS "min_age_to_force_merge_seconds=10,enable_replacing_merge_with_cleanup_for_min_age_to_force_merge=1"
Updated