> For the complete documentation index, see [llms.txt](https://docs.metarank.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.metarank.ai/development/changelog.md).

# Changelog

In a human-readable format. For a technical changelog for robots, see [github releases page](https://github.com/metarank/metarank/releases). Check our [blog](https://blog.metarank.ai) for more detailed updates.

## 0.7.9

* expose redis click-through store TTL to config

## 0.7.8

* a bigfix release: slash/semicolon in key/value, kinesis retries

## 0.7.7

* a bugfix release: race condition in cache invalidation, booster native memleak

## 0.7.6

* [API Warmup support](/reference/deployment-overview/warmup.md)
* Rate feature now can be scoped to [ranking.field + item](/reference/overview/feature-extractors/counters.md#grouping-by-ranking-field)
* You can now specify [which eval metrics](/reference/overview/supported-ranking-models.md#lambdamart) should be computed on training.
* Proper handling of [GCP Memorystore Redis](/reference/overview/persistence.md#redis-support-limitations)

## 0.7.5

* [Unbiased LTR support](/reference/overview/supported-ranking-models.md#xgboost-and-lightgbm-backend-options)
* [Train/test splitting strategy support](/reference/overview/supported-ranking-models.md#traintest-splitting-strategies)

## 0.7.4

* support for rocksdb-backed file storage

## 0.7.3

* a bugfix release

## 0.7.2

* Support for kv-granular Redis TTLs
* Support HF tokenizers for biencoders: now you can run a multi-lingual E5 model in Metarank!

## 0.7.1

* `/inference`: [Inference API](/reference/api.md#inference-with-llms) to expose bi- and cross-encoders.

## 0.7.0

* Support for [LLM embedding-based content recommendations](/reference/overview/recommendations/semantic.md)
* `field_match` support for [BM25](/reference/overview/feature-extractors/text.md#bm25-score)
* `field_match` support for [LLM bi-encoders](/reference/overview/feature-extractors/text.md#llm-bi-encoders)
* `field_match` support for [LLM cross-encoders](/reference/overview/feature-extractors/text.md#llm-cross-encoders)
* Relevance judgments can now also [be explicit](/reference/event-schema.md#ranking-event)

## 0.6.4

* a minor bugfix release

## 0.6.3

* [diversity](/reference/overview/feature-extractors/diversity.md) feature extractor
* [scoped rate](/reference/overview/feature-extractors/counters.md#field-scoped-rates) feature
* fixed an important bug with dataset preparation (symptom: NDCG reported by the booster was higher than NDCG computed after the training) - prediction quality should go up a lot.

## 0.6.2

* print NDCG before and after reranking
* print statistics for mem usage after training

## 0.6.1

* fix for crash when using file-based clickthrough store

## 0.6.0

Upgrading: note that redis state format has a non backwards compatible change, so you need to reimport the data when upgrading.

* [recommendations](/reference/overview/recommendations.md) support for similar and trending items models.
* Local caching for state, the import should be 2x-3x faster.

## 0.5.16

* [expose](/reference/deployment-overview/docker.md#memory) `JAVA_OPTS` env variable to control JVM heap size.
* fix bug for a case when there is a click on a non-existent item.

## 0.5.15

* `cache.maxSize` for redis now disables client-side caching altogether. Makes Metarank compatible with GCP Memstore Redis.
* fixed mem leak in clickthrough joining buffer.
* lower mem allocation pressure in interacted\_with feature.

## 0.5.14

* interacted\_with feature now supports string\[] fields
* fixed a notorious bug with local-file click-through store.

## 0.5.13

* XGBoost LambdaMART impl now supports categorical encoding.
* [Event selector support](/reference/overview/supported-ranking-models.md#event-selectors) when serving multiple models.
* [Clickthrough storage configuration](/reference/overview.md#training) for storing clickthrough data.

## 0.5.12

* Redis AUTH override with [env vars](/reference/overview/persistence.md#redis-persistence)
* Prometheus `/metrics` [endpoint](/reference/deployment-overview/prometheus.md)
* Per-item [ranking fields](/reference/event-schema.md#ranking-event)

## 0.5.11

* linux/aarch64 support, so docker images on Mac M1 are OK.
* \--split option for CLI with [multiple splitting strategies](/reference/cli.md#training-the-model)
* a ton of bugfixes

## 0.5.10

* Redis [TLS support](/reference/overview/persistence.md#tls-support)
* Redis [timeout configuration](/reference/overview.md#persistence)

## 0.5.9

* [`interacted_with`](/reference/overview/feature-extractors/user-session.md#interacted-with) feature now has much less overhead in Redis, and supports multiple fields in a single visitor profile.
* [click-through events now can be stored in a file](/reference/overview.md#training), and not inside Redis, also reducing the overall costs of running Metarank
* it is now possible to [export lightgbm/xgboost-compatible](/reference/cli.md#dataset-export) datasets for further hyper-parameter optimization.

## 0.5.8

* bugfix: add explicit sync on /feedback api call
* bugfix: config decoding error for field\_match over terms
* bugfix: version detect within docker was broken
* bugfix: issue with improper iface being bound in docker

## 0.5.7

* [Request latency benchmark](/introduction/performance.md), with ballpark estimations useful for resource planning.
* [Vector feature extractor](/reference/overview/feature-extractors/scalar.md#vector-extractor) with reducer and autofeature support.
* [Redis AUTH support](/reference/overview/persistence.md#redis-persistence) which is common on managed Redis setups.

## 0.5.6

* [Binary state serialization format](/reference/overview/persistence.md#state-encoding-formats), which is 2x faster and 4x more compact than JSON
* [Multi-arch docker images](/reference/deployment-overview/docker.md), so metarank can now be run natively on Mac M1/M2.
* [Kubernetes Helm chart](/reference/deployment-overview/kubernetes.md) and an official guide on how to do production deployment on k8s.
* [Training dataset export](/reference/cli.md#training-the-model) for further hyper-parameter tuning.

## 0.5.5

Notable features:

* [Rate normalization](/reference/overview/feature-extractors/counters.md#rate-normalization) support, so having 1 click over 2 impressions is not resulting in a 50% CTR anymore.
* [Position de-biasing](/reference/overview/feature-extractors/relevancy.md#position) based on a dynamic position feature.

## 0.5.4

Most notable improvements:

* [AutoML style feature engineering](/how-to/autofeature.md) based on an existing dataset

## 0.5.3

Highlights of this release are:

* added [historical data sorting](https://github.com/metarank/metarank/blob/stabledoc/doc/cli/README.md#historical-data-sorting) sub-command
* added [core.clickthrough.maxSessionLength and core.clickthrough.maxParallelSessions](/reference/overview.md#click-through-joining) parameters to improve memory consumption

## 0.5.2.

Highlights of this release are:

* added [analytics](/reference/overview.md#anonymous-usage-analytics) and [error tracking](/reference/overview.md#error-logging)

## 0.5.1

Highlights of this release are:

* Flink is rermoved. As a result only `memory` and `redis` [persistance](/reference/overview/persistence.md) modes are supported now.
* [Configuration file](https://github.com/metarank/metarank/blob/stabledoc/doc/configuration/sample-config.yml) now has updated structure and is not compatible with previous format.
* CLI is updated, most of the options are moved to [configuration](/reference/overview.md).
  * We have updated the [`validate`](/reference/cli.md#validation) mode of the CLI, so you can use it to validate your data and configuration.

## 0.4.0

Highlights of this release are:

* Kubernetes support: now it's possible to have a production-ready metarank deployment in K8S
* Kinesis source: on par with Kafka and Pulsar
* Custom connector options pass-through

### Kunernetes support

Metarank is a multi-stage and multi-component system, and now it's possible to get it deployed in minutes inside a Kubernetes cluster:

* Inference API is just a regular [Deployment](https://github.com/metarank/metarank/blob/master/deploy/kubernetes/deployment.yaml)
* Bootstrap, Upload and Update jobs can be run both locally (to simplify things up for small datasets) and inside the cluster in a distributed mode.
* Job mabagement is done with [flink-kubernetes-operator](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/)

See [this doc section](https://docs.metarank.ai/deployment/kubernetes) for details.

### Kinesis source

[Kinesis Streams](https://aws.amazon.com/kinesis/data-streams/) can also be used as an event source. It still has a couple of drawbacks compared with Kafka/Pulsar, for example, due to max 7 day data retention it cannot be effectively used as a permanent storage of historical events. But it's still possible to pair it with a AWS Firehose writing events to S3, so realtime events are coming from Kinesis, and historical events are offloaded to S3.

Check out [the corresponding part of docs](https://docs.metarank.ai/introduction/configuration/data-sources#aws-kinesis-streams) for details and examples.

### Custom connector options pass-through

As we're using Flink's connector library for pulling data from Kafka/Kinesis/Pulsar, there is a ton of custom options you can tune for each connector. It's impossible to expose all of them directly, so now in connector config there is a free-form `options` section, allowing to set any supported option for the underlying connector.

Example to force Kinesis use EFO consumer:

```yaml
type: kinesis
topic: events
region: us-east-1
offset: latest
options: 
  flink.stream.recordpublisher: EFO 
  flink.stream.efo.consumername: metarank 
```

See [this doc section](https://docs.metarank.ai/introduction/03_configuration/data-sources#common-options-for-bootstrapping-connectors) for details.