Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Warmup

When running Metarank in production, you may hit the cold-start problem:

  • after a restart (due to redeployment/autoscaling) Metarank pod starts cold with empty caches,

  • JVM also has not yet compiled most of the code,

  • due to this, reranking latency of first N requests is too high.

To handle this Metarank (starting from v0.7.6) supports explicit API warmup:

  • while training, sample N random but real reranking requests and persist them in store

  • on API startup, replay this random traffic sample

  • when replay done, bring up the API - so k8s readiness probe will be successful.

Configuring warmup

API warmup is only supported for LambdaMART models and is configured per-model:

models:
  my-super-model:
    type: lambdamart
    warmup:
      sampledRequests: 100 # how many requests sample during training
      duration: 5s # how long to replay the traffic during warmup
    # ...

Warmup is disabled by default, and you need to retrain your model if you trained it in Metarank prior to 0.7.6.

After the warmup is enabled, you will see the following log output when starting the API:

PreviousCustom loggingNextIntegrations

Last updated

Was this helpful?