SCIENCE@home

Delay-Adaptive Learning in Generalized Linear Contextual Bandits

J. Blanchet,Renyuan Xu,Zhengyuan Zhou

Published 2020 in Mathematics of Operations Research

ABSTRACT

In this paper, we consider online learning in generalized linear contextual bandits where rewards are not immediately observed. Instead, rewards are available to the decision maker only after some delay, which is unknown and stochastic. Such delayed feedback occurs in several active learning settings, including product recommendation, personalized medical treatment selection, bidding in first-price auctions, and bond trading in over-the-counter markets. We study the performance of two well-known algorithms adapted to this delayed setting: one based on upper confidence bounds and the other based on Thompson sampling. We describe modifications on how these two algorithms should be adapted to handle delays and give regret characterizations for both algorithms. To the best of our knowledge, our regret bounds provide the first theoretical characterizations in generalized linear contextual bandits with large delays. Our results contribute to the broad landscape of contextual bandits literature by establishing that both algorithms can be made to be robust to delays, thereby helping clarify and reaffirm the empirical success of these two algorithms, which are widely deployed in modern recommendation engines. Funding: This work was supported by the National Science Foundation [Grants 2118199, 1915967, and CCF-2106508], the Air Force Office of Scientific Research [Award FA9550-20-1-0397], a Digital Twin research grant from Bain & Company, and a faculty research grant from New York University’s Center for Global Economy and Business.

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-49 of 49 references · Page 1 of 1

CITED BY

Showing 1-8 of 8 citing papers · Page 1 of 1