Séminaire Probabilités et Statistiques
Frequency and cardinality recovery from sketched data: a novel approach bridging Bayesian and frequentist views
11
jan. 2024
logo_team
Intervenant : Stefano Favaro
Institution : University of Torino and Collegio Carlo Alberto
Heure : 15h45 - 16h45
Lieu : 3L15

We study how to recover the frequency of a symbol in a large discrete data set, using only a (lossy) compressed representation, or sketch, of those data obtained via random hashing. This is a classical problem at the crossroad of computer science and information theory, with various algorithms available, such as the count-min sketch. However, these algorithms often assume that the data are fixed, leading to overly conservative and potentially inaccurate estimates when dealing with randomly sampled data. In this talk, we consider the sketched data as a random sample from an unknown distribution, and then we introduce novel estimators that improve upon existing approaches. Our method combines Bayesian nonparametric and classical (frequentist) perspectives, addressing their unique limitations to provide a principled and practical solution. Additionally, we extend our method to address the related but distinct problem of cardinality recovery, which consists of estimating the total number of distinct objects in the data set.

Voir tous les événements