Hostname: page-component-77f85d65b8-v2srd Total loading time: 0 Render date: 2026-04-20T08:54:42.001Z Has data issue: false hasContentIssue false
  • English
  • Français

Enhanced upper confidence limits via randomized tests in random sampling without replacement

Part of: Parametric inference Sampling theory, sample surveys

Published online by Cambridge University Press:  09 October 2025

Zihao Li Open the ORCID record for Zihao Li [Opens in a new window] ,
Huangjun Zhu Open the ORCID record for Huangjun Zhu [Opens in a new window]  and
Masahito Hayashi Open the ORCID record for Masahito Hayashi [Opens in a new window]
Zihao Li*
Affiliation:
Fudan University
Huangjun Zhu*
Affiliation:
Fudan University
Masahito Hayashi*
Affiliation:
The Chinese University of Hong Kong
*
*Postal address: Department of Physics and State Key Laboratory of Surface Physics, Institute for Nanoelectronic Devices and Quantum Computing, Center for Field Theory and Particle Physics, Fudan University, Shanghai 200433, China.
*Postal address: Department of Physics and State Key Laboratory of Surface Physics, Institute for Nanoelectronic Devices and Quantum Computing, Center for Field Theory and Particle Physics, Fudan University, Shanghai 200433, China.
****Postal address: School of Data Science, The Chinese University of Hong Kong, Shenzhen, Longgang District, Shenzhen 518172, China; International Quantum Academy (SIQA), Futian District, Shenzhen 518048, China; and Graduate School of Mathematics, Nagoya University, Chikusa-ku, Nagoya 464-8602, Japan. Email: hmasahito@cuhk.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper we study one-sided hypothesis testing under random sampling without replacement, which frequently appears in the cryptographic problem setting, including the verification of measurement-based quantum computation. Suppose that $n+1$ binary random variables $X_1,\ldots, X_{n+1}$ follow a permutation invariant distribution and n binary random variables $X_1,\ldots, X_{n}$ are observed. Then, we propose randomized tests with a randomization parameter for the expectation of the $(n+1)$th random variable $X_{n+1}$ under a given significance level $\delta>0$. Our randomized tests significantly improve the upper confidence limit over deterministic tests. Our problem setting commonly appears in machine learning in addition to cryptographic scenarios by considering adversarial examples. Such studies are essential for expanding the applicable area of statistics. Although this paper addresses only binary random variables, a similar significant improvement by randomized tests can be expected for general non-binary random variables.

Keywords

Randomized testrandom sampling without replacementadversarial scenarioadversarial example

MSC classification

Primary: 62F05: Asymptotic properties of tests 62D05: Sampling theory, sample surveys
Secondary: 62F03: Hypothesis testing

Information

Type
Original Article
Information
Advances in Applied Probability , First View , pp. 1 - 51
Check for updates
Copyright
© The Author(s), 2025. Published by Cambridge University Press on behalf of Applied Probability Trust

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Ai, J., Kuželka, O. and Wang, Y. (2023). Hoeffding–Serfling inequality for U-statistics without replacement. J. Theoret. Probab. 36, 390–408.10.1007/s10959-022-01169-xCrossRefGoogle Scholar
Bardenet, R. and Maillard, O.-A. (2015). Concentration inequalities for sampling without replacement. Bernoulli 21, 1361–1385.10.3150/14-BEJ605CrossRefGoogle Scholar
Bartroff, J., Lorden, G. and Wang, L. (2023). Optimal and fast confidence intervals for hypergeometric successes. Am. Stat. 77, 151–159.10.1080/00031305.2022.2128421CrossRefGoogle Scholar
Becchetti, L., Colesanti, U. M., Marchetti-Spaccamela, A. and Vitaletti, A. (2011). Recommending items in pervasive scenarios: Models and experimental analysis. Knowl. Inf. Syst. 28, 555–578.10.1007/s10115-010-0338-4CrossRefGoogle Scholar
Ben-Hamou, A., Peres, Y. and Salez, J. (2018). Weighted sampling without replacement. Braz. J. Probab. Stat. 32, 657–669.10.1214/17-BJPS359CrossRefGoogle Scholar
Berry, A. C. (1941). The accuracy of the Gaussian approximation to the sum of independent variates. Trans. Am. Math. Soc. 49, 122–136.10.1090/S0002-9947-1941-0003498-3CrossRefGoogle Scholar
Bradley, J. R. and Blossom, A. P. (2023). The generation of visually credible adversarial examples with genetic algorithms. ACM Trans. Evol. Learn. Optim. 3, 1–44.10.1145/3582276CrossRefGoogle Scholar
Casiraghi, G. and Nanumyan, V. (2021). Configuration models as an urn problem. Sci. Rep. 11, 13416.10.1038/s41598-021-92519-yCrossRefGoogle Scholar
Chalé, M., Cox, B., Weir, J. and Bastian, N. D. (2023). Constrained optimization based adversarial example generation for transfer attacks in network intrusion detection systems. Optim. Lett. 18, 2169–2188.10.1007/s11590-023-02007-7CrossRefGoogle Scholar
Chiu, C.-H. (2023). A species richness estimator for sample-based incidence data sampled without replacement. Methods Ecol. Evol. 14, 2189–2510.10.1111/2041-210X.14146CrossRefGoogle Scholar
Choi, D. (2023). Estimating the prevalance of peer effects and other spillovers. arXiv: 2309.03969.Google Scholar
Covey, R. and Buonamano, L. (2023). Survey design and estimating equations when combining big data with probability samples. arXiv: 2307.11999.Google Scholar
Dembo, A. and Zeitouni, O. (2009). Large Deviations Techniques and Applications , Vol. 38. Springer Science & Business Media.Google Scholar
Dey, A. and Chaudhuri, P. (2023). A comparison of estimators of mean and its functions in finite populations. arXiv: 2305.15019.Google Scholar
Dong, C., Liu, L. and Shang, J. (2022). Label noise in adversarial training: A novel perspective to study robust overfitting. In Advances in Neural Information Processing Systems 35, pp. 17556–17567.Google Scholar
Feng, D., Du, Y., Gomes, C. P. and Selman, B. (2023). Weighted sampling without replacement for deep top-k classification. Proceedings of the 40th International Conference on Machine Learning, eds. A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato and J. Scarlett, Vol. 202 of Proceedings of Machine Learning Research. PMLR, pp. 9910–9920.Google Scholar
Gordon, R. D. (1941). Values of Mills’ ratio of area to bounding ordinate and of the normal probability integral for large values of the argument. Ann. Math. Stat. 12, 364–366.10.1214/aoms/1177731721CrossRefGoogle Scholar
Greene, E. and Wellner, J. A. (2016). Finite sampling inequalities: An application to two-sample Kolmogorov–Smirnov statistics. Stochastic Process. Appl. 126, 3701–3715.10.1016/j.spa.2016.04.020CrossRefGoogle ScholarPubMed
Greene, E. and Wellner, J. A. (2017). Exponential bounds for the hypergeometric distribution. Bernoulli 23, 1911–1950.10.3150/15-BEJ800CrossRefGoogle ScholarPubMed
Gubri, M., Cordy, M., Papadakis, M., Traon, Y. L. and Sen, K. (2022). LGV: Boosting adversarial example transferability from large geometric vicinity. In Computer Vision – ECCV 2022, eds. S. Avidan, G. Brostow, M. Cissé, G. M. Farinella and T. Hassner. Springer Nature Switzerland, Cham. pp. 603–618.10.1007/978-3-031-19772-7_35CrossRefGoogle Scholar
Hall, P. (1992). Principles of edgeworth expansion. In The Bootstrap and Edgeworth Expansion. Springer, pp. 39–81.10.1007/978-1-4612-4384-7_2CrossRefGoogle Scholar
Hayashi, M. and Morimae, T. (2015). Verifiable measurement-only blind quantum computing with stabilizer testing. Phys. Rev. Lett. 115, 220502.10.1103/PhysRevLett.115.220502CrossRefGoogle ScholarPubMed
Hayashi, M. and Tsurumaru, T. (2012). Concise and tight security analysis of the Bennett–Brassard 1984 protocol with finite key lengths. New J. Phys. 14, 093014.10.1088/1367-2630/14/9/093014CrossRefGoogle Scholar
Hodara, P. and Reynaud-Bouret, P. (2019). Exponential inequality for chaos based on sampling without replacement. Stat. Probab. Lett. 146, 65–69.10.1016/j.spl.2018.11.003CrossRefGoogle Scholar
Kaas, R. and Buhrman, J. M. (1980). Mean, median and mode in binomial distributions. Stat. Neerl. 34, 13–18.10.1111/j.1467-9574.1980.tb00681.xCrossRefGoogle Scholar
Karagiannidis, G. K. and Lioumpas, A. S. (2007). An improved approximation for the Gaussian Q-function. IEEE Commun. Lett. 11, 644–646.10.1109/LCOMM.2007.070470CrossRefGoogle Scholar
Lehmann, E. L. and Romano, J. P. (2005). Testing Statistical Hypotheses, Vol. 3. Springer Texts in Statistics.Google Scholar
Levine, A. and Feizi, S. (2020). Robustness certificates for sparse adversarial attacks by randomized ablation. Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, pp. 4585–4593.10.1609/aaai.v34i04.5888CrossRefGoogle Scholar
Li, Z., Zhu, H. and Hayashi, M. (2023). Robust and efficient verification of graph states in blind measurement-based quantum computation. npj Quantum Inf. 9, 115.10.1038/s41534-023-00783-9CrossRefGoogle Scholar
Lodato, M. A., Woodworth, M. B., Lee, S., Evrony, G. D., Mehta, B. K., Karger, A., Lee, S., Chittenden, T. W., D’Gama, A. M., Cai, X., Luquette, L. J., Lee, E., Park, P. J. and Walsh, C. A. (2015). Somatic mutation in single human neurons tracks developmental and transcriptional history. Science 350, 94–98.10.1126/science.aab1785CrossRefGoogle ScholarPubMed
Morimae, T., Takeuchi, Y. and Hayashi, M. (2017). Verification of hypergraph states. Phys. Rev. A 96, 062321.10.1103/PhysRevA.96.062321CrossRefGoogle Scholar
Motoyama, H. (2024). Extended Glivenko-Cantelli theorem for simple random sampling without replacement from a finite population. Commun. Stat. Theory Methods 53, 5924–5934.10.1080/03610926.2023.2238233CrossRefGoogle Scholar
Narodytska, N. and Kasiviswanathan, S. (2017). Simple black-box adversarial attacks on deep neural networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1310–1318.10.1109/CVPRW.2017.172CrossRefGoogle Scholar
Nowakowski, S. (2021). Uniqueness of a median of a binomial distribution with rational probability. Adv. Math. Sci. J. 10, 1951.10.37418/amsj.10.4.9CrossRefGoogle Scholar
O’Neill, T. J. and Stern, S. E. (2012). Finite population corrections for the Kolmogorov-Smirnov tests. J. Nonparam. Stat. 24, 497–504.10.1080/10485252.2011.650169CrossRefGoogle Scholar
Ouimet, F. (2024). Deficiency bounds for the multivariate inverse hypergeometric distribution. Stat. Papers 65, 3959–3969.10.1007/s00362-023-01524-yCrossRefGoogle Scholar
Papoulis, A. (2002). Probability, Random Variables, and Stochastic Processes . McGraw-Hill Europe, New York, NY, USA.Google Scholar
Sambale, H. and Sinulis, A. (2022). Concentration inequalities on the multislice and for sampling without replacement. J. Theoret. Probab. 35, 2712–2737.10.1007/s10959-021-01139-9CrossRefGoogle Scholar
Samohyl, R. W. (2018). Acceptance sampling for attributes via hypothesis testing and the hypergeometric distribution. J. Ind. Eng. Int. 14, 395–414.10.1007/s40092-017-0231-9CrossRefGoogle Scholar
Serfling, R. J. (1974). Probability inequalities for the sum in sampling without replacement. Ann. Stat. 2, 39–48.10.1214/aos/1176342611CrossRefGoogle Scholar
Shamir, O. (2016). Without-replacement sampling for stochastic gradient methods. In Advances in Neural Information Processing Systems 29, pp. 46–54.Google Scholar
Simon, H. U. and Telle, J. A. (2024). MAP-and MLE-based teaching. J. Mach. Learn. Res. 25, 1–34.Google Scholar
Sutter, T. M., Manduchi, L., Ryser, A. and Vogt, E., J. (2022). Learning group importance using the differentiable hypergeometric distribution. arXiv: 2203.01629.Google Scholar
Tanash, I. M. and Riihonen, T. (2021). Improved coefficients for the Karagiannidis–Lioumpas approximations and bounds to the Gaussian Q-function. IEEE Commun. Lett. 25, 1468–1471.10.1109/LCOMM.2021.3052257CrossRefGoogle Scholar
Withers, C. S. and Nadarajah, S. (2023). Unbiased estimates for products of moments and cumulants for finite populations. Mathematics 11, 3720.10.3390/math11173720CrossRefGoogle Scholar
Zhu, H. and Hayashi, M. (2019). Efficient verification of pure quantum states in the adversarial scenario. Phys. Rev. Lett. 123, 260504.10.1103/PhysRevLett.123.260504CrossRefGoogle ScholarPubMed
Zhu, H. and Hayashi, M. (2019). General framework for verifying pure quantum states in the adversarial scenario. Phys. Rev. A 100, 062335.10.1103/PhysRevA.100.062335CrossRefGoogle Scholar
Zhu, H., Li, Z. and Hayashi, M. (2022). Nearly tight universal bounds for the binomial tail probabilities. arXiv: 2211.01688.Google Scholar