Hostname: page-component-586b7cd67f-r5fsc Total loading time: 0 Render date: 2024-11-22T17:14:39.691Z Has data issue: false hasContentIssue false
  • English
  • Français

EXTREME VALUE ANALYSIS WITHOUT THE LARGEST VALUES: WHAT CAN BE DONE?

Published online by Cambridge University Press:  30 January 2019

Jingjing Zou Open the ORCID record for Jingjing Zou [Opens in a new window] ,
Richard A. Davis  and
Gennady Samorodnitsky
Jingjing Zou
Affiliation:
Department of Statistics Columbia University, New York, NY, USA E-mail: [email protected]; [email protected]
Richard A. Davis
Affiliation:
Department of Statistics Columbia University, New York, NY, USA E-mail: [email protected]; [email protected]
Gennady Samorodnitsky
Affiliation:
School of Operations Research and Information Engineering, Cornell University, Ithaca, NY, USA E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, we are concerned with the analysis of heavy-tailed data when a portion of the extreme values is unavailable. This research was motivated by an analysis of the degree distributions in a large social network. The degree distributions of such networks tend to have power law behavior in the tails. We focus on the Hill estimator, which plays a starring role in heavy-tailed modeling. The Hill estimator for these data exhibited a smooth and increasing “sample path” as a function of the number of upper order statistics used in constructing the estimator. This behavior became more apparent as we artificially removed more of the upper order statistics. Building on this observation we introduce a new version of the Hill estimator. It is a function of the number of the upper order statistics used in the estimation, but also depends on the number of unavailable extreme values. We establish functional convergence of the normalized Hill estimator to a Gaussian process. An estimation procedure is developed based on the limit theory to estimate the number of missing extremes and extreme value parameters including the tail index and the bias of Hill's estimator. We illustrate how this approach works in both simulations and real data examples.

Keywords

functional convergenceheavy-tailed distributionshill estimatormissing extremes
Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 34 , Issue 2 , April 2020 , pp. 200 - 220
Copyright
Copyright © Cambridge University Press 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Aban, I.B., Meerschaert, M.M., & Panorska, A.K. (2006). Parameter Estimation for the Truncated Pareto Distribution. Journal of the American Statistical Association 101(473): 270–277.CrossRefGoogle Scholar
2.Beirlant, J., Alves, I.F., & Gomes, I. (2016a). Tail fitting for truncated and non-truncated Pareto-type distributions. Extremes 19(3): 429–462.CrossRefGoogle Scholar
3.Beirlant, J., Alves, I.F., & Reynkens, T. (2016b) Fitting tails affected by truncation. arXiv.org, page arXiv:1606.02090.CrossRefGoogle Scholar
4.Benchaira, S., Meraghni, D., & Necir, A. (2016). Tail product-limit process for truncated data with application to extreme value index estimation. Extremes 19(2): 219–251.CrossRefGoogle Scholar
5.Burroughs, S.M. & Tebbens, S.F. (2001a). Upper-truncated power law distributions. Fractals 09(02): 209–222.CrossRefGoogle Scholar
6.Burroughs, S.M. & Tebbens, S.F. (2001b). Upper-truncated Power Laws in Natural Systems. Pure and Applied Geophysics 158(4): 741–757.CrossRefGoogle Scholar
7.Burroughs, S.M. & Tebbens, S.F. (2002). The upper-truncated power law applied to earthquake cumulative frequency-magnitude distributions: evidence for a time-independent scaling parameter. Bulletin of the Seismological Society of America 92(8): 2983–2993.CrossRefGoogle Scholar
8.Clark, D.R. (2013) A note on the upper-truncated Pareto distribution. Casualty Actuarial Society E-Forum.Google Scholar
9.Davis, R. & Resnick, S. (1984). Tail estimates motivated by extreme value theory. The Annals of Statistics 12(4): 1467–1487.CrossRefGoogle Scholar
10.de Haan, L. & Ferreira, A. (2006). Extreme value theory. Springer Series in Operations Research and Financial Engineering. New York: Springer.CrossRefGoogle Scholar
11.Drees, H. (1998). On smooth statistical tail functionals. Scandinavian Journal of Statistics 25(1): 187–210.CrossRefGoogle Scholar
12.Drees, H., de Haan, L., & Resnick, S. (2000). How to make a Hill plot. The Annals of Statistics 28(1): 254–274.Google Scholar
13.Embrechts, P., Klüppelberg, C., & Mikosch, T (1997). Modelling extremal events, volume 33 of Applications of Mathematics (New York). Berlin, Berlin, Heidelberg: Springer-Verlag.CrossRefGoogle Scholar
14.Hill, B.M. (1975). A simple general approach to inference about the tail of a distribution. The Annals of Statistics 3(5): 1163–1174.CrossRefGoogle Scholar
15.Kaufmann, E. & Reiss, R.D. (1998). Approximation of the Hill estimator process. Statistics & Probability Letters 39(4): 347–354.CrossRefGoogle Scholar
16.Komlós, J., Major, P., & Tusnády, G. (1975). An approximation of partial sums of independent RV's and the sample DF. I. Z. Wahrscheinlichkeitstheorie und Verwandte Gebiete 32(1–2): 111–131.CrossRefGoogle Scholar
17.Komlós, J., Major, P., & Tusnády, G. (1976). An approximation of partial sums of independent RV's, and the sample DF. II. Z. Wahrscheinlichkeitstheorie und Verwandte Gebiete 34(1): 33–58.CrossRefGoogle Scholar
18.Newman, M. (2010). Networks: An Introduction. New York: Oxford University Press.CrossRefGoogle Scholar
19.Reiss, R.D. (1989). Approximate distributions of order statistics. Springer Series in Statistics. New York, New York, NY: Springer-Verlag.CrossRefGoogle Scholar
Supplementary material: File

Zou et al. supplementary material

Zou et al. supplementary material 1

Download Zou et al. supplementary material(File)
File 1.3 KB
Supplementary material: File

Zou et al. supplementary material

Zou et al. supplementary material 2

Download Zou et al. supplementary material(File)
File 13.5 KB
Supplementary material: File

Zou et al. supplementary material

Zou et al. supplementary material 3

Download Zou et al. supplementary material(File)
File 39.3 KB