Book contents
- Frontmatter
- Contents
- Preface
- Part I Background
- Part II Applications, tools, and tasks
- Interlude — Good practices for scientific computing
- Part III Fundamentals
- Chapter 21 Networks demand network thinking: the friendship paradox
- Chapter 22 Network models
- Chapter 23 Statistical models and inference
- Chapter 24 Uncertainty quantification and error analysis
- Chapter 25 Ghost in the matrix: spectral methods for networks
- Chapter 26 Embedding and machine learning
- Chapter 27 Big data and scalability
- Conclusion
- Bibliography
- Index
Chapter 27 - Big data and scalability
from Part III - Fundamentals
Published online by Cambridge University Press: 06 June 2024
- Frontmatter
- Contents
- Preface
- Part I Background
- Part II Applications, tools, and tasks
- Interlude — Good practices for scientific computing
- Part III Fundamentals
- Chapter 21 Networks demand network thinking: the friendship paradox
- Chapter 22 Network models
- Chapter 23 Statistical models and inference
- Chapter 24 Uncertainty quantification and error analysis
- Chapter 25 Ghost in the matrix: spectral methods for networks
- Chapter 26 Embedding and machine learning
- Chapter 27 Big data and scalability
- Conclusion
- Bibliography
- Index
Summary
Networks can get big. Really big. Examples include web crawls, online social networks, and knowledge graphs. Networks from these domains can have billions of nodes and hundreds of billions of edges. Systems biology is yet another area where networks will continue to grow. As sequencing methods continue to advance, more networks and larger, denser networks will need to be analyzed. This chapter discusses some of the challenges you face and solutions you can try when scaling up to massive networks. These range from implementation details to new algorithms and strategies to reduce the burden of such big data. Various tools, such as graph databases, probabilistic data structures, and local algorithms, are at our disposal, especially if we can accept sampling effects and uncertainty.
Keywords
- Type
- Chapter
- Information
- Working with Network DataA Data Science Perspective, pp. 447 - 470Publisher: Cambridge University PressPrint publication year: 2024