Performance Optimization for Managing Massive Numbers of Small Files in Distributed File Systems

No abstract is available for this paper.

• Publication year

  2015

• Venue

  IEEE Transactions on Parallel and Distributed Systems

• Publication date

  2015-12-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/TPDS.2014.2377720
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• MilkyWay-2 supercomputer: system and application

  2014cited by this paper
• The TH Express high performance interconnect networks

  2014cited by this paper
• Research and Improvement of the Hot Small File Storage Performance under HDFS

  2013cited by this paper
• The TokuFS Streaming File System

  2012cited by this paper
• BlueSky: a cloud-backed file system for the enterprise

  2012cited by this paper
• Content Delivery Networks

  2012cited by this paper
• Characteristics of backup workloads in production systems

  2012cited by this paper
• Metadata Performance Optimization in Distributed File System

  2012cited by this paper
• A File Is Not a File: Understanding the I/O Behavior of Apple Desktop Applications

  2011cited by this paper
• DepSky: dependable and secure storage in a cloud-of-clouds

  2011cited by this paper
• Ext4 file system performance analysis in linux environment

  2011cited by this paper
• The TianHe-1A Supercomputer: Its Hardware and Software

  2011cited by this paper
• A study of practical deduplication

  2011cited by this paper
• HBase: The Definitive Guide

  2011cited by this paper
• Finding a Needle in Haystack: Facebook's Photo Storage

  2010influential reference
• RACS: a case for cloud storage diversity

  2010cited by this paper
• THE optimization of HDFS based on small files

  2010cited by this paper
• The Hadoop Distributed File System

  2010influential reference
• MongoDB: The Definitive Guide

  2010cited by this paper
• Implementing WebGIS on Hadoop: A case study of improving small file I/O performance on HDFS

  2009cited by this paper
• Improving metadata management for small files in HDFS

  2009cited by this paper
• Dynamo: amazon's highly available key-value store

  2007cited by this paper
• A five-year study of file-system metadata

  2007cited by this paper
• Improving Small File Performance in Object-based Storage (CMU-PDL-06-104)

  2006cited by this paper
• Bigtable: a distributed storage system for structured data

  2006cited by this paper
• UNIX Filesystems: Evolution, Design and Impemenation

  2003cited by this paper
• Journaling with ReisersFS

  2001cited by this paper
• PVFS: A Parallel Virtual File System for Linux Clusters

  2000cited by this paper
• Embedded Inodes and Explicit Grouping: Exploiting Disk Bandwidth for Small Files

  1997cited by this paper
• Scalability in the XFS File System

  1996influential reference
• Analysis of file I/O traces in commercial computing environments

  1992influential reference
• Proceedings of the Fast 2002 Conference on File and Storage Technologies Gpfs: a Shared-disk File System for Large Computing Clusters

  1992cited by this paper

• Information-Theoretic Security Problem in Cluster Distributed Storage Systems: Regenerating Code Against Two General Types of Eavesdroppers

  2025cites this paper
• Correlation aware solution for merging small files based on similarity and clustering in HDFS

  2025cites this paper
• Unlocking the power of machine learning in big data: a scoping survey

  2025cites this paper
• Merging and cooperative caching based HDFS small files access performance optimization

  2024cites this paper
• A Lightweight Approach for Large CAD Models Based on Lazy Loading

  2023cites this paper
• FSbrain: An intelligent I/O performance tuning system

  2022cites this paper
• A survey of data element perspective: Application of artificial intelligence in health big data

  2022cites this paper
• Enhancing HDFS with a full-text search system for massive small files

  2021cites this paper
• Performance Evaluation of Merging Techniques for Handling Small Size Files in HDFS

  2021cites this paper
• Data replication schemes in cloud computing: a survey

  2021cites this paper
• A Novel Technique for Handling Small File Problem of HDFS: Hash Based Archive File (HBAF)

  2021cites this paper
• Addressing the Small Files Issue in Hadoop

  2021cites this paper
• Batch-file Operations to Optimize Massive Files Accessing

  2020cites this paper
• Optimization Scheme of Massive Meteorological Data Storage Based on OpenStack Swift

  2020cites this paper
• Data Management Techniques in Hadoop Framework for Handling Small Files: A Survey

  2020cites this paper
• Crawler Framework for Category Search Engine

  2020cites this paper
• A Small File Merging Strategy for Spatiotemporal Data in Smart Health

  2019cites this paper
• Impact of Small Files on Hadoop Performance: Literature Survey and Open Points

  2019cites this paper
• BFO: Batch-File Operations on Massive Files for Consistent Performance Improvement

  2019cites this paper
• Hadoop Perfect File: A fast access container for small files with direct in disc metadata access

  2019influential citation
• Distributed Personal Cloud Storage without Third Parties

  2019cites this paper
• Size Matters: Improving the Performance of Small Files in Hadoop

  2018cites this paper
• Scaling Distributed Hierarchical File Systems Using NewSQL Databases

  2018cites this paper
• Weight‐based strategy for an I/O‐intensive application at a cloud data center

  2018influential citation
• Performance analysis and framework optimization of open source cloud storage system

  2016cites this paper
• StepAhead: Rethinking Filesystem Namespace Translations

  2016cites this paper
• Hitchhike: An I/O Scheduler Enabling Writeback for Small Synchronous Writes

  2016cites this paper
• Critical Analysis of Solutions to Hadoop Small File Problem

  year unknowncites this paper
• Critical Analysis of Solutions to Hadoop Small File Problem

  year unknowncites this paper