Class ParallelFileInputStream

java.lang.Object
java.io.InputStream
de.calamanari.pk.util.pfis.ParallelFileInputStream
All Implemented Interfaces:
java.io.Closeable, java.lang.AutoCloseable

public final class ParallelFileInputStream
extends java.io.InputStream
Parallel File Input Stream - input stream using a concurrent reader thread.
This class is useful for reading large files especially if processing takes longer than reading. In this case the reader thread can do a parallel read ahead while the main thread processes the data from the last chunk and so on.
Because the behavior of java NIO can differ from OS to OS the caller can choose which buffer type to be used (see BufferType).
Note:
  • For small files this technique is not recommended and may result in slow processing.
  • Instances of ParallelFileInputStream MUST NOT be accessed concurrently.
Author:
Karl Eilebrecht
  • Field Summary

    Fields
    Modifier and Type Field Description
    static int DEFAULT_MAX_BUFFER_SIZE
    Maximum buffer size, the default is 52428800 bytes.
  • Method Summary

    Modifier and Type Method Description
    int available()  
    void close()  
    static ParallelFileInputStream createInputStream​(java.io.File file)
    Creates a new Memory Mapped Parallel File Input Stream on the given file using default buffer size ( DEFAULT_MAX_BUFFER_SIZE).
    static ParallelFileInputStream createInputStream​(java.io.File file, int maxBufferSize)
    Creates a new Memory Mapped Parallel File Input Stream on the given file using the specified buffer size.
    static ParallelFileInputStream createInputStream​(java.io.File file, int maxBufferSize, BufferType bufferType)
    Creates a new Memory Mapped Parallel File Input Stream on the given file using the specified buffer size.
    Note: A small buffer may lead to extremely bad performance!
    static ParallelFileInputStream createInputStream​(java.io.File file, BufferType bufferType)
    Creates a new Memory Mapped Parallel File Input Stream on the given file using default buffer size ( DEFAULT_MAX_BUFFER_SIZE).
    static ParallelFileInputStream createInputStream​(java.lang.String fileName)
    Creates a new Memory Mapped Parallel File Input Stream on the given file using default buffer size ( DEFAULT_MAX_BUFFER_SIZE).
    static ParallelFileInputStream createInputStream​(java.lang.String fileName, int maxBufferSize)
    Creates a new Memory Mapped Parallel File Input Stream on the given file using the specified buffer size.
    static ParallelFileInputStream createInputStream​(java.lang.String fileName, int maxBufferSize, BufferType bufferType)
    Creates a new Memory Mapped Parallel File Input Stream on the given file using the specified buffer size.
    Note: A small buffer may lead to extremely bad performance!
    static ParallelFileInputStream createInputStream​(java.lang.String fileName, BufferType bufferType)
    Creates a new Memory Mapped Parallel File Input Stream on the given file using default buffer size ( DEFAULT_MAX_BUFFER_SIZE).
    long getNumberOfBytesDelivered()
    Returns the number of bytes that has been returned to the caller during read()-operations until now.
    This method does ignore skip() and mark(), so at the end of file the value may be smaller than the file size or even greater.
    void mark​(int readlimit)  
    boolean markSupported()
    Mark is supported.
    int read()  
    int read​(byte[] b)  
    int read​(byte[] b, int off, int len)  
    void repositionFileStream​(long positionAbs)
    Repositions the pointer on the stream so that the next buffer access will read at the given absolute position
    If this given position is greater than file size, the pointer will be set at the end of file.
    void reset()  
    long skip​(long bytesToBeSkipped)  

    Methods inherited from class java.io.InputStream

    nullInputStream, readAllBytes, readNBytes, readNBytes, skipNBytes, transferTo

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • Field Details

  • Method Details

    • createInputStream

      public static ParallelFileInputStream createInputStream​(java.io.File file, int maxBufferSize, BufferType bufferType) throws java.io.IOException
      Creates a new Memory Mapped Parallel File Input Stream on the given file using the specified buffer size.
      Note: A small buffer may lead to extremely bad performance!
      Parameters:
      file - underlying source file
      maxBufferSize - maximum size a buffer may have (0 or less means DEFAULT_MAX_BUFFER_SIZE)
      bufferType - the type of buffer to be used, see BufferType
      Returns:
      input stream
      Throws:
      java.io.IOException - on file access error
    • createInputStream

      public static ParallelFileInputStream createInputStream​(java.io.File file, int maxBufferSize) throws java.io.IOException
      Creates a new Memory Mapped Parallel File Input Stream on the given file using the specified buffer size. Uses memory mapped buffer (see BufferType).
      Note: A small buffer may lead to extremely bad performance!
      Parameters:
      file - underlying source file
      maxBufferSize - maximum size a buffer may have (0 or less means DEFAULT_MAX_BUFFER_SIZE)
      Returns:
      input stream
      Throws:
      java.io.IOException - on file access error
    • createInputStream

      public static ParallelFileInputStream createInputStream​(java.io.File file) throws java.io.IOException
      Creates a new Memory Mapped Parallel File Input Stream on the given file using default buffer size ( DEFAULT_MAX_BUFFER_SIZE). Uses memory mapped buffer (see BufferType).
      Parameters:
      file - underlying source file
      Returns:
      input stream
      Throws:
      java.io.IOException - on file access error
    • createInputStream

      public static ParallelFileInputStream createInputStream​(java.io.File file, BufferType bufferType) throws java.io.IOException
      Creates a new Memory Mapped Parallel File Input Stream on the given file using default buffer size ( DEFAULT_MAX_BUFFER_SIZE).
      Parameters:
      file - underlying source file
      bufferType - the type of buffer to be used, see BufferType
      Returns:
      input stream
      Throws:
      java.io.IOException - on file access error
    • createInputStream

      public static ParallelFileInputStream createInputStream​(java.lang.String fileName, int maxBufferSize, BufferType bufferType) throws java.io.IOException
      Creates a new Memory Mapped Parallel File Input Stream on the given file using the specified buffer size.
      Note: A small buffer may lead to extremely bad performance!
      Parameters:
      fileName - underlying source file's name
      maxBufferSize - maximum size a buffer may have
      bufferType - the type of buffer to be used, see BufferType
      Returns:
      input stream
      Throws:
      java.io.IOException - on file access error
    • createInputStream

      public static ParallelFileInputStream createInputStream​(java.lang.String fileName, int maxBufferSize) throws java.io.IOException
      Creates a new Memory Mapped Parallel File Input Stream on the given file using the specified buffer size. Uses memory mapped buffer (see BufferType).
      Note: A small buffer may lead to extremely bad performance!
      Parameters:
      fileName - underlying source file's name
      maxBufferSize - maximum size a buffer may have
      Returns:
      input stream
      Throws:
      java.io.IOException - on file access error
    • createInputStream

      public static ParallelFileInputStream createInputStream​(java.lang.String fileName) throws java.io.IOException
      Creates a new Memory Mapped Parallel File Input Stream on the given file using default buffer size ( DEFAULT_MAX_BUFFER_SIZE). Uses memory mapped buffer (see BufferType).
      Parameters:
      fileName - underlying source file's name
      Returns:
      input stream
      Throws:
      java.io.IOException - on file access error
    • createInputStream

      public static ParallelFileInputStream createInputStream​(java.lang.String fileName, BufferType bufferType) throws java.io.IOException
      Creates a new Memory Mapped Parallel File Input Stream on the given file using default buffer size ( DEFAULT_MAX_BUFFER_SIZE).
      Parameters:
      fileName - underlying source file's name
      bufferType - the type of buffer to be used, see BufferType
      Returns:
      input stream
      Throws:
      java.io.IOException - on file access error
    • getNumberOfBytesDelivered

      Returns the number of bytes that has been returned to the caller during read()-operations until now.
      This method does ignore skip() and mark(), so at the end of file the value may be smaller than the file size or even greater.
      Returns:
      number of bytes delivered to the caller
    • read

      public int read​(byte[] b) throws java.io.IOException
      Overrides:
      read in class java.io.InputStream
      Throws:
      java.io.IOException
    • read

      public int read​(byte[] b, int off, int len) throws java.io.IOException
      Overrides:
      read in class java.io.InputStream
      Throws:
      java.io.IOException
    • repositionFileStream

      public void repositionFileStream​(long positionAbs) throws java.io.IOException
      Repositions the pointer on the stream so that the next buffer access will read at the given absolute position
      If this given position is greater than file size, the pointer will be set at the end of file.
      Parameters:
      positionAbs - absolute position in file (if greater than file size, we will reposition at EOF)
      Throws:
      java.io.IOException - on file access error
    • skip

      public long skip​(long bytesToBeSkipped) throws java.io.IOException
      Overrides:
      skip in class java.io.InputStream
      Throws:
      java.io.IOException
    • available

      public int available() throws java.io.IOException
      Overrides:
      available in class java.io.InputStream
      Throws:
      java.io.IOException
    • close

      public void close() throws java.io.IOException
      Specified by:
      close in interface java.lang.AutoCloseable
      Specified by:
      close in interface java.io.Closeable
      Overrides:
      close in class java.io.InputStream
      Throws:
      java.io.IOException
    • mark

      public void mark​(int readlimit)
      Overrides:
      mark in class java.io.InputStream
    • reset

      public void reset() throws java.io.IOException
      Overrides:
      reset in class java.io.InputStream
      Throws:
      java.io.IOException
    • markSupported

      public boolean markSupported()
      Mark is supported.

      Overrides:
      markSupported in class java.io.InputStream
      Returns:
      true
    • read

      public int read() throws java.io.IOException
      Specified by:
      read in class java.io.InputStream
      Throws:
      java.io.IOException