IO

The io module manages input and output functions. It contains three main loader modules: loader, parallel_loader, and data_loader. The loader provides an interface for loading texts in a variety of formats, whether from local files or from urls. The parallel_loader module provides an optimized loader for large data sets using parallel I/O operations. The data_loader module provides method for loading or downloading large numbers of texts that are generally stored in a single file. All three modules inherit from a BaseLoader class, which provides common functionality for loading texts.