Skip to content

Home

Static Pre-release Python 3.12 Python wheels Code style: ruff license Coverage

Introduction¤

The Lexos Python library is designed to implement many common text analysis procedures in a way that saves the user having to re-invent the wheel or figure out how to combine multiple Python packages to achieve a given result. It is intended to be used as a library in other projects to build backend functions for applications, but it can be used in standalone scripts or in Jupyter notebooks. As with the original web app, it is designed to accessible to entry-level users whilst offering power functionality for students and researchers, particularly in the Humanities. It is also designed to be as language-agnostic as possible so that it can be used for a wide variety of historical and under-resourced languages. The Lexos library reproduces and extends most of the text analysis tools in the original Lexos web app.

Features¤

  • Loads texts from a variety of sources into a common data structure.
  • Manages a corpus of texts and generates statistics about the corpus.
  • Performs text pre-processing ("scrubbing") and splitting ("cutting").
  • Performs tokenization and trains language models using spaCy.
  • Creates assorted visualizations of term vectors.
  • Performs hierarchical and kmeans clustering with a variety of visualizations for text comparisons..
  • Performs classification using spaCy, scikit-learn, and custom architectures.
  • Generates topic models and topic model visualizations using MALLET and DFR Browser 2.

And more!

Changes in v0.2.0-beta¤

  • Performance optimizations and bug fixes througout.
  • A simplified public API.
  • Better HTML and XML tag handling in Scrubber.
  • New and improved clustering visualizations.
  • Improved plotting functions in Bootstrap Consensus clustering.
  • Silhouette score analysis for KMeans clustering.
  • More advanced topic modelling features, including termite plots, customisable labels, and LLM-generated labels with local or cloud-based models.
  • A new Classification module that performs text classification using a variety of architectures.
  • A new Structural Stylometry module that allows you to assess the significance of punctuation and whitespace patterns in your documentation.
  • Language model training features.
  • Improved documentation and tutorials for new users (now in a separate lexos-docs repository).

Project Status¤

The Lexos API is currently in beta. Most of the core functionality of the Lexos web app, along with new features, has been implemented and documented. As of August 2026, the API is considered feature complete and stable for general use, but some rough edges remain. Feedback is welcome.

The beta release coincides with at a time when the landscape of digital tools is rapidly evolving and AI-assisted coding is becoming more prevalent. I am waiting to see whether there is significant adoption of the Lexos library before investing more time in developing new features. If you like Lexos, you can help by requesting new features in the GitHub issues (labelled as "enhancement") or contributing them yourself.

Questions¤

If you are looking for help using Lexos, please post your question on the GitHub Discussions board.

Contributing¤

Lexos is an open-source project, and contributions from the public are welcome. If you are interested in contributing or have a bug to report, see the Development pages for information on how to get started.

License¤

This project is licensed under the MIT License - see the LICENSE.

Citation Information¤

Kleinman, S., (2026). Lexos. v0.2.0-pre-release https://github.com/scottkleinman/lexos.

Kleinman, S., (2026). Lexos. v0.1.0b3 https://github.com/scottkleinman/lexos. doi:10.5281/zenodo.18157848. -->