Axiomatic constraints for information retrieval and retrieval-augmented generation.
ir_axioms is a Python framework for experimenting with axioms in information retrieval in a declarative way.
It includes reference implementations of many commonly used traditional retrieval axioms as well as axioms for retrieval-augmented generation.
The library is well integrated with the PyTerrier framework and supports Pyserini retrieval indices too.
Use-cases of ir_axioms include search-result re-ranking, analyses of (neural) retrieval systems, and analyses of generated RAG answers.
Refer to the ir_axioms documentation for:
- Installation and a quickstart
- Axiom intuitions, formalizations, and implementations
- Related information retrieval literature
- Walkthroughs on re-ranking, post-hoc analyses, and RAG evaluation
- Instructions for developing new axioms
To get started without reading the documentation, follow these steps:
-
Install the package from PyPI:
pip install ir_axioms>=1.0 -
Install a spaCy model:
python -m spacy download en_core_web_sm
-
Load and combine axioms:
from ir_axioms.axiom import TFC1, PROX1, ORIG axiom = (TFC1() & PROX1()) | ORIG()
-
Define some test data:
from ir_axioms.model import Query, Document query = Query(id="q1", text="axiomatic information retrieval") documents = [ Document(id="d1", text="Retrieval axioms formalize constraints for retrieval models.", rank=1), Document(id="d2", text="Axiomatic information retrieval: axiomatic constraints guide retrieval.", rank=2), ]
-
Compute axiom preferences or re-rank documents:
axiom.preferences(input=query, outputs=documents) axiom.rerank_kwiksort(input=query, ranking=documents)
If you have found an important feature missing from our tool, please suggest it by creating an issue. We also gratefully accept pull requests! Get started by following our development guides.
This repository is released under the MIT license.
Files in the data/ directory are exempt from this license.
If you use ir_axioms in your research, consider citing us.