BEIR

Open source

Open-source benchmark and evaluation harness for heterogeneous information retrieval tasks.

by BEIR ProjectOfficial site ↗Source code ↗

Pricing: free
Free tier: true
License: open-source
Open source: Yes

Editorial review

BEIR has not yet been editorially reviewed. User reviews and published field data below are unaffected — we say so rather than implying an editorial verdict.

Facts & attributes

Every value carries its evidence state. “—” means unknown — never a silent default (P-02).

Identity · Taxonomy

Documentation(HTTPS URL): https://github.com/beir-cellar/beir/wikiVerified
Tags(open-source;cloud-native): information retrieval, benchmark, evaluation, NLP, dense retrievalVerification pending

Platform · Deployment

Delivery model(SaaS;self-hosted;desktop;mobile;embedded;managed-cloud;appliance): self-hostedVerification pending

Pricing · Licensing

Free plan(true/false): trueVerification pending
License model(proprietary;open-source;source-available;dual-license): open-sourceVerification pending
Pricing model(free;subscription;usage;seat;asset;transaction;revenue-share;perpetual;custom): freeVerified
Source repository(HTTPS URL): https://github.com/beir-cellar/beirVerified
SPDX license IDs(Apache-2.0;MIT): Apache-2.0Verified

Features · Product · Fit

Core capabilities(authoring;automation;analytics): Heterogeneous benchmark containing diverse IR tasks with 17 preprocessed benchmark datasets, Evaluates lexical, dense, sparse and reranking-based retrieval architectures, Evaluation metrics NDCG@k, MAP@k, Recall@k and Precision@k, plus custom metrics like MRR, Install via pip (pip install beir), tested with Python 3.9+Verification pending

Audience · Use · Case

Best for(Neutral editorial summary): Researchers benchmarking NLP retrieval models across 15+ diverse IR datasets with a common evaluation frameworkVerification pending
Primary use cases(build;monitor;automate): Zero-shot evaluation of information retrieval models, Academic and industry IR benchmarkingVerification pending
Skill level(beginner;intermediate;advanced;specialist): specialistVerification pending

Vendor · Maintainer

Vendor type(commercial;foundation;community;government;consortium;individual): communityVerified

User reviews

Review policy

No published reviews yet.