ChoiceLLM

Python command-line tool to use LLMs for rating items on a scale, comparative judgments, and multiple choice questions

Provided tools & services

ChoiceLLM

Type
  • Command-line Application
Executable name
choicellm

ChoiceLLM Aggregate

Type
  • Command-line Application
Executable name
choicellm-aggregate

ChoiceLLM Template

Type
  • Command-line Application
Executable name
choicellm-template

Citation

You can cite this software using the following citation generated from its metadata:

  • Westera, Matthijs
ChoiceLLM .
  • Leiden University
.

Logs & Reviews

Name
Automatic software metadata validation report for ChoiceLLM (unknown version)
Author
  • codemetapy validator using software.ttl
Date
2026-02-07 03:04:33
Review
Please consult the CLARIAH Software Metadata Requirements at https://github.com/CLARIAH/clariah-plus/blob/main/requirements/software-metadata-requirements.md for an in-depth explanation of any found problems

Validation of ChoiceLLM (unknown version) failed (score 1/5) due to one or more requirement violations:

1. Violation: Software source code *MUST* have a proper README. (This is missing in the metadata)
2. Violation: Software source code *MUST* state its version (This is missing in the metadata)
3. Info: Software source code *SHOULD* link to a continuous integration service that builds the software and runs the software's tests (This is missing in the metadata)
4. Info: Software source code *MAY* express the programming language(s) used (This is missing in the metadata)
5. Warning: Documentation *SHOULD* be expressed (This is missing in the metadata)
6. Info: Reference publications *SHOULD* be expressed, if any (This is missing in the metadata)
7. Info: The technology readiness level *SHOULD* be expressed (This is missing in the metadata)
8. Info: A research activity *SHOULD* be expressed as a category using the TaDiRaH vocabulary (This is missing in the metadata)
Rating
★ ☆ ☆ ☆ ☆
(log file starts at Sat Feb  7 03:04:29 UTC 2026)

[harvester info] --> Processing choicellm (https://github.com/mwestera/choicellm) [Sat Feb  7 03:04:29 UTC 2026]

[harvester info] Git updating cached clone of https://github.com/mwestera/choicellm...

[harvester info] No releases found, falling back to default git branch!

[harvester info] Using 'master'

[harvester info] Git reference: master

[harvester info] Scanning directory /tmp/codemeta-harvester.cache/choicellm for harvestable resources...

[harvester info] found codemeta.json for choicellm (md5sum 3f1576de114f760ad93f30fd91dcadf4); **NOTE: this is considered authoritative and most other detection methods will be skipped now!**

[harvester info] Inferring repostatus information from git activity (used only as a fallback if not explicitly provided)...

[harvester info] Inferred repostatus https://www.repostatus.org/#wip

[harvester info] Reconciliating: codemetapy  --baseuri https://tools.clariah.nl --baseuri https://tools.clariah.nl --includecontext --addcontext https://w3id.org/nwo-research-fields --addcontext https://w3id.org/research-technology-readiness-levels --addcontextgraph https://vocabs.dariah.eu/rest/v1/tadirah/data?format=text/turtle --trl --identifier "choicellm" --codeRepository "https://github.com/mwestera/choicellm" --validate /etc/software.ttl --enrich --textv "Please consult the CLARIAH Software Metadata Requirements at https://github.com/CLARIAH/clariah-plus/blob/main/requirements/software-metadata-requirements.md for an in-depth explanation of any found problems" -O /tmp/out/choicellm.codemeta.json /tmp/codemeta-harvester.cache//tmp/99-repostatus.choicellm.codemeta.json /tmp/codemeta-harvester.cache//tmp/10-jsonld.choicellm.codemeta.json 

-- begin log --

/usr/lib/python3.12/site-packages/pyshacl/extras/__init__.py:6: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.

  import pkg_resources

Passed 2 files/sources but specified 0 input types! Automatically guessing types...

Detected input types: [('/tmp/codemeta-harvester.cache//tmp/99-repostatus.choicellm.codemeta.json', 'json'), ('/tmp/codemeta-harvester.cache//tmp/10-jsonld.choicellm.codemeta.json', 'json')]

Adding to contextgraph: /tmp/turtle

Initial URI automatically generated, may be overriden later: https://tools.clariah.nl/choicellm

Processing source #1 of 2

Parsing json-ld file from /tmp/codemeta-harvester.cache//tmp/99-repostatus.choicellm.codemeta.json

    NOTE: Not a valid JSON-LD document, @context missing! Attempting to inject automatically...

    Injected (possibly temporary) URI https://tools.clariah.nl/choicellm

[CODEMETA COMPOSITION (https://tools.clariah.nl/choicellm)] processed 1 new triples, total is now 2

Processing source #2 of 2

Parsing json-ld file from /tmp/codemeta-harvester.cache//tmp/10-jsonld.choicellm.codemeta.json

    Found main resource with URI https://pypi.org/project/choicellm/

    Injected (possibly temporary) URI https://tools.clariah.nl/choicellm

[CODEMETA COMPOSITION (choicellm)] overriding old https://codemeta.github.io/terms/developmentStatus (https://www.repostatus.org/#wip -> https://www.repostatus.org/#active)

[CODEMETA CORRECTION (choicellm)] automatically converting spdx license URI from https:// to http:///

[CODEMETA COMPOSITION (choicellm)] processed 44 new triples, total is now 44

Remapping URI to (possibly) new identifier and version component: https://tools.clariah.nl/choicellm -> https://tools.clariah.nl/choicellm/snapshot

[CODEMETA VALIDATION (choicellm)] done

[CODEMETA ENRICHMENT (choicellm)] adding author https://tools.clariah.nl/stub/H46d07f2ea9e401ba as contributor

VALIDATION https://tools.clariah.nl/choicellm/snapshot #1: Violation: Software source code *MUST* have a proper README. (This is missing in the metadata)

VALIDATION https://tools.clariah.nl/choicellm/snapshot #2: Violation: Software source code *MUST* state its version (This is missing in the metadata)

VALIDATION https://tools.clariah.nl/choicellm/snapshot #3: Info: Software source code *SHOULD* link to a continuous integration service that builds the software and runs the software's tests (This is missing in the metadata)

VALIDATION https://tools.clariah.nl/choicellm/snapshot #4: Info: Software source code *MAY* express the programming language(s) used (This is missing in the metadata)

VALIDATION https://tools.clariah.nl/choicellm/snapshot #5: Warning: Documentation *SHOULD* be expressed (This is missing in the metadata)

VALIDATION https://tools.clariah.nl/choicellm/snapshot #6: Info: Reference publications *SHOULD* be expressed, if any (This is missing in the metadata)

VALIDATION https://tools.clariah.nl/choicellm/snapshot #7: Info: The technology readiness level *SHOULD* be expressed (This is missing in the metadata)

VALIDATION https://tools.clariah.nl/choicellm/snapshot #8: Info: A research activity *SHOULD* be expressed as a category using the TaDiRaH vocabulary (This is missing in the metadata)

-- end log --

[harvester info] Output written to /tmp/out/choicellm.codemeta.json

[harvester info] <-- Finished processing choicellm (https://github.com/mwestera/choicellm) [Sat Feb  7 03:04:34 UTC 2026]

        

Metadata Properties

Interface types
  • Command-line Application
Source code repository
 https://github.com/mwestera/choicellm  Stars are an indicator of the popularity of this project on GitHub
Category
  • Artificial intelligence, export systems
  • Computational linguistics and philology
  • Computers and the humanities
  • Linguistics
  • Software for humanities
  • Textual and content analysis
Development Status
  • Active: The project has reached a stable, usable state and is being actively developed.
Issue Tracker (Support)
https://github.com/mwestera/choicellm/issues  The number of open issues on the issue tracker  The number of closes issues on the issue tracker
License
Author(s)
  •   Matthijs Westera
Maintainer(s)
  •   Matthijs Westera
Contributor(s)
  •   Matthijs Westera
Producer
Metadata validation
★ ☆ ☆ ☆ ☆