Home
Softono
logikon

logikon

Open source Python
47
Stars
0
Forks
10
Issues
3
Watchers
1 year
Last Commit

About logikon

Analyzing and scoring reasoning traces of LLMs

Platforms

Web Self-hosted

Languages

Python
Logikon Logo

Logikon

AI Analytics for Natural Language Reasoning.

🕹️ Guided Reasoning™️ Demo | 📄 Technical Report

[!NOTE] 🎉  We're excited to announce the release of Logikon 0.2.0 –– a major update to our analytics toolbox for natural-language reasoning.

Main changes:

  • All LLM-based argument analysis pipelines are now built with LCEL/LangChain (and not with LMQL anymore).
  • We're introducing Guided Reasoning™️ (abstract interface and simple implementations) for walking arbitrary conversational AI agents through complex reasoning processes.
  • AGPL license.

Our short-term priorities are housekeeping, code cleaning, and documentation. Don't hesitate to reach out if you have any questions or feedback, or if you'd like to contribute to the project.

Installation

pip install git+https://github.com/logikon-ai/[email protected]

Basic usage

import os
from logikon import ascore, ScoreConfig

# 🔧 config 
config = ScoreConfig(
    global_kwargs={
        "expert_model": "meta-llama/Meta-Llama-3-70B-Instruct",
        "inference_server_url": "https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-70B-Instruct",
        "llm_backend": "HFChat",
        "api_key": os.environ["HF_TOKEN"],
        "classifier_kwargs": {
            "model_id": "MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli",
            "inference_server_url": "https://api-inference.huggingface.co/models/MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli",
            "api_key": os.environ["HF_TOKEN"],
            "batch_size": 8,
        },
    }
)

# 📝 input to evaluate
issue = "Should I eat animals?",
reasoning = "No! Animals can suffer. Animal farming causes climate heating. Meat is not healthy.",

# 🏃‍♀️ run argument analysis
result = await ascore(
    prompt = issue,
    completion = reasoning,
    config = config,
)

Guided Reasoning™️

sequenceDiagram
    autonumber
    actor User
    participant C as Client LLM
    User->>+C: Problem statement
    create participant G as Guide LLM
    C->>+G: Problem statement
    loop 
        G->>+C: Instructions...
        C->>+G: Reasoning traces...
        G->>+G: Evaluation
    end
    destroy G
    G->>+C: Answer + Protocol
    C->>+User: Answer (and Protocol)
    User->>+C: Why?
    C->>+User: Explanation (based on Protocol)