Ornith-1.5: How Self-Scaffolding Builds Self-Improving Small Language Models

What Happened: The Self-Scaffolding Pipeline

Ornith-1.5 is a 1.5-billion-parameter language model that demonstrates a practical path to self-improvement without relying on frontier models. The core insight: a small model can generate its own training data, evaluate that data, and iteratively improve—if you give it the right scaffolding.

The team at Ornith AI started with their base Ornith-1 model and built a three-stage pipeline:

1. Self-Scaffolding: The model generates multiple candidate responses to a prompt, then uses a rubric-based self-evaluation to rank them. This isn't just "pick the best one"—the model produces structured critiques and selects the response that scores highest across dimensions like correctness, completeness, and clarity.

2. Preference Optimization: The ranked pairs (chosen vs. rejected responses) feed into Direct Preference Optimization (DPO). This is key: rather than doing expensive RLHF with a reward model, DPO directly optimizes the policy from preference pairs. The model learns to produce more of what its own evaluator prefers.

3. Iterative Refinement: The process repeats. Each generation of the model becomes both the student and the teacher for the next iteration. The authors ran multiple cycles, with each cycle producing measurably better outputs on standard benchmarks.

The result: a 1.5B model that punches above its weight class, approaching the performance of models 3-4x its size on certain reasoning and instruction-following tasks. The full technical details are in the Ornith-1.5 release post.

The Numbers That Matter

On AlpacaEval 2.0, Ornith-1.5 achieves a length-controlled win rate that puts it in the same conversation as models like Gemma-2B and Qwen1.5-1.8B, while using fewer parameters. More importantly, the self-scaffolding process itself shows a clear upward trajectory: each DPO cycle adds roughly 2-4 percentage points of improvement before diminishing returns set in around cycle 4-5.

Why This Matters for Engineers and FDEs

If you're a forward deployed engineer or working engineer shipping AI features, this matters for three concrete reasons:

1. On-Device and Air-Gapped Deployments Become Real

A 1.5B parameter model runs comfortably on a laptop CPU, a Raspberry Pi 5, or a mid-range smartphone. No GPU required. For FDEs deploying AI in environments where cloud APIs are off-limits—defense, finance, healthcare—this is the difference between shipping and not shipping.

When you're embedding with a customer to unlock trapped value, the ability to say "this runs entirely on your hardware" eliminates entire categories of compliance and security objections. You're not sending sensitive data to OpenAI. You're not dependent on an internet connection. The model lives inside their perimeter.

2. Domain Adaptation Without Vendor Lock-In

Self-scaffolding means you can take a base model and adapt it to a customer's specific domain—legal contracts, medical records, proprietary codebases—without needing access to GPT-4 or Claude to generate synthetic training data. The model bootstraps itself.

This is the kind of capability that turns a 6-day LLM feature deployment from a prototype into a production system. You're not just wrapping an API; you're building a model that learns the customer's data distribution.

3. The Cost Math Shifts Dramatically

Self-scaffolding with a 1.5B model costs pennies in compute. Compare that to generating synthetic data through GPT-4 API calls at scale. For a customer processing millions of documents, the difference isn't marginal—it's the difference between a viable product and a line item that gets killed in procurement.

How to Run Ornith-1.5 Today

Let's get practical. Here's what you need to reproduce this or experiment with the approach.

Prerequisites

• Python 3.10+
• 16GB RAM minimum (32GB comfortable)
• Optional but recommended: CUDA-capable GPU with 8GB+ VRAM for faster inference
• The model weights from Hugging Face: ornithai/Ornith-1.5

Step 1: Load the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "ornithai/Ornith-1.5"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

Step 2: Basic Inference

def generate(prompt, max_new_tokens=512):
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    outputs = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        temperature=0.7,
        do_sample=True
    )
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

response = generate("Explain the difference between DPO and RLHF in simple terms.")
print(response)

Step 3: Implement Self-Scaffolding

The key innovation is the self-evaluation loop. Here's a minimal implementation:

def self_scaffold(prompt, num_candidates=4):
    candidates = []
    for _ in range(num_candidates):
        response = generate(prompt, max_new_tokens=256)
        candidates.append(response)
    
    # Self-evaluation: ask the model to score each candidate
    eval_prompt = f"""Rate the following responses on a scale of 1-10 for 
correctness, clarity, and completeness.

Prompt: {prompt}

Responses:
"""
    for i, cand in enumerate(candidates):
        eval_prompt += f"\nResponse {i+1}: {cand}\n"
    
    eval_prompt += "\nProvide scores in JSON format with keys: response_number, correctness, clarity, completeness, total"
    
    evaluation = generate(eval_prompt, max_new_tokens=512)
    
    # Parse evaluation and select best candidate
    # In practice, you'd parse the JSON and compute totals
    # For brevity, we return the raw evaluation
    return candidates, evaluation

candidates, scores = self_scaffold("What is a monad?")

Step 4: Run DPO Training

For the actual DPO training loop, you'll want to use the TRL library. The Ornith team used a setup similar to:

from trl import DPOTrainer
from datasets import Dataset

# Your preference pairs from self-scaffolding
preference_data = Dataset.from_dict({
    "prompt": [...],
    "chosen": [...],  # highest-scored response
    "rejected": [...]  # lowest-scored response
})

trainer = DPOTrainer(
    model=model,
    train_dataset=preference_data,
    tokenizer=tokenizer,
    args=training_args
)
trainer.train()

Quick Start with Pre-built Pipeline

The Ornith team has released their full pipeline on GitHub. Clone and run:

git clone https://github.com/ornithai/ornith-self-improvement
cd ornith-self-improvement
pip install -r requirements.txt
python run_pipeline.py --model ornithai/Ornith-1.5 --cycles 3

Architecture: The Data-Generation Flow

Understanding the flow helps you debug when things go wrong or adapt it to your own use case. Here's how the components connect:

The self-evaluation rubric is the secret sauce. It's not just asking "is this good?"—it's a structured prompt that forces the model to articulate specific criteria. This constraint is what makes the preference signal useful rather than noisy. Without structured evaluation, you're essentially amplifying the model's own biases. With it, you're teaching it to discriminate along dimensions that correlate with actual quality.

A Balanced Take: Strengths and Limits

What's Genuinely Impressive

The bootstrapping actually works. The improvement curves aren't flat or noisy—they show consistent gains across multiple cycles. This isn't a one-off cherry-picked result.

The approach is model-agnostic. You could apply this same pipeline to Mistral, Llama, or any other base model. The technique isn't tied to Ornith's architecture.

It solves a real deployment problem. Small models that can improve themselves on domain-specific data without external API calls are exactly what many enterprise deployments need.

Where to Be Skeptical

Diminishing returns hit fast. After 4-5 cycles, the gains flatten. Self-scaffolding isn't a perpetual motion machine for intelligence—it converges toward the model's inherent ceiling given its architecture and parameter count.

Evaluation quality is the bottleneck. If your self-evaluation rubric is poorly designed, you're optimizing for the wrong thing. Garbage in, garbage out applies doubly when the model is both student and teacher.

The benchmarks tell a partial story. AlpacaEval improvements are real but narrow. The model may get better at the specific style of instruction-following that AlpacaEval tests without generalizing to the messy, ambiguous tasks that real users throw at it.

Scale limits are real. A 1.5B model has fundamental ceiling constraints. It won't suddenly develop reasoning capabilities that require more parameters. Self-improvement optimizes within the model's existing capacity; it doesn't expand that capacity.

FAQ

Q: Can I use this technique with any small model, or does it require Ornith's architecture?

The self-scaffolding approach is architecture-agnostic. Any decoder-only transformer model that can follow instructions can be plugged into this pipeline. The Ornith team used a standard Llama-style architecture.

Q: How much compute does a full self-improvement cycle require?

For a 1.5B model on a dataset of ~10k prompts, expect roughly 4-6 hours on a single A100 or 12-18 hours on an RTX 4090. CPU-only is possible but painful—budget 3-5 days per cycle.

Q: Does this eliminate the need for human feedback entirely?

No. The self-evaluation rubric is designed by humans, and you should spot-check the preference pairs for quality. Think of it as reducing the human feedback burden by 90%, not eliminating it. For production systems, you'll still want human evaluation on a sample.

Q: What's the catch with self-evaluation? Doesn't the model just reinforce its own mistakes?

This is the central risk. If the model systematically misunderstands a concept, self-evaluation may reinforce that misunderstanding. The structured rubric helps—by forcing explicit criteria, it creates some distance between the model's generation behavior and its evaluation behavior. But it's not foolproof. This is why the approach works best when the base model already has reasonable capabilities and you're refining rather than teaching from scratch.

Q: How does this compare to distillation from a larger model?

Distillation from GPT-4 or Claude typically produces better results in fewer cycles—but requires API access, incurs per-token costs, and may violate terms of service for some providers. Self-scaffolding trades peak quality for independence and cost control. For FDEs working in air-gapped environments, it's often the only viable path.

Q: Can I deploy Ornith-1.5 in production today?

Yes, with caveats. The model is Apache 2.0 licensed. It's suitable for instruction-following, summarization, and basic reasoning tasks. For mission-critical applications, you'll want to run your own evaluation suite on your specific use case. The model is not a drop-in replacement for GPT-4, but for well-scoped tasks where latency and privacy matter, it's production-ready.