Technical report
Needle-1.0: a model for typed decisions
How a 395M-parameter model answers typed questions in one forward pass, what the experiments show, and where it struggles.
Current work · needle-1.0
Needle
A decision model that answers typed questions about a supplied state. Choices, scores, and probabilities in a single forward pass.
In development. The inference API is not yet available.
An example
Supply a situation and the questions you want answered. Each question defines its possible answers.
These are fixed examples showing the intended interaction, not live model inference.
1. Supplied state
User: How did last quarter's revenue compare to forecast? Tools: sql, web_search, calculator.
- tool
- choice
- clarify
- probability of true
- risk
- ordered score
Fixed example. No request is sent to a model.
2. Show illustrative response2. Illustrative response
All answers returned together in one forward pass.
toolchoice → sql
sql0.91
web_search0.06
calculator0.03
clarifyprobability of true → 0.12
P(true)0.12
riskordered score → 0.07
read only0.94
writes0.05
irreversible0.01
3. Application policy
Here, sql is the selected tool at probability 0.91. The application can run the query and use a language model to write the reply. Needle selects the tool, it does not execute it or compose the reply.
Inspect the proposed request
{
"model": "needle-1.0",
"state": "User: How did last quarter's revenue compare to forecast? Tools: sql, web_search, calculator.",
"questions": {
"tool": {"type": "choice",
"criteria": {"sql": null, "web_search": null,
"calculator": null}},
"clarify": {"type": "noul",
"instructions": "Ask a clarifying question first?"},
"risk": {"type": "score",
"criteria": ["read only", "writes", "irreversible"]}
}
}The model
Needle reads the supplied state and scores the candidate answers directly. It does not generate text or search through a sequence of responses.
- Model
- needle-1.0
- Architecture
- Bidirectional encoder, ModernBERT-large backbone
- Parameters
- 395 million
- Answers
- Choice, Score, and Noul (probability of true)
- Computation
- One forward pass, zero output tokens
Evaluation
Needle performs well on the tested decision families. Unfamiliar task families and reordered choices remain weaknesses.
| Evaluation panel | Needle | Reference |
|---|---|---|
| Blind test1,180 fields | 97.0% | 85.8% |
| New wording and layouts432 fields | 98.1% | 88.2% |
| Unseen task families675 fields | 80.4% | 96.1% |
| Reordered choices1,008 fields | 82.9% | 93.9% |
Field accuracy on four named panels, using the same requests for both models. One checkpoint and one seed, not a general reasoning score. The checkpoint label is a readable alias, not an artifact hash.
Research
Follow the work
Occasional updates on Needle’s development and new results.