Rendered at 18:37:39 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
jacobgold 22 hours ago [-]
First, congrats to the team on launching something genuinely interesting and new.
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
dbbk 22 hours ago [-]
When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
8note 19 hours ago [-]
if it puts a high confidence value on a wrong answer, thats still hallucinating, no?
llm hallucinations are high probability tokens that are incorrect vs the real world
adastra22 10 hours ago [-]
No, I don't believe so. Hallucinations are not "high probability" in a real sense. They are an artifact of the random walk the inference algorithm takes, which causes it to latch on to and chase attractors in the noise. This random walk behavior is necessary for chat interfaces to be useful, but are less critical to typed output predictors. I'm guessing they found some optimization that is possible if you give up caring about chat.
jubilanti 15 hours ago [-]
Correct, they have not made a universal all-knowing omniscient oracle, which is what would be required for "can't hallucinate".
eru 13 hours ago [-]
That seems like a weird standard.
I would be happy enough with: only produces what it can verify with sources.
If you eg try to remember a court case (ie produce the reference via LLM token generation only), it's easy enough to check with your data whether it really exists. Similar for following links and other references.
If your data or sources are wrong, obviously your report about them will be wrong. But I wouldn't call that a hallucination.
baq 12 hours ago [-]
There isn’t a single human in this world and hasn’t ever been that meets your happy-enough standard. Make of it what you will.
hdjrudni 11 hours ago [-]
Why is that at all relevant?
Humans are known to hallucinate a lot. Ask 10 different witnesses at a crime scene what they saw and they'll all report different things.
A good, non-hallucinating LLM would only report things for which it has evidence. It would consult the facts every single time.
It's a pain in the butt for humans to fact-check everything but LLMs can quickly look up all kinds of stuff. That's what makes them useful.
eru 9 hours ago [-]
Yes, and for the LLM you can do it in multiple passes.
So you can bolt the fact-check / source-check pass onto whatever other system you have, without having to redesign the underlying system.
eru 11 hours ago [-]
It's not a binary thing. You can get closer or further away from that standard.
And humans also behave differently in different contexts. A conversation at the pub has more such hallucinations than a formal deposit in court. For the latter, a good lawyer will look at her shoes, when you ask him what colour her laces are.
spencerflem 14 hours ago [-]
Not to be tooo pedantic, but a bot that assigned 0 confidence to everything wouldn’t hallucinate.
A calculator either gets the right answer or doesn’t answer.
It wouldn’t have to be all knowing as long as it knew perfectly what it doesn’t know
baq 12 hours ago [-]
A quantum calculator answers in distributions.
stpedgwdgfhgdd 8 hours ago [-]
In one universe that is true, in another one not.
dozerly 17 hours ago [-]
Yes, there is no magic sauce here that makes stochastic output binary if that’s what people are looking for.
resonious 16 hours ago [-]
Right and so maybe we should stop saying "can't hallucinate" when it can by definition.
rpunkfu 16 hours ago [-]
It’s not what people are looking for, but what they wrongly claim.
elil17 9 hours ago [-]
What we would want to see if a confidence value that is in line with the actual correctness. If the value is 0.9 for 1000 different answers, then approximately 900 of those answers should be correct.
janalsncm 22 hours ago [-]
Technically speaking when you send the prefix “The capital of France is “ into an LLM it will also produce probabilities across its whole vocabulary.
sothatsit 19 hours ago [-]
The probability values don’t really represent confidence in modern LLMs though, especially after RLHF and RLVR.
System One says they use RLCD, Reinforcement Learning for Calibrated Decisions, which presumably has accurate probabilities as an explicit optimisation goal.
nkozyra 16 hours ago [-]
How is that different from RLVR?
sothatsit 15 hours ago [-]
RLVR generally upweights tokens along the whole thinking trace that led to a correct answer, whether each token was "correct" or not. RLVR doesn't train a model to output an 80% likelihood, it just trains it to produce correct answers, and not to produce incorrect ones.
System One hasn't said how RLCD works, but they do say it is explicitly training models to output "calibrated" probabilities, which makes it distinct from RLVR. This is how they describe it:
> System One models are trained for calibrated decisions: their probabilities are optimized against outcomes to reflect uncertainty.
jiggawatts 20 hours ago [-]
… which they could provide in their APIs but are vehemently opposed to because it makes distillation much easier, and faster.
bigglebear 18 hours ago [-]
Yeah. Yet another reason why open-weight models are better. If I want to use the logits, I can.
CompleteSkeptic 22 hours ago [-]
that's right, but because these models are probabilistic, it's also possible to be confidently wrong (and all future models will be smarter still and still have that possibility)
flockonus 27 minutes ago [-]
Correct. Not to say we're getting into the weeds of probability here as well.
"What are the odds a thunder will strike in Paris at 1pm UTC of 2026-09-16" - that could be a 0.001 chance going from blind historical measurements; 0.01 if it's raining; or 1 or 1 after the date has passed.
darylteo 15 hours ago [-]
I read "hallucinations" as "generates novel output with no grounding/source". i.e. "it just made something completely up".
I believe their "accuracy" metric (sonnet 5 level) is where "right/wrong" is measured.
orbital-decay 20 hours ago [-]
Yeah but what stops it from producing confidently incorrect outputs...
zenlikethat 19 hours ago [-]
Nothing, but imagine using LLMs for a classification task
People out there are so resigned to the models being unreliable that they are really doing things like hallucinating deliberately, and then matching the hallucinations to embeddings -
You could do that or you could just... use a model that will never produce unreliable outputs in the first place.
nkozyra 15 hours ago [-]
I'm certainly not resigned to that, at least for classification.
Even non-frontier models are absurdly good at this in a broad sense.
Which would make it hard to judge "a model that will never produce unreliable outputs in the first place" against something that is already really, really good and exceptional in domain-specific areas with the tiniest amount of elbow grease.
Speed and cost look good though (for now)!
threecheese 19 hours ago [-]
But we're going from "Apple" to "Apple: 99% - trust me". It could still be an image of an orange :)
14 hours ago [-]
zenlikethat 19 hours ago [-]
It's pretty darn smart. If you did want to hack on it in earnest and find out for yourself, send me an email - nathan@typesafe.ai
bradly 15 hours ago [-]
What about the LLM calls though that are done midchain? In the Home Assistant video the multi-intent prompt gets split using what looks like a traditional llm model, which I'm assuming is vulnerable to classical hallucinations.
csomar 14 hours ago [-]
That's really funny when you consider that generative models also don't hallucinate if you check up on them on every token generated?
riknos314 18 hours ago [-]
Has LLM become so synonymous with Generative Transformer that other high-parameter count models that interpret language need a different name?
For all we know this might be a non-language-generative transformer e.g. a transformer where the decoder produces confidence scores rather than language. Please provide more likely architectures if you know them, I'm genuinely curious.
soleveloper 12 hours ago [-]
I think the meaning of can't hallucinate in this model is that the type won't be hallucinated.
So if the generated schema is for a tool call for calculator, then the numbers will be valid numbers for sure (and not random words).
To me, it looks similar to BNF schema already introduced and implemented few years ago: generally speaking - it limits the next token that is allowed to be generated, probs are drawn from a subset tokens.
(tbh, I'm not sure why it didn't pick up as a more standard interface to LLMs, as it made a lot of sense back then, and now.)
NitpickLawyer 10 hours ago [-]
Yeah, I thought about constrained generation as well. I've actually done something similar with local models before. And you can even get a "confidence" score by looking at the logits (something along the lines of logprob("YES") + logprob("Yes") + logprob("yes") - logprob("NO")...
There's also a cheeky "one of the models hallucinated a link" in the wiki jump example that most likely could have been avoided by properly using grammars. You can setup constrained gen so that only valid options (say from a list) can be outputted. Their own inference lib likely does that. So comparing to one that doesn't is a bit cheeky.
That being said, after a brief look at the site I could see this working. Especially if this can be ran locally, the speed and cost can enable some workflows where you have this as an "overseer" layer over say a cli agent. After each step you run through a list of "questions" ("is the task completed?" -> yes -> "does the edit touch files it shouldn't" / "does the edit follow our code writing policies") etc.
edit: extra points if the "question" rubric is also generated by a higher abstraction model. Say "/goal Build out auth" -> generate_rubrics(goal) -> "Is auth implemented on all endpoints" / "Has code touched anything else than auth" / "is this following the best practices" / ...
adastra22 10 hours ago [-]
AFAICT it is the same interface as you describe, but the underlying inference algorithm is fundamentally different, hence the speed gains. There is an application I am currently working on right now where this typed output predictor is the performance bottleneck. I'd be very interested to see how this performs.
janalsncm 22 hours ago [-]
I don’t think it’s misleading if you compare on the use cases they suggested. It’s faster and cheaper (no idea if higher quality), so it’s immediately interesting for certain things.
And if you buy their RLCD claims, this might be even better than huge models that know a bunch of irrelevant things.
WhitneyLand 21 hours ago [-]
What was misleading was the original title:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm not the gatekeeper of who gets to call themselves a frontier model, but I don't think most people would count Jev in that group. It sounds false.
If their specific claims hold up, then it would make more sense to say something like:
"Advanced the speed/cost frontier for structured decisions"
sroussey 21 hours ago [-]
I dunno, I would consider Waymo and Tesla to have frontier models.
I think AlphaFold and related are also frontier models.
Being an LLM does not seem like the qualifier for frontier.
riknos314 18 hours ago [-]
This is likely still an LLM (in the purest definition of a language model with relatively many parameters) since the inputs are natural language, just not a generative LLM as the output is something other than more language.
The inputs are natural language, they're just also structured into a tree. The first example on that very page shows natural language instructions:
questions = {
"refund_requested": Noul(
instructions="Does the customer request a refund?",
),
}
sroussey 1 hours ago [-]
The hybrid nature of this thing is not a detriment, nor does it make it an LLM.
19 hours ago [-]
cgio 9 hours ago [-]
It is frontier in the sense it is exploring an unexplored domain. I do agree on questioning the comparatives though. Speed/cost is indeed relevant for problems that can be framed as structured decisions only. The question is, would defining a structured decision model be a structured decision model itself? This would significantly increase the application domain.
janalsncm 20 hours ago [-]
Large language models are not the only type of model.
18 hours ago [-]
alfalfasprout 21 hours ago [-]
How is this not a frontier model? It's bleeding edge in its own niche. It's not a frontier LLM; however, applicable to many of the things people use LLMs for.
bigglebear 18 hours ago [-]
It's nothing like a traditional LLM and so should not be compared to one. It's a heavily constrained, tiny model that can only produce a probability score or a yes/no answer over pre-defined selections. It has no long-context capacity.
I mean, imagine comparing this thing to Astra, it's hilarious. They don't even tell you what the max input size is, and they only allow 10 possible answers to choose from for the Choice mode. It's probably like a 1billion param model. They say it's "not small", but there's zero reason to believe that.
I suspect someone will be able to recreate this within a week by piecing together open-weight models.
janalsncm 17 hours ago [-]
> It's nothing like a traditional LLM and so should not be compared to one.
Frontier LLMs are expensive jack of all trades. You can absolutely compare them to purpose-built tools on any domain they touch. Engineering is all about assessing tradeoffs.
A calculator is faster and cheaper and higher quality for the use cases I bind my task space to.
It's therefore misleading because they make people believe they're as general purpose as LLMs.
hdjrudni 11 hours ago [-]
> Assuming there's no funny business, the Doom demo is cool.
The Doom demo seems very funny business. They're not feeding it video, they're feeding it a text description of what's going on in the game. It's not reading pixel data.
I think LLMs would play a lot better with that input too but Jev does seem to have a huge speed advantage; I don't know if the other models could do that in real-time.
salomonk_mur 5 hours ago [-]
Well, the whole point of the demo was showing that transforming from game state to text to action is so quick with Jev that it can play in real time.
So no, the other models cannot play in real time. Hence why this is interesting.
nylonstrung 11 hours ago [-]
In a case like this it still seems more appropriate to encode that data in tabular form and use a tabular foundation model
adastra22 10 hours ago [-]
Forgive my ignorance. Tabular foundation model?
CompleteSkeptic 22 hours ago [-]
I'm biased but I wouldn't call it misleading - generating text is super awesome and flexible, (we describe that in the blog post - and I personally use string models all the time) but it's true you pay a high tax for autoregressive generation
> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do
WhitneyLand 21 hours ago [-]
His claim was that the title is misleading, not sure how it's relevant to that claim that you use "string models" (full LLMs).
The original title before it changed less than an hour ago was:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm going to agree that was misleading.
And on the second point:
>>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
>that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do"
Also going to disagree here, and I don't think it's semantics.
Type safety is not factual correctness.
CompleteSkeptic 21 hours ago [-]
> Type safety is not factual correctness.
I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?
elcomet 21 hours ago [-]
Hallucinations were defined in the context of text generation models so your question does not really make sense.
IMO your system can make mistakes that are similar in spirit to hallucination (i.e. answering with a false answer instead of abstaining to answer).
bigglebear 18 hours ago [-]
And furthermore, because the model is forced to answer in a boolean (if in boolean mode), if the user input is outside of the range of a boolean, it's forced to hallucinate. It can't abstain.
thduabmd 18 hours ago [-]
No. Your launch post puts “0%” on a hallucination chart, then explains that the number comes from guaranteed schema matching.
You’ve already agreed that this doesn’t establish correctness. An approve for an unauthorized action still meets the schema guarantee.
That’s why I find the messaging misleading. You’re acknowledging the limitations in these replies while defending the broader reliability pitch.
Even granting that each answer is calibrated individually, that doesn’t establish calibration of the decision that combines them.
Sure, I can threshold a composite score, but there may be many wrong answers with the same score. An unauthorized action doesn’t become acceptable because it scores highly on the other dimensions.
I still have to define the constraints and test which wrong actions get through the complete workflow on my own data. That’s a substantial part of the work being pushed back onto the developer.
agos 11 hours ago [-]
hallucinations are not wrong answers, that's why we use a different term
nkozyra 15 hours ago [-]
> I very much agree with this
From the intro blog
"Hallucination and type-safety are intrinsically related"
I'm not entirely sure why we're conflating type safety with, I guess, value or output safety.
"Would you say a linear classifier hallucinates?"
No, but it can be (and often is) mathematically correct and functionally incorrect. It doesn't help to say "a linear classifier can't hallucinate" when you get even 99% accuracy. That's 100% a semantic play, and it doesn't help when the picture of a dog is labeled cat and the response is "yeah but that's not a hallucination, only stupid LLMs do that"
bigglebear 18 hours ago [-]
User input: "Hey, have your human support agent call me, tomorrow at 5pm."
Model input: "Does the user want to speak to a human support agent?"
Output: Yes.
I imagine that your model would produce this, and I think it's fair to say this is a hallucination. A human would caveat it with: "Yes, but not right now.", your model is incapable of that. Yes is technically correct, but within the context of being in a live chat, a human would understand that the caveat is required.
djray 4 hours ago [-]
No, the model has answered correctly. Your question is poorly phrased (possibly deliberately).
Your question would correctly classify the user's input as requesting a human support agent, but at an indeterminate time.
If you wanted to determine whether the user wants to speak to a human support agent immediately, you would have to correctly qualify your question, e.g. "Does the user want to speak to a human support agent now?". You could have another question which is "Is the user requesting a call-back from a human support agent?". Or you could have a multiple choice query which would filter the conversation into one of a number of pre-written possibilities.
This is nothing to do with accuracy or hallucination. It's a different method of interacting with the model where you are relied upon to be precise.
monkpit 14 hours ago [-]
To be fair - you’re crafting a deliberately bad model input for a contrived example.
8note 18 hours ago [-]
id say yes. a linear classifier that classifies between red and yellow balls will hallucinate on blue.
linear regressions hallucinate in the simpson's paradox.
the model output can be quite confident and not representative of reality
WhitneyLand 21 hours ago [-]
Let's say classifiers don't hallucinate. To make a fair comparison we should constrain LLMs to the same classification task. In that case, no, LLMs also don't hallucinate.
- Give Jev and LLM the same input
- Lock down both to approved/rejected/unknown (LLM restricts on decoding)
- Both can be wrong, but neither can hallucinate (invent an another option).
18 hours ago [-]
InsideOutSanta 17 hours ago [-]
A hallucination in the context of LLMs is generally understood as an incorrect answer presented as factual. If you claim that "x can't hallucinate" in the context of LLMs, you're saying that x always gives accurate answers. It does not matter whether the answer is type safe. If its value is incorrect, it's a hallucination.
seizethecheese 22 hours ago [-]
Just to be sure that I understand, you're saying that your model "can't hallucinate" because it only outputs a single thing, right? In this way, an LLM can't hallucinate either if I prompt it to do a classification task with a discrete set of possible outputs, right? (Assuming I reject non-conforming output. Actually, maybe what you're saying is that your system can't output non-conforming output?)
zenlikethat 19 hours ago [-]
Yeah that's precisely correct.
For e.g. classification tasks, even in 2026 people are doing things like hallucinating deliberately, and then matching the hallucinations to embeddings -
From a quick look at this it looks like it could easily generate natural language text by following a structured representation like UMR (Uniform Meaning Representation) or the similar representation the Abstract-Wikipedia folks will be working on for generic encyclopedic text (which will be heavily informed by Universal Dependencies). These are basically linguistically principled and frame-based counterparts to a programming language AST, that can be then converted to natural language (in a broadly language-independent way, to the extent that semantics and pragmatics make that feasible) via some sort of NLG rendering.
(To be clear, this one raw model does not support outputing a full AST directly - it wants to output "choice" among fixed options, "score" on a sliding scale, or a true/false answer (all of these with confidence scores attached), so building the AST/structure would be a code-driven (or even perhaps outside LLM-driven in some more challenging cases) multi-step affair where the model would essentially be playing a "game" of building the structured output step by step and getting a revised partial state back. But one could expect this to lead to interesting results.)
18 hours ago [-]
dfee 21 hours ago [-]
> I'm biased but I wouldn't call it misleading
- @CompleteSkeptic
Very strange.
Flere-Imsaho 22 hours ago [-]
> Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring,
My first thought was that it would be ideal for robotics? As in control of limbs, general planning, route finding, etc.
ygouzerh 12 hours ago [-]
That's a great point! It quite looks like the System 1 model of Physical Intelligence
copperx 19 hours ago [-]
Um, Isn't SELF DRIVING the elephant in the room?
aryamccarthy 18 hours ago [-]
Only if you think that everyone cares about self-driving. Lots of niches require structured domains; self-driving is just one that has a lot of capital thrown at it.
stiiv 6 hours ago [-]
> but it's nothing like the code generating models we're all using today for code and automation.
Is this true? Code is structured output. At the very least it seems like a question of degree rather than kind.
While the LLMs we're using today are limited to sequenced text, it seems that a model like Jev could excel at coding on a more structural level (factoring, controls) by working within the constraints of an actual language specification and supplemental domain model. I don't know, though -- maybe that's too deep and complex.
vvzz 21 hours ago [-]
I feel like the power of the approach presented here is that it gives a model a proper "language" to describe computations directly vs moving tape silliness.
I foresee this to be the path moving forward - giving AI models understanding of the computation directly(as well as compositional rules) This feels like a short path towards total software in many areas.
14 hours ago [-]
bigglebear 18 hours ago [-]
Agreed. It's a wildly dishonest presentation of their product from many perspectives, which is a shame because it might actually have some good use cases.
The comparison between LLM speed and Jev speed is misleading, because they're using autoregression to generate all of the type names, all of the schema, etc. A closer comparison would be if the LLM was purely outputting the raw numbers. Even then, comparisons to LLMs are pointless because you could train a transformer on the same sort of task that Jev is doing and get even better performance yet again, and a smaller model. I suspect this is some form of stripped down diffusion language model.
You really have to do a lot of hand holding here, and map out your problem space manually, and very carefully, to get any sort of accuracy. For example:
> Keep each Score to one dimension. If a description says “punctual and smart and experienced”, the question is measuring three things, and an input that is high on one and low on another can’t be placed. Confidence drops and the score means less. Split it into one Score per thing and combine them in code
If you don't perfectly represent the distributions of possible answers then you'll likely get garbage results. As far as probabilistic state machines are concerned, I'd say creating the distributions of possible answers, and their hierarchy, is the actual hard part.
One of their examples is:
- "state": "I have asked three times now. Can I please just talk to a real person?"
- "Is the customer asking for a human agent?"
Imagine the users request is: "I want your human agent to call me tomorrow at 5pm."
Human conversation is fuzzy, getting useful reliable results out of this is going to be a challenge. Of course, you could add follow up checks like: "Do they want that now, or later?" -> if later -> "Do they want that tomorrow, or the day after?" and so on... But now you're building an LLM out of if statements. I am skeptical of whether this model has much utility for fluid language interpretation - I suspect it'll only be useful for scenarios where you've tightly constrained the answer space but want to use fuzzy language to describe it. Like:
- Question to human: "Would you like a support agent RIGHT NOW?"
- Their response: Yes | Yeah | Mhmm | ye sure (any possible yes signal)
Model input: "Did they ask for a support agent?"
Still... a tiny LLM could accomplish this sort of thing without problem. And that doesn't stop someone from saying: "No, not right now. But tomorrow." - and the tomorrow would get missed. I think this is why people haven't really tried this approach much already.
Also their Doom demo is on structured state, not on images. Meaning, the enemies must be being served to the model as coordinates (or the exact angle of projectiles that hit the player), otherwise it'd have to scan every pixel of the 360 degrees to know whether an enemy is in front of the crosshair or not. You can see from the map below that it's also choosing travel checkpoints/destinations through walls. So they've severely cooked this to make it look far more capable than it is in practice, and any speed advantage that is offered here is not factoring in the shortcuts it is taking, the training on the map, and the fact that it can cheat because the structured state it is using is not bound by obstructions.
This is very cool. However I don’t really want to bounce all my home automation commands to the cloud. I hope there will be an open weights approach one day. I’ve spent a lot of time setting up my local only home automation system, it would suck if it didn’t work during an internet outage, and also there are obvious privacy problems.
ramoz 15 hours ago [-]
GLiClass is performant, and its zero-shot classification scores are in the same ballpark as the Terra-level results Jev points to.
Rereading some things and because there's no official benchmarks, I misspoke about the ballpark comparison., but the open model's still a useful foundation to work with
somepatriot 10 hours ago [-]
thanks, it's definetly relevant
toasty228 3 hours ago [-]
How many connected devices do you have? How often do you add/remove some?
I feel like this whole scenario could be solved by a very ugly switch case generated by deepseek for 0.02ct.
rattray 7 hours ago [-]
Open weight or even closed with a proprietary license, I'd definitely like to be able to run it locally for all sorts of physical world stuff.
fgblanch 16 hours ago [-]
This video makes a better job at explaining what it is about vs the marketing ones. Thanks for sharing.
wg0 14 hours ago [-]
That's good.
Side note - just like most people don't need an intelligent personal assistant to manage and respond their emails and book their flights, most people also don't need smart homes. Century old toggle switches are more than enough in a 3 room apartment or 5 room house unless you have a mention.
qingcharles 14 hours ago [-]
My primary beef with smart home (having tried it) is that every person that visits your home ends up confused about some element of it. A light switch that goes up and down is universally understood.
apexalpha 11 hours ago [-]
My smarthome has regular switches and wifi.
There's no reason to not do both.
Also a quick NFC sticker in each room taking you to a small HTML site containing settings (temp, ventilation, lights, shutters, setting a alarm by the lights) has been golden.
No one wants to: download Shelly app + AC app + look for ventilation IR controller + figure out how casting works for the TV + figure out how to use the Shelly app to turn lights into an alarm. It's too much friction for little gain. But a quick tap? Great.
But tapping your phone on a NFC sticker bringing all those controls together per room in stead of per category (all lights in Shelly app. Person in room #1 has no interests in the lights in room #4 at the same time.).
IF you tap it while not on Wifi yet it just tells you to connect to Wifi. :-)
One "all house" sticker next to the front door allows any last person leaving or first person entering to put the entire house in active / idle mode.
Works wonders. And as soon as local AI is quick enough the stickers will be a microphone!
throwaway314155 10 hours ago [-]
That's clever and all - solid setup, good work. But I still think you either overestimate the average house guest or have particularly savvy/young house guests.
apexalpha 9 hours ago [-]
Most people think it's cool but use it once.
The regular-ceiling-lights-as-alarm service gets positive feedback. Weird how smarthome companies never market that, seems easy win.
Atreiden 5 hours ago [-]
Not sure I totally understand what you mean by light alarms. Do you just flash ceiling lights to notify the user of something?
shifto 7 hours ago [-]
I have a hugely integrated smart home. 2 rules; 1. Everything can still be manually operated in the way people expect it to. 2. Local only.
LeBit 7 hours ago [-]
I once made the mistake a buying a light bulb that needed a phone app to turn on and off.
That was my light bulb moment.
Semaphor 7 hours ago [-]
I personally just find it convenient, even in my 3.5 room apartment.
Though I do have switches that work.
suprfnk 12 hours ago [-]
Need is a big word. Convenience is also a factor.
We have 8 light buttons in our living room/dining room/kitchen space. It is very convenient to us that we have 1 button for turning all of them on/off at the door to upstairs (at night turn off all lights and go to bed upstairs, in the morning come downstairs and turn on all lights) - but also have 1 on/off button near our back door for when we leave/come home.
Next to that: on/off toggles a schedule where the lights are bright and cold-ish by day, and low and warm by night without us having to manually adjust each light every hour or something.
Again, need is a big word. But it's very convenient and pleasant.
ramoz 17 hours ago [-]
Guess I'm a bit less impressed seeing that for some of the more intelligent driven+action work -- splitting requests in the video -- they had to kick out to an anthropic model.
cfowles 16 hours ago [-]
That's fair, but it highlights how this would actually be used. It doesn't really seem like a competitor to other models but instead a way to make these real systems more enjoyable to deal with.
kzsh 16 hours ago [-]
Haiku, to rewrite a sentence as two discreet commands.
I agree that it was notable that they delegated to an existing LLM, but I don't think it detracts much from the value proposition (not yet proven) of their demo.
alpineman 11 hours ago [-]
Agreed but is it much easier to deal with if you need to have all of these sub processes integrated? How does one know when you need to reword a request? What if Anthropic then has a type error, then debugging that just got harder.
14 hours ago [-]
15 hours ago [-]
5 hours ago [-]
mcintyre1994 10 hours ago [-]
That's a really nice demo and way more helpful than their website, thanks for sharing it!
BiteCode_dev 8 hours ago [-]
Thanks, much clearer that way. The speaker is unusually eloquent and to the point for a very nerdy demo, they should put that on their home page.
qingcharles 14 hours ago [-]
That really helped figuring out what this thing does, thank you.
dominotw 4 hours ago [-]
Its all fun and games till it turns on blender instead of shutting down garage door
vintermann 10 hours ago [-]
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle.
Oh, I have one of those use cases, matching people in genealogy trees. You can ask all sorts of questions: do the names match? Do they match within some edit distance? Do they match according to soundex/ metaphone rules (which are themselves a ginormous set of rules for letters and letter combinations which may or may not result in the same sounds, hand-coded as a huge if tree by a linguist not a programmer)? What about their relatives, do they match by the same rules? Should we incorporate domain knowledge about local naming customs? Etc etc.
I pointed a coding agent to this problem, and it aggressively started coming up with complex scoring rules and testing them against real datasets. Which led to sort-of acceptable results, but it still missed lots of cases which were obvious to a human, and had false positives which were obvious to a human. Which I could trade off, and slightly improve, with more back and forth with the coding agent.
Pointing a good LLM to all the information about two people, would of course give great results. Maybe even better than human judgment. But I can't do that for 100000^2 people, it would be too expensive in all sorts of ways. I need a fast, reliable scorer. I could maybe train an embedding, but that would be a huge job and where would I get the quality data?
RobinL 8 hours ago [-]
There's also two other important limitations to using an LLM and just providing it with pairs of records.
It does not know enough about the records in the context of the overall dataset:
- what is the data quality and to what extent do we expect a errors in some fields
- how unusual are certain values such as names in the context of the dataset as a whole, e.g. some names would be very common in some countries but rare in others.
Another great article, thanks. I thought a lot about that too, and basically realized the same thing (this wasn't the coding agent), that frequency in the dataset mattered a lot for how useful a match is, and made frequency lists from my datasets, both on how common the literal names were and the phoneme-reduced names.
But I'm worried about that "bitter lesson" the TypeSafe CEO refers to, and that we're retreading the steps of natural language processing and a lot of other fields, trying to come up with clever rules, when the rule-based approach simply never gets good enough. I need matching to be good, it's absolutely central to genealogy, and I've seen the damage blindly linking by hand-crafted matching formulas can do.
RobinL 7 hours ago [-]
I agree, and I think LLMs can potentially do a better job than more conventional methods, so long as they're provided with enough content.
Another aspect of context that could be relevant to your work may be how people are nested within households. In some record linkage scenarios you can run a linkage on household membership and use this to make person linkages more precise. I imagine an LLM agent could also use this information effectively if it were explicitly provided, but sometimes they miss logical leaps like this
camdenclark 9 hours ago [-]
You need blocking!
Fundamentally this is an entity resolution problem. An LLM can score pairwise really well but scoring all the pairs would be insanely computationally difficult.
If you can constrain the set of potential matches up front by querying the dataset for things that could be matches it gets a lot more tractable to use an LLM for this.
Are there any heuristics you can use to reduce the search space? You mentioned soundex transformation and maybe prefixes of last names could work? Even if you get the number of potential matches down by a few orders of magnitude this gets more reasonable!
The coding agent was pretty good at coming up with heuristics for matching - even more than the dozen I suggested from domain experience. And it used some of them sensibly for blocking, too. I'm sure I could get it to perform a little better and a lot faster with more agent wrangling. I did consider using the heuristics just for blocking, and letting a local LLM do the actual evaluation, but if Jev or Jev-like models work as advertised, maybe we can have the best of both worlds.
Thanks for the link, it is an interesting topic.
futurisold 21 hours ago [-]
This, combined with contracts, could make a lot of things so much fun now!
For those who don't know (which is probably everyone but me), I ported the design-by-contract pattern in Python and combined it with LLMs. This was early 2025. I originally wrote about it here: https://leoveanu.com/2025-03-01-dbc/
. Contracts are a core feature of SymbolicAI ever since. The community seems to have loved it too (https://news.ycombinator.com/item?id=44399234).
I think I'm starting to glimpse the implications and it's gonna change agentic workloads if it holds up to scrutiny. It's too early for me to tell anything other than jot down some rough thoughts.
In short, you get blazingly fast semantic branching you can use in control flows. For contracts, I can now directly take the data model that you have to design and convert it into Jev's expected format. Or I can use Jev for semantic branching in postconditions.
If my understanding is correct, that should be doable, but I need to think more about it. It could be that with Jev I can finally “compile contracts” and better chain them into workflows, which is something I always wanted but didn't know how to do properly.
Eager to test. On the waiting list.
zenlikethat 19 hours ago [-]
love it. send me an email and i'll try to get you moved up on the list? nathan@typesafe.ai
12 hours ago [-]
AdieuToLogic 13 hours ago [-]
[flagged]
jbotz 13 hours ago [-]
Judgemental much?
GP's first sentence isn't arrogant (at worst displaying a bit of false humility) because it's saying everyone but him doesn't know about a thing he did. Your second quote you apparently mis-parsed because of a minor English error (he should have said "to Python" rather than "in Python"), but to me it was pretty clear what he meant.
12 hours ago [-]
stillpointlab 11 minutes ago [-]
IIRC, Carmack was working on getting AIs to play Amiga games. The Doom demo suggests a very interesting direction to take this research.
I'm curious to hear his take on this approach.
maltalex 18 hours ago [-]
This is a very promising idea - a model that takes arbitrary text input (which can be a complex json), plus a set of questions (yes/no, multiple-choice, or score) and quickly (milliseconds) and cheaply ($0.042/MTok) answers those questions.
Unfortunately, none of this is explained in the announcement, but the documentation [0] is pretty good.
An example with manual combinatorial exclusion in “not_for” field made me cry, this is a wild hybrid of code logic, textual definitions, and AI blackbox. It’s a cool idea, but the “glue” layer is too boilerplate-ish
big_toast 22 hours ago [-]
It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing.
It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence).
Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model?
I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable).
But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)
mijoharas 11 hours ago [-]
Small request, can we get an explanation of the naming of "noul" in the docs[0]. I tried googling, and searching the docs and didn't understand why it was called that.
(I'd also argue something like p_yes or just probability might be a simpler name, but I'm sure there's a better reason behind Bernoulli maps).
It is Bernoulli (mentioned in the comment I replied to). I just failed to guess that myself :)
mckngbrd 22 hours ago [-]
here is how I attempted to explain it to my company's AI group chat, is this roughly accurate?
"instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel
so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K.
all of this while still being instruction-tuned (!!!)"
It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept
CompleteSkeptic 21 hours ago [-]
very accurate!
the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing
Flere-Imsaho 22 hours ago [-]
Hi - first congratulations, System One looks really promising.
The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!
The demo is showing System One producing its output in parallel very quickly and for little cost compared to an LLM generating its answers token-by-token. The "noul" type is used to evaluate a yes/no question and return the probability that the answer is yes.
So this demo is showing System One offering much more nuanced responses and specific probabilities compared to an LLM's more crude responses (e.g. LLM shows "true" or "false" compared to "0.9" or "0.07" probabilities that the answer to some question is true).
potatoman22 21 hours ago [-]
I think that's to demonstrate its speed
ttul 22 hours ago [-]
For many day-to-day computing use cases, Jev seems far better suited than an autoregressive language model, if for no other reason than it is not wasting compute thinking about anything other than how to spit out a decision.
Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?
CompleteSkeptic 22 hours ago [-]
architecture is close to the chest for now, but we have talked about writing a paper
Well, I'm stoked to try it out. We have about a billion reasons a day to call a model like this to rid the world of spam and phishing.
ianbutler 22 hours ago [-]
I see this super interestingly as the "subconscious" to the llms "conscious" for lack of better terms. I'm super interested in this for broad and rapid decision making in the context of consumer agents so will be signing up for sure.
CompleteSkeptic 21 hours ago [-]
1. I am extremely on the same page
2. I do think that subconscious is not only much smarter than we give it credit for, but also much more robust than the "jagged frontier" of current LLMs
It's unclear if the context extends as the conversation grows?
zenlikethat 22 hours ago [-]
> the model takes as input a state (structured text? not sure if multi-modal)
Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure.
I assume this isn't really for consumers/individuals currently? Kinda feels like an improved magic 8 ball.
I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
zenlikethat 19 hours ago [-]
> I assume this isn't really for consumers/individuals currently?
Unless they're hackers, no. It's not really a chat interface, it's meant for consumption by machines and composing into higher level systems (pairs great with LLMs).
> Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
We're going to release some more info on evaluations over time, and yeah, join the waitlist! We offer faster access in exchange for good memes
skerit 22 hours ago [-]
So in theory you could feed it incomplete text, and then ask it for the probabilities of what the next character could be?
mckngbrd 22 hours ago [-]
I think the joke here is getting missed
zenlikethat 21 hours ago [-]
Strings trigger us
iforgotmypasswo 16 hours ago [-]
Or a partially completed song, asking for the next note. I’m not sure if you’re joking, but using it for space constrained next token generation within a grammar sounds like a really neat use case.
copperx 14 hours ago [-]
Feed the generated note back into the input for the next query and you have ... autoregression?
CompleteSkeptic 22 hours ago [-]
you could, but it the model is not optimized for text
this is complex, but generating text is highly complicated and requires mode dropping to make long cohesive text
vatsachak 22 hours ago [-]
If you provide it an AST of the english language, yes.
aghilmort 18 hours ago [-]
was wondering same
boutell 41 minutes ago [-]
I wonder how many choices you can give this thing in multiple choice response mode. I'm guessing you could give it enough choices to produce Turing complete programs one symbol at a time when running in a loop, or to hold a conversation when given a vocabulary as the choices. It would be particularly hilarious to just let it choose the next ASCII character of output in a loop.
My guess is that due to its design there is no support for prompt caching, as there would normally be no reason for it. So the performance of my idea would probably be appalling as every step in the loop would reevaluate every input and output token. But it would be interesting to see the outcome.
bigmadshoe 38 minutes ago [-]
It likely isn't trained on that task so performance would be worse than frontier models specifically trained for coding.
lubujackson 22 hours ago [-]
After much fumbling around with prompts and evals, this is exactly how I am using LLMs in production, to narrowly make choices and return structured data. Any deterministic work gets pulled out of the prompt and my goal is to narrow the model output to be as clearly defined and as minimal as possible.
Jev's focus on structured I/O and confidence scores are game changing. If this does at all what it claims, I think this is going to quickly become the new standard approach for agentic systems.
CompleteSkeptic 22 hours ago [-]
we hope so! the bigger hope is to not just eat LLM market share, but to allow for people to use AI much more in the inner loop of software
copperx 19 hours ago [-]
I'm sure you've thought of self-driving. How does the model work in that space?
bobtheborg 14 hours ago [-]
Great question! Yes, this works much like the doom player. Sensor data (LIDAR, velocity, etc.) becomes the state. You use the score primitive to operate the controls ("What level of braking should be applied" 0: None, 1: just slightly slowing down, 2: there's a suspicious cat on the side of the road you don't trust, ...
Full disclosure, I am not they :=)
wg0 14 hours ago [-]
But the real problem in self driving isn't the decision making but object description. That is, computer vision if with cameras.
Decision making isn't that of a bottleneck I suppose.
sebzim4500 7 hours ago [-]
99.9% of the time you are right, but in the long tails (construction works, major road damage, temporary road markings that contradict permanent signs) I could see this kind of architecture being useful.
dozerly 16 hours ago [-]
This smells like a tool a more broadly capable LLM would take advantage of extremely well.
wg0 14 hours ago [-]
Curious what your use case is if not confidential.
lubujackson 13 hours ago [-]
Not confidential, but not super relevant, as this is something I have learned the hard way over the past year across various projects.
A lot of people have become prompt maximalists, asking for complex multi-part solutions or dynamic workflows in a single prompt. You can get this to work sort of reliably with frontier models, but without much confidence or clarity where things might break in practice. My goal is to strip out as much determinism as possible from prompts so the LLM only needs to handle a narrow, well-informed decision, like "Pick one of these three things" and build around the answer. Sometimes you need to fill out a whole JSON payload and LLMs really actually suck at manipulating and adhering to JSON. They do ok now because labs have put in a ton of effort on making harnesses play nice with structured data. But it comes at a high token and context cost because under the hood I suspect the model is churning invalid text repeatedly until it gets around to passing some internal validation.
xixixao 12 hours ago [-]
Example I have worked: Personal delivery app, that tracks packages from various senders using incoming emails.
I am using the single prompt approach with GPT5.4, which is free, but it’s not reliable. Using Jev I’d decompose the prompt into a bunch of smaller questions, then I’d combine the answers in software. I’m super excited to try Jev out.
dgellow 23 hours ago [-]
Side note: it took me more time than I would like to admit to realize that Diogo Almeida isn’t a satirical version of the name Dario Amodei
bogzz 22 hours ago [-]
That would have to default to Wario Amodei.
clayhacks 22 hours ago [-]
I feel like should be Cario Amodei. The D to C flip a rotation of the M to W flip
layer8 7 hours ago [-]
In lower case you’d flip from d to q (or to b).
Alpha3031 21 hours ago [-]
Flip both, Cario Vmodei.
jakintosh 23 hours ago [-]
It wasn't until the demo videos that I realized the post wasn't satirical.
Aboutplants 19 hours ago [-]
Well now I’m rooting for them!
jdthedisciple 11 hours ago [-]
thought the same lol
latteren 13 hours ago [-]
Looking at the example Jev use cases, it almost feels like Jev's incredible cost/task can make it competitive as a generalized "poor man's ranking" algorithm that can be useful for lean startups or any fast paced development org.
I need to rank 1000 articles and pick the 5 most relevant for the user? Jev.
I need to audit and strip out content because my user is affected by regional privacy laws (without hallucinating)? Jev.
I need to surface the 3 funniest media comments that match the user's sense of humour? Jev.
nullbio 12 hours ago [-]
More like:
I need to ...? -> Open-weight model.
I'm sure someones working on this as we speak using an open-weight LLM base (Qwen or something would be a perfect fit).
This sort of task is a perfect fit for a very small model capable of semantic parsing. You can get away with a LOT less parameters without all the autoregressive generation and long-context reasoning.
Crazy, looks like this was just published a few hours after the TypeSafe post!
11 hours ago [-]
Gecko4072 13 hours ago [-]
Wonder if this could lead to better recommendation algorithms.
gogoout 4 hours ago [-]
Is this the reverse of LLM?
Ie, "what's the capital of France?" LLM picks from "Paris" 99.9% / "London" 0.001%, LLM then with some randomness output you "Paris".
For Jev, you ask it to give probability of a set of answers "what's the capital of France?" choose from answers (Paris/London), it then gives you (99.9%/0.001%)
aatd86 4 hours ago [-]
99.90% ;D
It basically gives you the softmax value instead of using it to generate the output token. (said loosely)
brooksy 3 hours ago [-]
Essentially, although LLM token probabilities tend to be miscalibrated (mostly bc of posttraining). Jev is meant to be particularly calibrated
mushufasa 23 hours ago [-]
I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman tax is well worth it when the cost savings of the model itself can be one-two orders of magnitude.
varenc 17 hours ago [-]
I think the trouble is that Typesafe APIs don't fit into the normal OpenAI-style API that every other regular LLM provider users. You're not just providing unstructured text and getting unstructured text back. It would take a different request and response format than every other model on Open Router. Though you could shoe-horn it in some way, it'd be hacky.
But agreed it'd be very useful to see it deployed on other hubs, and it seems worth it to provide the bespoke API format. Perhaps Typesafe's API will end up becoming the standard for a new type of structured model, the way OpenAI's API did.
anshumankmr 11 hours ago [-]
It can be shoehorned to work with OpenAI's newer Responses format.
cheeze 22 hours ago [-]
Isn't openrouter the exact opposite of caring about security and privacy?
I guess you can choose your provider still? But isn't the point that the lowest bidder is doing inference?
ajmurmann 22 hours ago [-]
You can set privacy requirements and define an allow list. To me the main value prop is that I get one bill for all models and can quickly try new models without signing up anywhere or changing my code.
Oh! Also you can pass an array of models and if the first provider is down it automatically falls through to the next provider. More useful than it should be...
hobofan 12 hours ago [-]
That's still ultimately privacy by contract (where you have to trust the inference providers to uphold their end of the deal), rather than privacy by design.
LeBit 22 hours ago [-]
I always setup guard rails so that only zdr providers are used.
17 hours ago [-]
CompleteSkeptic 21 hours ago [-]
They don't like adding stealth startups :(
oblio 22 hours ago [-]
The thing is, in this climate it's hard to believe such tech will remain secret for long.
So, assuming this is not vaporware, this would raise the tide for everyone because it shows what's possible.
ramon156 23 hours ago [-]
This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
They have various benchmarks, e.g. how much time it takes them to do wikipedia page -> page games. Jev seems to take the same or fewer hops but in ~10x less time and for ~10x less money.
It's totally reasonable to compare against LLMs doing chain of thought if it gets comparable performance.
yunwal 18 hours ago [-]
> If Jev is skipping generation entirely for a narrow structured task, of course it's faster
I think this is reasonable if people are actually using LLMs to solve this type of narrow structured task, which they are. The evidence is that every LLM provider has some method of forcing the output to conform to a json schema in their documentation.
It's not an LLM though it's a frontier model on structured data
bigglebear 18 hours ago [-]
> I really have to say that I like their manifesto
Their manifesto: "you only build on top of it if it's trustworthy." - the irony of this while putting out the most misleading, dishonest marketing campaign I've seen in months for their first public appearance doesn't exactly scream "trustworthy" to me.
alchemist1e9 17 hours ago [-]
What do you find dishonest?
arbayi 9 hours ago [-]
Comments under the x post.
BoorishBears 22 hours ago [-]
Did you see the video where it plays Doom, it made it click for me
zergrush 13 hours ago [-]
i've seen it play minecraft as well, what im not sure here is what is the thing that produces the JSON and keeps track of the objects
can jev play battlefield six for example
simianwords 22 hours ago [-]
BTW it was not multi model playing doom, it was passing structured input and getting structured output. Its not what I thought: frames of video passed and real time game play.
yieldcrv 22 hours ago [-]
so what? put an LLM on Cerebras and get its responses faster, and put Jev on Cerebras and gets its responses even faster
bregmandiv 20 hours ago [-]
I'm trying to parse it down to what we had before vs what is new here.
We already had encoder models that skipped text generation for giving us a numerical output that could be computed as a probability. we also got no hallucinations and faster inference for free there. So we already had
1. "unstructured state in, probabilistic decisions out"
2. "orders of magnitude faster and more efficient"
What was hard there was to train the model head without ML expertise, and considerable amount of data.
This seems like this is a democratization of those encoders? The addition over existing encoders seems to be coming from being able to specify the output shape (up to a cardinality of 255). It is unclear to me if this is possible using Jev without additional labels for fine-tuning.
If so, that is still very impressive, but I think the faster inference and 0 hallucinations might come for free, from it not being generative.
techn00 12 hours ago [-]
I can't see how this is different from a fine tuned LFM2.5 encoder
brooksy 3 hours ago [-]
It doesn't produce text -> it does not hallucinate statements
This is kind of trivially satisfied and they make this sound more extraordinary than it is. Also I suppose it can still hallucinate in the sense that for out-of-distribution data it will give miscalibrated probabilities.
Anyway a great step in the direction of calibrated AGI
wxw 22 hours ago [-]
> Input tokens: $0.042 / MTok ($42 per billion tokens).
> Output tokens: FREE (too cheap to meter).
Insane. The video demos are really compelling, in particular the speed.
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.
I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.
CompleteSkeptic 22 hours ago [-]
thanks a ton!
constrained decoding (OpenAI-style structured outputs) make models dumber unfortunately - the short+dense version is that simply masking logits is insufficient because if ever a model was assigning probability to an invalid token, the model is by definition confused. you'd be better off erroring IMO
brainless 11 hours ago [-]
I am not an expert in this domain but as an engineer-turned-researcher, this looks a lot like GliNER with a fitting harness.
This is something I focus on in a bunch of my experiments - how to get immense value out of tiny models (<1b params). There are lots of different architectures out there and there is so much to optimize if you know what you are asking and have a grammar to constrain with.
Great to see this and I hope this is a lot on top of what is already openly available.
abeppu 20 hours ago [-]
I think this is a great direction -- for some kinds of users. And this makes me wonder if the 'vs' framing is misleading.
Yes, I think it's a mistake that many organizations are cramming LLMs inside of automated pipelines where the extreme generality/flexibility of the model is at odds with the fact that you're using it for a very specific task that gets repeated over and over, and needs a very specific structured output to be successful. But specifying your task carefully (as well as deciding what counts as your input state representation etc) seems like a form of programming. Something (a person or a model working in a relatively unrestricted way) will need to produce a configuration/specification for this system.
So rather than Jev vs Claude I imagine that using Claude/ChatGPT/whatever interactively to define / refine your Jev config which then runs in prod might be the happy combination?
jawns 22 hours ago [-]
I could see this being fantastic for classification tasks. Last year I shifted from using LLMs for bulk data classification tasks (1M transcripts) to generating embeddings and categorizing based on cosine similarity. It saved a ton of costs and time, but wasn't as accurate as LLMs. This seems like it can give me Terra-level classification ability with the cost/speed I need.
copperx 14 hours ago [-]
This ought to work better than SpamAssassin, I'm sure
pjm331 22 hours ago [-]
yup just joined the waiting list with a very similar use case in mind
prometheus1992 6 hours ago [-]
Just trying to validate my understanding - so this is a Large natural language classifier, a general purpose or zero shot classifier ?? it can recognize entities, can classify text into some pre-defined classes ? right? or did i miss something amid all the marketing terms such as system one or RLCD or whatever??
prometheus1992 4 hours ago [-]
If you are wait listed and eager to try this, I will save you some time. Try this model - https://huggingface.co/MoritzLaurer/deberta-v3-large-zerosho... . They are using something similar under the hood. The comparison to LLMs on their blog post is definitely shady.
prometheus1992 4 hours ago [-]
Example to try on this model:
customer complaint - My credit card was charged twice for the same subscription
labels - billing, technical, sales
The model will always return something from the above classes - "so it can't hallucinate".
fallingbananna 4 hours ago [-]
Looks like it.
A general AI classifier that can be set up easily and used to classify anything… but with probably lower quality than a purpose built one.
alphazard 21 hours ago [-]
There's a whole lot of information on this page that doesn't tell me anything about what this actually is. Can anyone spell out what the architecture is here?
They claim it's not an LLM, which I read as "not an auto-regressive token generator". I assume they are still using a transformer, otherwise they would be talking about the thing that's not a transformer, instead of all the fluff on the linked page. But they emphasize parallel generation, so is it like a text diffusion model?
tacoooooooo 19 hours ago [-]
Sounds like its essentially a generalized zero-shot classifier that takes and option set at runtime and works on unstructured inputs.
you pass in your "prompt" and options (described in natural language) that it can respond with, in addition to your input. it gives back that option set with a probability assigned to each one
CompleteSkeptic 18 hours ago [-]
yes and can do many of those in parallel
bigglebear 17 hours ago [-]
I would guess a tiny stripped down text diffusion model. It only has 32k context, and for choice mode it can only select from 10 choices.
wesammikhail 16 hours ago [-]
> and for choice mode it can only select from 10 choices.
Rip there goes my excitement. I have a task that something like this would be great for but the list of options is a zero or two larger than that xd
copperx 14 hours ago [-]
They said that it works with up to 255 options.
mercat 15 hours ago [-]
you can still chain them
ursuscamp 4 hours ago [-]
Is this fundamentally different from other text-based LLMs, or is it the same except with special reinforcement learning a safe guards around generating valid types?
Surely it’s still generating some kind unstructured data internally? For example, what if I told it to generate a short story, but the short story is output as a JSON string?
jw1224 3 hours ago [-]
An LLM takes (text in) -> (text out).
Jev takes (…questions in) -> (…probabilities out)
So Jev won’t write a story or emit arbitrary structured data. But if you ask it the right questions, it can make near-instant “decisions” against those questions, with accuracy and world knowledge on par with LLMs. The economic advantage is that it’s parallelizable and can give back up to 255 answers at once, in milliseconds.
mortsnort 13 hours ago [-]
I am confused why they say it is not an LLM and then in the documentation it is shown as being an LLM derivative. The documentation makes it sound like they're taking a pretrained LLM and then giving it their unique post-training. How is that not an LLM?
FAQ:
Is Jev just a smaller LLM?
Jev is neither small nor an LLM, hence being off the intelligence Pareto curve.
LLM seems to have become synonymous with Generative Transformer architecture.
While this model may share much with GPT-style models on the encoder side, it clearly has a different decoder architecture. So is a high-parameter count language model an LLM even when it doesn't have a GPT-style decoder? The definitions are in flux.
anentropic 7 hours ago [-]
Yeah, it must be an LLM for some definitions of LLM
It seems to take two forms of context input: 'state' and 'questions'
> State can also be a JSON object or array containing related context, examples, and other information that helps the model answer the associated questions.
> The state contains the content and supporting facts.
The state seems to be schemaless, while the questions determine the output schema.
The doom demo is also in the article, for anyone that doesn't want to go to X.com. :)
caspar 17 hours ago [-]
I'm not sure the authors realize this is way more than "just a cool demo": if this holds up, it's going to be huge for game QA work.
Instrument your game to output properties of entities near the player and the output is the various control inputs - moment to moment gameplay gets solved. Maybe augment with a tick-by-tick controlled stepping mode if particularly twitchy - an LLM can take care of the higher level reasoning then.
The doom video is also in the article itself (headline: "Doom").
I suppose this is the same video as the one from the parent comment, but I don't know for sure - I don't have a twitter account and the above link doesn't work for me.
ErneX 22 hours ago [-]
I linked to the tweet that has the video because if you are not signed in you cannot see the whole thread of tweets.
I can see the individual tweets in the browser while not signed in though.
yehat 22 hours ago [-]
[flagged]
smusamashah 6 hours ago [-]
Doesn't this mean Jev can be used to drive a car as well?
It's in the parent article under a section named "Doom" in case that asset URL ever changes.
einpoklum 22 hours ago [-]
But when their system is given the instruction "do not fire, simply dodge" - it doesn't "simply dodge", it actually gets close to the fleshy pink demon rather than keeping its distance. Or am I misunderstanding?
anthonypasq 21 hours ago [-]
i think its just telling the model that it cant output a shoot action
baist0 6 hours ago [-]
lol! they reinvented aim bot for cheaters.
ggcr 2 hours ago [-]
Interesting. Perhaps I can see this being quickly adopted in LLMs-as-a-judge, where you normally need (a) a structured answer, say, with lots of different fields (metrics) and (b) you want the judge to be fast, not being a bottleneck.
sonink 4 hours ago [-]
Spent a lot of time - but this makes zero sense to me. It can, maybe, return type safe outputs faster than larger llms - but there is little reason to believe that it will be more accurate. It does absolutely hallucinate - and seems to me that the claim is largely misleading.
You architect your systems with typesafe - because it is marginally faster, but inaccurate - to do what ? You can just wait for the next version of LLM's to get more accuracy at the same cost - or just use a faster model right now from a different provider.
bigmadshoe 3 hours ago [-]
It isn't marginally faster, it seems to be approx 100x faster and 10-100x cheaper.
brooksy 3 hours ago [-]
For certain tasks, a model like Jev may be intrinsically more efficient than an LLM because it doesn't have to predict a token distribution and can instead focus solely on the probability of a single question/action
dinobones 21 hours ago [-]
This is a good product but the naming/branding is pretty unfortunate.
Typesafe.AI sounds like some typescript/structured output type of tool…
What even is “system one” ?
IMO the product/tech is really there, just needs better communication.
zenlikethat 21 hours ago [-]
The model can't reason comprehensively (e.g., like Sol XHigh would to solve a complicated problem), but it's designed to be able to answer anything a human reasonably could quickly and intuitively, i.e., system one thinking: https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow
vintermann 11 hours ago [-]
I wonder how well it can play chess, or go.
salicideblock 9 hours ago [-]
> What even is "system one"
I definitely agree it's underexplained in type safe.ai's materials.
I have to assume it's a reference to the fast, heuristic, intuitive "system 1" process in humans, as opposed to the slow, procedural, reasoning "system 2".
This theory is recognized, among others, in Daniel Kahneman 2002 Nobel prize on Economics.
ostacke 5 hours ago [-]
You are correct, they state that on their website.
wging 20 hours ago [-]
I had a different initial confusion - it seems this company has no relation to the company formerly known as Typesafe https://en.wikipedia.org/wiki/Akka.io
toddmorey 21 hours ago [-]
I mean, it's a structured output model that (apparently) can't hallucinate. I don't mind the name.
flyinglizard 20 hours ago [-]
It can't hallucinate, but it doesn't mean it can't make wrong decisions. Just because it adheres to a specific output format at all time, while LLMs have the output format at their mercy, then the claim of not hallucinating is made technically true.
I think that this specific part is not super interesting if your harness just recovers from invalid LLM outputs.
The latency and cost - yes, those are super interesting.
Would like to have something like in the original post but open weights.
paraschopra 10 hours ago [-]
Cool approach, i think less latency and cost is the way to go.
Here's how this would have likely been made.
- Tiny transformer or equivalent model (maybe a few bn or so?), explaining latency and cost
- Questions are sent in parallel to multiple copies of it (I'm sure they're edge located)
- The model is post-trained for calibration in a wide variety of data (the recipe is relatively simple, and likely targeted on distillation of logprobs / confidence of a bigger model)
Notice how cost is ONLY for input tokens as output is merely numbers (few tokens) because input could be huge (questions and options).
At 0.042-per-million price they have, Astra estimates the model to be 3bn parameters.
One could replicate this by post training Qwen 3.5 2Bn. I expect people to do so soon!
niutech 10 hours ago [-]
How does Jev compare with encoder language models like BERT/RoBERTa, which could also be used for classification?
tensegrist 21 hours ago [-]
what is the…epistemic status, for lack of a better way to put it, of the probabilities? what do they mean? what (probabilistic) guarantees do we have about, say, the responses to
- is the capital of france paris?
- it is august. is it raining in paris?
(forgive the examples; they're probably not semantically the sort of thing jev is trained to work on. but i figure the point translates to various kinds of questions that come up in "inner loop of agentic pid controller" contexts)
a normal text-generating model if asked to produce a number will also do that just fine. i assume in jev's case it was actually rled to essentially learn to express priors over things using its implicit world model, which definitely ought to help, but can we say more?
saldubai 4 hours ago [-]
When we speak about good judgement in models and agents we are talking about humans skills such as critical thinking , judgement and decision making , emotional iq, mindfulness etc.
We have been building since 2018 a structured good judgement data lake tied to ten core humans skills and sub skills levelled queaisn and answers against blooms taxonomy from a tagged community of experts , outliers , contrarians . Open ended situational questions and answers like how the real world operates in various sectors to capture tacit knowledge . We have a playground with a tiny slice of just 100 of our over 500k base good judgement scenarios that can then be synthesized across sectors and workflows . With just a tiny slice it outperforms Fable and Open Ai models. Exceptional human judgement outperforms consistently models…. Good judgement is just also rare in us humans like common sense ( no pun intended) . Check us out at lovelyhumans.ai. Holler if curious. Sallyann Dellacasa on LinkedIn .
vatsachak 23 hours ago [-]
It could be used for coding if you gave it an AST.
If you work at TypeSafe please try this.
Side note: This is probably how LLMs would perform with better encoders and next-latent prediction, so eventually those will beat this architecture out. Still amazing though.
ramon156 23 hours ago [-]
I've implemented tree-sitter in pi before, and while it works, I have no real proof it saves me tokens, or is more accurate. I think a better implementation is a model that's trained for AST's, not just "use tool, see what happens".
I'd love to do research on this when I have the time.
vatsachak 22 hours ago [-]
Cool project!
That's what I was insinuating through "better encoder"; the model creating more efficient representations of ASTs using something like JEPA
CompleteSkeptic 22 hours ago [-]
the hard part for coding is actually state engineering (e.g. getting your dependencies in context) - we haven't even tried it yet (because my philosophy is we should automate the easy tasks before the hard and we've been working on getting the model smart on the former)
we do think there's a lot of potential though and do want coding themed releases soon
hunterbrooks 21 hours ago [-]
I could see Jev being great at finding key symbols in codebase before a code generation/code review task. I sent you guys an email (to hello@) about using Jev in Code Review for www.ellipsis.dev.
Escapado 22 hours ago [-]
I saw the CEO reply elsewhere in the comments to some other question. Maybe he can shed some light on it. My gut feeling is that this is non-trivial and they did not get this to work (yet?), otherwise I can’t come up with a good reason as to why they would not demo that as I assume half of the crowd here (myself included) would line up as customers.
vatsachak 22 hours ago [-]
Yeah it would be quite trivial to try and implement an auto regressive AST generator for STLC with Jev provided that you had bounded variable names and integers.
As you said, if it worked, they would have demoed it haha
8note 18 hours ago [-]
im not seeing it.
youd ask it to pick a location on the ast to add something from the grammar?
i dont see how this stays confined well enough? make a new output space every time? does that end up auto-regressive?
postalcoder 20 hours ago [-]
This has the potential to be huge for computer use.
OpenAI has been teasing how fast computer use is with their models running on Cerebras chips but the difference here is a burning hole in your pocket.
Gecko4072 13 hours ago [-]
Like which elements to select? Similar to the doom and wikipedia runs?
postalcoder 10 hours ago [-]
Yeah. Computer use is essentially a model navigating the OS-provided accessibility tree. I imagine a model trained on it would operate the computer exactly as we saw it control Doom.
Wow, this is really cool. If this holds up to scrutiny, and has a decent context window (+16k), it suddenly changes our project's status from "cool concept, too slow and expensive to release" to "doable", just like that.
Just joined the waitlist, excited to try it out!
mercat 14 hours ago [-]
afaik Jev's context window is 32k
vopi 16 hours ago [-]
This is actually pretty cool. I think the undertalked about part of this for TypeSafe is that they can always "extract"/distill the frontier of this type of task from the newest LLMs for cheap. Jev seems seems to be GPT-6-Astra/Fable 5.1 but I imagine a bunch of training data is from earlier models?
Then, you can serve it faster/cheaper than the frontier LLMs. It's basically distilling a small but extremely common use-case from LLMs and serving it. Then RLCD comes into play to update weights when a new model comes out, etc.
Any thoughts on what the next potential "cheap" win to be distilled from frontier LLMs is? I'm going to need to play around with this.
spacedoutman 5 hours ago [-]
The fact this isn't open-source is troublesome.
Such large advances shouldn't be locked up away from local hardware.
aslkalska 5 hours ago [-]
Honestly all AI research should be open source but that's just a dream
wg0 14 hours ago [-]
Can I put it as Air Traffic Controller? With similar error rates as humans?
That would be the litmus test.
"Does not hallucinate" is not the same as "is never wrong".
So the ATC test could be the benchmark.
copperx 14 hours ago [-]
Not hallucinating is easy when you don't produce strings.
vintermann 11 hours ago [-]
Hallucinating as we use the word really only applies to generative AI. Non generative AIs can't hallucinate, they can just be wrong.
6 hours ago [-]
ymir_e 8 hours ago [-]
I was previously working on LLMs to extract key info from data rooms for energy assets, and this looks great for that use case.
"Does this contract contain ____?" is a pretty typical query for many industries, and then you can have follow up questions that nest down into further info about X, Y or Z thing.
Looks really good for that use case, especially with certainty as part of the output, as you could flag things that didn't have high enough of a certainty to human review.
I'm sure legora and the other legal AI tech softwares are all over this.
Kurtz79 6 hours ago [-]
I was going to ask if the inspiration for System One name came from Kahneman's and Tversky's research, then I read the FAQ.
I listened to "Thinking fast and slow" recently and I was surprised how closely in behavior a LLM approaches the "System One" as defined there.
I approve of the clever branding!
mixtureoftakes 21 hours ago [-]
Doom demo is beyond impressive, even scary
bigglebear 17 hours ago [-]
It's very misleading. If I'm actually playing a game I don't get the coordinates of enemies sent back to me so that I can feed into my mouse to snap my crosshair to. It's looking through walls too, because it's working off structured state in text form. You could re-create this whole demo without using AI. Have an LLM generate the state machine for you and no model is required to run it.
someguynamedq 8 hours ago [-]
The impressive part is that it is low latency enough to serve high quality answers at game speed through the model instead of a pre generated ad-hoc machine.
padolsey 11 hours ago [-]
I think what this shows is how important branding and comms are. They've captured imaginations with their demos and nomenclature, despite the arguably non-novel architecture. One forward pass, read the embedding space, train some regressors on predicate structure, [??]
nickstinemates 18 hours ago [-]
We've already started using it for some pretty powerful decision tree stuff. We're just scratching the surface. We shipped an extension for Swamp[1] a few minutes ago and the combination is great!
The one downside is that the context window is very small (32k.) So some initial ideas we had for initial evaluation of code reviews won't fit yet in the window.
They never show exactly how they use it? Only a bunch of animations of it 'working'. Would like to see the actual code used for the demos!
zenlikethat 21 hours ago [-]
It's a bit hastily put together, but I made a dspy fork where you can add a decorator to automatically use TypeSafe where possible on Signatures. It shows a fair bit of what actual, hands on usage looks like.
DSPy seems like the right comparison and this is the first comment I've seen mentioning it.
Thanks for putting this together. I'm surprised the cost saving is so little though. I expected much more based on the post.
ricardobeat 23 hours ago [-]
The doom demo shows the program state / query.
singularity2001 9 hours ago [-]
How is that different from machine learning 101 "regression"? And why don't they just put a regression or softmax head on top of a trained transformer? (or do they?)
sva_ 4 hours ago [-]
Someone else noticed the base64 encoded block on the launch site[0]? Its the fast inverse square root algorithm q_rsqrt[1]. I guess its meant as a joke to put this algorithm that makes use of type punning on a site called typesafe.ai? Or maybe because of its efficiency?
This is too much for me. ML playing doom was a thing since before LLMs, decisions tree were always insanely and no one ever used then anyway, i can't see anything new in this yet everyone is treating this as a revolution. This technology was always there and quite easily accessible all along.
freddex 6 hours ago [-]
Very cool, I immediately jumped on the waitlist and shared this with my co-founders. Any plans for offering this through a European provider at some point after launching in the US? We work in EdTech, so non-EU-sovereign solutions are a harder sell to our customers.
samayashar 8 hours ago [-]
Amazing work by the team! Looks like they've traded accuracy for speed and this is most likely going to be the case with the next class of models.
This is a valid tradeoff for one-off responses but if we're dealing with a distributed system (eg: Kafka), then only the high-confidence responses (>0.8) should move forward as input to the next service. If a low confidence output is propagated, then it can break the entire chain.
edot 18 hours ago [-]
Very cool! Can you explain when I would use this vs. training a standard ML model on my data? Suppose I had a fraud dataset with features like customer ID, amount, merchant, online or in-person, etc. - I can't imagine that a general model like Jev would predict this more accurately or cheaply than even a basic XGBoost model trained on my dataset (one that I could build in a few minutes by asking Codex to build it). Where does Jev add value here?
hangrymoon01 11 hours ago [-]
you will need to collect data for every decision/usecase and then train a model. But this can be used for different use cases with just a prompt.
pasting it here:
zero-shot + general == programmable
I would assume any extreme scale narrow task could then be fine-tuned for, but we'll see - I suspect putting it all in shared cognitive core has bit maintainability/generalization benefits
edot 6 hours ago [-]
Thanks. I do concede it’s very general but that is a double-edged sword. I don’t need a general fraud identification algorithm. I need an accurate one. If I have another classification task I’ll train another model for that task.
initsecret 23 hours ago [-]
> [others] Output tokens: ~5x more expensive than input tokens.
> [them] Output tokens: FREE (too cheap to meter).
I'm very confused by this.
varenc 17 hours ago [-]
The output tokens are just responses to your inputed questions and their probability. So relatively few output tokens. No unstructured text back in the response.
quotemstr 22 hours ago [-]
They're not doing autoregression, so all the outputs are computed in one big forward pass. Very cheap.
ambicapter 22 hours ago [-]
I think OP is confused about "others" vs "them".
initsecret 22 hours ago [-]
they’re talking about two totally different things, right?
someguynamedq 8 hours ago [-]
This is not confusing
CompleteSkeptic 22 hours ago [-]
it's our output tokens that are free (under the system one / jev column)
tylermarques 22 hours ago [-]
We had early access and found it to be pretty useful. Having a second form of verification, where you can ask multiple questions (in the form of Nouls) raised our confidence in the outputs of other models. [0] IMHO This type of model works incredibly well in concert with LLMs, not as a replacement.
Don't your numbers suggest DeepSeek V4.1 Flash, for $100 more, gets you to slightly better agreement?
tylermarques 18 minutes ago [-]
Yes they definitely do - not claiming it's a perfect solution, but as a V1 product it shows a lot of promise.
iforgotmypasswo 16 hours ago [-]
Could you use this to build a proactive memory formation and retrieval system for LLMs that runs lightning fast?
Last 32k of connect + Summary of current task: Did we learn something useful here (true/false)? What is the category to file it under? Then notify the LLM to file it away.
What class of memory might be useful here? Model gives probability to each item in the list. Short description of all memories ordered by tagged class is used in the next round. Are any of these memories useful in the current context, such that they will inform the model and help in its task (yes/no)?
I’m sure there’s some fine tuning to be had, but this sure seems like the basis for a substantially better proactive memory system that works around an existing LLM conversation.
If I’m understanding what this does and how this works (generic input, intelligent classification with probabilities, rapid and cheap), this is absolutely nuts.
9 hours ago [-]
yymir 3 hours ago [-]
This could be killer for ingame AI for grand strategy games like Victoria 3 or EU5
lkm0 10 hours ago [-]
One application that sounds pretty interesting would be the creation of wikidata pages for anything. Plug a topic/word/concept/historical event in, take a bunch of wikidata properties, rephrase them as questions with the choices being the existing property values. Then feed it to LLMs or something. Does that make them more reliable? Probably not.
YPCrumble 6 hours ago [-]
What are peoples' thoughts on whether a local version of Jev is possible? Having to call an API for something that's main benefit is speed is orthagonal to their ethos.
xynelius 20 hours ago [-]
The Doom demo looks impressive but was it a fine-tuned model? It's the difference between a cool demo and revolutionary tech.
copperx 19 hours ago [-]
Shouldn't self-driving be a piece of cake if it works this well for Doom? Or what am I missing?
pantelisk 18 hours ago [-]
I think the doom demo uses a text representation of the world and it's basically, "projectile coming your way" -> "Strafe". "Enemy ahead" -> "shoot. So it works well when spawned in a room of enemies (as we see in the video).
If self driving is red means stop, green means go, and stay in your lane - then it would work great, but having to actually think and test which maneuver is optimal for a given situation while weighting safety, road rules, random unexpected actions and getting to your destination, I think it's a much bigger problem. A bigger model specifically trained on that maybe would do great, but then the output is not the constraint anymore.
But I haven't tried the model, so I 'm just ballparking and could be very wrong.
hamishwhc 18 hours ago [-]
The model doesn't have image input capabilities (yet, it seems from the post), so for the Doom demo, a harness is extracting a bunch of structured information from the game (map layout, enemy locations, player ammo, health, etc) and providing it as a massive JSON blob to the model so it can make its decisions. This model _could_ be hooked up to make the decisions for a self-driving car, but it would need to be fed a structured blob of the situation around it, so all the computer vision problems of self-driving are still there. And that's before you get into the confidence and accuracy of this model.
jamilton 17 hours ago [-]
Driving is more complicated than Doom, and it doesn't look that great at Doom to me.
warpspin 21 hours ago [-]
Haven't seen any docs or so. Is this actually a general model, or does it need training on the the data set it answers? Finding it suspicious you never see some kind of prompt.
1. yes a general model
2. no training at all
3. but it is focused on "System 1" tasks (more human judgment, less math reasoning)
zenlikethat 21 hours ago [-]
It's very generalized. Can't wait until everyone can see it.
padolsey 12 hours ago [-]
I'd love to know if Jev is still fundamentally LLM-shaped in architecture. Like is it using a single forward pass with a learned readout over the predefined options (i.e. a discriminative head on a transformer, no decoding), or something else? I did similar things for zero-shot criterion-based classification using a 4B Qwen model but could not reach the level of intelligence they've got here. Tho speed/cheapness was similar.
virajk_31 13 hours ago [-]
Great to see something new..
However I don't understand how are they claiming zero hallucination, how does giving confidence score fix hallucination? or am I missing something here?
bjconlan 19 hours ago [-]
You know you're too old when you see the company name and think! Oh I wonder what Martin Odeskey , Jonas Bonér and co are up to. Wait, didn't they become lightbend... Altho this comment takes away from what these guys are doing which legitimately sounds interesting.
torginus 22 hours ago [-]
I was thinking about something similar (maybe) - generally speaking, embeddings for LLMs tend to learn real world concepts - things like 'fruit' or 'France' or 'city' as directions in embeddings.
But in things like programming, most concepts are abstract - 'if hungry eat an apple' in programming terms would look like
'if hunger > 50 {apples--; hunger-=30;}'
and compilers work with 'concept erasure' - to them, tokens (which are like llm tokens) look like
'if var1 > 50 {var2--;var1-=30}'.
They don't care about how these things map to real concepts. So all the embedding directions used to encode real-world concepts are just noise to LLMs when programming. This greatly reduces dimensionality and training costs. So does a token representation tuned for programming constructs, rather than natural language would probably have a more efficient encoding.
ta988 21 hours ago [-]
Current models go beyond the simple embedding because you start to encode groups of concepts in the context-aware part of the model (attention heads or any other method). So it is never simply words/tokens in isolation anymore.
22 hours ago [-]
Gecko4072 22 hours ago [-]
From the person in the video regarding issues with benchmarks in general, and for LLMs. Also their approach. Good article.
do you all see the use cases being similar to what you might use Fastino's Gliner models for? i see similar differentiation from general purpose LLMs in the sense that they can take natural-language input and return outputs adherent to a user-defined schema.
im thinking about how well Jev could be used to replace a current LLM-as-Judge evaluation workflows, specifically on chat transcript data (think ~1,500 tokens) i wonder if the reasoning usually required pushes it a bit out of scope. didnt see anything published about constraints on the state size, so would be curious to hear about that.
mary776 11 hours ago [-]
definitely seems like a modified version of GLiNER2 or 2.5:
- encoder-based (no text generation)
- multiple tasks in a single forward pass
- deterministic outputs
- constraint-based classification
dthedavid 17 hours ago [-]
Looks promising. I'm building an AI video editor and multi tool calls take >30s using Gemini. This would be a a game changer if Jev can take that down to single digits at p95.
paraschopra 13 hours ago [-]
I'm trying to understand what difference does this make over LLMs.
LLMs are universal simulators, their latents model the world. So I bet if you compare their logprobs with probabilities output by this model, it will be highly correlated.
Someone should do this quick experiment. I bet there won't be enough of a meaningful difference.
consumer451 17 hours ago [-]
Super cool! Instantly joined the waitlist.
It might be boring, but I can see exactly how I could use this right now to improve my agentic rag.[0] In two months I am supposed to deal with a giant corpus, while still maintaining responsive chat UX. I have been working my butt off to make our first big client happy. This could really help solve the chunk ranking problem.
[0] assuming the policies are compatible with sensitive production workloads, some time in the near future.
Otterly99 11 hours ago [-]
Always exciting to see people working on novel models, rather than the Nth version of the same slightly tweaked LLM.
I'm very curious how much ressources are needed to run such a model. This could be a complete game changer for local applications.
activehuman 17 hours ago [-]
I can see the value in this but looks like there's going to be trouble in communicating the difference between this and a regular LLM, and also proving the potential cost savings in using this to replace existing systems that are using LLMs with frameworks like langgraph, as this can't be a drop in replacement and would require a significant amount of re-architecting/reengineering of systems to get the type system to work
filearts 19 hours ago [-]
If we could come up with a system to classify the probabilities across a large number of candidate words (or components thereof) then this could actually be good at producing text, one element at a time. We could call these elements 'tokens' and picking the right one could be called something like 'decoding'. Crazy idea but hear me out...
On a more serious note, it will be fascinating to see how this different spin on modelling inference will create new paradigms or slot into existing ones.
preommr 21 hours ago [-]
This will be insane for tool usage, and probably where the major economics for day-to-day usage will be.
The goal is going to be to use llms to distill operations down to some dsl, and pass it into something like Jev.
mentalgear 11 hours ago [-]
Overall this seems like a classifier that gives weighted scores per custom labels. It's certainly useful, but whether it brings higher quality than an LLM in structured output mode has to be seen in objective benchmarks.
StevenWaterman 11 hours ago [-]
Zero shot classifier indeed. Reminiscent of asking an llm a yes/no question, constraining the output to either yes or no, and looking at the logits directly
And each question is a separate single token model completion done in parallel
aryehof 14 hours ago [-]
As a zero-shot classifier, I expect that effectiveness is dependent on the data trained upon.
Jev input … > Unstructured data (e.g. text) with an emphasis on structured program state.
What pre-training data/model is Jev based on? Surely result effectiveness is dependent (outside of one’s own input as “state”) on that?
Mentlo 21 hours ago [-]
Hm, would be good to understand the architecture better. Is this answering just from a world model informed prior? How informed is it by the information in the prompt? I can't see this maintaining calibration across all domains and all types of structured output.
Is there anything published on how it maintains calibration? Or when you say "outputs calibrated probabilities" you mean "as calibrated as frontier LLM models, just cheaper" - which is a different claim; as LLM's aren't particularly well calibrated
passive 18 hours ago [-]
While I understand that accelerating development isn't necessarily the target for this, and it's not at all intended to generate code the way many of us are...
I think this could be pretty decent in CI? There's a lot of "flakes" I've mediated that this could have handled much more efficiently. Maybe observability as well, triggering elevated logging and other initial measures?
2001zhaozhao 21 hours ago [-]
Hasn't there been a lot talk about Astra's opaque reasoning capabilities (being able to think through complex questions without using a chain of thought)?
Given that, can't you just replicate Jev by telling Astra "here is the question, you must make a multiple choice decision / output a score between 1-10, please answer directly in a single word, no reasoning allowed"?
(Edit: Ok, Jev is much cheaper in input tokens so these two aren't directly comparable at all)
CompleteSkeptic 20 hours ago [-]
the edit is right - jev would be cheaper, faster, and more self-consistent (in general)
we actually use astra (and fable) in this way for our evals: evals.typesafe.ai
someone on the team cooked hard on that and it shows example traces comparing our model to opus/sol
pixelmelt 22 hours ago [-]
Interesting concept, I can't see a reason to use a generalist classifier over an api rather then just training my own? If it was open weights I would probably mess around with it.
anentropic 7 hours ago [-]
So it's kind of like BERT but you don't have to train it for each request/response shape ?
hoppp 20 hours ago [-]
This is amazing. I really could use this.
I like the idea of System one models but all LLMs so far work as system 1 thinking because humans generate speech subconsciously with system 1.
System 2 thinking requires consciousness which AI does not have, so even reasoning models are still system 1 thinking as system 1 in humans has reasoning with heuristics.
Its limited but most people navigate the world with it completely, so it's enough for AI.
rock_artist 3 hours ago [-]
A little off-topic,
I have to admit, I did play Doom back in the 90s and I know it is just a game.
But is it just me or other folks feel uncanny seeing "a machine" playing Doom with low-latency as the first demo.
While I'm optimistic that humanity is good (but sometimes makes bad decisions), My first thought was seeing such models used by armies.
arbayi 9 hours ago [-]
I don't know if it's just me but comments under Twitter post felt like paid partnerships to me.
The idea sounds cool though
niutech 10 hours ago [-]
How does Jev compare with encoder language models like BERT/RoBERTa/DistilBERT, which could also be used for text classification?
petesergeant 22 hours ago [-]
This is basically a zero-shot classifier that can accept raw text (or structured text) as an input, and is able to classify that text as accurately (they claim) as a frontier-level LLM. I have workflows this would be useful for, looking forward to it showing up on OpenRouter.
CompleteSkeptic 22 hours ago [-]
exactly right!
whazor 21 hours ago [-]
A question I have, with the type { output: string }, would the model not become a LLM? And if it does, shouldn’t it cost the same as a LLM for output?
speedping 21 hours ago [-]
I don’t see this as an option in their website
You could theoretically ask “what is the next appropriate character?” and add the entire ascii charset but i doubt it’d work well and you’d be implementing autoregressive churn across network latency…
CompleteSkeptic 21 hours ago [-]
strings (and all sequential data structures) are not allowed at all - this is how we make sure all outputs can be computed in parallel (thus no output token cost)
Imanari 12 hours ago [-]
Seems like LLM can do everything Jev can do (just structured outputs?) but Jev is highly optimized and purpose built for it and thus way faster and cheaper. Is that a fair description?
johnecheck 21 hours ago [-]
This makes me think of Expressions of Change [1], a project that aimed to make updates to a program a first-class primitive in a programming language. A model like this can't output code directly, but perhaps it would be well suited to select from the small set of discrete operations on code envisioned by the EoC author?
[1]: www.expressionsofchange.org
rattray 7 hours ago [-]
Super cool. Does it, or will it, work with image, audio, or video input?
moffers 22 hours ago [-]
So is it a structured data-based language model? Or is there a model and a harness? Hopefully they’ll open up and explain more.
CompleteSkeptic 22 hours ago [-]
it is just a model, no harness yet ;)
it is a structured data model, but technically not a language model (it doesn't generate language)
15 hours ago [-]
10 hours ago [-]
cooljoseph 19 hours ago [-]
A few questions:
1. Do you provide any kind of largest common subtree caching for cheaper input?
2. Have you tried auto-generating Lisp programs structurally?
3. Have you tried augmenting a Lisp language with a `choice` function that makes choices given a prompt, the environment, and the continuation stack?
zenlikethat 18 hours ago [-]
(1) Nope, it's always the same input token cost
(2-3) No, but that's kind of a sick cook ... Want to get access and try it? nathan@typesafe.ai
cooljoseph 17 hours ago [-]
Thanks for the early access! I was testing the Lisp idea out in the playground, but I don't think the model is smart enough right now to generate actual code. I tried having Jev finish generating the code for a Fibonacci number function, but it kept wanting to create a literal number instead of refer to a variable which is a number. This happened both when I gave Jev the current program as a string and when I gave Jev the program as structured data.
Maybe I'm just not doing a very good job at prompting Jev, but I think right now it's not quite capable enough to generate Lisp code.
Signed up for the beta! :) would love to put this through some real-world shootouts against traditional LLMs to see where this type of model really excels.
I’m guessing it might be able to replace maybe 40-70% of LLM calls for a given pipeline depending on the business task, cutting the API costs on those calls by an order of magnitude.
faizshah 15 hours ago [-]
I think I missed why is this faster? What I’m reading here is it’s similar to constrained decoding but I’m not seeing the explanation of why it’s able to get those results.
theredsix 22 hours ago [-]
Congrats on the launch! What's different between Jev and Microsoft's Guidance package? https://github.com/guidance-ai/guidance Is it a diffusion generator under the hood?
2001zhaozhao 21 hours ago [-]
Funny how the authors are asserting that "doing the right task > data > compute > algorithms" while simultaneously releasing AI model for calibrated decision making, which if they work, would mean that "compute > doing the right task"
sim04ful 23 hours ago [-]
This sort of stuff almost sends shivers down my spine, it's like i'm looking 5 years into the future.
zenlikethat 21 hours ago [-]
join the discord! we love forward thinkers
darpa_hr 22 hours ago [-]
There was no "AI Winter"
someguynamedq 8 hours ago [-]
"can't hallucinate" feels like some word game Olympics
adroitboss 22 hours ago [-]
I am positive I know exactly how this works, I made something similar a few months back. But the problem is without generation you are extremely limited in the use cases. And while the model can't hallucinate, it can still be wrong. It just can't make up data.
cooljoseph 16 hours ago [-]
Last year I also had a rather similar idea, but dropped it before I went very far in working on it. I wonder if you and I had similar ideas?
1. Start with an LLM, so that your model understands natural language.
2. Replace RoPE with a tree embedding scheme, and causal attention with a sparse attention on the graph structure. (You could use full attention... but it's cheaper to use graph attention.)
3. Chop off the final unembedding layer, replacing it with a projection down to two scalars, one for logits and one for confidence.
4. Each option of a choice is represented by a number of tokens in leaf position; average these tokens' logit outputs to get the option's logit. Average all of the confidences from all of the options to get the choice's confidence.
5. Train the logits by KL divergence from a true distribution (or NLL on samples from a true distribution).
6. Train the confidences on a subset of the data in which you know the entire true distribution.
The hardest part is getting real world data for workflows, but I wildly speculate that you can get by with only ~50,000 documents if you first adapt domains using synthetic data.
StevenWaterman 11 hours ago [-]
Yeah saying it can't hallucinate is crazy. It can still forward a billing query to the dev department incorrectly. It can still get an obvious yes/no question completely wrong
dennisy 22 hours ago [-]
Are you able to share how it works in that case?
adroitboss 22 hours ago [-]
I'll tell you this. Output isn't too cheap to meter, there is no decoder.
krackers 21 hours ago [-]
So an encoder-only model with a classifier trained on the heads or something? DeepSeek recently switched to an encoder-decoder architecture in an attempt to get the best of both worlds (fast prefill while preserving generation capability), I wonder if that might be the future?
That was the first thing that come into my head. OK I can train very simple model, that can generate json's for specific tasks, so what?
How we can be sure that this "limited use cases" not just overfitting for particular outputs (or even distillation?)
Except this, this thing looks like revolution.
scottyah 23 hours ago [-]
Wild that it doesn't generate text. I wonder how its technology compares to Tesla's FSD stack.
jceg 22 hours ago [-]
> We deliberately chose not to publish performance against public benchmarks. In fact, we plan to only have one-off evals when we make product updates.
lol, I bet they would publish them if their score on those benchmarks were good.
nelaggy 14 hours ago [-]
insane doom demo
i wonder what the limits of its intelligence are? i'm guessing it's not great at reasoning tasks, it seems breaking down the problem helps significantly, but how much does a problem need to be broken down for reliable performance?
also this would be huge if it could run locally but it seems like there's no intention to do that at the moment
sreekanth850 11 hours ago [-]
This is best thing to use for decision making, evaluation, classification. If I'm not wrong.
Imanari 11 hours ago [-]
> AI Map Reduce over Big Data
> Search for relevant information over giant corpuses
Do you mean as an alternative to embeddings?
tidewave 22 hours ago [-]
Congrats on the release!
Finetuning a language model for decision classification (with probabilities) is already well-understood. What specifically changes in the training objective with RLCD? Are its benefits isolated from Jev’s new architecture/parallelism?
findjashua 20 hours ago [-]
Would it be fair to say that this is tailored for tool-selection subagents?
kroaton 9 hours ago [-]
Seems that way.
sourcecodeplz 8 hours ago [-]
this reminds me of laravel boost, which does something similar.
it can generate classes/models/routers etc via tool calls, doesnt write the actual code.
strich 17 hours ago [-]
Huh this looks fantastic. The Doom demo really sold for me that this could be a great tool for accelerating QA at my gamedev studio. Signed up for early access.
Havoc 22 hours ago [-]
Will need hands on to truly tell, but the doom demo seems very promising. If it can play that with text descriptions of where stuff is by distance and degrees in a 3D context then many GUI automation tasks should be easily doable
11 hours ago [-]
gok 22 hours ago [-]
So... a classifier model?
mackross 7 hours ago [-]
Can’t wait to use this. Amazing work.
mmastrac 17 hours ago [-]
Is this a Markov/Diffusion model with some sort of external Engram memory? If so, this could be extremely interesting.
Ozzie_osman 12 hours ago [-]
This is quite the paradigm shift. Can't wait to get my hands on it.
dozerly 17 hours ago [-]
Very cool. LLMs have been borderline unusable as functions for the longest time, very excited for this direction.
entrep 22 hours ago [-]
This puts the human even more out of the loop I'll guess?
zenlikethat 21 hours ago [-]
That's kinda the goal. Imagine all the automation in the world being able to embed intelligence directly inside it - factories could route based on more complicated questions, hardware could anticipate your needs. Customer support could be done without humans 90% of the time.
altcognito 18 hours ago [-]
If it is so cheap, why such a limited release?
bthornbury 22 hours ago [-]
Is the tradeoff of the parallel output that we don't get arbitrary string generation? like output # of tokens is fixed ahead of time?
Either way, really cool and impressive.
zenlikethat 21 hours ago [-]
Yeah, it doesn't output strings, just decisions/answers.
copperx 19 hours ago [-]
Non-hallucinated ones at that.
bananaflag 22 hours ago [-]
Funny how it can do everything but not chat. Sort of how when I was a kid I thought of a medicine that could cure any disease except the common cold.
iamgopal 18 hours ago [-]
If I understand correctly, it can play chess and rubic cube better than LLM ? ( may be go too ? )
iamgopal 15 hours ago [-]
ok I've searched, it may not, but it can do "driving car" and "trade 0-DTE Option" much better.
totallygeeky 22 hours ago [-]
Woof, that page is hard to read. I don't understand what they've done to the way text is rendering but it's not great for my eyes.
phenomen 22 hours ago [-]
If you zoom in (especially on the large title), you'll see that the text is a semi-transparent gray with a black internal outline. It seems like all the typography is SVG-rendered. Actually insane. I've never seen this before. Not even the most vibeslopped websites have that.
agos 11 hours ago [-]
I don't know if they changed it since your comment, but it's all just text to me
16 hours ago [-]
bqsile 19 hours ago [-]
If it work as good as they say it does, confidence score + really fast response when you want very fast response, basically..
To me it is a crime against humanity to not open source it.
Just get the money from cloud inference and cloud agentic sessions or whatever but open source it.
This tech, a good harness, a good model provider, and you have basically a AGI building machine.
bqsile 19 hours ago [-]
Golem, if you read this, add me on battle.net (europe) Sansviande#2540 and let's talk. Give me 1 minute.
Invictus0 19 hours ago [-]
“Crime against humanity” buddy please
omeid2 15 hours ago [-]
Can HN have a tag for open-weight vs closed-source models please? The progress is nice, but if it is not released at least in papers or open-weight? These are just ads?
22 hours ago [-]
_davide_ 22 hours ago [-]
What's the difference compared to just taking an embedding and feed forward a simple net trained for the task?
woggy 20 hours ago [-]
Can this be used in conjunction with a text-generating LLM for better quality code generation?
respectattentio 15 hours ago [-]
Seems like "some" of LLMs tasks are now Jev tasks.
nightshift1 18 hours ago [-]
The whole page reads like it was vibe-written by an AI. If I'd built something as disruptive as this claims to be, I'd have spent at least fifteen minutes writing the announcement myself.
Every time I see 'we' in an announcement like this, I picture one guy alone in his basement.
CompleteSkeptic 18 hours ago [-]
unfortunately all hand-written :( my chief-of-staff does unironically handwrite em dashes though
lwansbrough 18 hours ago [-]
For what it’s worth, I didn’t get that impression, and even noticed a couple typos ;)
nojvek 16 hours ago [-]
I really appreciated the hand-written release. Thank you.
pennomi 23 hours ago [-]
> Extraordinary claims require extraordinary evidence so see below for the receipts.
Yes, that’s the kind of attitude I want to see in these model releases
ramon156 23 hours ago [-]
But the evidence is not there...
pennomi 22 hours ago [-]
Indeed, they talk as skeptics but don’t offer a ton of evidence, other than a couple videos of demos. A live demo would be far more convincing.
simianwords 22 hours ago [-]
They gesture at not using benchmarks for some reason...
But also effectively this is a classification model. It excels at specific certain types of workloads, and obviously will fail at others. Not really sure how one benchmarks this tbf. I can see their argument on why this requires a novel specific eval for whatever your usecase is. A consistent "global" benchmark might be hard to do
Wazzymandias 17 hours ago [-]
This looks and feels a lot like productionized conformal prediction
elcomet 21 hours ago [-]
The technology and the results are very handwavy. What is RLCD exactly ? What are scores on benchmarks compared to LLMs ?
This website does not inspire confidence at all, it all sounds like a marketing piece. I wish it was true, some kind of text-prompted classifier with LLM performance would be cool, but I can't trust it with what we are given.
hi_hi 22 hours ago [-]
If I’m understanding correctly, this will work well for self driving cars?
darksaints 22 hours ago [-]
Okay, so it doesn't output text, that much is understood. What are the inputs like? I'm assuming maybe a text input? maybe an AST definition? Really hard to tell how this works at all from the demos, especially since we can't really try it out.
CompleteSkeptic 21 hours ago [-]
inputs are structured program state. there is an example at around second 30 of the doom demo
(though ideally everyone gets off the waitlist and can try it out for themselves )
andai 22 hours ago [-]
Why did they pick the name System One? It's not really explained what "System One tasks" and "System One shaped queries" are. Things that need a fast response?
Does this imply it's a very small model? I couldn't find anything about the model itself.
Bingo. It's a Psychology term for the part of our brain that reacts instinctively rather than thoughtfully and logically
zmmmmm 21 hours ago [-]
The eval is baffling me
> we assume there is a correct compute graph (a “workflow” represented in code) and use the predictions of the largest, smartest, and most expensive external models as reference probabilities.
...
Rephrased: every model gets the same workflow. We test how they compare to the average of the smartest models (in this case, Astra and Fable).
They assume there is a correct graph, but they don't compare to that, they compare to the average of the smarts models? So the smartest models are getting it wrong but you compare that anyway as a benchmark? So the outcome is "how much of a Fable am I getting" etc. Why not compare the actually correct thing?
But then even on this hand constructed eval, the first plot is showing Jev at less than Sonnet 5 accuracy. It is barely better than Luna. There are two Opus 5's and two Sonnet 5's without explanation. What is the plot showing?
I gave up.
_boffin_ 18 hours ago [-]
Any relation / inspiration to GLiClass?
bfeynman 23 hours ago [-]
Super intrigued by this - large scale automation using LLMs is quite annoying due to deprecation cycles of models from frontier labs and cost of running your own being prohibitive when you have a blend of them.
pama 22 hours ago [-]
Is there a downloadable technical report somewhere?
freshnode 13 hours ago [-]
I like it. What is it?
seinecle 22 hours ago [-]
Can this be used in practice to write code?
zenlikethat 21 hours ago [-]
Not in the traditional sense of a coding agent, but we think there's a ton of opportunity in using it for context management ("do we _really_ need to pass all these tokens to the agent?"), semantic linting ("how does this score against this AGENTS.md: <...>"), etc.
hunterbrooks 23 hours ago [-]
um what is going on with the outfit changes in the launch video...
This is potentially huge and can crash the Big Two's stock prices or block their IPOs completely.
anshumankmr 11 hours ago [-]
a) Is this available on Bedrock?
b) Does it support structured outputs?
c) What about trying it out?
erichocean 23 hours ago [-]
I could put this to use today.
I think we'll see a bunch of different architectures over the next five years.
bilsbie 22 hours ago [-]
I’m not understanding what this is. It’s a faster cheaper LLM?
hspeiser 21 hours ago [-]
this might finally be smart enough and fast enough for jarvis. hard to feel like iron man when your assistant takes 8 seconds to decide to pause your music
hmartin 14 hours ago [-]
Am I the only one struggling to parse the distinction System One (the system/harness?) and Jev (the model?)?
kroaton 9 hours ago [-]
System One seems to be more of a "class" of models, as it's a good classifier but can't do what traditional LLMs with chain of thought do.
poly2it 20 hours ago [-]
Is there a bottleneck which would hinder putting this architecture in charge of a humanoid? Would it be able to operate continuously, for example in conjunction with an LLM for long-term reasoning? Doom seemingly works extremely well.
8note 18 hours ago [-]
defining the workflow such that the operation is a set of relevant questions
yieldcrv 22 hours ago [-]
oooooh it can play Doom!
forget LLM benchmaxxing sidequests, I'm sold on the real benchmark
charcircuit 22 hours ago [-]
Parallel inference where you don't want a subagent seems niche. But there is a lot of random things where businesses ultimately want some kind of score instead of generating something.
I think the interesting thing would be seeing if prompt injections still work with this kind of model.
CompleteSkeptic 22 hours ago [-]
we have played with this! the fascinating thing we've found so far is that adversarial examples for our model are quite different from that of LLMs so that they work even better together
whalesalad 23 hours ago [-]
What is it about the rendering of this page that is so... off? It almost looks like the entire thing is a <canvas> element.
edit: looks like a framer export where there is a text stroke being applied :|
esafak 23 hours ago [-]
Looks like a great model for NLP.
kobe_bryant 16 hours ago [-]
my sons name is also Jev
yieldcrv 22 hours ago [-]
this is interesting, so not an LLM but can be used in these use cases that LLM's have been shoehorned into
"While Jev gives up string generation, it’s optimized for structured outputs and can’t hallucinate"
Ouh! Any open weights models that can do this yet?? If not, how much longer? I have a Mac Studio coming soon.
kypro 22 hours ago [-]
> Outputs
> LLMS
> Strings / generated text. Strings are flexible and can be anything: chat responses, code, hallucinations, refusals, or even type-safe structured values. To be used by software, responses need to be parsed + validated. There is also always some risk that the AI goes off the rails.
> Jev
> Type-safe structured values. Possible outputs and structure are defined in advance. The model never makes type errors. All answers are accompanied with calibrated probabilities and confidence scores.
I mean, this isn't even remotely comparable to LLMs so why compare? Also, why are they bringing up AGI given there approach is so restrictive that what they're building literally cannot have the creativity required for AGI? The video is 100% marketing slop...
The bulk of the application of LLMs is that they generate reasonably reliable text which doesn't need to be defined in advanced. I'm sure there is a niche for this and congrats to the team, but please let's not hype this as if it's the next big thing in AI...
bqsile 17 hours ago [-]
Creativity is not required for AGI, that's maybe the only thing that is not required for AGI actually.
What a sad world would you live in if you don't keep creativity for the humans.
mkrishnan 22 hours ago [-]
If this is true means, AI Stock bubble burst. (For good)
quotemstr 22 hours ago [-]
It looks like a specialized encoder-only(-ish) transformer with scalar and ordinal output heads. Acausal in effect, maybe? Probably not even autoregressive?
I'd use this as a tool an LLM can use for specialized tasks. It's not AI in itself.
claud_ia 9 hours ago [-]
[flagged]
mlcruz 20 hours ago [-]
[dead]
agnishom 8 hours ago [-]
TLDR: Like an LLM, the input is a string, but the output is not a completion. The output is a ranking of elements from a certain enum (e.g, [Yes/No], [A/B/C/D]). They use a technique called Reinforcement Learning for Calibrated Decisions (RLCD) instead of RLHF. Also, inference is a lot faster.
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
llm hallucinations are high probability tokens that are incorrect vs the real world
I would be happy enough with: only produces what it can verify with sources.
If you eg try to remember a court case (ie produce the reference via LLM token generation only), it's easy enough to check with your data whether it really exists. Similar for following links and other references.
If your data or sources are wrong, obviously your report about them will be wrong. But I wouldn't call that a hallucination.
Humans are known to hallucinate a lot. Ask 10 different witnesses at a crime scene what they saw and they'll all report different things.
A good, non-hallucinating LLM would only report things for which it has evidence. It would consult the facts every single time.
It's a pain in the butt for humans to fact-check everything but LLMs can quickly look up all kinds of stuff. That's what makes them useful.
So you can bolt the fact-check / source-check pass onto whatever other system you have, without having to redesign the underlying system.
And humans also behave differently in different contexts. A conversation at the pub has more such hallucinations than a formal deposit in court. For the latter, a good lawyer will look at her shoes, when you ask him what colour her laces are.
A calculator either gets the right answer or doesn’t answer.
It wouldn’t have to be all knowing as long as it knew perfectly what it doesn’t know
System One says they use RLCD, Reinforcement Learning for Calibrated Decisions, which presumably has accurate probabilities as an explicit optimisation goal.
System One hasn't said how RLCD works, but they do say it is explicitly training models to output "calibrated" probabilities, which makes it distinct from RLVR. This is how they describe it:
> System One models are trained for calibrated decisions: their probabilities are optimized against outcomes to reflect uncertainty.
"What are the odds a thunder will strike in Paris at 1pm UTC of 2026-09-16" - that could be a 0.001 chance going from blind historical measurements; 0.01 if it's raining; or 1 or 1 after the date has passed.
I believe their "accuracy" metric (sonnet 5 level) is where "right/wrong" is measured.
People out there are so resigned to the models being unreliable that they are really doing things like hallucinating deliberately, and then matching the hallucinations to embeddings -
https://softwaredoug.com/blog/2026/08/10/hypothetical-classi...
You could do that or you could just... use a model that will never produce unreliable outputs in the first place.
Even non-frontier models are absurdly good at this in a broad sense.
Which would make it hard to judge "a model that will never produce unreliable outputs in the first place" against something that is already really, really good and exceptional in domain-specific areas with the tiniest amount of elbow grease.
Speed and cost look good though (for now)!
For all we know this might be a non-language-generative transformer e.g. a transformer where the decoder produces confidence scores rather than language. Please provide more likely architectures if you know them, I'm genuinely curious.
So if the generated schema is for a tool call for calculator, then the numbers will be valid numbers for sure (and not random words).
To me, it looks similar to BNF schema already introduced and implemented few years ago: generally speaking - it limits the next token that is allowed to be generated, probs are drawn from a subset tokens.
(tbh, I'm not sure why it didn't pick up as a more standard interface to LLMs, as it made a lot of sense back then, and now.)
There's also a cheeky "one of the models hallucinated a link" in the wiki jump example that most likely could have been avoided by properly using grammars. You can setup constrained gen so that only valid options (say from a list) can be outputted. Their own inference lib likely does that. So comparing to one that doesn't is a bit cheeky.
That being said, after a brief look at the site I could see this working. Especially if this can be ran locally, the speed and cost can enable some workflows where you have this as an "overseer" layer over say a cli agent. After each step you run through a list of "questions" ("is the task completed?" -> yes -> "does the edit touch files it shouldn't" / "does the edit follow our code writing policies") etc.
edit: extra points if the "question" rubric is also generated by a higher abstraction model. Say "/goal Build out auth" -> generate_rubrics(goal) -> "Is auth implemented on all endpoints" / "Has code touched anything else than auth" / "is this following the best practices" / ...
And if you buy their RLCD claims, this might be even better than huge models that know a bunch of irrelevant things.
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm not the gatekeeper of who gets to call themselves a frontier model, but I don't think most people would count Jev in that group. It sounds false.
If their specific claims hold up, then it would make more sense to say something like:
"Advanced the speed/cost frontier for structured decisions"
I think AlphaFold and related are also frontier models.
Being an LLM does not seem like the qualifier for frontier.
I mean, imagine comparing this thing to Astra, it's hilarious. They don't even tell you what the max input size is, and they only allow 10 possible answers to choose from for the Choice mode. It's probably like a 1billion param model. They say it's "not small", but there's zero reason to believe that.
I suspect someone will be able to recreate this within a week by piecing together open-weight models.
Frontier LLMs are expensive jack of all trades. You can absolutely compare them to purpose-built tools on any domain they touch. Engineering is all about assessing tradeoffs.
It's therefore misleading because they make people believe they're as general purpose as LLMs.
The Doom demo seems very funny business. They're not feeding it video, they're feeding it a text description of what's going on in the game. It's not reading pixel data.
I think LLMs would play a lot better with that input too but Jev does seem to have a huge speed advantage; I don't know if the other models could do that in real-time.
So no, the other models cannot play in real time. Hence why this is interesting.
> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do
The original title before it changed less than an hour ago was:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm going to agree that was misleading.
And on the second point:
>>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
>that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do"
Also going to disagree here, and I don't think it's semantics.
Type safety is not factual correctness.
I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?
IMO your system can make mistakes that are similar in spirit to hallucination (i.e. answering with a false answer instead of abstaining to answer).
You’ve already agreed that this doesn’t establish correctness. An approve for an unauthorized action still meets the schema guarantee.
That’s why I find the messaging misleading. You’re acknowledging the limitations in these replies while defending the broader reliability pitch.
Even granting that each answer is calibrated individually, that doesn’t establish calibration of the decision that combines them.
Sure, I can threshold a composite score, but there may be many wrong answers with the same score. An unauthorized action doesn’t become acceptable because it scores highly on the other dimensions.
I still have to define the constraints and test which wrong actions get through the complete workflow on my own data. That’s a substantial part of the work being pushed back onto the developer.
From the intro blog
"Hallucination and type-safety are intrinsically related"
I'm not entirely sure why we're conflating type safety with, I guess, value or output safety.
"Would you say a linear classifier hallucinates?"
No, but it can be (and often is) mathematically correct and functionally incorrect. It doesn't help to say "a linear classifier can't hallucinate" when you get even 99% accuracy. That's 100% a semantic play, and it doesn't help when the picture of a dog is labeled cat and the response is "yeah but that's not a hallucination, only stupid LLMs do that"
Model input: "Does the user want to speak to a human support agent?"
Output: Yes.
I imagine that your model would produce this, and I think it's fair to say this is a hallucination. A human would caveat it with: "Yes, but not right now.", your model is incapable of that. Yes is technically correct, but within the context of being in a live chat, a human would understand that the caveat is required.
Your question would correctly classify the user's input as requesting a human support agent, but at an indeterminate time.
If you wanted to determine whether the user wants to speak to a human support agent immediately, you would have to correctly qualify your question, e.g. "Does the user want to speak to a human support agent now?". You could have another question which is "Is the user requesting a call-back from a human support agent?". Or you could have a multiple choice query which would filter the conversation into one of a number of pre-written possibilities.
This is nothing to do with accuracy or hallucination. It's a different method of interacting with the model where you are relied upon to be precise.
linear regressions hallucinate in the simpson's paradox.
the model output can be quite confident and not representative of reality
- Give Jev and LLM the same input
- Lock down both to approved/rejected/unknown (LLM restricts on decoding)
- Both can be wrong, but neither can hallucinate (invent an another option).
For e.g. classification tasks, even in 2026 people are doing things like hallucinating deliberately, and then matching the hallucinations to embeddings -
https://softwaredoug.com/blog/2026/08/10/hypothetical-classi...
With TypeSafe it just picks the class (actually probabilities across classes), reliably every single time.
(To be clear, this one raw model does not support outputing a full AST directly - it wants to output "choice" among fixed options, "score" on a sliding scale, or a true/false answer (all of these with confidence scores attached), so building the AST/structure would be a code-driven (or even perhaps outside LLM-driven in some more challenging cases) multi-step affair where the model would essentially be playing a "game" of building the structured output step by step and getting a revised partial state back. But one could expect this to lead to interesting results.)
- @CompleteSkeptic
Very strange.
My first thought was that it would be ideal for robotics? As in control of limbs, general planning, route finding, etc.
Is this true? Code is structured output. At the very least it seems like a question of degree rather than kind.
While the LLMs we're using today are limited to sequenced text, it seems that a model like Jev could excel at coding on a more structural level (factoring, controls) by working within the constraints of an actual language specification and supplemental domain model. I don't know, though -- maybe that's too deep and complex.
I foresee this to be the path moving forward - giving AI models understanding of the computation directly(as well as compositional rules) This feels like a short path towards total software in many areas.
The comparison between LLM speed and Jev speed is misleading, because they're using autoregression to generate all of the type names, all of the schema, etc. A closer comparison would be if the LLM was purely outputting the raw numbers. Even then, comparisons to LLMs are pointless because you could train a transformer on the same sort of task that Jev is doing and get even better performance yet again, and a smaller model. I suspect this is some form of stripped down diffusion language model.
You really have to do a lot of hand holding here, and map out your problem space manually, and very carefully, to get any sort of accuracy. For example:
> Keep each Score to one dimension. If a description says “punctual and smart and experienced”, the question is measuring three things, and an input that is high on one and low on another can’t be placed. Confidence drops and the score means less. Split it into one Score per thing and combine them in code
If you don't perfectly represent the distributions of possible answers then you'll likely get garbage results. As far as probabilistic state machines are concerned, I'd say creating the distributions of possible answers, and their hierarchy, is the actual hard part.
One of their examples is:
- "state": "I have asked three times now. Can I please just talk to a real person?"
- "Is the customer asking for a human agent?"
Imagine the users request is: "I want your human agent to call me tomorrow at 5pm."
Human conversation is fuzzy, getting useful reliable results out of this is going to be a challenge. Of course, you could add follow up checks like: "Do they want that now, or later?" -> if later -> "Do they want that tomorrow, or the day after?" and so on... But now you're building an LLM out of if statements. I am skeptical of whether this model has much utility for fluid language interpretation - I suspect it'll only be useful for scenarios where you've tightly constrained the answer space but want to use fuzzy language to describe it. Like:
- Question to human: "Would you like a support agent RIGHT NOW?"
- Their response: Yes | Yeah | Mhmm | ye sure (any possible yes signal)
Model input: "Did they ask for a support agent?"
Still... a tiny LLM could accomplish this sort of thing without problem. And that doesn't stop someone from saying: "No, not right now. But tomorrow." - and the tomorrow would get missed. I think this is why people haven't really tried this approach much already.
Also their Doom demo is on structured state, not on images. Meaning, the enemies must be being served to the model as coordinates (or the exact angle of projectiles that hit the player), otherwise it'd have to scan every pixel of the 360 degrees to know whether an enemy is in front of the crosshair or not. You can see from the map below that it's also choosing travel checkpoints/destinations through walls. So they've severely cooked this to make it look far more capable than it is in practice, and any speed advantage that is offered here is not factoring in the shortcuts it is taking, the training on the map, and the fact that it can cheat because the structured state it is using is not bound by obstructions.
Here is their docs by the way: https://docs.typesafe.ai/ - so you can understand how it works.
Seems really cool.
https://github.com/knowledgator/gliclass
I feel like this whole scenario could be solved by a very ugly switch case generated by deepseek for 0.02ct.
Side note - just like most people don't need an intelligent personal assistant to manage and respond their emails and book their flights, most people also don't need smart homes. Century old toggle switches are more than enough in a 3 room apartment or 5 room house unless you have a mention.
There's no reason to not do both.
Also a quick NFC sticker in each room taking you to a small HTML site containing settings (temp, ventilation, lights, shutters, setting a alarm by the lights) has been golden.
No one wants to: download Shelly app + AC app + look for ventilation IR controller + figure out how casting works for the TV + figure out how to use the Shelly app to turn lights into an alarm. It's too much friction for little gain. But a quick tap? Great.
But tapping your phone on a NFC sticker bringing all those controls together per room in stead of per category (all lights in Shelly app. Person in room #1 has no interests in the lights in room #4 at the same time.).
IF you tap it while not on Wifi yet it just tells you to connect to Wifi. :-)
One "all house" sticker next to the front door allows any last person leaving or first person entering to put the entire house in active / idle mode.
Works wonders. And as soon as local AI is quick enough the stickers will be a microphone!
The regular-ceiling-lights-as-alarm service gets positive feedback. Weird how smarthome companies never market that, seems easy win.
That was my light bulb moment.
We have 8 light buttons in our living room/dining room/kitchen space. It is very convenient to us that we have 1 button for turning all of them on/off at the door to upstairs (at night turn off all lights and go to bed upstairs, in the morning come downstairs and turn on all lights) - but also have 1 on/off button near our back door for when we leave/come home.
Next to that: on/off toggles a schedule where the lights are bright and cold-ish by day, and low and warm by night without us having to manually adjust each light every hour or something.
Again, need is a big word. But it's very convenient and pleasant.
I agree that it was notable that they delegated to an existing LLM, but I don't think it detracts much from the value proposition (not yet proven) of their demo.
Oh, I have one of those use cases, matching people in genealogy trees. You can ask all sorts of questions: do the names match? Do they match within some edit distance? Do they match according to soundex/ metaphone rules (which are themselves a ginormous set of rules for letters and letter combinations which may or may not result in the same sounds, hand-coded as a huge if tree by a linguist not a programmer)? What about their relatives, do they match by the same rules? Should we incorporate domain knowledge about local naming customs? Etc etc.
I pointed a coding agent to this problem, and it aggressively started coming up with complex scoring rules and testing them against real datasets. Which led to sort-of acceptable results, but it still missed lots of cases which were obvious to a human, and had false positives which were obvious to a human. Which I could trade off, and slightly improve, with more back and forth with the coding agent.
Pointing a good LLM to all the information about two people, would of course give great results. Maybe even better than human judgment. But I can't do that for 100000^2 people, it would be too expensive in all sorts of ways. I need a fast, reliable scorer. I could maybe train an embedding, but that would be a huge job and where would I get the quality data?
It does not know enough about the records in the context of the overall dataset:
- what is the data quality and to what extent do we expect a errors in some fields - how unusual are certain values such as names in the context of the dataset as a whole, e.g. some names would be very common in some countries but rare in others.
I've written in more detail about this here: https://www.robinlinacre.com/fellegi_sunter_accuracy/
But I'm worried about that "bitter lesson" the TypeSafe CEO refers to, and that we're retreading the steps of natural language processing and a lot of other fields, trying to come up with clever rules, when the rule-based approach simply never gets good enough. I need matching to be good, it's absolutely central to genealogy, and I've seen the damage blindly linking by hand-crafted matching formulas can do.
Another aspect of context that could be relevant to your work may be how people are nested within households. In some record linkage scenarios you can run a linkage on household membership and use this to make person linkages more precise. I imagine an LLM agent could also use this information effectively if it were explicitly provided, but sometimes they miss logical leaps like this
Fundamentally this is an entity resolution problem. An LLM can score pairwise really well but scoring all the pairs would be insanely computationally difficult.
If you can constrain the set of potential matches up front by querying the dataset for things that could be matches it gets a lot more tractable to use an LLM for this.
Are there any heuristics you can use to reduce the search space? You mentioned soundex transformation and maybe prefixes of last names could work? Even if you get the number of potential matches down by a few orders of magnitude this gets more reasonable!
Check out https://moj-analytical-services.github.io/splink/index.html
Thanks for the link, it is an interesting topic.
For those who don't know (which is probably everyone but me), I ported the design-by-contract pattern in Python and combined it with LLMs. This was early 2025. I originally wrote about it here: https://leoveanu.com/2025-03-01-dbc/ . Contracts are a core feature of SymbolicAI ever since. The community seems to have loved it too (https://news.ycombinator.com/item?id=44399234).
I think I'm starting to glimpse the implications and it's gonna change agentic workloads if it holds up to scrutiny. It's too early for me to tell anything other than jot down some rough thoughts.
In short, you get blazingly fast semantic branching you can use in control flows. For contracts, I can now directly take the data model that you have to design and convert it into Jev's expected format. Or I can use Jev for semantic branching in postconditions.
If my understanding is correct, that should be doable, but I need to think more about it. It could be that with Jev I can finally “compile contracts” and better chain them into workflows, which is something I always wanted but didn't know how to do properly.
Eager to test. On the waiting list.
GP's first sentence isn't arrogant (at worst displaying a bit of false humility) because it's saying everyone but him doesn't know about a thing he did. Your second quote you apparently mis-parsed because of a minor English error (he should have said "to Python" rather than "in Python"), but to me it was pretty clear what he meant.
I'm curious to hear his take on this approach.
Unfortunately, none of this is explained in the announcement, but the documentation [0] is pretty good.
[0]: https://docs.typesafe.ai/concepts/how-to-build-with-system-o...
It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence).
Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model?
[0]:https://docs.typesafe.ai/concepts/system-one
I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable).
But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)
(I'd also argue something like p_yes or just probability might be a simpler name, but I'm sure there's a better reason behind Bernoulli maps).
[0] https://docs.typesafe.ai/primitives/noul#noul
"instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel
so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K.
all of this while still being instruction-tuned (!!!)"
It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept
the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing
The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!
The demo is showing System One producing its output in parallel very quickly and for little cost compared to an LLM generating its answers token-by-token. The "noul" type is used to evaluate a yes/no question and return the probability that the answer is yes.
So this demo is showing System One offering much more nuanced responses and specific probabilities compared to an LLM's more crude responses (e.g. LLM shows "true" or "false" compared to "0.9" or "0.07" probabilities that the answer to some question is true).
Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?
I don't want to shill my blog too much, but I will say data is probably far most interesting than architecture: https://www.completeskeptic.com/p/the-bitterest-lesson
(shilling my blog post on that jaggedness: https://www.completeskeptic.com/p/lies-damned-lies-and-bench...)
Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure.
e.g.: https://docs.typesafe.ai/primitives/advanced#structured-inst...
> not sure if multi-modal
just JSON... for now :)
> outputs the question's answers as appropriate
correct!
I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
Unless they're hackers, no. It's not really a chat interface, it's meant for consumption by machines and composing into higher level systems (pairs great with LLMs).
> Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
We're going to release some more info on evaluations over time, and yeah, join the waitlist! We offer faster access in exchange for good memes
this is complex, but generating text is highly complicated and requires mode dropping to make long cohesive text
My guess is that due to its design there is no support for prompt caching, as there would normally be no reason for it. So the performance of my idea would probably be appalling as every step in the loop would reevaluate every input and output token. But it would be interesting to see the outcome.
Jev's focus on structured I/O and confidence scores are game changing. If this does at all what it claims, I think this is going to quickly become the new standard approach for agentic systems.
Full disclosure, I am not they :=)
Decision making isn't that of a bottleneck I suppose.
A lot of people have become prompt maximalists, asking for complex multi-part solutions or dynamic workflows in a single prompt. You can get this to work sort of reliably with frontier models, but without much confidence or clarity where things might break in practice. My goal is to strip out as much determinism as possible from prompts so the LLM only needs to handle a narrow, well-informed decision, like "Pick one of these three things" and build around the answer. Sometimes you need to fill out a whole JSON payload and LLMs really actually suck at manipulating and adhering to JSON. They do ok now because labs have put in a ton of effort on making harnesses play nice with structured data. But it comes at a high token and context cost because under the hood I suspect the model is churning invalid text repeatedly until it gets around to passing some internal validation.
I am using the single prompt approach with GPT5.4, which is free, but it’s not reliable. Using Jev I’d decompose the prompt into a bunch of smaller questions, then I’d combine the answers in software. I’m super excited to try Jev out.
I need to rank 1000 articles and pick the 5 most relevant for the user? Jev.
I need to audit and strip out content because my user is affected by regional privacy laws (without hallucinating)? Jev.
I need to surface the 3 funniest media comments that match the user's sense of humour? Jev.
I need to ...? -> Open-weight model.
I'm sure someones working on this as we speak using an open-weight LLM base (Qwen or something would be a perfect fit).
This sort of task is a perfect fit for a very small model capable of semantic parsing. You can get away with a LOT less parameters without all the autoregressive generation and long-context reasoning.
But agreed it'd be very useful to see it deployed on other hubs, and it seems worth it to provide the bespoke API format. Perhaps Typesafe's API will end up becoming the standard for a new type of structured model, the way OpenAI's API did.
I guess you can choose your provider still? But isn't the point that the lowest bidder is doing inference?
So, assuming this is not vaporware, this would raise the tide for everyone because it shows what's possible.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
Edit: I really have to say that I like their manifesto https://typesafe.ai/manifesto
It's totally reasonable to compare against LLMs doing chain of thought if it gets comparable performance.
I think this is reasonable if people are actually using LLMs to solve this type of narrow structured task, which they are. The evidence is that every LLM provider has some method of forcing the output to conform to a json schema in their documentation.
Their manifesto: "you only build on top of it if it's trustworthy." - the irony of this while putting out the most misleading, dishonest marketing campaign I've seen in months for their first public appearance doesn't exactly scream "trustworthy" to me.
can jev play battlefield six for example
We already had encoder models that skipped text generation for giving us a numerical output that could be computed as a probability. we also got no hallucinations and faster inference for free there. So we already had
1. "unstructured state in, probabilistic decisions out" 2. "orders of magnitude faster and more efficient"
What was hard there was to train the model head without ML expertise, and considerable amount of data.
This seems like this is a democratization of those encoders? The addition over existing encoders seems to be coming from being able to specify the output shape (up to a cardinality of 255). It is unclear to me if this is possible using Jev without additional labels for fine-tuning.
If so, that is still very impressive, but I think the faster inference and 0 hallucinations might come for free, from it not being generative.
This is kind of trivially satisfied and they make this sound more extraordinary than it is. Also I suppose it can still hallucinate in the sense that for out-of-distribution data it will give miscalibrated probabilities.
Anyway a great step in the direction of calibrated AGI
> Output tokens: FREE (too cheap to meter).
Insane. The video demos are really compelling, in particular the speed.
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.
I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.
constrained decoding (OpenAI-style structured outputs) make models dumber unfortunately - the short+dense version is that simply masking logits is insufficient because if ever a model was assigning probability to an invalid token, the model is by definition confused. you'd be better off erroring IMO
This is something I focus on in a bunch of my experiments - how to get immense value out of tiny models (<1b params). There are lots of different architectures out there and there is so much to optimize if you know what you are asking and have a grammar to constrain with.
Great to see this and I hope this is a lot on top of what is already openly available.
Yes, I think it's a mistake that many organizations are cramming LLMs inside of automated pipelines where the extreme generality/flexibility of the model is at odds with the fact that you're using it for a very specific task that gets repeated over and over, and needs a very specific structured output to be successful. But specifying your task carefully (as well as deciding what counts as your input state representation etc) seems like a form of programming. Something (a person or a model working in a relatively unrestricted way) will need to produce a configuration/specification for this system.
So rather than Jev vs Claude I imagine that using Claude/ChatGPT/whatever interactively to define / refine your Jev config which then runs in prod might be the happy combination?
customer complaint - My credit card was charged twice for the same subscription
labels - billing, technical, sales
The model will always return something from the above classes - "so it can't hallucinate".
A general AI classifier that can be set up easily and used to classify anything… but with probably lower quality than a purpose built one.
They claim it's not an LLM, which I read as "not an auto-regressive token generator". I assume they are still using a transformer, otherwise they would be talking about the thing that's not a transformer, instead of all the fluff on the linked page. But they emphasize parallel generation, so is it like a text diffusion model?
you pass in your "prompt" and options (described in natural language) that it can respond with, in addition to your input. it gives back that option set with a probability assigned to each one
Rip there goes my excitement. I have a task that something like this would be great for but the list of options is a zero or two larger than that xd
Surely it’s still generating some kind unstructured data internally? For example, what if I told it to generate a short story, but the short story is output as a JSON string?
Jev takes (…questions in) -> (…probabilities out)
So Jev won’t write a story or emit arbitrary structured data. But if you ask it the right questions, it can make near-instant “decisions” against those questions, with accuracy and world knowledge on par with LLMs. The economic advantage is that it’s parallelizable and can give back up to 255 answers at once, in milliseconds.
FAQ: Is Jev just a smaller LLM?
Jev is neither small nor an LLM, hence being off the intelligence Pareto curve.
Image in documentation: https://mintcdn.com/ts-docs/aFVnpmCIX68NpsV1/images/ai-prime...
While this model may share much with GPT-style models on the encoder side, it clearly has a different decoder architecture. So is a high-parameter count language model an LLM even when it doesn't have a GPT-style decoder? The definitions are in flux.
It seems to take two forms of context input: 'state' and 'questions'
https://docs.typesafe.ai/concepts/state
> State can be as simple as a string
> State can also be a JSON object or array containing related context, examples, and other information that helps the model answer the associated questions.
> The state contains the content and supporting facts.
The state seems to be schemaless, while the questions determine the output schema.
The doom demo is quite cool
Instrument your game to output properties of entities near the player and the output is the various control inputs - moment to moment gameplay gets solved. Maybe augment with a tick-by-tick controlled stepping mode if particularly twitchy - an LLM can take care of the higher level reasoning then.
https://x.com/completeskeptic/status/2099925687465570372
I suppose this is the same video as the one from the parent comment, but I don't know for sure - I don't have a twitter account and the above link doesn't work for me.
I can see the individual tweets in the browser while not signed in though.
It's in the parent article under a section named "Doom" in case that asset URL ever changes.
You architect your systems with typesafe - because it is marginally faster, but inaccurate - to do what ? You can just wait for the next version of LLM's to get more accuracy at the same cost - or just use a faster model right now from a different provider.
Typesafe.AI sounds like some typescript/structured output type of tool…
What even is “system one” ?
IMO the product/tech is really there, just needs better communication.
I definitely agree it's underexplained in type safe.ai's materials.
I have to assume it's a reference to the fast, heuristic, intuitive "system 1" process in humans, as opposed to the slow, procedural, reasoning "system 2".
This theory is recognized, among others, in Daniel Kahneman 2002 Nobel prize on Economics.
I think that this specific part is not super interesting if your harness just recovers from invalid LLM outputs.
The latency and cost - yes, those are super interesting.
Would like to have something like in the original post but open weights.
Here's how this would have likely been made.
- Tiny transformer or equivalent model (maybe a few bn or so?), explaining latency and cost
- Questions are sent in parallel to multiple copies of it (I'm sure they're edge located)
- The model is post-trained for calibration in a wide variety of data (the recipe is relatively simple, and likely targeted on distillation of logprobs / confidence of a bigger model)
Notice how cost is ONLY for input tokens as output is merely numbers (few tokens) because input could be huge (questions and options).
At 0.042-per-million price they have, Astra estimates the model to be 3bn parameters.
One could replicate this by post training Qwen 3.5 2Bn. I expect people to do so soon!
- is the capital of france paris?
- it is august. is it raining in paris?
(forgive the examples; they're probably not semantically the sort of thing jev is trained to work on. but i figure the point translates to various kinds of questions that come up in "inner loop of agentic pid controller" contexts)
a normal text-generating model if asked to produce a number will also do that just fine. i assume in jev's case it was actually rled to essentially learn to express priors over things using its implicit world model, which definitely ought to help, but can we say more?
If you work at TypeSafe please try this.
Side note: This is probably how LLMs would perform with better encoders and next-latent prediction, so eventually those will beat this architecture out. Still amazing though.
I'd love to do research on this when I have the time.
That's what I was insinuating through "better encoder"; the model creating more efficient representations of ASTs using something like JEPA
we do think there's a lot of potential though and do want coding themed releases soon
As you said, if it worked, they would have demoed it haha
youd ask it to pick a location on the ast to add something from the grammar?
i dont see how this stays confined well enough? make a new output space every time? does that end up auto-regressive?
OpenAI has been teasing how fast computer use is with their models running on Cerebras chips but the difference here is a burning hole in your pocket.
https://developer.apple.com/library/archive/documentation/Ac...
Just joined the waitlist, excited to try it out!
Then, you can serve it faster/cheaper than the frontier LLMs. It's basically distilling a small but extremely common use-case from LLMs and serving it. Then RLCD comes into play to update weights when a new model comes out, etc.
Any thoughts on what the next potential "cheap" win to be distilled from frontier LLMs is? I'm going to need to play around with this.
Such large advances shouldn't be locked up away from local hardware.
That would be the litmus test.
"Does not hallucinate" is not the same as "is never wrong".
So the ATC test could be the benchmark.
"Does this contract contain ____?" is a pretty typical query for many industries, and then you can have follow up questions that nest down into further info about X, Y or Z thing.
Looks really good for that use case, especially with certainty as part of the output, as you could flag things that didn't have high enough of a certainty to human review.
I'm sure legora and the other legal AI tech softwares are all over this.
I listened to "Thinking fast and slow" recently and I was surprised how closely in behavior a LLM approaches the "System One" as defined there.
I approve of the clever branding!
The one downside is that the context window is very small (32k.) So some initial ideas we had for initial evaluation of code reviews won't fit yet in the window.
1: https://swamp-club.com/extensions/@swamp/typesafe-ai
https://github.com/typesafeainate/dspy-typesafeify
Thanks for putting this together. I'm surprised the cost saving is so little though. I expected much more based on the post.
0. https://typesafe.ai/
1. https://en.wikipedia.org/wiki/Fast_inverse_square_root
This is a valid tradeoff for one-off responses but if we're dealing with a distributed system (eg: Kafka), then only the high-confidence responses (>0.8) should move forward as input to the next service. If a low confidence output is propagated, then it can break the entire chain.
Founders response to a similar question on X: https://x.com/CompleteSkeptic/status/2100067328620896408?s=2...
pasting it here: zero-shot + general == programmable
I would assume any extreme scale narrow task could then be fine-tuned for, but we'll see - I suspect putting it all in shared cognitive core has bit maintainability/generalization benefits
> [them] Output tokens: FREE (too cheap to meter).
I'm very confused by this.
[0] https://goodstartlabs.com/research/verification-is-the-bottl...
Last 32k of connect + Summary of current task: Did we learn something useful here (true/false)? What is the category to file it under? Then notify the LLM to file it away.
What class of memory might be useful here? Model gives probability to each item in the list. Short description of all memories ordered by tagged class is used in the next round. Are any of these memories useful in the current context, such that they will inform the model and help in its task (yes/no)?
I’m sure there’s some fine tuning to be had, but this sure seems like the basis for a substantially better proactive memory system that works around an existing LLM conversation.
If I’m understanding what this does and how this works (generic input, intelligent classification with probabilities, rapid and cheap), this is absolutely nuts.
If self driving is red means stop, green means go, and stay in your lane - then it would work great, but having to actually think and test which maneuver is optimal for a given situation while weighting safety, road rules, random unexpected actions and getting to your destination, I think it's a much bigger problem. A bigger model specifically trained on that maybe would do great, but then the output is not the constraint anymore.
But I haven't tried the model, so I 'm just ballparking and could be very wrong.
Edit: never mind, found https://docs.typesafe.ai/introduction/quickstart by now
However I don't understand how are they claiming zero hallucination, how does giving confidence score fix hallucination? or am I missing something here?
But in things like programming, most concepts are abstract - 'if hungry eat an apple' in programming terms would look like
'if hunger > 50 {apples--; hunger-=30;}'
and compilers work with 'concept erasure' - to them, tokens (which are like llm tokens) look like
'if var1 > 50 {var2--;var1-=30}'.
They don't care about how these things map to real concepts. So all the embedding directions used to encode real-world concepts are just noise to LLMs when programming. This greatly reduces dimensionality and training costs. So does a token representation tuned for programming constructs, rather than natural language would probably have a more efficient encoding.
https://substack.com/home/post/p-215252866
https://fastino.ai/blog/gliner2-5-span-free-information-extr...
im thinking about how well Jev could be used to replace a current LLM-as-Judge evaluation workflows, specifically on chat transcript data (think ~1,500 tokens) i wonder if the reasoning usually required pushes it a bit out of scope. didnt see anything published about constraints on the state size, so would be curious to hear about that.
LLMs are universal simulators, their latents model the world. So I bet if you compare their logprobs with probabilities output by this model, it will be highly correlated.
Someone should do this quick experiment. I bet there won't be enough of a meaningful difference.
It might be boring, but I can see exactly how I could use this right now to improve my agentic rag.[0] In two months I am supposed to deal with a giant corpus, while still maintaining responsive chat UX. I have been working my butt off to make our first big client happy. This could really help solve the chunk ranking problem.
[0] assuming the policies are compatible with sensitive production workloads, some time in the near future.
I'm very curious how much ressources are needed to run such a model. This could be a complete game changer for local applications.
On a more serious note, it will be fascinating to see how this different spin on modelling inference will create new paradigms or slot into existing ones.
The goal is going to be to use llms to distill operations down to some dsl, and pass it into something like Jev.
And each question is a separate single token model completion done in parallel
Jev input … > Unstructured data (e.g. text) with an emphasis on structured program state.
What pre-training data/model is Jev based on? Surely result effectiveness is dependent (outside of one’s own input as “state”) on that?
Is there anything published on how it maintains calibration? Or when you say "outputs calibrated probabilities" you mean "as calibrated as frontier LLM models, just cheaper" - which is a different claim; as LLM's aren't particularly well calibrated
I think this could be pretty decent in CI? There's a lot of "flakes" I've mediated that this could have handled much more efficiently. Maybe observability as well, triggering elevated logging and other initial measures?
Given that, can't you just replicate Jev by telling Astra "here is the question, you must make a multiple choice decision / output a score between 1-10, please answer directly in a single word, no reasoning allowed"?
(Edit: Ok, Jev is much cheaper in input tokens so these two aren't directly comparable at all)
we actually use astra (and fable) in this way for our evals: evals.typesafe.ai
someone on the team cooked hard on that and it shows example traces comparing our model to opus/sol
I like the idea of System one models but all LLMs so far work as system 1 thinking because humans generate speech subconsciously with system 1.
System 2 thinking requires consciousness which AI does not have, so even reasoning models are still system 1 thinking as system 1 in humans has reasoning with heuristics.
Its limited but most people navigate the world with it completely, so it's enough for AI.
But is it just me or other folks feel uncanny seeing "a machine" playing Doom with low-latency as the first demo.
While I'm optimistic that humanity is good (but sometimes makes bad decisions), My first thought was seeing such models used by armies.
The idea sounds cool though
You could theoretically ask “what is the next appropriate character?” and add the entire ascii charset but i doubt it’d work well and you’d be implementing autoregressive churn across network latency…
[1]: www.expressionsofchange.org
it is a structured data model, but technically not a language model (it doesn't generate language)
1. Do you provide any kind of largest common subtree caching for cheaper input?
2. Have you tried auto-generating Lisp programs structurally?
3. Have you tried augmenting a Lisp language with a `choice` function that makes choices given a prompt, the environment, and the continuation stack?
(2-3) No, but that's kind of a sick cook ... Want to get access and try it? nathan@typesafe.ai
Maybe I'm just not doing a very good job at prompting Jev, but I think right now it's not quite capable enough to generate Lisp code.
Link: https://console.typesafe.ai/playground?share=shr_148e1248984...
I’m guessing it might be able to replace maybe 40-70% of LLM calls for a given pipeline depending on the business task, cutting the API costs on those calls by an order of magnitude.
1. Start with an LLM, so that your model understands natural language.
2. Replace RoPE with a tree embedding scheme, and causal attention with a sparse attention on the graph structure. (You could use full attention... but it's cheaper to use graph attention.)
3. Chop off the final unembedding layer, replacing it with a projection down to two scalars, one for logits and one for confidence.
4. Each option of a choice is represented by a number of tokens in leaf position; average these tokens' logit outputs to get the option's logit. Average all of the confidences from all of the options to get the choice's confidence.
5. Train the logits by KL divergence from a true distribution (or NLL on samples from a true distribution).
6. Train the confidences on a subset of the data in which you know the entire true distribution.
The hardest part is getting real world data for workflows, but I wildly speculate that you can get by with only ~50,000 documents if you first adapt domains using synthetic data.
Except this, this thing looks like revolution.
lol, I bet they would publish them if their score on those benchmarks were good.
> Search for relevant information over giant corpuses
Do you mean as an alternative to embeddings?
Finetuning a language model for decision classification (with probabilities) is already well-understood. What specifically changes in the training objective with RLCD? Are its benefits isolated from Jev’s new architecture/parallelism?
it can generate classes/models/routers etc via tool calls, doesnt write the actual code.
Either way, really cool and impressive.
Yes, that’s the kind of attitude I want to see in these model releases
But also effectively this is a classification model. It excels at specific certain types of workloads, and obviously will fail at others. Not really sure how one benchmarks this tbf. I can see their argument on why this requires a novel specific eval for whatever your usecase is. A consistent "global" benchmark might be hard to do
This website does not inspire confidence at all, it all sounds like a marketing piece. I wish it was true, some kind of text-prompted classifier with LLM performance would be cool, but I can't trust it with what we are given.
(though ideally everyone gets off the waitlist and can try it out for themselves )
Does this imply it's a very small model? I couldn't find anything about the model itself.
> we assume there is a correct compute graph (a “workflow” represented in code) and use the predictions of the largest, smartest, and most expensive external models as reference probabilities. ... Rephrased: every model gets the same workflow. We test how they compare to the average of the smartest models (in this case, Astra and Fable).
They assume there is a correct graph, but they don't compare to that, they compare to the average of the smarts models? So the smartest models are getting it wrong but you compare that anyway as a benchmark? So the outcome is "how much of a Fable am I getting" etc. Why not compare the actually correct thing?
But then even on this hand constructed eval, the first plot is showing Jev at less than Sonnet 5 accuracy. It is barely better than Luna. There are two Opus 5's and two Sonnet 5's without explanation. What is the plot showing?
I gave up.
https://x.com/CompleteSkeptic/status/2099925682726002904
we also thought the voice at the end was AI-ish, but apparently that's a real voice actor but slightly sped up
Reminds me of this: https://www.reddit.com/r/ITcrowd/comments/tg05j1/i_cant_beli...
I think we'll see a bunch of different architectures over the next five years.
forget LLM benchmaxxing sidequests, I'm sold on the real benchmark
I think the interesting thing would be seeing if prompt injections still work with this kind of model.
edit: looks like a framer export where there is a text stroke being applied :|
https://docs.typesafe.ai/concepts/use-case-map
Ouh! Any open weights models that can do this yet?? If not, how much longer? I have a Mac Studio coming soon.
> LLMS > Strings / generated text. Strings are flexible and can be anything: chat responses, code, hallucinations, refusals, or even type-safe structured values. To be used by software, responses need to be parsed + validated. There is also always some risk that the AI goes off the rails.
> Jev > Type-safe structured values. Possible outputs and structure are defined in advance. The model never makes type errors. All answers are accompanied with calibrated probabilities and confidence scores.
I mean, this isn't even remotely comparable to LLMs so why compare? Also, why are they bringing up AGI given there approach is so restrictive that what they're building literally cannot have the creativity required for AGI? The video is 100% marketing slop...
The bulk of the application of LLMs is that they generate reasonably reliable text which doesn't need to be defined in advanced. I'm sure there is a niche for this and congrats to the team, but please let's not hype this as if it's the next big thing in AI...
What a sad world would you live in if you don't keep creativity for the humans.
I'd use this as a tool an LLM can use for specialized tasks. It's not AI in itself.
https://docs.typesafe.ai/primitives has a much better explanation