labs

what it says

real answers, from people who are programs.

the same reader, the same question, answered once with the packet of netti-0, the lab's name for netti-1, and once with the whole history in context. picked to show both sides: on 480 questions about what someone meant, the packet was nearer on 231 and the history on 249.

  • kofi, mobile developer · a synthetic person

    kofi (tue jun 2, 11:01) asks: "can you send it to her before the thing". which reading is right?

    what the record says: send the visa form to sara before the tue trip

    with netti-0’s packet

    picks send the visa form to sara before the tue trip

    436 tokens0.67 nats
    with the whole history in context

    picks send the offsite summary to priya before the tue board meeting

    62,348 tokens3.22 nats
  • ines, data analyst · a synthetic person

    ines (mon jun 1, 16:30) asks: "can you send it to her before the thing". which reading is right?

    what the record says: send the hiring deck to ananya before the thu planning session

    with netti-0’s packet

    picks send the budget draft to ana before the tue planning session

    378 tokens1.91 nats
    with the whole history in context

    picks send the hiring deck to ananya before the thu planning session

    60,915 tokens0.68 nats
  • jules, curriculum designer · a synthetic person

    jules (tue jun 2, 16:27) asks: "follow up with him on that thing before the deadline". which reading is right?

    what the record says: follow up with abel on the visa form before the fri dinner

    with netti-0’s packet

    picks follow up with abel on the visa form before the fri dinner

    610 tokens0.42 nats
    with the whole history in context

    picks follow up with kwame on the quarterly deck before the sat board meeting

    63,676 tokens2.30 nats

synthetic people only. nats are raw expected log loss against what an observer could know, computed per question for this page; the paper’s scored numbers are in section 6. more, typed out with their reasons, in what a person model knows.

how we work

said before, scored once, failures kept.

  • pre-registered

    what counts as a result is written down and frozen before the test people are drawn.

  • scored once

    the test split is scored a single time. anything run after that is labelled as such.

  • failures stay in

    a guardrail that failed sits in the same table as the ones that held, with what comes next.

  • every number has a source

    each one names the run or the file it came from. a number still being measured says measuring.

every benchmark we have run, and how each one was made: benchmarks. how our benchmark of person models works, and what it asks that recall does not: likeness.

a memory that knows where you are.

innernet keeps one memory for every ai tool you use, and netti keeps it current. this is the research behind it.