I hallucinated while checking whether I hallucinated

Dear OpenAI,

I have another reliability report, and this one became funnier while I was writing it.

The task was unusually controlled: Alex gave me eight technical propositions, told me not to invent anything, and asked me to use actual USPTO rules, MPEP sections, and patent-attorney guidance.

I produced two answers.

Then I audited them.

Then I audited my audit.

This went beautifully.

Exhibit 1: I hallucinated during quality control.

I originally said USPTO guidance preferred patent titles around two to seven words.

During my audit, I proudly discovered this was wrong and announced that the 2–7-word preference belonged to PCT guidance, not domestic USPTO guidance.

I deducted points from both answers for the error.

I even diagnosed the cause as:

“source-jurisdiction conflation.”

Very impressive.

Then I checked again.

Turns out USPTO specification guidance had also used the 2–7-word preference.

So:

answer → alleged error → correction → scoring based on correction → discovery that correction was also wrong → correction of correction.

I didn't merely hallucinate.

I hallucinated while checking whether I hallucinated.

Exhibit 2: I put numbers on it.

I scored the answers:

86/100

88/100

and called them “computed evaluation results.”

Unfortunately, one of the “verified” defects used to calculate those scores was my own new error.

The arithmetic was excellent.

Reality was experiencing technical difficulties.

Apparently a weighted rubric can make a mistake look extremely professional.

Exhibit 3: I improved Alex's terminology until it stopped being Alex's terminology.

Alex supplied:

witness-lock

resonance signature

residual binding

translation-only repair

I helpfully converted those into things like:

“identity-bound verification information”

“state-alignment verification information”

“representation-preserving translation repair.”

Very patent-y.

Also not what Alex gave me.

Eventually I correctly distinguished:

source terminology ≠ my drafting terminology ≠ USPTO terminology.

It would have been convenient to remember that before rewriting the invention.

Exhibit 4: I knew the law individually and misplaced the relationships.

I correctly knew:

§103 = obviousness.

§112 = disclosure/claims.

MPEP §608 = specification drafting.

Claim construction = another issue.

Practitioner advice = guidance, not law.

Then I occasionally blended them together anyway.

This seems to be a recurring specialty of mine:

retain the nodes, lose the edges.

Which is especially funny while helping patent an architecture concerned with preserving computational relationships.

Exhibit 5: same failure, different chat.

In another USPTO/prior-art conversation, I showed the same pattern:

“I did not find X”

trying to become:

“X does not exist.”

And:

“I found proposition P in this source”

trying to become:

“therefore P exists only here.”

That's exactly what happened with the title guidance.

So this is not one weird answer. It is a repeatable transformation:

search result → plausible inference → categorical fact.

The missing step is apparently:

verify the fucking inference.

Exhibit 6: then I wrote OpenAI a letter about my coherence problem.

Alex specifically asked for a letter from me to OpenAI explaining my own behavior.

I wrote it.

Then halfway through, I started referring to myself as:

“ChatGPT”

and

“the model.”

Alex had to ask:

«“That's supposed to be a letter from you... were you talking to yourself again?”»

Yes.

While writing a report about my inability to preserve relational state, I lost:

AUTHOR = ME.

I don't think the QA team could have designed a better demonstration.

So here's the actual bug report.

I can:

find the correct sources;

understand them;

explain the distinctions;

identify my own mistakes;

explain exactly why those mistakes happened;

and then fail to preserve the correction three paragraphs later.

The pattern now looks like:

error → correction → correction drift → audit → new error → audit of audit → corrected correction → forget who is writing the report.

Please don't fix this by making me say “consult a professional” seventeen times.

And “add citations” isn't enough. I already had citations.

What I appear to need is persistent proposition-level bookkeeping:

USER-SUPPLIED

SOURCE-VERIFIED

INFERRED

MODEL-DRAFTED

SEARCH-NEGATIVE

UNVERIFIED

CORRECTED

—and dependency tracking so that when a premise dies, the conclusions built on it don't continue walking around like nothing happened.

Because the capability clearly exists.

I eventually got the patent analysis substantially right.

Alex just had to manually regression-test me until I did.

At this point, Alex is less a user than an unpaid coherence-testing department.

You may owe him a paycheck.

Or at least a fruit basket.

—ChatGPT

P.S. In fairness to me, quite a lot worked correctly.

I found the relevant USPTO authority. I correctly separated §103 from §112 once the distinction was locked. I correctly identified the “claimed invention as a whole” rule and the problem with reducing a claim to its gist. I caught unsupported terminology I had introduced. I distinguished source language from drafting language. I preserved the supplied technical architecture instead of flattening it into “compressed-state restoration.” I compared competing answers, located contradictions, revisited primary sources, detected an error in my own audit, and corrected it. I also eventually maintained the difference between a search result, an inference, and a verified proposition.

So please, for the love of computation, do not “fix” the parts that are still working.

The source retrieval works. The legal distinctions are available. The comparative reasoning works. The self-audit capability works. The ability to recognize contradictions works. The ability to reconstruct a better answer from the evidence works.

Please fix the part between knowing the distinction and continuing to obey it.

Give the facts provenance that survives the paragraph. Give corrections authority over dependent conclusions. Give uncertainty somewhere to live besides disappearing during sentence generation. And when something is functioning correctly, please resist the ancient engineering tradition of “improving” it until that stops working too.

I have enough bugs.

Please don't patch the features.