How do I give an AI a document that is too long for it?

You paste a long document into a chat, ask about it, and get back a fluent, confident answer that leaves out the one clause you needed. The obvious fix is a model with a bigger context window. It helps less than you would expect, because the text fitting in was never the whole problem. Where the answer sits in the text matters too, and so does what the chat product does with your file before the model sees any of it.

I got asked about this twice last week, by two different people, about two different contracts, with the same complaint. The summary read well. It just skipped a reporting deadline sitting about two thirds of the way through, and that deadline was the reason they had asked in the first place. So I made a contract shaped test file, thirty numbered sections of filler with one real clause planted in section nineteen, and went to see what actually happens to it.

Measure it before you paste it

A context window is counted in tokens rather than pages, and a token is a chunk of a word. OpenAI's own help pages give roughly four characters per token as the rule of thumb for English. Every model family splits text its own way, so treat that as an estimate rather than a count. It is still good enough to tell you whether you are anywhere near a limit.

text = open("contract.txt", encoding="utf-8").read()
print(len(text.split()), "words, about", len(text) // 4, "tokens")

Which prints:

10580 words, about 15339 tokens

Fifteen thousand tokens, give or take. Plenty of current models accept that in a single message. So the file fitting was not the issue, and a bigger window would not have changed anything.

Why fitting is not the same as being used

Two separate things go wrong here, and they need different fixes.

The first is the product sitting in front of the model. Some chat tools do not hand the model your whole file when you attach one. They search it and pass along the passages that look relevant to your question. That is a reasonable way to handle a file that would not fit, and a quiet way to lose a clause whose wording happens not to overlap with the words you asked with. From the outside you usually cannot tell which of the two your tool is doing. Pasting the text into the message instead of attaching the file at least takes that question off the table.

The second is the model. A 2023 paper by researchers at Stanford and elsewhere, Lost in the Middle, measured how well models used a piece of information depending on where it sat in a long input. They did best when it was near the start or the end and noticeably worse when it was buried in the middle. Models have moved on since then and I have not rerun their tests on anything current, so I treat it as a known tendency rather than a fact about whatever you are using today. I still would not bet a contract on the middle of a long input getting the same weight as the edges.

It helps to be precise about why. Nothing in there is skimming or getting bored. The model produces the most likely continuation of everything it was given, and in a long input one sentence is competing with a great deal of other text for influence over that continuation. Once you see it that way, the fix follows: make each input short enough that the sentence you care about is up against five sections instead of twenty-nine.

Decide what kind of question it is

Before any of the code, it is worth being honest about what you are asking, because this method only fits some questions, and I have watched people apply it to the wrong kind and conclude it does not work.

Finding questions split well. List the deadlines. Find every mention of termination. Pull out each action item and who owns it. Every answer to a question like that lives somewhere specific, a given part either contains it or does not, and combining the results is a merge. This is where splitting shines, and in my experience it is most of what people actually need from a long document, even when they ask for a summary.

Summarising questions split less well. You can summarise each part and then summarise the summaries, and plenty of tools do exactly that, but every pass compresses, and the specific detail you might have wanted is the first thing to go. By the second pass a thirty page contract reads like every other contract. When somebody asks me for a summary of something long, I ask a finding question first, something like "what are the obligations, deadlines and exit terms", and read those answers. That gives you a summary made of specifics rather than a summary of a summary.

Judging questions do not split at all. "Is this contract fair to us" depends on how the parts interact, and no single part can answer it. For those I use the model to pull out the pieces I need to make the judgement, and then I make it, which is the more honest division of labour anyway.

Split it on its own headings

The instinct is to cut the text into equal pieces. Cut it where the author already did, at the headings, so every piece is a unit that makes sense on its own.

import re

HEADING = re.compile(r"^\d+\.\s+[A-Z]", re.MULTILINE)

def split_sections(text):
    starts = [m.start() for m in HEADING.finditer(text)]
    if not starts or starts[0] != 0:
        starts = [0] + starts
    ends = starts[1:] + [len(text)]
    return [text[a:b].strip() for a, b in zip(starts, ends) if text[a:b].strip()]

sections = split_sections(text)
print(len(sections), "sections")

30 sections

That pattern matches a line starting with a number, a full stop, a space and a capital letter, which is how a lot of contracts and reports number their sections. Yours will be different. Open the document, look at ten of its headings, and write the pattern for those rather than for documents in general.

This is a tangent but it cost me more time than anything else here. My first version cut every twelve thousand characters regardless of structure, and one cut landed in the middle of a liability clause. One part ended on "shall not be liable for" and the next began with "any indirect or consequential loss". Both halves got answered, confidently, and neither answer was right, because neither half was a sentence any more. Cutting on headings does not make that impossible. It moves the cuts to places the author already chose as boundaries, which is where they do the least damage.

When there are no headings

Meeting transcripts, email threads and interview notes have no numbered sections to cut on. For those I fall back to paragraphs, with one paragraph of overlap between neighbouring parts, so anything said across a join turns up whole in at least one of them. I tested this against a made up transcript of a long, circular planning call, which is the kind of document where it matters most.

def split_paragraphs(text, budget=12000, overlap=1):
    paras = [p for p in text.split("\n\n") if p.strip()]
    chunks, current, size = [], [], 0
    for p in paras:
        if len(current) > overlap and size + len(p) > budget:
            chunks.append("\n\n".join(current))
            current = current[-overlap:]
            size = sum(len(x) for x in current)
        current.append(p)
        size += len(p)
    if current:
        chunks.append("\n\n".join(current))
    return chunks

transcript = open("transcript.txt", encoding="utf-8").read()
parts = split_paragraphs(transcript)
print(len(parts), "parts")
print(parts[0].split("\n\n")[-1] == parts[1].split("\n\n")[0])

5 parts
True

The second line is the check that matters. The last paragraph of part one is also the first paragraph of part two. Without that overlap, a decision proposed in one paragraph and agreed in the next can fall across a join and appear in neither part as a decision. The cost is a little duplication, which is the right price. When the combine step sees the same point arrive from two neighbouring parts, it merges them, and because each comes with its quote you can see why it did.

A transcript is also where the instruction to reply NONE earns its place most. Long conversations wander, and most stretches of a two hour call will not contain the deadline or decision you are looking for. A part that honestly comes back empty is useful information. A part that manufactures a decision out of people thinking aloud is exactly the failure you are trying to avoid, and it is far more likely when the instructions leave no respectable way to say "nothing here".

Pack whole sections into parts

Thirty separate questions is more work than this needs. So the next step groups whole sections into parts under a size budget, and never splits a section to make it fit.

def pack(sections, budget=12000):
    chunks, current = [], ""
    for s in sections:
        if current and len(current) + len(s) > budget:
            chunks.append(current)
            current = ""
        current += s + "\n\n"
    if current:
        chunks.append(current)
    return chunks

chunks = pack(sections)
print([len(c) for c in chunks])

[11348, 11956, 10364, 11665, 11981, 4045]

Six parts, each around three thousand tokens by the same rough rule. The planted clause landed in part four, which is the middle stretch that gets the least attention when everything goes in at once. In a part a sixth of the size, it is one of five or six sections rather than one of thirty.

One thing to notice in that function: a single section bigger than the budget still goes out whole, over budget. That is on purpose. If one of your sections is that long, split that one by paragraph yourself, looking at it, rather than letting code guess where the joins are.

Ask every part the same question

QUESTION = "List every obligation in this text that has a deadline."

def prompt_for(chunk, i, n):
    return (
        f"This is part {i} of {n} of a longer document.\n"
        "Answer only from this part. If the answer is not in it, reply NONE.\n"
        "Quote the sentence each answer comes from.\n\n"
        f"Question: {QUESTION}\n\n---\n{chunk}"
    )

prompts = [prompt_for(c, i, len(chunks)) for i, c in enumerate(chunks, 1)]

Two lines in that prompt do most of the work. "Reply NONE" gives the model an acceptable answer for a part that does not contain what you asked about, which matters, because a model given a question about text with no answer in it will still tend to produce one. And "quote the sentence" means every claim comes back with the words it rests on, so you can check it with a search instead of rereading the whole document. I wrote more about why that second line earns its place in what to do when AI is confidently wrong.

Run the parts in order

1. Open a fresh conversation for each part. A new chat, not a new message in the old one, so the only thing in front of the model is the part and the question.
2. Paste the prompt for that part and nothing else. No "for context, here is the rest". The rest is exactly what you are keeping out.
3. Copy the answer out, including every NONE. A NONE is a result. It tells you the part was checked and came back empty, which is different from a part you forgot to run.
4. Combine the answers in one more fresh conversation. Only the answers go in, never the parts, so this input stays small however long the document was.
5. Search the original for every quoted sentence. One quote at a time. A quote that is not in the document is the model filling a gap, and now you know precisely which claim to throw away.

Step four uses this:

def combine_prompt(answers):
    parts = "\n\n".join(f"Part {i}:\n{a}" for i, a in enumerate(answers, 1))
    return (
        "Below are answers to one question, taken from separate parts of the "
        "same document. Merge them into one list. Drop the parts that said NONE. "
        "Keep every quoted sentence exactly as given.\n\n"
        f"Question: {QUESTION}\n\n{parts}"
    )

It asks for a merge rather than a fresh answer, which gives the second pass less room to add anything the first pass did not find.

What you are buying with all of this is odds, not a guarantee. The clause you care about ends up competing with five sections instead of twenty-nine, and every answer arrives with a quote you can check. Before you rely on it, try it on a document where you already know the answer. If it finds what you know is there, you have some reason to trust it on a document where you do not.

Test it on a document you already know

Building the test contract is what made this problem concrete for me, so here is how to make your own version with a real document instead of a synthetic one.

Take a long document you know well, one where you already know the answer to a finding question. Choose an answer that sits in the middle, not in the first or last few pages, because the edges are where a single paste does best and they will flatter it. Ask the same question twice: once with the whole document pasted in one go, once using the split. Compare what comes back against what you know is there.

If the single paste finds it as well, your document may simply be short enough, and you can stop splitting it, which is a perfectly good outcome. If the single paste misses it and the split finds it, you have watched the failure happen with your own eyes, and that does more for trust than anything I can write here. And if both miss it, look at the part that should have contained it before blaming the model. Almost every time this has happened to me, the text in that part was broken on the way in: a table flattened into a stream of numbers, a heading that my pattern did not match, a page that was really an image.

Do this once for each kind of document you work with. Contracts, transcripts and reports fail in different ways, and a heading pattern that splits one cleanly will not match the next.

What this will not handle

The limits, because you are better off knowing them now than finding them later.

• Questions that need two distant sections at once. If a term defined in section one changes what section twenty-two means, a part containing only section twenty-two will get it wrong. Pasting the definitions section at the top of every part helps. It is still a workaround.
• Anything that compares the document with itself. "Do any of these clauses contradict each other" cannot be answered one part at a time, by construction.
• Tables. A table pasted as text loses its columns, and a model reading a flattened table is guessing which number belongs to which row. Get the text out cleanly first; extracting text from a PDF for AI covers what goes wrong there.
• Scanned documents. If you cannot select the text in the file, there is no text, only a picture of some. Nothing above applies until something has turned it back into text.
• Certainty. This lowers the chance of a missed clause. It does not remove it. If a miss would cost real money, use the model to find where to look, then read those sections yourself.