Source code as structured text

Source code is plain text, the same kind of characters you would type in a note or an email. What makes it different is that those characters follow a strict grammar, and that grammar is exactly what lets a machine rebuild the structure of a program from the flat stream of letters, structure that a simple text search never sees.

Code is just characters

Open a source file in the plainest editor you have and there is nothing magic inside. It is a sequence of characters: letters, digits, spaces, line breaks, punctuation. The file that holds a thousand line program and the file that holds a shopping list are the same kind of object to your computer. Both are text. Neither one, at the level of the raw bytes, knows anything about what it means.

This is worth sitting with, because it is easy to imagine that code is stored in some special structured form, the way a spreadsheet stores cells or a database stores rows. It is not. A program on disk is a run of characters, one after another, and the program that runs it has to read those characters and figure out what they were supposed to say. The structure we care about, the functions and the loops and the nesting, is not sitting in the file as structure. It is implied by the characters and their order.

What a character actually is

Since code is characters, it helps to know what a character is to a computer. A computer stores everything as numbers. A character is a number that has been agreed, by a standard, to stand for a particular symbol. The letter capital A is the number 65. A space is 32. The agreement that maps numbers to symbols is called a character encoding. For decades the common one was ASCII, which covered the English alphabet, digits, and basic punctuation in the numbers 0 to 127.

ASCII could not represent most of the world’s writing. The modern answer is Unicode, a much larger agreement that assigns a number to essentially every symbol in every script, from accented letters to Chinese characters to emoji. The most common way to store Unicode in a file is an encoding called UTF-8, which writes the common English characters as a single byte each, exactly as ASCII did, and uses more bytes for the rarer symbols. This backward compatibility is why an old ASCII file is already a valid UTF-8 file.

Why does this matter for code? Because before anything can understand a program, it has to read the file with the right encoding, or the characters come out wrong. A tool that assumes ASCII and meets a file full of accented names or non-English comments will misread bytes and can choke on them. A serious code tool has to treat every file as Unicode text and get the decoding right first, before it can even begin to look for structure. The characters have to be correct before the grammar built on top of them can be found.

Text, but not free text

Here is the pivot. Code is text, but it is not the kind of text a person writes freely. When you write a sentence in English, the rules are loose. You can drop a comma, invert a clause, coin a word, run two ideas together, and a human reader still recovers your meaning. Natural language is forgiving because a mind on the other end fills the gaps. Code has no such luxury. It is read, in the end, by a machine that will do precisely what the characters say and nothing they do not say.

So a programming language comes with a rigid set of rules for what a valid program may look like. These rules are called a syntax: the grammar of the language. Where an English sentence can bend, a line of code cannot. A parenthesis that opens must close. A string of text has to be wrapped in matching quotes. A statement ends where the language says it ends, by a semicolon or a line break or an indent, depending on the language. Break one of these rules and the program is not slightly wrong in a way the machine will forgive. It is not a program at all, and the tools reject it outright.

Natural language is forgiving because a human fills the gaps. Code is unforgiving because a machine will not. That strictness is a cost when you write, and a gift when you analyze.

This strictness feels like a burden when you are typing and the tool complains about a missing bracket. But turn it around. Because the rules are strict and non-negotiable, a program that reads the code can rely on them completely. The very rigidity that annoys the author is the thing that makes the code recoverable. There is exactly one grammar, it is published, and every valid file obeys it. Free text gives you no such guarantee.

A short comparison makes the gap vivid. If a friend texts you “meet me their at noon,” you read past the wrong their without a pause, because you supply the correction yourself. A language does no such thing. Write the wrong keyword, or forget a quote, and there is no charitable reading waiting to rescue you. The program either matches the rules or it does not, and the answer is binary. This is not a flaw in programming languages. It is the price of being read by something that must act on exactly what you wrote, and it is the same property that lets a tool later recover your structure without having to guess at your intent.

What a human reads versus what a machine can recover

When a person reads code, a great deal of what they take from it is not in the syntax at all. They read the names of variables and infer intent. They read comments written in plain English. They notice that a function is called handlePayment and assume, correctly or not, that it handles payments. They bring years of experience about how software is usually shaped. Much of a person’s understanding of a file is this soft, inferred, human layer, sitting on top of the hard structure.

A machine cannot reliably recover the soft layer. It does not know that handlePayment is about payments in any deep sense, only that it is a name. It cannot judge whether a comment still describes what the code does or is three years out of date. But the machine can recover the hard structure completely and without guessing, precisely because the grammar is strict. It can know, for certain, that this block of characters is a function, that these lines are its body, that this name inside it is a call to another function, that this loop contains that statement. The recoverable structure is smaller than a human’s full understanding, but it is exact, and exactness at scale is its own kind of power.

Think of it like the difference between the meaning of a poem and its grammar. A machine will not tell you what the poem is about. It can tell you, flawlessly, which words are the verbs and where each sentence ends. Code is unusual in that its grammar carries so much of what actually matters. The wiring of a program, which piece calls which, is structure, not sentiment, and structure is exactly what the strict grammar lets a machine pull back out.

Why searching the text loses the structure

The simplest way to look through code is to search it as text. The classic tool for this is called grep, which finds every line where a run of characters appears. It is fast, it works on any file, and it is genuinely useful many times a day. But it operates on the flat characters, not on the structure hidden in them, and that limit is the whole reason more is needed.

Suppose you search a project for the word total. Grep will return every line where those five letters appear in that order. That includes a variable named total, but also the word total inside a comment, the word total inside a longer word like subtotal, the word total in a string of user-facing text, and a completely unrelated function in another part of the system that also happens to use the name. To the search, all of these are the same, because to the search they are just matching characters. It has no way to tell a definition from a mention, a real use from a word in a comment, or one meaning of a name from another.

The deeper loss is that search has no idea about nesting or connection. Consider this tiny snippet:

function price(item) {
  return item.cost + tax(item);
}

A person sees at once that price is a function, that it takes an item, and that inside its body it calls another function named tax. A text search for the word tax finds the characters t, a, x on that line. It does not know that this is a call, that the call sits inside the function price, or that price and tax are therefore connected. All of that structure is really there in the file, guaranteed by the grammar, but a search over characters steps right past it. The structure is present; the tool is simply not built to see it.

Strictness is what makes recovery possible

Put the pieces together and the shape of the whole field appears. Code is text. Text alone, searched as characters, gives you matches but not meaning. Yet the same code obeys a strict, published grammar, and that grammar is precisely the ladder that lets a machine climb from flat characters back up to structure. Because the rules cannot be bent, a program can read the characters, check them against the rules, and rebuild, with certainty, the nested shape the author wrote: this is a function, this is its body, this line is a call, this call is inside that loop.

This is the difference between reading the code as a string and reading it as a language. A string is a line of beads. A language has a shape, and the shape is a tree of things contained in things: statements inside functions, functions inside files, calls inside statements. The strictness of the syntax is what guarantees that tree is always there to be found, in every valid file, the same way every time. Loose text would give you no such promise. The unforgiving grammar, the thing that makes a missing bracket an error rather than a shrug, is exactly the thing that makes the structure recoverable at all.

It is worth being precise about what recoverable means here, because it is a strong claim. It does not mean a tool can divine what you were trying to accomplish or whether your logic is correct. It means that from the characters alone, without running the program and without talking to the author, a tool can rebuild the exact structural facts the grammar defines: this run of characters is a function, these lines are its body, this name is a call, that block is a loop. Those facts are not opinions and they are not guesses. They follow from the grammar the way the meaning of a sentence follows from its words, and they are the same no matter who reads the file or when. That reliability, repeated across every file in a project, is what makes it possible to build anything trustworthy on top of source code at all.

So the reason to care that source code is structured text is not academic. It is the foundation everything useful is built on. If code were merely text, the best any tool could do is match characters, and we would be stuck with the limits of search. Because code is text with a recoverable grammar, a machine can go further: it can read a file the way the language defines it and rebuild the author’s structure exactly.

The act of reading text against a grammar to rebuild that structure has a name, and it is the engine behind every compiler, every code editor’s smart features, and every serious code tool. It is called parsing, and grammars are the rules it reads against. That is the subject of the next idea in this hub: parsing and grammars, where we take a tiny language and watch a flat string become a nested shape.