Bit rot is real. Here is the evidence.

Same file. Same byte count. Different hash.

An enterprise storage node with two redundant server modules. One module came back from a routine restart and refused to start its application stack — seven consecutive failed starts, each dying on a class the runtime could not load. The other module, running the identical build, was fine and carried the workload throughout.

The two modules ship byte-identical copies of the same Java archive. Comparing them took one command.

Identical size, different content. That pairing is the whole finding, because it eliminates every ordinary explanation. A truncated write produces a shorter file. An interrupted copy produces a shorter or longer one. A bad deployment changes the modification time and usually the size. A partial overwrite changes the length.

None of that happened here. The file was written once, months earlier, and never touched again — same size, same timestamp, same everything the filesystem tracks. Yet the bytes on disk no longer matched the bytes that were written. The archive's own internal CRC, on a single compressed entry, is what finally exposed it: the entry was listed in the central directory at 8,547 bytes and read back as zero.

The data rotted in place.

Everything else was healthy.

Corruption is an easy thing to assert and a hard thing to prove, because almost every other explanation is more likely. Each of these was checked and ruled out before bit rot was named.

• ✓The driveSMART self-assessment PASSED. Zero grown defect count. Zero uncorrected errors. No reallocations, no pending sectors.
• ✓The filesystemKernel ring buffer clean. Mounted read-write, ordered journal mode. No I/O errors, no journal aborts, no read-only remount.
• ✓The write pathFile size unchanged and modification time unchanged since original install. Nothing wrote to it, so nothing could have written it badly.
• ✓The software buildThe healthy twin module runs the identical version from the identical source. Its copy is intact. A packaging defect would have corrupted both.
• ✓Blast radiusA full integrity scan of the library directory found exactly one bad archive. Widespread damage would indicate a failing device; a single artifact does not.

With the disk healthy, the filesystem clean, the file untouched and the build verified against a good twin, one explanation remains: the stored bits changed on media, silently, with nothing in the stack positioned to notice.

It was already broken for eighty-four days.

This is the part that should worry you more than the corruption itself.

The application had been running continuously for roughly eighty-four days. It loaded that archive once, at startup, and the operating system kept it in page cache from then on. Every subsequent read was served from memory — from a correct copy that had been read off disk before the rot occurred.

The corruption on the platter and the good copy in RAM coexisted for nearly three months. Monitoring was green. The application was healthy. Every health check passed, because every health check was exercising memory, not media.

The fault only surfaced on the first cold read. A clean shutdown flushed the cache; eleven minutes later the process tried to start, went to disk for the first time in three months, and got the rotted bytes. The restart did not cause the corruption — it revealed it.

Which raises the uncomfortable question: how many long-uptime systems are serving a cached copy of a file that no longer exists intact on disk? You would not know. Nothing in a normal monitoring stack looks.

Flip one bit. Watch the whole hash avalanche.

Here is why "same size, different hash" is a corruption fingerprint rather than a coincidence. Type anything below, then flip a single bit. The byte length will not change. The MD5 will change almost completely. That is the avalanche effect, and it is exactly what one rotted bit does to a file's checksum. Real MD5, computed live in your browser. Nothing is sent anywhere.

Run it a few times. The byte length never moves. The hash rarely keeps more than half its characters. That is the property that makes checksums useful for detecting corruption — and it is why a matching size with a mismatched hash is not ambiguous evidence. It is the signature.

The facts, unembellished.

Everything load-bearing in one place. Node identifiers, hostnames and organizational detail removed.

The repair took minutes: stage the twin's copy, verify it passes integrity before touching anything, back up the corrupt file for the case record, install atomically preserving ownership and mode, verify again. The module rejoined on the next supervisor cycle. Redundancy carried the service throughout, and no data was lost.

What made this catchable.

Not luck, exactly — but not design either. Two properties happened to be in place, and neither was there because someone planned for this.

1. The file carried its own checksumA compressed archive stores a CRC per entry. That is the only reason the corruption announced itself instead of loading as silently wrong data. A plain binary with no internal integrity structure would have failed in some stranger, later, harder-to-diagnose way.
2. A provably-good twin existedRedundancy gave both a comparison and a repair source. Without a second copy there is no way to tell which version is right — only that something is broken. Redundancy you can verify is worth considerably more than redundancy you assume.
3. Nothing was scrubbingThis is the gap. Filesystems with end-to-end checksumming and periodic scrub — ZFS, Btrfs, or an object layer that verifies on a schedule — would likely have caught this months earlier, on a background pass, instead of at startup during a maintenance window.
4. Monitoring was measuring the wrong layerEvery check was green for eighty-four days while the file was already bad. Process health, port checks and application probes all exercise cached memory. None of them read from media. If your integrity assurance depends on something noticing, confirm that something actually reads the disk.

The uncomfortable general case

This one was found because a restart forced a cold read on a file with an internal CRC and a good twin to compare against. Remove any one of those three conditions and the corruption stays invisible — or worse, gets read, used, and propagated as if it were correct.

Bit rot is not a folk tale told by storage people to justify budget. It is a low-probability event operating across an enormous number of stored bits for a very long time, which is a combination that makes rare things routine. The question is not whether it happens to your data. It is whether anything in your stack is positioned to tell you when it does.