Hello everyone! This is my first post! I am a 5th year PhD candidate currently exploring career paths in biosecurity. I'd love your feedback on this side project of mine.
@Tessa A 🔸 asked on X how to easily differentiate biological DNA from abiological DNA (ie DNA data storage, DNA origami) if someone is ordered a DNA sequence and collected possible options here. She also suggested folks work on it in this post.
I decided to test out some of the ideas. I used a two-trial investigation using six-frame ORF finding, codon adaptation index (CAI), an ESM-2 protein language model, and sequence-complexity metrics (Shannon entropy and k-mer repeat content), applied to real Addgene plasmid sequences and to DNA generated by 2-bit-encoding random Wikipedia article text, a stand-in for DNA data storage.1,2
Why do this? Well, DNA synthesis companies want to screen DNA that people order for potentially dangerous sequences. But now, AI can design toxic proteins with high structural homology to existing toxins (i.e. it acts like a toxin), but low sequence homology (i.e. the DNA does not look like a toxin). This requires more intensive (and computationally expensive) screening to catch and doing this for every order may not be feasible. If a pre-screening pipeline could definitively state that DNA is abiologic (i.e. for DNA data storage), then screening of that sequence could stop there, saving the more intense screening for biologic DNA.
Five metrics were tried across two trials. ORF density, as expected, provided the best differentiation. Real plasmids contain far more open reading frames than text-derived DNA, in both versions of the analysis. k-mer repeat content is also a good metric: test-derived DNA is much more repetitive than biologic DNA due to the patterns of language. CAI (codon usage bias) is distinct between the datasets but notably, Wikipedia-derived DNA still averages just above 50% CAI match to E. coli, which tempers how much weight codon bias alone can carry as a biologic/abiologic signal. Shannon entropy, as implemented here, was not useful for this purpose. This is unsurprising in hindsight, since both real and text-derived DNA use all four bases at close to equal frequency. And ESM-2 perplexity does not cleanly separate biologic from abiologic DNA either — Wikipedia-derived ORFs did not score reliably worse (more “non-protein-like”) than real plasmid ORFs on average. Why exactly this happened requires more investigation, but may be a limitation of the model I used.
Two independent batches of each dataset were generated and analyzed at different points (“Trial 1” and “Trial 2” below); file counts differ slightly between trials because of how random sampling and file-parsing failures landed each time (Addgene: 484 files in Trial 1, 483 in Trial 2; Wikipedia: 500 files in both).
Both trials use a 6-frame open reading frame search (3 forward frames, 3 reverse-complement frames). An ORF is defined as a run from a start codon to the next in-frame stop codon (TAA/TAG/TGA), and only ORFs translating to more than 80 amino acids are kept — this is similar to NCBI ORFfinder's default behavior.
Each ORF's own codon usage is compared against a real reference table for E. coli (strain W3110, Kazusa Codon Usage Database, 4,332 CDS's / 1,372,057 codons). CAI is reported as a percentage, where 100% means every codon in the ORF is E. coli's single most-preferred synonymous codon for that amino acid. It is a measure of how “E. coli-like” a sequence's codon choices are, not a measure of protein quality or authenticity in general. Notably, the Addgene plasmids were not restricted to those for bacterial expression or only carrying E. coli genes.
Each candidate protein was scored with ESM-2 (facebook/esm2_t6_8M_UR50D), a protein language model trained on real protein sequences.4 We report a pseudo-perplexity: each residue is masked one at a time, the model predicts it from context, and the negative log-likelihoods are averaged and exponentiated.5 Lower perplexity means the model finds a sequence more consistent with real proteins it was trained on.
Trial 2 drops ESM-2 (no GPU/model download required) and adds two sequence-complexity checks, computed both for each whole input sequence and for each individual ORF:
The code used for these analyses can be found here. Both the code, results visualization, and writing of this document were assisted by Claude Sonnet 5.0.
Real plasmids contain far more ORFs than text-derived DNA, both in raw count and normalized by length:
Figure 1. Distribution of ORF counts per file (ATG-only search), Addgene vs Wikipedia-derived DNA. Wikipedia's distribution is a thin spike pinned near zero (356 of 500 files have zero qualifying ORFs); Addgene's is a broad distribution centered around 8 ORFs/file.
Figure 2. Same comparison normalized to ORFs per kb, removing sequence-length as a confound. Addgene averages roughly 8x the ORF density of Wikipedia-derived DNA (1.25 vs 0.15 ORFs/kb).
Figure 3. Per-ORF CAI codon match (left) and ESM-2 perplexity (right), Addgene vs Wikipedia-derived DNA.
CAI is informative here — Addgene sits higher and wider (mean 56.9%, median 56.6%) than Wikipedia (mean 51.7%, median 51.2%). Wikipedia-derived DNA, which has no biological origin whatsoever, still lands just above 50% CAI match on average, which is fairly surprising. It is unlikely that this metric is useful since many Addgene plasmids do not contain E. coli genes. So while this differentiates the two groups at a bulk level, the overlapped distributions make it less useful for differentiating a single sequence.
ESM-2 perplexity gave confusing results. Wikipedia's ORFs (mean perplexity 9.43) actually score lower, i.e., more “protein-like” by the model's own metric, than Addgene's real ORFs (mean 13.99) on average. Read naively, that would suggest text-derived DNA looks more like a real protein than real plasmid DNA does, which is not a sound conclusion. The more defensible read is that ESM-2 perplexity, at least at this model size and scoring approach, is not a reliable dataset-level biologic/abiologic classifier.
The two-lobed shape in Figure 3 turned out to be real, not a plotting artifact. A kernel density estimate over all 4,064 Addgene ORFs identifies two genuine modes, a smaller one around perplexity ≈ 10 (22.0% of ORFs) and a larger one around ≈ 14–16 (78.0%), separated by a valley (density drops to 23–36% of the peak heights at perplexity ≈ 11.4).
Figure 4. Histogram and kernel density estimate of ESM-2 perplexity across all 4,064 Addgene ORFs, with detected modes marked. The distribution is bimodal.
Figure 5. The equivalent distribution for Wikipedia-derived DNA (n=210 ORFs) is a single broad, roughly flat spread with no comparable second mode.
Splitting Addgene's ORFs at the valley (perplexity = 11.38) and comparing the two groups confirms these are meaningfully different populations, not sampling noise:
| Group | n | % of ORFs | Median length (aa) | Mean CAI (%) |
| Low-perplexity mode (< 11.38) | 894 | 22.0% | 286 | 59.1 |
| High-perplexity mode (≥ 11.38) | 3,170 | 78.0% | 127 | 56.3 |
Both differences are significant (Mann-Whitney U, length p = 2.2×10⁻¹²⁵; CAI p = 6.0×10⁻⁴⁶). The low-perplexity mode is made of substantially longer ORFs with modestly higher codon bias — consistent with genuine coding sequence (resistance markers, proteins to be expressed, other real genes). The high-perplexity majority has a median length of just 127 aa, quite short for a real protein and thus consistent with ORFs that arise by chance in non-coding backbone DNA.
Figure 6. ORF length (log scale) vs ESM-2 perplexity for both datasets. Wikipedia's ORFs occupy only the short end (max 274 aa) and overlap with Addgene's shorter, higher-perplexity ORFs; Addgene's long tail (up to 3,185 aa) is unique to the real-DNA dataset.
Both datasets show a modest negative correlation between length and perplexity (Addgene: Spearman ρ = −0.29, p = 8.1×10⁻⁸¹; Wikipedia: ρ = −0.25, p = 3.0×10⁻⁴). Length is a contributing factor to perceived protein-likeness in both, but there is substantial independent variation in perplexity at any given length.
All Trial 1 scoring used facebook/esm2_t6_8M_UR50D, the smallest available ESM-2 checkpoint (8M parameters). Larger ESM-2 models (35M up to 15B parameters) are documented to capture real-protein statistics more accurately, and would plausibly show a sharper separation between the two modes than what we observed here. I did not have the ability to test this within the current environment; it's a reasonable next step for anyone looking to firm up these findings.
Trial 2 reran the analysis on freshly generated datasets, searching for ATG plus the alternative bacterial start codons GTG, TTG, and CTG, and replacing ESM-2 with two lightweight sequence-complexity checks (Shannon entropy and k-mer repeat fraction). Because more start codons mean more opportunities for any given stretch of DNA to qualify as an ORF, raw counts here are not directly comparable to Trial 1's ATG-only numbers.
Figure 7. ORFs per file (left) and ORF density (right) with all four start codons enabled. The same qualitative gap persists — Addgene still averages roughly 3x Wikipedia's ORF density (2.57 vs 0.88 ORFs/kb) — though Wikipedia's zero-ORF spike is less extreme than in Trial 1 (113 of 500 files vs 356 of 500), because there are now more ways for a short ORF to arise by chance.
71.8% of Addgene's ORFs and 87.6% of Wikipedia's used a non-canonical (non-ATG) start codon — alternative starts account for the majority of ORFs in both datasets once they're allowed. Splitting each dataset by start-codon type shows a small effect:
Figure 8. CAI (left) and Shannon entropy (right) split by canonical (ATG) vs non-canonical (GTG/TTG/CTG) start, for both datasets. Canonical starts score modestly higher on CAI in both datasets; entropy is essentially indifferent to start-codon type.
Figure 9. k-mer repeat fraction split the same way. The canonical/non-canonical difference is statistically significant but small (Addgene p = 0.039, Wikipedia p = 0.0036); the much larger, more informative gap is between datasets. Wikipedia's repeat fraction (≈ 0.35–0.38) runs roughly 3x higher than Addgene's (≈ 0.12) regardless of start codon.
Whole-sequence Shannon entropy is nearly identical between the two datasets (Addgene 1.993 bits, Wikipedia 1.954 bits, out of a max of 2.0), both are close to perfectly balanced in raw base composition, because this order-0 formula only checks whether A/C/G/T proportions are roughly even, and says nothing about repetition or structure. The k-mer repeat fraction tells a different story: Addgene sequences are only about 50% repeat-content on average (whole-sequence), while Wikipedia-derived sequences are about 68–69%. This makes sense given how the Wikipedia dataset was built: English text has letter-frequency skew and recurring short patterns (common words, spacing) that get carried through the fixed 2-bit encoding into detectable nucleotide-level repetition, even though the overall base composition still comes out close to even.
For anyone wanting to cheaply distinguish real biological sequence from synthetic/random DNA, ORF density and k-mer repeat content do all the useful work. CAI adds a smaller amount of corroborating evidence. Shannon entropy (at least the order-0 version used here) and ESM-2 perplexity (at least at this model size, used as a between-dataset comparison) are not pulling their weight, despite both being intuitively appealing metrics.
(1) Gervasio, J. H. D. B.; da Costa Oliveira, H.; da Costa Martins, A. G.; Pesquero, J. B.; Verona, B. M.; Cerize, N. N. P. How Close Are We to Storing Data in DNA? Trends in Biotechnology2024, 42 (2), 156–167. https://doi.org/10.1016/j.tibtech.2023.08.001.
(2) Startup Catalog has jammed all 16GB of Wikipedia’s text onto DNA strands. CNET. https://www.cnet.com/tech/computing/startup-packs-all-16gb-wikipedia-onto-dna-strands-demonstrate-new-storage-tech/ (accessed 2026-07-17).
(3) Peabody, D. S. Translation Initiation at Non-AUG Triplets in Mammalian Cells. J Biol Chem 1989, 264 (9), 5031–5035.
(4) Lin, Z.; Akin, H.; Rao, R.; Hie, B.; Zhu, Z.; Lu, W.; Smetanin, N.; Verkuil, R.; Kabeli, O.; Shmueli, Y.; dos Santos Costa, A.; Fazel-Zarandi, M.; Sercu, T.; Candido, S.; Rives, A. Evolutionary-Scale Prediction of Atomic-Level Protein Structure with a Language Model. Science 2023, 379 (6637), 1123–1130. https://doi.org/10.1126/science.ade2574.
(5) Salazar, J.; Liang, D.; Nguyen, T. Q.; Kirchhoff, K. Masked Language Model Scoring. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics; Jurafsky, D., Chai, J., Schluter, N., Tetreault, J., Eds.; Association for Computational Linguistics: Online, 2020; pp 2699–2712. https://doi.org/10.18653/v1/2020.acl-main.240.