Model type
Masked-token RNA transformer encoder
Configuration
RNA-FM
RNA-FM learns contextual representations of RNA nucleotides for downstream RNA analyses.
Sources (2)
ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison tableHow it worksRNA-FM workflow
Conceptual summary of the documented data flow; optional inputs and configured downstream stages must be reported for a reproducible evaluation.
Sources (2)
ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison tableOverview
Inputs
RNA sequences tokenized at nucleotide resolution.
Outputs
Contextual token embeddings for a specified downstream RNA task.
Access
Official project documentation and implementation: https://github.com/ml4bio/RNA-FM
Sources (2)
ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison tablelimited source coverage · Automated source review, 2026-09-23. All specifications and missing details
Evaluations and results
13 evaluations · 13 results. Different protocols are not a single leaderboard.
Filter evaluations
Applied filters: All linked evaluations
| Tested configuration | Protocol and dataset | Finding | Evidence and details |
|---|---|---|---|
| Configuration: RNA-FM | Task: BEACON APA: Alternative polyadenylation isoform prediction Dataset subset: APARENT (BEACON split) | 70.32(0.97)% r2 percent · higher Uncertainty: type: standard_deviation; value: 0.97 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON APA: Alternative polyadenylation isoform prediction BEACON harness, fixed downstream head per task. Train/validation/test 145,463/33,170/49,755; dataset APARENT. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(APA) |
| Configuration: RNA-FM | Task: BEACON CMP: Contact map prediction Dataset subset: RNAcontact (BEACON split) | 47.56(6.73)% precision_at_l percent · higher Uncertainty: type: standard_deviation; value: 6.73 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON CMP: Contact map prediction BEACON harness, fixed downstream head per task. Train/validation/test 188/23/80; dataset RNAcontact. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(CMP) |
| Configuration: RNA-FM | Task: BEACON CRI-Off: CRISPR off-target effect prediction Dataset subset: DeepCRISPR (BEACON split) | 2.49(1.56)% spearman_corr percent · higher Uncertainty: type: standard_deviation; value: 1.56 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON CRI-Off: CRISPR off-target effect prediction BEACON harness, fixed downstream head per task. Train/validation/test 14,223/2,032/4,064; dataset DeepCRISPR. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(CRI-Off) |
| Configuration: RNA-FM | Task: BEACON CRI-On: CRISPR on-target efficiency prediction Dataset subset: DeepCRISPR (BEACON split) | 31.62(1.16)% spearman_corr percent · higher Uncertainty: type: standard_deviation; value: 1.16 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON CRI-On: CRISPR on-target efficiency prediction BEACON harness, fixed downstream head per task. Train/validation/test 1,453/207/416; dataset DeepCRISPR. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(CRI-On) |
| Configuration: RNA-FM | Task: BEACON DMP: Distance map prediction Dataset subset: RNAcontact (BEACON split) | 51.45(0.51)% r2 percent · higher Uncertainty: type: standard_deviation; value: 0.51 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON DMP: Distance map prediction BEACON harness, fixed downstream head per task. Train/validation/test 188/23/80; dataset RNAcontact. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(DMP) |
| Configuration: RNA-FM | Task: BEACON Modif: RNA modification site prediction Dataset subset: MultiRM (BEACON split) | 94.98(0.042)% auc percent · higher Uncertainty: type: standard_deviation; value: 0.042 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON Modif: RNA modification site prediction BEACON harness, fixed downstream head per task. Train/validation/test 304,661/3,599/1,200; dataset MultiRM. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(Modif) |
| Configuration: RNA-FM | Task: BEACON MRL: Mean ribosome loading prediction Dataset subset: Optimus (BEACON split) | 79.47(0.47)% r2 percent · higher Uncertainty: type: standard_deviation; value: 0.47 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON MRL: Mean ribosome loading prediction BEACON harness, fixed downstream head per task. Train/validation/test 76,319/7,600/7,600; dataset Optimus. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(MRL) |
| Configuration: RNA-FM | Task: BEACON ncRNA: Non-coding RNA family classification Dataset subset: Noorul's ncRNA set (BEACON split) | 96.81(0.061)% accuracy percent · higher Uncertainty: type: standard_deviation; value: 0.061 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON ncRNA: Non-coding RNA family classification BEACON harness, fixed downstream head per task. Train/validation/test 5,679/650/2,400; dataset Noorul's ncRNA set. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(ncRNA) |
| Configuration: RNA-FM | Task: BEACON PRS: Programmable RNA switch prediction Dataset subset: Angenent-Mari's switch set (BEACON split) | 55.98(0.09)% r2 percent · higher Uncertainty: type: standard_deviation; value: 0.09 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON PRS: Programmable RNA switch prediction BEACON harness, fixed downstream head per task. Train/validation/test 73,227/9,153/9,154; dataset Angenent-Mari's switch set. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(PRS) |
| Configuration: RNA-FM | Task: BEACON SPL: Splice site prediction Dataset subset: SpliceAI (BEACON split) | 34.84(0.87)% top_k_accuracy percent · higher Uncertainty: type: standard_deviation; value: 0.87 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON SPL: Splice site prediction BEACON harness, fixed downstream head per task. Train/validation/test 144,628/18,078/16,505; dataset SpliceAI. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(SPL) |
| Configuration: RNA-FM | Task: BEACON SSI: Structure score imputation Dataset subset: StructureImpute (BEACON split) | 42.36(0.24)% r2 percent · higher Uncertainty: type: standard_deviation; value: 0.24 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON SSI: Structure score imputation BEACON harness, fixed downstream head per task. Train/validation/test 14,049/1,756/3,095; dataset StructureImpute. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(SSI) |
| Configuration: RNA-FM | Task: BEACON SSP: Secondary structure prediction Dataset subset: bpRNA-1m (BEACON split) | 68.50(0.54)% f1 percent · higher Uncertainty: type: standard_deviation; value: 0.54 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON SSP: Secondary structure prediction BEACON harness, fixed downstream head per task. Train/validation/test 10,814/1,300/1,305; dataset bpRNA-1m. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(SSP) |
| Configuration: RNA-FM | Task: BEACON VDP: Vaccine degradation prediction Dataset subset: OpenVaccine (BEACON split) | 0.347(0.003) mcrmse error · lower Uncertainty: type: standard_deviation; value: 0.003 Coverage: Not reported scored / Not reported eligible | Author-reported evaluation · Source checkedMethods, coverage and sourceRNA-FM on BEACON VDP: Vaccine degradation prediction BEACON harness, fixed downstream head per task. Train/validation/test 2,155/245/629; dataset OpenVaccine. Aggregation: Not reported BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Table3,p.8,row(RNA-FM),column(VDP) |
Source checking is not independent reproduction. Release 2026-09-29-06401fd5b220.
Use this model
How it works, versions and access
Related profile: RNA-FM. This page retains the exact record and its evaluation context.
This configuration
Published RNA language model backbone, fine-tuned per task by the BEACON authors under their protocol.
- record
- RNA-FM
- configuration
- Not reported
- entity type
- Configuration
How it works
How it works
RNA-FM converts an RNA sequence into one token per nucleotide. Twelve transformer encoder blocks use self-attention to produce a 640-dimensional representation at each position. During pretraining, the model learns to recover masked nucleotides from their surrounding sequence. The resulting representations can be supplied to a separately specified downstream model; they are not, by themselves, a structure or functional prediction.
Sources
RNA-FM original methods, arXiv v5 · arXiv:2204.00300v5, Methods: ncRNA data collection and preprocessing and RNA foundation model training details (p.22)Versions and reproducibility
The nucleotide-based rna_fm_t12 and codon-based mrna_fm_t12 interfaces are distinct. The original RNA-FM paper sets a training input-length limit of 1,024 and describes a usable limit of 1,022 nucleotides. That limit should not be assigned to mRNA-FM without checking its separate configuration.
Sources (3)
ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py; RNA-FM original methods, arXiv v5 · Official README: Quick Start and RNA-FM/mRNA-FM examples; arXiv:2204.00300v5, Methods: training input length and SARS-CoV-2 genome embedding extraction (pp.22–23)Strengths, limitations and unresolved questions
Strengths and limitations
Strengths and considerations
- The repository exposes embedding extraction and examples for downstream RNA analyses.
Sources (2)
ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison table
Limitations and conditions
- Base-level RNA-FM and codon-level mRNA-FM are not interchangeable. The task head and tokenization must be specified in each evaluation.
Sources (2)
ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison table
Profile review details
Follow-up review of Checkpoint identification, Training cutoff, Weights licence. Source locations and before/after decisions are recorded in the 23 September profile-evidence audit. Other explanatory content retains its earlier source scope. No human scientific review or independent reproduction is implied.
Stable record: discovery-model-rna-fmSpecifications
Inputs, training, access and other details
Explanatory profile: limited source coverage · Automated source review, 2026-09-23. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.
| Property | Description and evidence |
|---|---|
| Model type | Masked-token RNA transformer encoderSources (2)ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison table |
| Architecture | 12-layer masked-token transformer encoder with hidden width 640 and 20 attention heads; nucleotide tokens produce contextual representations.SourcesRNA-FM original methods, arXiv v5 · arXiv:2204.00300v5, Methods: ncRNA data collection and preprocessing and RNA foundation model training details (p.22) |
| Inputs | RNA sequences tokenized at nucleotide resolution.Sources (2)ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison table |
| Outputs | Contextual token embeddings for a specified downstream RNA task.Sources (2)ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison table |
| Parameters | 99M, as printed in the official Foundation Models table.Sources (2)ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison table |
| Known versions | rna_fm_t12 and mrna_fm_t12 are separate pretrained interfaces.Sources (2)ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison table |
| Training data | 23.7 million non-coding RNA sequences collected from RNAcentral. The authors replace T with U and remove identical sequences using CD-HIT-EST at 100% identity, naming the resulting corpus RNAcentral100.SourcesRNA-FM original methods, arXiv v5 · arXiv:2204.00300v5, Methods: ncRNA data collection and preprocessing and RNA foundation model training details (p.22) |
| Training cutoff | The original Methods describe RNAcentral100 preprocessing but do not establish a dated RNAcentral release. RNAcentral100 is a processed-corpus label, not a release number. · Not reported in inspected sourcesSourcesRNA-FM original methods, arXiv v5 · arXiv2204.00300v5, Methods: Large-scale pre-training dataset, p22 |
| Context limits | The original paper sets a training input-length limit of 1,024 and describes a usable input limit of 1,022 nucleotides. These are the original RNA-FM settings, not a validated limit for later codon-based mRNA-FM checkpoints.SourcesRNA-FM original methods, arXiv v5 · arXiv:2204.00300v5, Methods: RNA foundation model training details (pp.22–23) and RNA-FM application input-limit statement (p.23) |
| Weights licence | The official model-comparison table labels RNA-FM MIT, while the README footer explicitly refers to source code. Separate checkpoint-distribution terms remain unverified; the linked weight repository could not be retrieved during this review. · Not reported in inspected sourcesSources (3)ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: LICENSE; rnafm loader: primary artifact · README: Related RNA Language Models, RNA-FM row; License footer; LICENSE. Linked https://huggingface.co/ml4bio/RNA-FM returned HTTP401 during the review. |
| Access | Official project documentation and implementation: https://github.com/ml4bio/RNA-FMSources (2)ml4bio/RNA-FM: README.md; ml4bio/RNA-FM: fm/pretrained.py · README.md: Quick Start, embedding examples and RNA foundation model comparison table |
| Code licence | MITSourcesml4bio/RNA-FM: LICENSE · LICENSE: licence text |
| Checkpoint identification | The official rna_fm_t12 loader downloads RNA-FM_pretrained.pth from the authors’ server. The inspected loader does not pin a checkpoint revision or publish a digest; a code commit alone cannot identify the bytes used by a published evaluation. · Not reported in inspected sourcesSourcesrnafm loader: primary artifact · load_fm_model_and_alphabet_hub: rna_fm_t12 branch, lines155–162 |
Evidence
Source checking verifies the cited claim or transcription. It does not establish independent reproduction.
Evidence table
Inspect claims, sources and review details
Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.
One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.
2 evidence rows matching the loaded filters
| Property and statement | Original source and location | Review and provenance |
|---|---|---|
| Relationship: family discovery-model-rna-fm Individual claims | ml4bio/RNA-FM: README.md BEACON Table 2 and Appendix A.3.2 RNAcentral ncRNA model, Table 3; mRNABench Table 2 and Appendix inventory; NABench model inventory; source-labelled configuration RNA-FM | Existing reviewed locator: Table3,p.8,row(RNA-FM) Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: 348951516e0963d22bbb33b3c9fc18c89081d38e | source checked automated source review · 2026-09-23 Audit detailsSource review establishes this relationship only. Exact evaluated configurations and original numerical review status remain unchanged. RNA-FM ncRNA backbone explicitly identified; mRNA-FM is distinct and must not inherit these scores. Field: Claim: model-evaluation-identity-bef319624c59d1475fe2 Source artifact SHA-256: Hash scope: SHA-256 of retrieved original artifact bytes Format: original_artifact |
| Relationship: family discovery-model-rna-fm Individual claims | BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) BEACON Table 2 and Appendix A.3.2 RNAcentral ncRNA model, Table 3; mRNABench Table 2 and Appendix inventory; NABench model inventory; source-labelled configuration RNA-FM | Existing reviewed locator: Table3,p.8,row(RNA-FM) Shared locator for this statement’s cited sources; not a separate locator for each citation. Version: v2, 12 December 2024 | source checked automated source review · 2026-09-23 Audit detailsSource review establishes this relationship only. Exact evaluated configurations and original numerical review status remain unchanged. RNA-FM ncRNA backbone explicitly identified; mRNA-FM is distinct and must not inherit these scores. Field: Claim: model-evaluation-identity-bef319624c59d1475fe2 Source artifact SHA-256: Hash scope: Hash scope not separately documented; inspect source record |
Sources and history
View linked audit checks and correction history
Release 2026-09-29-06401fd5b220 · Record review: source checked
1 source records and release history
- BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (arXiv:2406.10391v2) · Original source · v2, 12 December 2024
Technical metadata and extraction receipts
Stable ID: beacon-model-rna-fm
- areas
- rna-transcriptomes
- family
- pretrained
- source locator
- Table3,p.8,row(RNA-FM)
- missing metadata
- checkpoint revision: unreported; parameters: unextracted
Related records
- family: RNA-FM
- model: RNA-FM on BEACON APA: Alternative polyadenylation isoform prediction
- model: RNA-FM on BEACON CMP: Contact map prediction
- model: RNA-FM on BEACON CRI-Off: CRISPR off-target effect prediction
- model: RNA-FM on BEACON CRI-On: CRISPR on-target efficiency prediction
- model: RNA-FM on BEACON DMP: Distance map prediction
- model: RNA-FM on BEACON Modif: RNA modification site prediction
- model: RNA-FM on BEACON MRL: Mean ribosome loading prediction
- model: RNA-FM on BEACON ncRNA: Non-coding RNA family classification
- model: RNA-FM on BEACON PRS: Programmable RNA switch prediction
- model: RNA-FM on BEACON SPL: Splice site prediction
- model: RNA-FM on BEACON SSI: Structure score imputation
- model: RNA-FM on BEACON SSP: Secondary structure prediction
- model: RNA-FM on BEACON VDP: Vaccine degradation prediction
- subject: RNA-FM: family RNA-FM