songlab-cal/tape: tape/models/modeling_bert.py

What this source contributed

Records in this release that cite this source. A record can cite several sources.

• 5 models
• 1 method
• 6 evidence claims

Claims

Statements this source supports, each with where in the source it was checked.

• TAPE Bepler: description: Description Protein language model of Bepler et al. as benchmarked in TAPE: a two-layer bidirectional language model followed by three bidirectional LSTMs with 512 hidden units, trained by next-token prediction. (evidence-expansion-evidence-discovery-final-tape-6ee0c3e6: Models and Experimental Setup, Architectures and Training)
• TAPE LSTM: description: Description Protein language model in the TAPE benchmark: two three-layer LSTMs with 1,024 hidden units for the forward and backward directions, pretrained by next-token prediction. (evidence-expansion-evidence-discovery-final-tape-6ee0c3e6: Models and Experimental Setup, Architectures and Training)
• TAPE One Hot: description: Description Baseline featurisation in the TAPE benchmark that encodes the protein's amino-acid sequence one-hot, with no learned representation. (evidence-expansion-evidence-discovery-final-tape-6ee0c3e6: Models and Experimental Setup, Baselines ('a one-hot encoding of the input amino acid sequence, which provides a simple baseline'))
• TAPE ResNet: description: Description Protein language model in the TAPE benchmark: a dilated residual network of 35 blocks, each with two convolutional layers of 256 filters, kernel size 9 and dilation rate 2, pretrained by masked-token prediction. (evidence-expansion-evidence-discovery-final-tape-6ee0c3e6: Models and Experimental Setup, Architectures and Training)
• TAPE Transformer: description: Description Protein language model in the TAPE benchmark: a 12-layer Transformer with 512 hidden units and 8 attention heads (about 38 million parameters), pretrained by masked-token prediction. (evidence-expansion-evidence-discovery-final-tape-6ee0c3e6: Models and Experimental Setup, Architectures and Training)
• TAPE Unirep: description: Description Protein language model of Alley et al. (UniRep) as benchmarked in TAPE: a unidirectional multiplicative LSTM with 1,900 hidden units, trained by next-token prediction. (evidence-expansion-evidence-discovery-final-tape-6ee0c3e6: Models and Experimental Setup, Architectures and Training)

Models, benchmarks and other records

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.