[NeurIPS 2023 Main Track] This is the repository for the paper titled "Don’t Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner"

Repository for My HuggingFace Natural Language Processing Projects

PyTorch implementation for "Training and Inference on Any-Order Autoregressive Models the Right Way", NeurIPS 2022 Oral, TPM 2023 Best Paper Honorable Mention

11th place solution of NeurIPS 2024 - Predict New Medicines with BELKA competition on Kaggle: https://www.kaggle.com/competitions/leash-BELKA

Implementation of Transformer Encoders / Masked Language Modeling Objective

Measuring Biases in Masked Language Models for PyTorch Transformers. Support for multiple social biases and evaluation measures.

Official implementation of 'Masked Language Modeling Becomes Conditional Density Estimation for Tabular Data Synthesis' (MaCoDE) with pytorch (AAAI 2025 accepted paper).

The default way to fine-tune BERT is wrong. Here is why

Code for the publication of WWW'22

Continue T5 MLM pre-training on verbalized ConceptNet and fine-tune for commonsense question-answering

Transformer for Automatic Speech Recognition

External Knowledge Infusion using INCIDecoder into BERT for Chemical Mapping

Pre-training a Transformer from scratch.

Evaluation of zero-shot classification models on Turkish datasets.

Raw Torch, heavily annotated, pretrainable MPNet

An empirical study of Transformer adaptation techniques. Pre-training from scratch (MLM), classic fine-tuning, and from-scratch implementations of PEFT methods (LoRA, Adapters). Tuning both encoder (BERT) and decoder (OPT) models.

Data-driven approaches for identifying depression severity in social media posts.

Implicitly Adaptive Refinement Model

Use BERTRAM to get single-token embeddings for idioms on the MAGPIE dataset.

A Pre-trained Language Model for Semantic Similarity Measurement of Persian Informal Short Texts