Simple Text Generation using FLAN-T5

A text generation implementation using Google's FLAN-T5-small model via Hugging Face Transformers for text completion and basic chatbot functionality.

Overview

This project contains a Jupyter notebook (SimpleTextGeneration.ipynb) that demonstrates text generation capabilities using a pre-trained FLAN-T5 model. The notebook shows basic text completion and simple chatbot interactions.

Technology Stack

Core Technologies

• PyTorch: Deep learning framework for tensor operations and model handling
• Hugging Face Transformers: Provides pre-trained models and tokenizers
• FLAN-T5-small: Google's instruction-tuned T5 model for text-to-text generation

Internal Tool Components

• AutoModelForSeq2SeqLM: Hugging Face's automatic model loading for sequence-to-sequence tasks
• AutoTokenizer: Automatic tokenizer selection matching the model
• Pipeline: High-level interface for text2text-generation tasks
• MPS Backend: Utilizes Apple Metal Performance Shaders for GPU acceleration (mps:0)

Additional Dependencies

• torch: PyTorch core library
• torchvision: Computer vision utilities (imported but not actively used)
• matplotlib: Plotting library (imported but not actively used)

Model Details

• Model: google/flan-t5-small
• Type: Sequence-to-sequence (text-to-text) generation
• Architecture: T5 (Text-To-Text Transfer Transformer)
• Size: Small variant (efficient for basic tasks)
• Instruction-tuned: Optimized for following natural language instructions

Functionality Demonstrated

1. Text Completion

prompt = "Complete this sentence: 'Taylor Swift is a'"
response = generator(prompt, max_length=50)
# Output: "singer"

2. Basic Chatbot - Factual Questions

chatbot_prompt = "You are a friendly AI assistant. Answer the user's question with a helpful response."
messages = [{"role": "user", "content": "Tell me a fact about the Sun."}]
# Output: "the sun is a source of heat"

3. Factual Information Retrieval

messages = [{"role": "user", "content": "Tell me a fact about the Taylor Swift."}]
# Output: "Taylor Swift is an American singer."

Usage

Prerequisites

pip install torch torchvision transformers matplotlib

Running the Notebook

1. Open the Jupyter notebook:

jupyter notebook SimpleTextGeneration.ipynb

2. Run the cells sequentially to:
   • Load the FLAN-T5-small model and tokenizer
   • Create a text generation pipeline
   • Test text completion functionality
   • Demonstrate basic chatbot capabilities

Using the Text Generator

Basic Setup

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, pipeline

model_name = "google/flan-t5-small"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
generator = pipeline("text2text-generation", model=model, tokenizer=tokenizer)

Text Completion

prompt = "Your prompt here"
response = generator(prompt, max_length=50)
print(response[0]["generated_text"])

Chatbot Interaction

chatbot_prompt = "You are a friendly AI assistant. Answer the user's question with a helpful response."
user_question = "Your question here"
response = generator(f"{chatbot_prompt} {user_question}", max_length=50)
print(response[0]["generated_text"])

Configuration Parameters

• max_length: Maximum length of generated text (set to 50)
• max_new_tokens: Default 256 tokens (takes precedence over max_length)
• device: Automatically uses MPS (Metal Performance Shaders) on compatible Apple devices

System Requirements

Hardware Acceleration

• MPS Support: Utilizes Apple Silicon GPU when available
• Device: Automatically detected and set to mps:0

Model Loading

• Automatic Download: Model and tokenizer downloaded from Hugging Face Hub
• Local Caching: Models cached locally after first download

Example Outputs

The notebook demonstrates these specific interactions:

1. Text Completion:

   • Input: "Complete this sentence: 'Taylor Swift is a'"
   • Output: "singer"

2. Sun Fact Query:

   • Input: "Tell me a fact about the Sun."
   • Output: "the sun is a source of heat"

3. Taylor Swift Fact Query:

   • Input: "Tell me a fact about the Taylor Swift."
   • Output: "Taylor Swift is an American singer."

Technical Notes

Warnings

• The notebook shows warnings about conflicting max_new_tokens and max_length parameters
• max_new_tokens takes precedence when both are specified

Performance

• Uses Apple's MPS for GPU acceleration on compatible hardware
• Small model variant ensures fast inference times
• Efficient memory usage suitable for local development

Potential Extensions

The current implementation can be extended for:

• Longer Conversations: Maintaining conversation context
• Custom Prompting: Different instruction templates
• Batch Processing: Multiple text generation requests
• Fine-tuning: Adapting the model for specific domains
• Interactive Interface: Building a web-based chatbot

Getting Started

1. Clone this repository
2. Install required dependencies
3. Open and run the SimpleTextGeneration.ipynb notebook
4. The model will automatically download on first run
5. Experiment with different prompts and questions

Model Limitations

• Short Responses: FLAN-T5-small generates concise answers
• Context Window: Limited input length for prompts
• Domain Knowledge: Based on training data cutoff
• Instruction Following: Works best with clear, direct instructions