A Rails gem that adds AI-powered screen analysis to any application. With a simple helper, you can capture the visual content of any page, send it for analysis with GPT-4V, and display the result in an interactive sidebar.
- Customizable Floating Button: A sleek button that appears on any page.
- Automatic Screenshot Capture: Uses
html2canvasto capture the visual content of the page. - AI Analysis: Integration with OpenAI's GPT-4V for intelligent analysis.
- Responsive Sidebar: Displays results in an elegant, non-intrusive sidebar.
- Customizable Prompts: Configure different prompts for different use cases.
- Easy Integration: Just add one line of code to your layout.
- No External Dependencies: Uses only standard Rails libraries.
Add the gem to your Gemfile:
gem 'ai_screen_analyzer'Run bundler:
bundle installRun the installation generator:
rails generate ai_screen_analyzer:installThe generator automatically mounts the engine in your host application. After installation, these routes are available by default:
- Engine mounted at:
/ai_screen_analyzer - Analysis endpoint:
POST /ai_screen_analyzer/analyze-screen
If you need to mount manually (or customize the path), add this to your app's config/routes.rb:
# Mount the AiScreenAnalyzer engine, exposing its routes under /ai_screen_analyzer
mount AiScreenAnalyzer::Engine => '/ai_screen_analyzer', as: 'ai_screen_analyzer_engine'
# (Optional) Direct alias for compatibility with existing calls
post '/ai_screen_analyzer/analyze-screen', to: 'ai_screen_analyzer/ai_analyzer#analyze_screen'Notes:
- The gem's view uses route helpers to resolve the endpoint dynamically according to the mounted path, avoiding 404s when you customize the path.
- If you change the mount path, you don't need to change anything in the view; the helper will point to the new path.
Set your OpenAI API key as an environment variable:
export OPENAI_API_KEY='your_secret_key_here'Or configure it in config/initializers/ai_screen_analyzer.rb:
AiScreenAnalyzer.configure do |config|
config.api_key = ENV['OPENAI_API_KEY']
config.model = 'gpt-4-turbo'
endOpen your main layout (app/views/layouts/application.html.erb) and add:
<body>
<%= yield %>
<%= ai_screen_analyzer %>
</body><%= ai_screen_analyzer %><%= ai_screen_analyzer(
prompt: "You are a UX expert. Analyze this interface and provide feedback.",
button_text: "Review Design",
sidebar_title: "UX Feedback"
) %>| Option | Description | Default |
|---|---|---|
prompt |
The prompt text sent to the AI along with the image | "Analyze the visible content on this screen..." |
button_text |
Text displayed on the floating button | "Analyze with AI" |
sidebar_title |
Title of the results sidebar | "AI Analysis" |
<%= ai_screen_analyzer(
prompt: "You are a UX/UI expert. Analyze this interface and provide feedback on: 1) Visual hierarchy, 2) Accessibility, 3) Design consistency, 4) Improvement suggestions.",
button_text: "Review Design",
sidebar_title: "UX/UI Feedback"
) %><%= ai_screen_analyzer(
prompt: "You are a web accessibility (WCAG) specialist. Analyze this page and identify potential accessibility issues.",
button_text: "Check Accessibility",
sidebar_title: "Accessibility Analysis"
) %><%= ai_screen_analyzer(
prompt: "Analyze the textual content of this page. Evaluate: 1) Clarity of message, 2) Writing quality, 3) Tone appropriateness.",
button_text: "Analyze Content",
sidebar_title: "Content Analysis"
) %>- The user clicks the "Analyze with AI" button.
- The
html2canvasJavaScript library captures an image of the page body. - The image is converted to base64 and sent to the Rails backend.
- The controller processes the image and sends it to the GPT-4V API.
- The AI analyzes the image according to the provided prompt.
- The result is returned and displayed in the sidebar.
ai_screen_analyzer/
├── app/
│ ├── controllers/
│ │ └── ai_screen_analyzer/
│ │ └── ai_analyzer_controller.rb
│ ├── helpers/
│ │ └── ai_screen_analyzer/
│ │ └── ai_screen_analyzer_helper.rb
│ └── views/
│ └── ai_screen_analyzer/
│ └── shared/
│ └── _ai_screen_analyzer.html.erb
├── config/
│ └── routes.rb
├── lib/
│ ├── ai_screen_analyzer.rb
│ ├── ai_screen_analyzer/
│ │ ├── engine.rb
│ │ └── version.rb
│ └── generators/
│ └── ai_screen_analyzer/
│ └── install_generator.rb
├── ai_screen_analyzer.gemspec
├── Rakefile
├── README.md
├── LICENSE.txt
└── CHANGELOG.md
- Rails 6.0 or higher
- Ruby 2.7.0 or higher
- Valid OpenAI API key (with access to the gpt-4-turbo model)
Make sure the helper is being called correctly in your layout and there are no errors in the browser console.
Ensure that:
- The OpenAI API key is configured correctly
- You have sufficient credits in your OpenAI account
- The route is registered correctly
-
Ensure the engine is mounted in the host app. By default, the generator already adds this line to your
config/routes.rb:mount AiScreenAnalyzer::Engine => '/ai_screen_analyzer', as: 'ai_screen_analyzer_engine'
-
If you removed the automatic mount, add it manually as above.
-
Restart the server after changing routes.
-
Run
bin/rails routes | grep ai_screen_analyzerto confirm the presence of theanalyze-screenendpoint.
If you receive CORS errors when capturing the screen, ensure that all external resources are using HTTPS.
Contributions are welcome! Please open an issue or submit a pull request.
This gem is licensed under the MIT License. See LICENSE.txt for details.
For support, open an issue in the GitHub repository or contact the maintainers.
See the CHANGELOG.md file for a list of all changes.
- Support for other AI models (Claude, Gemini, etc.)
- Cache of recent analyses