Feature: Automated Job Board with RSS Feed Integration

[Manmit124]

Overview

Build an automated job board that fetches jobs from RSS feeds daily using Vercel Cron Jobs. Display jobs on /jobs page following NDC architecture patterns.

---

Tech Stack

• Database: Supabase (jobs table)
• Automation: Vercel Cron Jobs (free tier)
• Data Fetching: rss-parser npm package
• State Management: React Query (following NDC pattern)
• Frontend: Next.js App Router

---

Implementation Steps

1. Database Setup

• Create jobs table in Supabase with fields: id, title, company, description, location, remote, apply_url, source, date_posted, created_at
• Add unique constraint on apply_url (prevent duplicates)
• Add types to types/database.ts (Job, JobInsert, JobUpdate)
• Add query keys to lib/queryKeys.ts

2. Cron Job (Backend)

• Install rss-parser package
• Create app/api/cron/fetch-jobs/route.ts
• Fetch from RSS feeds (We Work Remotely, Remotive, etc.)
• Parse job data and save to Supabase
• Add duplicate prevention
• Secure with CRON_SECRET env variable

3. Vercel Cron Configuration

• Create vercel.json in project root
• Configure daily cron schedule (5 AM UTC)
• Add CRON_SECRET to Vercel environment variables

4. Data Layer (Hooks)

• Create hooks/api/useJobs.ts
• Add useJobs(filters) hook to fetch jobs
• Add useJob(id) hook for single job
• Follow existing NDC React Query patterns

5. UI Components

• Update app/jobs/page.tsx with real job listings
• Create components/Jobs/JobCard.tsx for job display
• Create components/Jobs/JobFilters.tsx for filtering
• Add loading states, error handling, empty states

6. Features

• Display jobs in card grid layout
• Filter by: remote, location, source
• Show source attribution (required)
• "Apply Now" button links to original posting
• Mobile responsive design

---

File Structure

types/database.ts                    ← Add Job types
lib/queryKeys.ts                     ← Add jobs keys
hooks/api/useJobs.ts                 ← New file
app/api/cron/fetch-jobs/route.ts     ← New file
app/jobs/page.tsx                    ← Update existing
components/Jobs/JobCard.tsx          ← New file
components/Jobs/JobFilters.tsx       ← New file
vercel.json                          ← New file

---

Dependencies

npm install rss-parser

---

Environment Variables

CRON_SECRET=your-random-secret-key

---

Timeline

• Database & Types: 1-2 hours
• Cron Job API: 2-3 hours
• Hooks: 1-2 hours
• UI Components: 3-4 hours
• Total: 7-11 hours

---

Sample of code

// Import the Parser library
import Parser from 'rss-parser';

// Define the URL of the RSS feed you want to fetch
const FEED_URL = 'https://weworkremotely.com/remote-jobs.rss';

// Define a TypeScript interface for our clean job data.
// This is what we will save to our database.
interface Job {
  title: string;
  applyUrl: string;
  datePosted: Date | null;
  companyName?: string; // We'll try to get this
  jobType?: string;     // From <type>
  category?: string;    // From <category>
  region?: string;      // From <region>
  descriptionHtml: string; // The full HTML description
  descriptionText: string; // The clean, plain-text description
}

/**
 * Main function to fetch, parse, and process jobs
 */
async function fetchJobs() {
  console.log('Starting job fetch...');
  
  // 1. Initialize the RSS parser
  // We can pass options here, but the defaults are often fine.
  const parser = new Parser();

  try {
    // 2. Fetch and parse the feed from the URL
    const feed = await parser.parseURL(FEED_URL);
    
    console.log(`Feed loaded: ${feed.title}`);
    console.log(`Found ${feed.items.length} jobs.`);
    console.log('---');

    const allJobs: Job[] = [];

    // 3. Loop through each item in the feed
    for (const item of feed.items) {
      
      // The `item` object from rss-parser is very rich.
      // Let's log the first item to see all available fields
      if (allJobs.length === 0) {
        console.log('Logging the first job item to show all fields:');
        console.log(item);
        console.log('---');
      }

      // 4. Create our own clean Job object
      const newJob: Job = {
        title: item.title || 'No title',
        applyUrl: item.link || '',
        datePosted: item.isoDate ? new Date(item.isoDate) : null,
        
        // This is the full HTML from the <description> tag
        descriptionHtml: item.content || '', 
        
        // This is a plain-text version generated by rss-parser!
        // It automatically strips the HTML tags.
        descriptionText: item.contentSnippet || '', 

        // 5. Accessing the custom fields from your XML:
        // rss-parser adds them directly to the item object.
        jobType: item.type,     // From <type>
        category: item.category, // From <category>
        region: item.region,   // From <region>
      };

      allJobs.push(newJob);
    }

    // 6. At this point, you would save `allJobs` to your database.
    // For this example, we'll just log the title and text of the first 3 jobs.
    
    console.log('---');
    console.log('Processing complete. Showing first 3 jobs:');

    for (let i = 0; i < 3 && i < allJobs.length; i++) {
      const job = allJobs[i];
      console.log(`\nJob Title: ${job.title}`);
      console.log(`Type: ${job.jobType}`);
      console.log(`Category: ${job.category}`);
      console.log(`Region: ${job.region}`);
      console.log(`Apply: ${job.applyUrl}`);
      console.log(`Description (Plain Text): ${job.descriptionText.substring(0, 150)}...`);
    }
    
    return allJobs;

  } catch (error) {
    console.error('Error fetching or parsing RSS feed:', error);
  }
}

// Run the main function
fetchJobs();

Success Criteria

• ✅ Cron runs daily and fetches jobs
• ✅ Jobs saved to database without duplicates
• ✅ Jobs display on /jobs page
• ✅ Filters work correctly
• ✅ Source attribution visible
• ✅ Mobile responsive