Skip to content

Adding a webhook breaks scraper settings page until the next run #1457

@ianheggie-oaf

Description

@ianheggie-oaf

Describe the bug

I add a webhook url so my private scrapers alert plannies-mate know when they run. A few days later all but one (https://morph.io/ianheggie-oaf/multiple_civica-prs ) is back and working as the scrapers (aside from civica) have run in the mean time.

To Reproduce
Steps to reproduce the behavior:

  1. Go to a scraper
  2. Click on 'Settings'
  3. Scroll down to 'webhooks'
  4. Add a webhook like https://plannies-mate.example.com/callback?api_key=VerySecretPassword&private=1&scrape_tables=1&scraper=your-name/your-scraper-name
  5. This returns you to the scraper page
  6. Click Settings button again
  7. You will see We're sorry, but something went wrong.

Expected behavior

Should get the settings page again (it worked a few times, but most of the time it crashed, eg
https://morph.io/ianheggie-oaf/multiple_cianywhere-prs/settings
and
https://morph.io/ianheggie-oaf/multiple_civica-prs/settings

The webhook should have an indication beside it if the last callback was successful or not (or has yet to occur).

Screenshots/Output

We're sorry, but something went wrong.
If you are the application owner check the logs for more information.

Additional context

I need this to work, as I need someway of knowing that my private (multiple_*) scrapers are being run for my plannies-mate site - this is a result of making them private (I can't scrape the list of scrapers and their test results pages)

  • TypeError: Return value: Expected type WebhookDelivery, got type NilClass
  • Caller: app/helpers/scrapers_helper.rb:75
  • Definition: app/models/webhook.rb:34
Backtrace:
line 75 of [PROJECT_ROOT]/app/helpers/scrapers_helper.rb: webhook_last_delivery_status
line 41 of [PROJECT_ROOT]/app/views/scrapers/_settings.html.haml: block (3 levels) in _app_views_scrapers__settings_html_haml___3400369562261668251_407734160
line 40 of [PROJECT_ROOT]/app/views/scrapers/_settings.html.haml: block (2 levels) in _app_views_scrapers__settings_html_haml___3400369562261668251_407734160

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions