-
Notifications
You must be signed in to change notification settings - Fork 17
Use Google Calendar API calendar invites / Implement Push Notifications #1529
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
davinotdavid
wants to merge
48
commits into
main
Choose a base branch
from
google-ics-cards
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
48 commits
Select commit
Hold shift + click to select a range
1276b7a
Update mailer attachments and ics file handling
davinotdavid 1823cdd
Add insert_event method to GoogleClient and format body accordingly
davinotdavid f965c5d
Split logic between Google accounts vs non-Google accounts
davinotdavid ce53dc2
Update google connector tests
davinotdavid bba6711
Add google_calendar_channels table and model repo methods
davinotdavid 1e212e0
Implement Google Calendar's Push Notifications API methods
davinotdavid ebb8bed
Add POST to /google-calendar as webhook to receive calendar notificat…
davinotdavid b7452bc
Setup watch channels when connecting a google account
davinotdavid 1b8d5a8
Move google calendar watch methods into its own controller
davinotdavid 6aea4a1
Setup and teadown watch channels when calendar connection changes
davinotdavid 034d50c
Add backfill (to be ran once) and renew (periodically) to maintain ex…
davinotdavid 12ae74d
Add tests
davinotdavid 106fc7e
Fix lint
davinotdavid 2e85bf1
Create google events as tentative and add patch_event method
davinotdavid 8573587
Skip HOLD event for Google events, and confirm through event through …
davinotdavid d8c1b68
Only care about the declined in webhook
davinotdavid acfa2d2
Update tests
davinotdavid 46e1f47
Only create watch channels for default connected calendars
davinotdavid 7ad57f8
Add tests for backfill and renew channel commands
davinotdavid a929403
Fix tests regarding setup and teardown of watch channels during calen…
davinotdavid 195cfa5
Fix schedule tests to not depend on a real clock anymore
davinotdavid a6e85a0
Remove unnecessary setup_watch_channels_for_connection method
davinotdavid 257c3a7
Revert changes to the ics and mailer
davinotdavid a46d3f2
Add Subscriber to attendee list with NeedsAction if booking_confirmation
davinotdavid d60de30
Subscribers actions in Google Calendar should backtrack the booking s…
davinotdavid 18f1c67
Add tests for Susbcribers actions in Google Calendar
davinotdavid eb03569
Adding the organizer in the google event regardless of booking confir…
davinotdavid e7d32e6
Also make a remote event when there is no need to require booking con…
davinotdavid be50f5f
More test coverage
davinotdavid 547b9c9
Add GOOGLE_INVITE_ENABLED env var to control feature
davinotdavid ce5f105
Add tests for GOOGLE_INVITE_ENABLED fallback into branded emails
davinotdavid 508bb6f
Add or create Zoom meeting link to the invite for Google Invite flows
davinotdavid 72651fd
Update test suite with new coverage for Zoom meeting links
davinotdavid 096abbb
Attempt to fix CodeQL false positive in test
davinotdavid 022371c
No need for a helper function to check for GOOGLE_INVITE_ENABLED
davinotdavid 00b940e
Consolidate 200 successful response in a variable
davinotdavid ec36a80
Remove early success return in webhook in case the channel don't have…
davinotdavid 3ff14c2
Share code for creating zoom meeting link
davinotdavid 522e171
Update tests
davinotdavid eb12551
Add GOOGLE_CHANNEL_TTL_IN_SECONDS to safeguard against unexpected API…
davinotdavid 49ecaa0
Better query for google calendars in a schedule
davinotdavid 3b25ac0
Only sync watch channels if GOOGLE_INVITE_ENABLED is True
davinotdavid a9af4b1
Add state to the google_calendar_channels table, populate it and chec…
davinotdavid a8de1db
Better returns and erroring logs
davinotdavid 9e669f9
Make db query do the filtering instead of looping myself
davinotdavid 24b59dd
Update tests
davinotdavid 1db272e
Add renewal of google channels to Celery
davinotdavid 29c348d
(Re)fixing the schedule_calendar_ids query
davinotdavid File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -40,6 +40,9 @@ def create_celery_app() -> Celery: | |
| sentry_sdk.set_extra('CELERY_RESULT_EXPIRES', result_expires) | ||
| sentry_sdk.set_extra('CELERY_TASK_ALWAYS_EAGER', task_always_eager) | ||
|
|
||
| google_channel_ttl = float(os.getenv('GOOGLE_CHANNEL_TTL_IN_SECONDS', 604800)) | ||
| google_channel_renew_interval = google_channel_ttl - 86400 # 1 day buffer | ||
|
|
||
| app = Celery('appointment') | ||
|
|
||
| app.config_from_object({ | ||
|
|
@@ -59,6 +62,10 @@ def create_celery_app() -> Celery: | |
| 'task': 'appointment.tasks.health.heartbeat', | ||
| 'schedule': 60.0, | ||
| }, | ||
| 'renew-google-channels': { | ||
| 'task': 'appointment.tasks.google.renew_google_channels', | ||
| 'schedule': google_channel_renew_interval, | ||
| }, | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should probably store this outside of this object, but we can do that later. |
||
| }, | ||
| 'beat_schedule_filename': 'celerybeat-appointment-schedule', | ||
| }) | ||
|
|
||
101 changes: 101 additions & 0 deletions
101
backend/src/appointment/commands/backfill_google_channels.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| """One-off command to set up Google Calendar watch channels for existing connected calendars.""" | ||
|
|
||
| import json | ||
| import logging | ||
| import uuid | ||
| from datetime import datetime, timezone | ||
|
|
||
| from google.oauth2.credentials import Credentials | ||
|
|
||
| from ..controller.google_watch import get_webhook_url | ||
| from ..database import repo, models | ||
| from ..dependencies.database import get_engine_and_session | ||
| from ..dependencies.google import get_google_client | ||
| from ..main import _common_setup | ||
|
|
||
|
|
||
| def run(): | ||
| _common_setup() | ||
| google_client = get_google_client() | ||
|
|
||
| _, SessionLocal = get_engine_and_session() | ||
| db = SessionLocal() | ||
|
|
||
| webhook_url = get_webhook_url() | ||
| if not webhook_url: | ||
| print('BACKEND_URL not set, aborting.') | ||
| db.close() | ||
| return | ||
|
|
||
| # Find connected Google calendars that are the default in a schedule | ||
| # and don't yet have a watch channel | ||
| schedule_calendar_ids = [ | ||
| s.calendar_id for s in | ||
| db.query(models.Schedule.calendar_id).filter(models.Schedule.calendar_id != None).all() # noqa: E711 | ||
| ] | ||
|
|
||
| all_calendars = db.query(models.Calendar).filter( | ||
| models.Calendar.provider == models.CalendarProvider.google, | ||
| models.Calendar.connected == True, # noqa: E712 | ||
| models.Calendar.id.in_(schedule_calendar_ids), | ||
| ).all() | ||
|
|
||
| candidates = [] | ||
| for cal in all_calendars: | ||
| existing = repo.google_calendar_channel.get_by_calendar_id(db, cal.id) | ||
| if not existing: | ||
| candidates.append(cal) | ||
|
|
||
| print(f'Found {len(candidates)} connected Google calendar(s) without a watch channel.') | ||
|
|
||
| created = 0 | ||
| skipped = 0 | ||
| failed = 0 | ||
|
|
||
| for calendar in candidates: | ||
| ext_conn = calendar.external_connection | ||
| if not ext_conn or not ext_conn.token: | ||
| print(f' Calendar {calendar.id}: no external connection or token, skipping.') | ||
| skipped += 1 | ||
| continue | ||
|
|
||
| try: | ||
| token = Credentials.from_authorized_user_info( | ||
| json.loads(ext_conn.token), google_client.SCOPES | ||
| ) | ||
| except Exception as e: | ||
| print(f' Calendar {calendar.id}: failed to parse token ({e}), skipping.') | ||
| skipped += 1 | ||
| continue | ||
|
|
||
| try: | ||
| state = str(uuid.uuid4()) | ||
| response = google_client.watch_events(calendar.user, webhook_url, token, state=state) | ||
| if not response: | ||
| print(f' Calendar {calendar.id}: watch_events returned no response.') | ||
| failed += 1 | ||
| continue | ||
|
|
||
| expiration_ms = int(response.get('expiration', 0)) | ||
| expiration_dt = datetime.fromtimestamp(expiration_ms / 1000, tz=timezone.utc) | ||
|
|
||
| sync_token = google_client.get_initial_sync_token(calendar.user, token) | ||
|
|
||
| repo.google_calendar_channel.create( | ||
| db, | ||
| calendar_id=calendar.id, | ||
| channel_id=response['id'], | ||
| resource_id=response['resourceId'], | ||
| expiration=expiration_dt, | ||
| state=state, | ||
| sync_token=sync_token, | ||
| ) | ||
| created += 1 | ||
| print(f' Calendar {calendar.id}: channel created (expires {expiration_dt}).') | ||
| except Exception as e: | ||
| print(f' Calendar {calendar.id}: failed ({e}).') | ||
| logging.error(f'[backfill_google_channels] Error for calendar {calendar.id}: {e}') | ||
| failed += 1 | ||
|
|
||
| db.close() | ||
| print(f'\nBackfill complete: {created} created, {skipped} skipped, {failed} failed.') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| """Renew expiring Google Calendar push notification channels. | ||
|
|
||
| Run periodically (e.g., daily) to ensure channels don't expire. | ||
| Google channels typically last ~7 days, so daily renewal keeps a buffer. | ||
| """ | ||
|
|
||
| import json | ||
| import logging | ||
| import uuid | ||
| from datetime import datetime, timedelta, timezone | ||
|
|
||
| from google.oauth2.credentials import Credentials | ||
|
|
||
| from ..controller.google_watch import get_webhook_url | ||
| from ..database import repo | ||
| from ..dependencies.database import get_engine_and_session | ||
| from ..dependencies.google import get_google_client | ||
| from ..main import _common_setup | ||
|
|
||
|
|
||
| def run(): | ||
| _common_setup() | ||
| google_client = get_google_client() | ||
|
|
||
| _, SessionLocal = get_engine_and_session() | ||
| db = SessionLocal() | ||
|
|
||
| webhook_url = get_webhook_url() | ||
| if not webhook_url: | ||
| logging.error('[renew_google_channels] BACKEND_URL not set, aborting') | ||
| db.close() | ||
| return | ||
|
|
||
| # Renew channels that expire within the next 24 hours | ||
| threshold = datetime.now(tz=timezone.utc) + timedelta(hours=24) | ||
| channels = repo.google_calendar_channel.get_expiring(db, before=threshold) | ||
|
|
||
| renewed = 0 | ||
| failed = 0 | ||
|
|
||
| for channel in channels: | ||
| calendar = channel.calendar | ||
| if not calendar or not calendar.connected: | ||
| repo.google_calendar_channel.delete(db, channel) | ||
| continue | ||
|
|
||
| external_connection = calendar.external_connection | ||
| if not external_connection or not external_connection.token: | ||
| repo.google_calendar_channel.delete(db, channel) | ||
| continue | ||
|
|
||
| token = Credentials.from_authorized_user_info( | ||
| json.loads(external_connection.token), google_client.SCOPES | ||
| ) | ||
|
|
||
| # Stop the old channel | ||
| try: | ||
| google_client.stop_channel(channel.channel_id, channel.resource_id, token) | ||
| except Exception as e: | ||
| logging.warning(f'[renew_google_channels] Failed to stop old channel {channel.channel_id}: {e}') | ||
|
|
||
| # Create a new channel | ||
| try: | ||
| new_state = str(uuid.uuid4()) | ||
| response = google_client.watch_events(calendar.user, webhook_url, token, state=new_state) | ||
| if response: | ||
| expiration_ms = int(response.get('expiration', 0)) | ||
| expiration_dt = datetime.fromtimestamp(expiration_ms / 1000, tz=timezone.utc) | ||
|
|
||
| repo.google_calendar_channel.update_expiration( | ||
| db, | ||
| channel, | ||
| new_channel_id=response['id'], | ||
| new_resource_id=response['resourceId'], | ||
| new_expiration=expiration_dt, | ||
| new_state=new_state, | ||
| ) | ||
| renewed += 1 | ||
| else: | ||
| repo.google_calendar_channel.delete(db, channel) | ||
| failed += 1 | ||
| except Exception as e: | ||
| logging.error(f'[renew_google_channels] Failed to renew channel for calendar {calendar.id}: {e}') | ||
| failed += 1 | ||
|
|
||
| db.close() | ||
| print(f'Channel renewal complete: {renewed} renewed, {failed} failed, {len(channels)} total processed') |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have the ability to handle watch channels locally? If not then we should default this to false.
We might want to split the variables for google invites vs handling watch channels though.