Add support for ingesting etag data from API endpoints #445

eddorre · 2025-06-07T09:05:46Z

Most of the YouTube API endpoints that return resources expose an etag property. It is highly desireable for consumers to be able to have access to this property in order to perform conditional requests (using If-Match / If-None-Match) or to verify that resources have changed.

Some examples of YouTube API endpoints that support etag:

The following sampling of commands should retrieve the etag for a resource (assume that the local variable account is an instance of a Yt::Models::Account). Note that I'm just using playlists here as an example. It should work for any resource that supports etags in the YouTube Data API.

Yt::Playlist.new(id: youtube_playlist_id, auth: account).etag #(this only works if there is an id) ✅ 

account.playlists.etag ✅

account.playlists.first.etag ✅

account.playlists.first.playlist_items.etag ✅

account.playlists.first.playlist_items.first.etag ✅

account.playlists.first.playlist_items.first.video.etag ✅

account.playlists.first.playlist_items.first.video.captions.etag ✅

account.playlists.first.playlist_items.first.video.captions.first.etag ✅

kangkyu · 2025-06-09T20:21:21Z

How about etag for a list response? channelListResponse etc

eddorre · 2025-06-11T21:46:07Z

@kangkyu Working on it right now. 😁

eddorre · 2025-06-12T06:33:58Z

@kangkyu I've update the PR (note that I tested changes in a local branch, merged them and rebased them into this one).

kangkyu · 2025-06-16T05:12:20Z

Thank you. I still want to try YouTube API requests as a test. You don't have to make all the existing integration tests pass, but you can run the tests you need with VCR locally (you don't have to share your cassettes) so I can run it locally too. We don't use mocks for Yt::Request normally.

eddorre · 2025-06-19T10:51:24Z

@kangkyu I ran the following:

bundle exec rspec spec/models/channel_spec.rb spec/models/playlist_spec.rb spec/models/video_spec.rb spec/collections/channels_spec.rb spec/collections/playlists_spec.rb spec/collections/playlist_items_spec.rb spec/collections/subscriptions_spec.rb spec/collections/videos_spec.rb spec/collections/comment_threads_spec.rb

197 examples, 0 failures

Let me know if you need anything else.

kangkyu · 2025-06-19T16:28:33Z

Sorry maybe I wasn't clear. Your tests in this pr didn't include calling youtube api directly. But I think we need some.

eddorre · 2025-06-22T01:37:15Z

@kangkyu Totally misunderstood what you were asking for, that's my fault. I have created request specs that you should be able to generate your own local cassettes for. Let me know how it looks.

kangkyu · 2025-06-30T03:58:28Z

Hi, when I called channel.etag for example

channel = Yt::Channel.new(id: 'UCJkWoS4RsldA1coEIot5yDA') # MotherGooseClub
channel.etag

With your code it returns "channelListResponse" etag, not "channel" etag. I am wondering if that intended. Because there are two etags for one response (one for channel list, and one for each channel) I am not sure which one fits your need @eddorre

        {
          "kind": "youtube#channelListResponse",
          "etag": "NEj_NTR9QZSTjmVYensEZdOiZRo",
          "pageInfo": {
            "totalResults": 1,
            "resultsPerPage": 50
          },
          "items": [
            {
              "kind": "youtube#channel",
              "etag": "9YK18ywsZCCx2mVnFMVj5fqZ-Co",
              "id": "UCJkWoS4RsldA1coEIot5yDA",
              "snippet": {
                "title": "Mother Goose Club",
...

eddorre · 2025-07-25T00:55:54Z

@kangkyu Sorry, got a bit busy there. You are absolutely right. I'm calling the where method on a resource collection where the id matches the id of the resource. Just like ActiveRecord this is a collection of entities and not a singular one. I'll create a patch for this and do some testing.

Most of the YouTube API endpoints that return resources expose an etag property. It is highly desirable for consumers to be able to have access to this property in order to perform conditional requests (using If-Match / If-None-Match) or to verify that resources have changed.

eddorre · 2025-07-28T06:33:11Z

@kangkyu I've pushed up the latest iteration. Here is an example of how some common resources working. I'm showing both raw JSON response and the result of an IRB session:

Video

JSON

{
  "kind": "youtube#videoListResponse",
  "etag": "_v_AVE2jJr8NSI8VjSfn8dav8fI",
  "items": [
    {
      "kind": "youtube#video",
      "etag": "pMtrD11jJwfAb08mUCrAm18HSXo",
      "id": "dQw4w9WgXcQ",
      "snippet": {
        "publishedAt": "2009-10-25T06:57:33Z",
        "channelId": "UCuAXFkgsw1L7xaCfnd5JJOw",
        "title": "Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster)",
        "description": "The official video for “Never Gonna Give You Up” by Rick Astley. \n\nNever: The Autobiography 📚 OUT NOW! \nFollow this link to get your copy and listen to Rick’s ‘Never’ playlist ❤️ #RickAstleyNever\nhttps://linktr.ee/rickastleynever\n\n“Never Gonna Give You Up” was a global smash on its release in July 1987, topping the charts in 25 countries including Rick’s native UK and the US Billboard Hot 100.  It also won the Brit Award for Best single in 1988. Stock Aitken and Waterman wrote and produced the track which was the lead-off single and lead track from Rick’s debut LP “Whenever You Need Somebody”.  The album was itself a UK number one and would go on to sell over 15 million copies worldwide.\n\nThe legendary video was directed by Simon West – who later went on to make Hollywood blockbusters such as Con Air, Lara Croft – Tomb Raider and The Expendables 2.  The video passed the 1bn YouTube views milestone on 28 July 2021.\n\nSubscribe to the official Rick Astley YouTube channel: https://RickAstley.lnk.to/YTSubID\n\nFollow Rick Astley:\nFacebook: https://RickAstley.lnk.to/FBFollowID \nTwitter: https://RickAstley.lnk.to/TwitterID \nInstagram: https://RickAstley.lnk.to/InstagramID \nWebsite: https://RickAstley.lnk.to/storeID \nTikTok: https://RickAstley.lnk.to/TikTokID\n\nListen to Rick Astley:\nSpotify: https://RickAstley.lnk.to/SpotifyID \nApple Music: https://RickAstley.lnk.to/AppleMusicID \nAmazon Music: https://RickAstley.lnk.to/AmazonMusicID \nDeezer: https://RickAstley.lnk.to/DeezerID \n\nLyrics:\nWe’re no strangers to love\nYou know the rules and so do I\nA full commitment’s what I’m thinking of\nYou wouldn’t get this from any other guy\n\nI just wanna tell you how I’m feeling\nGotta make you understand\n\nNever gonna give you up\nNever gonna let you down\nNever gonna run around and desert you\nNever gonna make you cry\nNever gonna say goodbye\nNever gonna tell a lie and hurt you\n\nWe’ve known each other for so long\nYour heart’s been aching but you’re too shy to say it\nInside we both know what’s been going on\nWe know the game and we’re gonna play it\n\nAnd if you ask me how I’m feeling\nDon’t tell me you’re too blind to see\n\nNever gonna give you up\nNever gonna let you down\nNever gonna run around and desert you\nNever gonna make you cry\nNever gonna say goodbye\nNever gonna tell a lie and hurt you\n\n#RickAstley #NeverGonnaGiveYouUp #WheneverYouNeedSomebody #OfficialMusicVideo",
        "thumbnails": {
          "default": {
            "url": "https://i.ytimg.com/vi/dQw4w9WgXcQ/default.jpg",
            "width": 120,
            "height": 90
          },
          "medium": {
            "url": "https://i.ytimg.com/vi/dQw4w9WgXcQ/mqdefault.jpg",
            "width": 320,
            "height": 180
          },
          "high": {
            "url": "https://i.ytimg.com/vi/dQw4w9WgXcQ/hqdefault.jpg",
            "width": 480,
            "height": 360
          },
          "standard": {
            "url": "https://i.ytimg.com/vi/dQw4w9WgXcQ/sddefault.jpg",
            "width": 640,
            "height": 480
          },
          "maxres": {
            "url": "https://i.ytimg.com/vi/dQw4w9WgXcQ/maxresdefault.jpg",
            "width": 1280,
            "height": 720
          }
        },
        "channelTitle": "Rick Astley",
        "tags": [
          "rick astley",
          "Never Gonna Give You Up",
          "nggyu",
          "never gonna give you up lyrics",
          "rick rolled",
          "Rick Roll",
          "rick astley official",
          "rickrolled",
          "Fortnite song",
          "Fortnite event",
          "Fortnite dance",
          "fortnite never gonna give you up",
          "rick roll",
          "rickrolling",
          "rick rolling",
          "never gonna give you up",
          "80s music",
          "rick astley new",
          "animated video",
          "rickroll",
          "meme songs",
          "never gonna give u up lyrics",
          "Rick Astley 2022",
          "never gonna let you down",
          "animated",                                                                                                                                                                                                                                             "rick rolls 2022",
          "never gonna give you up karaoke"
        ],
        "categoryId": "10",
        "liveBroadcastContent": "none",
        "defaultLanguage": "en",
        "localized": {
          "title": "Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster)",
          "description": "The official video for “Never Gonna Give You Up” by Rick Astley. \n\nNever: The Autobiography 📚 OUT NOW! \nFollow this link to get your copy and listen to Rick’s ‘Never’ playlist ❤️ #RickAstleyNever\nhttps://linktr.ee/rickastleynever\n\n“Never Gonna Give You Up” was a global smash on its release in July 1987, topping the charts in 25 countries including Rick’s native UK and the US Billboard Hot 100.  It also won the Brit Award for Best single in 1988. Stock Aitken and Waterman wrote and produced the track which was the lead-off single and lead track from Rick’s debut LP “Whenever You Need Somebody”.  The album was itself a UK number one and would go on to sell over 15 million copies worldwide.\n\nThe legendary video was directed by Simon West – who later went on to make Hollywood blockbusters such as Con Air, Lara Croft – Tomb Raider and The Expendables 2.  The video passed the 1bn YouTube views milestone on 28 July 2021.\n\nSubscribe to the official Rick Astley YouTube channel: https://RickAstley.lnk.to/YTSubID\n\nFollow Rick Astley:\nFacebook: https://RickAstley.lnk.to/FBFollowID \nTwitter: https://RickAstley.lnk.to/TwitterID \nInstagram: https://RickAstley.lnk.to/InstagramID \nWebsite: https://RickAstley.lnk.to/storeID \nTikTok: https://RickAstley.lnk.to/TikTokID\n\nListen to Rick Astley:\nSpotify: https://RickAstley.lnk.to/SpotifyID \nApple Music: https://RickAstley.lnk.to/AppleMusicID \nAmazon Music: https://RickAstley.lnk.to/AmazonMusicID \nDeezer: https://RickAstley.lnk.to/DeezerID \n\nLyrics:\nWe’re no strangers to love\nYou know the rules and so do I\nA full commitment’s what I’m thinking of\nYou wouldn’t get this from any other guy\n\nI just wanna tell you how I’m feeling\nGotta make you understand\n\nNever gonna give you up\nNever gonna let you down\nNever gonna run around and desert you\nNever gonna make you cry\nNever gonna say goodbye\nNever gonna tell a lie and hurt you\n\nWe’ve known each other for so long\nYour heart’s been aching but you’re too shy to say it\nInside we both know what’s been going on\nWe know the game and we’re gonna play it\n\nAnd if you ask me how I’m feeling\nDon’t tell me you’re too blind to see\n\nNever gonna give you up\nNever gonna let you down\nNever gonna run around and desert you\nNever gonna make you cry\nNever gonna say goodbye\nNever gonna tell a lie and hurt you\n\n#RickAstley #NeverGonnaGiveYouUp #WheneverYouNeedSomebody #OfficialMusicVideo"                             },                                                                                                                                                                                                                                                      "defaultAudioLanguage": "en"                                                                                                                                                                                                                          }                                                                                                                                                                                                                                                     }                                                                                                                                                                                                                                                     ],
  "pageInfo": {
    "totalResults": 1,
    "resultsPerPage": 1
  }
}

IRB

> video.id
=> "dQw4w9WgXcQ"
> video.etag
=> "pMtrD11jJwfAb08mUCrAm18HSXo"

Channel

JSON

{
  "kind": "youtube#channelListResponse",
  "etag": "5FAmNCyydD_wtrHKf9sKnt4MsqA",
  "pageInfo": {
    "totalResults": 1,
    "resultsPerPage": 50
  },
  "items": [
    {
      "kind": "youtube#channel",
      "etag": "IqzanLpWXrHcLHywOZrx5cUSOv0",
      "id": "UC38IQsAvIsxxjztdMZQtwHA",
      "snippet": {
        "title": "RickAstleyVEVO",
        "description": "The official home of Rick Astley on Vevo.",
        "customUrl": "@rickastleyvevo",
        "publishedAt": "2009-10-15T19:11:57Z",
        "thumbnails": {
          "default": {
            "url": "https://yt3.ggpht.com/K2ecE5j90a_DFzugHo0bW98vFlIQ1JJgs9mbcav7RGy1t7adJRnd2jaIv-oc6XzTRvDdWlFCAfc=s88-c-k-c0x00ffffff-no-rj",
            "width": 88,
            "height": 88
          },
          "medium": {
            "url": "https://yt3.ggpht.com/K2ecE5j90a_DFzugHo0bW98vFlIQ1JJgs9mbcav7RGy1t7adJRnd2jaIv-oc6XzTRvDdWlFCAfc=s240-c-k-c0x00ffffff-no-rj",
            "width": 240,
            "height": 240
          },
          "high": {
            "url": "https://yt3.ggpht.com/K2ecE5j90a_DFzugHo0bW98vFlIQ1JJgs9mbcav7RGy1t7adJRnd2jaIv-oc6XzTRvDdWlFCAfc=s800-c-k-c0x00ffffff-no-rj",
            "width": 800,
            "height": 800
          }
        },
        "defaultLanguage": "en",
        "localized": {
          "title": "RickAstleyVEVO",
          "description": "The official home of Rick Astley on Vevo."
        }
      },
      "status": {
        "privacyStatus": "public",
        "isLinked": true,
        "longUploadsStatus": "longUploadsUnspecified",
        "madeForKids": false
      }
    }
  ]
}

IRB

> channel.id
=> "UC38IQsAvIsxxjztdMZQtwHA"
> channel.etag
=> "IqzanLpWXrHcLHywOZrx5cUSOv0"

Playlist

{
  "kind": "youtube#playlistListResponse",
  "etag": "pKNuFuHQPPFL_SotQ41iOuNXXLk",
  "pageInfo": {
    "totalResults": 1,
    "resultsPerPage": 50
  },
  "items": [
    {
      "kind": "youtube#playlist",
      "etag": "mAF5QMTRDmF18MPgmre9HzEr-v0",
      "id": "PLBCF2DAC6FFB574DE",
      "snippet": {
        "publishedAt": "2011-09-23T04:52:01.295891Z",
        "channelId": "UCvceBgMIpKb4zK1ss-Sh90w",
        "title": "Google Search Stories",
        "description": "Searches can become stories. Some are inspiring, some change the way we see the world, and some just make us laugh. Here are a few of our favorites.",
        "thumbnails": {
          "default": {
            "url": "https://i.ytimg.com/vi/GvgqDSnpRQM/default.jpg",
            "width": 120,
            "height": 90
          },
          "medium": {
            "url": "https://i.ytimg.com/vi/GvgqDSnpRQM/mqdefault.jpg",
            "width": 320,
            "height": 180
          },
          "high": {
            "url": "https://i.ytimg.com/vi/GvgqDSnpRQM/hqdefault.jpg",
            "width": 480,
            "height": 360
          },
          "standard": {
            "url": "https://i.ytimg.com/vi/GvgqDSnpRQM/sddefault.jpg",
            "width": 640,
            "height": 480
          }
        },
        "channelTitle": "Google Search Stories",
        "localized": {
          "title": "Google Search Stories",
          "description": "Searches can become stories. Some are inspiring, some change the way we see the world, and some just make us laugh. Here are a few of our favorites."
        }
      },
      "status": {
        "privacyStatus": "public"
      }
    }
  ]
}

> playlist.id
=> "PLBCF2DAC6FFB574DE"
> playlist.etag
=> "mAF5QMTRDmF18MPgmre9HzEr-v0"

Playlist Item

JSON

{
  "kind": "youtube#playlistItemListResponse",
  "etag": "LxWy7sOso5S0O3EhoQmQomTQ-T0",
  "items": [
    {
      "kind": "youtube#playlistItem",
      "etag": "J72aKg7T88TviXJbgamom4N9iQ4",
      "id": "UExPVTJYTFl4bXNJS0xrdTN6S2hvRl9wVkhzeGtUdW40Uy4wMTcyMDhGQUE4NTIzM0Y5",
      "snippet": {
        "publishedAt": "2025-05-21T00:16:27Z",
        "channelId": "UC_x5XG1OV2P6uZZ5FSM9Ttw",
        "title": "#GoogleIO 2025 Developer Keynote Recap",
        "description": "Did someone say Google I/O recap? Yep, so make sure you catch the highlights from this year’s Developer Keynote. Discover how you can build across Android and web platforms, unlock the power of Gemini 2.5 Flash, and more! See what you missed at #GoogleIO!\n\nCheck out all the recap videos from Google I/O 2025 → https://goo.gle/IO25-Recaps\nCheck out all of the sessions from Google I/O 2025 → https://goo.gle/io25-sessions-yt\n\nSubscribe to Google for Developers → https://goo.gle/developers \n\nEvent: Google I/O 2025",
        "thumbnails": {
          "default": {
            "url": "https://i.ytimg.com/vi/EMZuKqr69Y8/default.jpg",
            "width": 120,
            "height": 90
          },
          "medium": {
            "url": "https://i.ytimg.com/vi/EMZuKqr69Y8/mqdefault.jpg",
            "width": 320,
            "height": 180
          },
          "high": {
            "url": "https://i.ytimg.com/vi/EMZuKqr69Y8/hqdefault.jpg",
            "width": 480,
            "height": 360
          },
          "standard": {
            "url": "https://i.ytimg.com/vi/EMZuKqr69Y8/sddefault.jpg",
            "width": 640,
            "height": 480
          },
          "maxres": {
            "url": "https://i.ytimg.com/vi/EMZuKqr69Y8/maxresdefault.jpg",
            "width": 1280,
            "height": 720
          }
        },
        "channelTitle": "Google for Developers",
        "playlistId": "PLOU2XLYxmsIKLku3zKhoF_pVHsxkTun4S",
        "position": 0,
        "resourceId": {
          "kind": "youtube#video",
          "videoId": "EMZuKqr69Y8"
        },
        "videoOwnerChannelTitle": "Google for Developers",
        "videoOwnerChannelId": "UC_x5XG1OV2P6uZZ5FSM9Ttw"
      },
      "status": {
        "privacyStatus": "public"
      }
    }
  ],
  "pageInfo": {
    "totalResults": 1,
    "resultsPerPage": 1
  }
}

IRB

> playlist_item.id
=> "UExPVTJYTFl4bXNJS0xrdTN6S2hvRl9wVkhzeGtUdW40Uy4wMTcyMDhGQUE4NTIzM0Y5"
> playlist_item.etag
=> "J72aKg7T88TviXJbgamom4N9iQ4"

eddorre force-pushed the etag-support branch from 922c1f2 to a05bbea Compare June 12, 2025 06:21

eddorre force-pushed the etag-support branch from a05bbea to 78b6202 Compare June 12, 2025 11:30

eddorre force-pushed the etag-support branch 2 times, most recently from 8df8af8 to 5b11716 Compare June 19, 2025 10:46

eddorre force-pushed the etag-support branch from 5b11716 to c4492a4 Compare June 19, 2025 23:13

eddorre force-pushed the etag-support branch from c4492a4 to 6fb3563 Compare July 28, 2025 06:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add support for ingesting etag data from API endpoints #445

Add support for ingesting etag data from API endpoints #445

Uh oh!

eddorre commented Jun 7, 2025 •

edited

Loading

Uh oh!

kangkyu commented Jun 9, 2025

Uh oh!

eddorre commented Jun 11, 2025

Uh oh!

eddorre commented Jun 12, 2025 •

edited

Loading

Uh oh!

kangkyu commented Jun 16, 2025

Uh oh!

eddorre commented Jun 19, 2025

Uh oh!

kangkyu commented Jun 19, 2025

Uh oh!

eddorre commented Jun 22, 2025

Uh oh!

kangkyu commented Jun 30, 2025 •

edited

Loading

Uh oh!

eddorre commented Jul 25, 2025

Uh oh!

eddorre commented Jul 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add support for ingesting etag data from API endpoints #445

Are you sure you want to change the base?

Add support for ingesting etag data from API endpoints #445

Uh oh!

Conversation

eddorre commented Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kangkyu commented Jun 9, 2025

Uh oh!

eddorre commented Jun 11, 2025

Uh oh!

eddorre commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kangkyu commented Jun 16, 2025

Uh oh!

eddorre commented Jun 19, 2025

Uh oh!

kangkyu commented Jun 19, 2025

Uh oh!

eddorre commented Jun 22, 2025

Uh oh!

kangkyu commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eddorre commented Jul 25, 2025

Uh oh!

eddorre commented Jul 28, 2025

Video

JSON

IRB

Channel

JSON

IRB

Playlist

Playlist Item

JSON

IRB

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

eddorre commented Jun 7, 2025 •

edited

Loading

eddorre commented Jun 12, 2025 •

edited

Loading

kangkyu commented Jun 30, 2025 •

edited

Loading