Corporations do not like it though, because they want to control spread of information. They prefer you re-enter page, be targeted with more ads, or spoonfed with their news algorithm on-site.
However RSS only works, if someone knows about your page. YouTube is quite good with RSS though. You can have channel RSS, be notified about new videos, while channels still are discoverable.
Also shameless plug to my own database of RSS feeds
h ttps://github.com/rumca-js/awesome-database-feeds
In that parallel universe web sites stop publishing RSS because they are overwhelmed with the polling traffic. The world really does need something like ActivityPub as much as Dave Winer denies it.
RSS has two polling speeds: too fast and too slow and it can even be both at the same time.
Those Planets are great, I wish we had more of them!
It costs me 10 cents/(month*feed) for superfeedr to ingest an RSS feed; it fills up an SQS queue with content items which is a model I find easy because I can drain that queue at my leisure.
It would be very cost effective for me to add a Planet and have my recommender system discover articles I like from it, that is, I'm happy to ingest 1000s of items a day to my reader.
Yes. Superfeedr makes RSS feeds look more like WebSub feeds and it's a great product. 10 cents/(month*feed) is a bargain if you want to subscribe to 100 high volume feeds but prohibitive for the model of "I want to subscribe to 10,000 indy blogs"
I understand your database of feeds is curated but you might be interested in another database listing feeds: Wikidata. See https://w.wiki/MpjN for a simple example of twenty (random) feeds (obviously, you can add filters for e.g. a country, language, etc.). (There are currently 18598 feeds there in total.)
Something I read here on HN some time ago and that can be of your interest: you can get specific Atom feeds from Youtube channels (only long format videos, only shorts, only live streams, only popular, etc)
I think many platform did support RSS, or API, but at some time it was dropped. It is not that hard to provide RSS. It is not like it has to be implemented from scratch. It is just dropped by platforms. There must be reasons for dropping. One may argue it is not worth it, but even to drop functionality there needs to be a decision from management. The management always think about money. Is RSS or API monetized correctly? Free? No? Then we drop it, because data from algorithms serving user contents can be easily monetized. Just follow the money, the oldest truth.
Both ideas can be true. It’s not on their radar because despite their popularity in consumer space they can’t find a business purpose that aligns with their self interests that require such user information. If I’m running a free podcast, in contrast, I might be happy anyone’s even bothering to visit and listen to what I have to say compared to who they are and whether I can assign a monetary value to their attention because spending money on something without clear, intentional, measurable ROI is anathema to our predominant modus operandi in business
I have to admit my interest rapidly waned when I realized I'd have to inspect a SQL database to look at this. Is it so enormous that a regular text file won't work?
There’s non-corpo reasons you might not want to expose all the content into an RSS feed. https://ciechanow.ski/ can only really be appreciated from the browser. In the salad days of blogging, it was as much about what kind of fun shit you could do with HTML or CSS as it was about writing and distributing your thoughts. The Yamauchi family office website could have just been a templates squarespace page, but they opted to make it look like this instead: https://www.y-n10.com/
There's a ton of selection bias going on here. Terence writes mostly about open web standards and topics that would be of significant interest to folks with an RSS feed reader.
Also, RSS readers are generally automated. I know I've had them around for years pulling in articles that I never read. Like a podcast "listen" is actually just an automated download, RSS traffic does not necessarily involve anyone actually reading the article, whereas search traffic is generally high intent and is at least resulting in eyeballs on the site, if not actual readers.
The data doesnt purport to cover any more than the 1 website. Its not like there are any generalisations about other websites derived from the data. Its just "These are where my hits come from"
> RSS readers are generally automated. I know I've had them around for years pulling in articles that I never read
They try to address that
> I added RSS and Newsletter tracking. These data are very lossy. If someone is subscribed to my RSS feed and opens a post and their client downloads a lazy-loaded image at the end of the post, I get a hit.
Right, I'd say RSS numbers (for a given blog post) mostly reflect cumulative visitors over the lifetime of the RSS feed while number of clicks coming in from a search engine reflect instantaneous popularity.
Same pattern as say a sub-reddit subscriber count compared to the number of nicks currently in an associated IRC channel, though IRC lurkers soften that distinction.
I protest the modern web by trying to consume all content via RSS.
The feed reader shall be my main window to the world, and I am sorry that it's not obvious to content creators that I read them so I often send an email on the note of "I enjoyed this article you wrote, thanks".
I write a small blog myself and I see the other side of it, but I just gave up on SEO, metrics, etc. I want to be the change I'd like see in the world: I publish full content RSS, I remove all analytics, make the website as lean as I can, put out my contact data and my only success metric is # of interactions I get with occasional readers.
Same experience. RSS is by far the most requested resource in the server logs. Sometimes up to 70% of the total traffic is coming from RSS. That said though, I wonder how much of that traffic is biological and how much is bots.
I also wonder how much is dead traffic. Dead as in people who add a ton of stuff in their RSS readers but don’t actually read.
RSS is a bit of a black box when it comes to this but maybe that's a good thing.
I've been toying with LLM based agents for staying on top of feeds lately. Staying on top of the fire hose was always the challenge with RSS readers, a lot of interesting low frequency stuff from people you follow gets buried under the non stop feed of stuff from high volume news that is actively trying to drown everyone out with many updates per day. Back in the day I solved it by simply not using RSS feeds for high volume stuff. But I was following a lot of interesting people. I miss that.
This is what lead to algorithm based filtering. Hacker News uses a simplistic algorithm but it is definitely using one and it works well enough. It's why I come here. We all collectively vote things up and what remains is nominally interesting enough to skim from the front page. With a bit of editorializing.
Social networks tried to game the algorithms for ad revenue. Which is why they are a lot less popular these days. Sites like Medium, Substack, Tumblr, etc. took over from simple blogs and immediately started raising walled gardens around them to become discovery platforms, have recommendations, etc.
But at least they support RSS. A lot of websites still do. If you run any kind of website publishing regular news or article content and you don't support a feed, you are being an idiot. It's easy, doesn't really cost anything, and you might actually get people using your feed once in a while. Your site might actually have one without you realizing. Most news papers have feeds. They are everywhere. The main issue isn't finding them but sifting through them. It always was.
With agent based approaches, you control the algorithm. That wasn't possible in the past. LLMs can summarize, aggregate, categorize, group, filter, etc.
> a lot of interesting low frequency stuff from people you follow gets buried under the non stop feed of stuff from high volume news that is actively trying to drown everyone out with many updates per day.
Every major RSS reader supports folders. Your problem is that you engage with RSS as if it were a social media feed, with it's single monolithic reverse-chronological feed.
Just don't do that. Stick all the high volume news feeds in a folder, and you can skim read the headlines & hit "mark all as read" once you're done or for whatever other reason don't want to look at the news anymore.
Stick the low volume things you care about in their own folder, and those will remain unread, in their own ordering for you to read at your own leasure.
Even for sites that don't offer granular feeds, every major feed reader offers filtering options, a lot of them offer fairly complex regex filtering.
> This is what lead to algorithm based filtering.
Feed aggregators (and most social media) exist because of discoverability, finding new stuff from new people you hadn't heard about before.
> With agent based approaches, you control the algorithm. That wasn't possible in the past. LLMs can summarize, aggregate, categorize, group, filter, etc.
You'd be spending tens of dollars of compute on something that every major RSS client was doing back in 2006 with the equivalent of less than a single penny worth of current day compute.
Just as a suggestion, but I've been thinking in terms of clustering. If I can figure out that five different posts from five different feeds are about one story, I can collapse them into a single UI element.
I curate my subscriptions pretty heavily. I scan the feed and triage to my linkding instance. I have a headless app that fetches unread items, uses a voice model from Hugging Face to audio using piper and then adds them to audiobookshelf as a "podcast". I catch up on things when I'm on a walk or doing the dishes.
I'm building Subweb.net (not ready yet, it's just a few test feeds without the LLM pipeline turned on yet) to LLM-tag RSS feed items with topic, relevance/interest, location, and translations, and present them as feeds. I'm thinking I could maybe let users specify their preferred custom prompts and ranking params or similar, though the standard prompt is already fine.
I think the open web needs to come back, but in a fair way for everyone, giving readers control over their feeds while also sending traffic and comments back to the original sources. Not quite sure how to do that yet.
Is it possible that the users who used to find us through Google are now satisfied with AI chat summaries and no longer feel the need to click through to the actual page?
Meanwhile, the long-time users who subscribed via RSS are still showing up like they always have. If this is the case, it’s a bit of a sad reality for content creators.
Yes. It caused a major drop in CTR for search. Google Zero is the term for it. It’s murderous for information websites. On my side, traffic is down roughly 70% on the website I live from.
It’s still part of the training data though. Not that I consented to this.
Very rarely. AI traffic is a tiny tiny fraction of total traffic. ChatGPT and Gemini mostly spit out unattributed information, except when queries trigger a web search.
I notice many 404s from ChatGPT to hallucinated URLs on my website. Sometimes it hallucinates guides about topics I never covered. So I'm getting cited for things I didn't even say!
I also notice that for other topics, the cited sources are crap. The best shaver is a Braun shaver according to Braun.com, and two comments on a 6 years old reddit thread about a different topic.
Whilst anecdotal, this supports the idea that I should de-index my personal site when I relaunch it. There's no value for me to get indexed by Google or Bing. The only index I care about is Wayback Machine.
This is probably the case for most personal websites with considerable cachet in their niche.
+ With a strong enough social network you probably don't have to care about SEO as much
You can title your post about bad customer service practices in a unique way without a second thought [0] and your more traditionally titled posts can still make the first page of a Google search with a reasonable query [1].
+ Depending on your niche your target audience is likely to already be tapped in well enough to not have to rely on search engines for content catering to their interests.
I feel like search engine practices trend along the curve shown in that meme where it's the "fool" on one end and then the "normie" in the middle and then the "Jedi" on the other end who does the same thing as the idiot. Except in this case "Jedis" only search for what's not present in their feeds (which doesn't have to be only RSS feeds) and fools can eventually cultivate their own feeds for their interests and reserve search engine use for mundane purposes that essentially fulfill the responsibility of some kind of pop culture almanac, phonebook and portal to Wikipedia.
I toyed around with using Language Embeddings as a way to categorize my RSS Feeds.
It works pretty well. But importantly, it's so cheap that I have never really seen it on my bill. An earlier prototype used OpenAI embeddings. I loaded 5$ API credits and after a year the credits expired.
Oh yeah, your feed is definitely in my reader. When I click through to your site, in the URL I pass ?rss_ref=mydomain This way you - and other domains I visit - can know where the RSS-visitor is coming from (not foolproof of course, because your average spammer can also put their domain in the URL).
More important, when you see my traffic your post is definitely read on your website itself (or at least opened to read). There's no background stuff going on in my reader, pulling in feeds without reading them.
My way of reading feeds is more Tinder-style: the latest post from a random feed per time, then another one, etc. I feel irky even thinking about a firehose of content coming my way.
Note I discover content with an RSS reader and then post it to HN and sometimes it frontpages and that traffic ought to be attributed to RSS when it comes to decisions like "should we publish RSS?"
For science fiction short stories I get all notifications of new issues and stories via RSS. There are some sites like https://www.freesfonline.net/NewAdditions.html that posts new stories every month or so. There are always news sites in that list to get more entries for my RSS reader, so no need to ever search web/agents for more content.
From my analytics, around 85% of visits to my sites are now from bots. It's hard to say if these are oldschool bots or people just browsing the web via chatgpt now, but the reality is that very few actually visit the actual sites I have anymore.
The universal contact function is usually there, and it's email. I tried many times: it works. Authors respond. Only very few people seem to want to keep it hidden.
I'm dumbfounded by the number of times I see comments of the form "if the author is reading this ..." on a 3rd party comment side, with a link posted by somebody else, on a forum the author is likely never going to watch, followed by an actually useful comment that you could have _ensured_ the author reads by you know... just contacting him?
Forum comments are just recipe for instant spam, and have been so in the last 10+ years. If you want to make them useful, it currently needs to be actively policed (not to mention: you can be responsible for the content posted as well in several countries now). As an author, only if you're trying to create an audience around your blog, all the hassle around it might be worth it.
Does it download the web version, or does it copy the content from the RSS feed itself? I believe that Terence's RSS feed includes a full copy of the content you'd see from visiting his site.
Wow. Dude creates an account more than 5 years ago, doesn't say a single thing until today, posts a completely innocuous thank-you, and gets downvoted into the gray.
Shame on whoever did that. You should lose your downvoting rights. The community deserves better.
Corporations do not like it though, because they want to control spread of information. They prefer you re-enter page, be targeted with more ads, or spoonfed with their news algorithm on-site.
However RSS only works, if someone knows about your page. YouTube is quite good with RSS though. You can have channel RSS, be notified about new videos, while channels still are discoverable.
Also shameless plug to my own database of RSS feeds
h ttps://github.com/rumca-js/awesome-database-feeds
RSS has two polling speeds: too fast and too slow and it can even be both at the same time.
Nostr works on a similar basis but you push to the aggregator instead of them pulling.
It costs me 10 cents/(month*feed) for superfeedr to ingest an RSS feed; it fills up an SQS queue with content items which is a model I find easy because I can drain that queue at my leisure.
It would be very cost effective for me to add a Planet and have my recommender system discover articles I like from it, that is, I'm happy to ingest 1000s of items a day to my reader.
What was good was when they had discovery. The RSS button would show up in the menu bar and you could click it and add to your RSS reader.
>https://stackoverflow.com/questions/71192605/how-do-i-get-yo...
I think many platform did support RSS, or API, but at some time it was dropped. It is not that hard to provide RSS. It is not like it has to be implemented from scratch. It is just dropped by platforms. There must be reasons for dropping. One may argue it is not worth it, but even to drop functionality there needs to be a decision from management. The management always think about money. Is RSS or API monetized correctly? Free? No? Then we drop it, because data from algorithms serving user contents can be easily monetized. Just follow the money, the oldest truth.
That said, this seems quaint in the modern era where trillion-dollar tech companies are doing that to publishers now, too.
Though some seem to go out of their way to make RSS impossible.
Also, RSS readers are generally automated. I know I've had them around for years pulling in articles that I never read. Like a podcast "listen" is actually just an automated download, RSS traffic does not necessarily involve anyone actually reading the article, whereas search traffic is generally high intent and is at least resulting in eyeballs on the site, if not actual readers.
The data doesnt purport to cover any more than the 1 website. Its not like there are any generalisations about other websites derived from the data. Its just "These are where my hits come from"
They try to address that
> I added RSS and Newsletter tracking. These data are very lossy. If someone is subscribed to my RSS feed and opens a post and their client downloads a lazy-loaded image at the end of the post, I get a hit.
Same pattern as say a sub-reddit subscriber count compared to the number of nicks currently in an associated IRC channel, though IRC lurkers soften that distinction.
The feed reader shall be my main window to the world, and I am sorry that it's not obvious to content creators that I read them so I often send an email on the note of "I enjoyed this article you wrote, thanks".
I write a small blog myself and I see the other side of it, but I just gave up on SEO, metrics, etc. I want to be the change I'd like see in the world: I publish full content RSS, I remove all analytics, make the website as lean as I can, put out my contact data and my only success metric is # of interactions I get with occasional readers.
I also wonder how much is dead traffic. Dead as in people who add a ton of stuff in their RSS readers but don’t actually read.
RSS is a bit of a black box when it comes to this but maybe that's a good thing.
It's not, if you add a tracking parameter in the posted URLs and you keep track of that in your analytics.
This is what lead to algorithm based filtering. Hacker News uses a simplistic algorithm but it is definitely using one and it works well enough. It's why I come here. We all collectively vote things up and what remains is nominally interesting enough to skim from the front page. With a bit of editorializing.
Social networks tried to game the algorithms for ad revenue. Which is why they are a lot less popular these days. Sites like Medium, Substack, Tumblr, etc. took over from simple blogs and immediately started raising walled gardens around them to become discovery platforms, have recommendations, etc.
But at least they support RSS. A lot of websites still do. If you run any kind of website publishing regular news or article content and you don't support a feed, you are being an idiot. It's easy, doesn't really cost anything, and you might actually get people using your feed once in a while. Your site might actually have one without you realizing. Most news papers have feeds. They are everywhere. The main issue isn't finding them but sifting through them. It always was.
With agent based approaches, you control the algorithm. That wasn't possible in the past. LLMs can summarize, aggregate, categorize, group, filter, etc.
Every major RSS reader supports folders. Your problem is that you engage with RSS as if it were a social media feed, with it's single monolithic reverse-chronological feed.
Just don't do that. Stick all the high volume news feeds in a folder, and you can skim read the headlines & hit "mark all as read" once you're done or for whatever other reason don't want to look at the news anymore.
Stick the low volume things you care about in their own folder, and those will remain unread, in their own ordering for you to read at your own leasure.
Even for sites that don't offer granular feeds, every major feed reader offers filtering options, a lot of them offer fairly complex regex filtering.
> This is what lead to algorithm based filtering.
Feed aggregators (and most social media) exist because of discoverability, finding new stuff from new people you hadn't heard about before.
> With agent based approaches, you control the algorithm. That wasn't possible in the past. LLMs can summarize, aggregate, categorize, group, filter, etc.
You'd be spending tens of dollars of compute on something that every major RSS client was doing back in 2006 with the equivalent of less than a single penny worth of current day compute.
https://aws.amazon.com/blogs/machine-learning/use-language-e...
I think the open web needs to come back, but in a fair way for everyone, giving readers control over their feeds while also sending traffic and comments back to the original sources. Not quite sure how to do that yet.
Meanwhile, the long-time users who subscribed via RSS are still showing up like they always have. If this is the case, it’s a bit of a sad reality for content creators.
It’s still part of the training data though. Not that I consented to this.
I notice many 404s from ChatGPT to hallucinated URLs on my website. Sometimes it hallucinates guides about topics I never covered. So I'm getting cited for things I didn't even say!
I also notice that for other topics, the cited sources are crap. The best shaver is a Braun shaver according to Braun.com, and two comments on a 6 years old reddit thread about a different topic.
+ With a strong enough social network you probably don't have to care about SEO as much
You can title your post about bad customer service practices in a unique way without a second thought [0] and your more traditionally titled posts can still make the first page of a Google search with a reasonable query [1].
+ Depending on your niche your target audience is likely to already be tapped in well enough to not have to rely on search engines for content catering to their interests.
I feel like search engine practices trend along the curve shown in that meme where it's the "fool" on one end and then the "normie" in the middle and then the "Jedi" on the other end who does the same thing as the idiot. Except in this case "Jedis" only search for what's not present in their feeds (which doesn't have to be only RSS feeds) and fools can eventually cultivate their own feeds for their interests and reserve search engine use for mundane purposes that essentially fulfill the responsibility of some kind of pop culture almanac, phonebook and portal to Wikipedia.
[0]: https://shkspr.mobi/blog/2026/03/bored-of-eating-your-own-do...
[1]: https://shkspr.mobi/blog/2026/04/does-mythos-mean-you-need-t... — I Googled "mythos and open source". Interestingly, a forum discussion about this post came before it: https://itsfoss.community/t/does-mythos-mean-you-need-to-shu...
It works pretty well. But importantly, it's so cheap that I have never really seen it on my bill. An earlier prototype used OpenAI embeddings. I loaded 5$ API credits and after a year the credits expired.
https://aws.amazon.com/blogs/machine-learning/use-language-e...
https://github.com/aws-samples/rss-aggregator-using-cohere-e...
More important, when you see my traffic your post is definitely read on your website itself (or at least opened to read). There's no background stuff going on in my reader, pulling in feeds without reading them.
My way of reading feeds is more Tinder-style: the latest post from a random feed per time, then another one, etc. I feel irky even thinking about a firehose of content coming my way.
I'm dumbfounded by the number of times I see comments of the form "if the author is reading this ..." on a 3rd party comment side, with a link posted by somebody else, on a forum the author is likely never going to watch, followed by an actually useful comment that you could have _ensured_ the author reads by you know... just contacting him?
Forum comments are just recipe for instant spam, and have been so in the last 10+ years. If you want to make them useful, it currently needs to be actively policed (not to mention: you can be responsible for the content posted as well in several countries now). As an author, only if you're trying to create an audience around your blog, all the hassle around it might be worth it.
Shame on whoever did that. You should lose your downvoting rights. The community deserves better.
https://brynet.ca/feed.xml
Also the embedded mastodon feed on my site uses rss.