This is about text submissions. More like if Google Trends counted word occurrences on webpages. Or if Google Ngrams counted webpages instead of books
People don't write much about non-newsworthy things whereas many people search "burger" anytime they want a burger delivery. The datasets aren't usable in the same way
If this project is an ad for their product (Upstash, promising "Highly Available, Infinitely Scalable"), then the last thing they'd want is a hug of death :/
/api/hn -> 502 {"error":"Search entry should have an initialized schema, command was: [\"SEARCH.AGGREGATE\",\"hn\",\"{\\\"$or\\\":[{\\\"title\\\":{\\\"$eq\\\":\\\"anthropic\\\",\\\"$boost\\\":5}},{\\\"text\\\":{\\\"$eq\\\":\\\"anthropic\\\"}}]}\",\"{\\\"by_month\\\":{\\\"$dateHistogram\\\":{\\\"field\\\":\\\"time\\\",\\\"fixedInterval\\\":\\\"30d\\\"}},\\\"top_authors\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"by\\\",\\\"size\\\":6}},\\\"by_type\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":4}}}\"]"}
Our startup (~20 people) got slashdotted in 1998 or so. I was the only one randomly awake at the time. Remember watching all the logs from our web server in realtime, ready to immediately kill anything or anyone threatening the overall availability.
512 kbps uplink, I think. Even accidental DoS was trivial. We had a self-hosted little data center at our office with the only available stupidly expensive commercial connection.
Felt some dread having to restart the main (async, single-process) web server a few times to keep things going due to bugs in our code. So many* people on dial-up patiently waiting for the page to load.
Its funny that these days the bottleneck is usually the data layer. Servers are so powerful now that even your average $5 server can handle HN levels of load if configured correctly.
The huge spike of "lk-99" in science & frontier tech is amusing...
This is cool concept, would love a positive/negative sentiment computed for each comment that refers to a given word, so you can see trends of "cloudflare (positive)" vs "cloudflare (negative)" where first one counts comments only if sentiment confidence is greater than say 0.6 and the other one counts comments only if sentiment is less than 0.4 (assuming [0,1] sentiment score)
Almost all of the major vulnerability and hack are just single spikes at the time it happened and it tails off after that… except Stuxnet. Stuxnet is was much more interesting that most other attacks since it was very political and openly published. Of course, the thing that attack was about is still a news headline today as well
>Stuxnet is was much more interesting that most other attacks since it was very political and openly published
it was also super cool, from a technical perspective. and was one of the first (to my knowledge) to really bring the concept of large-scale attacks against industrial control systems to light.
Nice! Would love a brief explanation of the infrastructure. I see the Powered by "Upstash Redish Search", but why choose Upstash Redis Search vs something else?
It looks like some of these terms aren't indexed (or the site is just too hug of deathed right now), but I'd like to see the graph of like, social media, iot, cryptocurrency, ai.
One useful feature would be to normalize by total so that I can see changes in something as opposed to just total site growth. Right now I have to chart a single generic parameter but if I pick poorly it’ll confuse the issue.
It's a HN clone, that syncs with HN that allows you to basically establish smaller private communities who can discuss anything that's on HN without actually being on HN.
It also indexes and let's you search through the DB which I find is really useful to find things that peak my interest.
one subtle consistency bug that made it hard for me to interpret when I was clicking around: the small thumbnail plot vs the full plot often (always?) seem to use different colors.
The blue / orange gets assigned to the opposite labels in the A vs. B when you click, which made it confusing to understand.
Great job! I've also been wanting to do similar statistics recently, wanting to know when LLMs becoming the absolute dominant topic on HN. Now it seems like half of the posts were about LLMs.
IMO, using AI to assign keywords to a broader group of strict synonymous keywords would make the comparison much more helpful.
Because in general we want to know the trend of categories more than of a word, asking for “auto pilot” for ex. should include “self driving”, FSD etc.
I would not like this. This is the kind of change that made google search so annoying. (Eg what if I want to track the history of 'self-driving' vs 'auto pilot' in sales pitches? Or more basically, what if the system wrongly interprets me wrongly?) Better to support | or similar old-fashioned search engine syntax and dwis and not dwim.
I also have a seperate page for the "Who is Hiring?" posts, here is the distribution of programming languages over each monthly "Who is hiring?" post in HN ever.
https://hackernewstrends.com/who-is-hiring
A minor suggestion - I'd like to be able to render the current graph taller (full height of my browser window).
Also some sentiment analysis on the "people" graphs would be very insightful (particularly for the likes of Edward Snowdon, Julian Assange, Elon Musk and Sam Altman). Perhaps colour the area under the graph red-orange-green based on the sentiment?
Just my idea. I'm working on a side project https://newsavista.com/invite/ASAD68923E that aggregates news and tracks news trends and changing sentiment on the major stories. With cheap cloud LLMs (and "free" local LLMs) it turns out to be a trivial feature to build.
There are a few technologies with pretty generic names which don’t lend themselves so well to this kind of trend analysis.
I was curious about Atom. According to the trend it’s still neck and neck with VS Code. But are people really talking about Atom the text editor that much still, or other types of atoms?
So you can create any sort of similar services in a single SQL query and an HTML page.
I also hosted it as a publicly accessible data lake, which you can query from everywhere: https://github.com/ClickHouse/ClickHouse/issues/29693#issuec...
It is also updated in real-time.
This is about text submissions. More like if Google Trends counted word occurrences on webpages. Or if Google Ngrams counted webpages instead of books
People don't write much about non-newsworthy things whereas many people search "burger" anytime they want a burger delivery. The datasets aren't usable in the same way
` /api/hn -> 504 An error occurred with your deployment FUNCTION_INVOCATION_TIMEOUT cle1::c8vgv-1782399959042-aeba3cae05ff `
/api/hn -> 502 {"error":"Search entry should have an initialized schema, command was: [\"SEARCH.AGGREGATE\",\"hn\",\"{\\\"$or\\\":[{\\\"title\\\":{\\\"$eq\\\":\\\"anthropic\\\",\\\"$boost\\\":5}},{\\\"text\\\":{\\\"$eq\\\":\\\"anthropic\\\"}}]}\",\"{\\\"by_month\\\":{\\\"$dateHistogram\\\":{\\\"field\\\":\\\"time\\\",\\\"fixedInterval\\\":\\\"30d\\\"}},\\\"top_authors\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"by\\\",\\\"size\\\":6}},\\\"by_type\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":4}}}\"]"}
512 kbps uplink, I think. Even accidental DoS was trivial. We had a self-hosted little data center at our office with the only available stupidly expensive commercial connection.
Felt some dread having to restart the main (async, single-process) web server a few times to keep things going due to bugs in our code. So many* people on dial-up patiently waiting for the page to load.
It was exhilarating though :).
*) Surely at least a hundred!
This is cool concept, would love a positive/negative sentiment computed for each comment that refers to a given word, so you can see trends of "cloudflare (positive)" vs "cloudflare (negative)" where first one counts comments only if sentiment confidence is greater than say 0.6 and the other one counts comments only if sentiment is less than 0.4 (assuming [0,1] sentiment score)
it was also super cool, from a technical perspective. and was one of the first (to my knowledge) to really bring the concept of large-scale attacks against industrial control systems to light.
The transition between crypto and ai on the graphs is already pretty funny. https://hackernewstrends.com/?q=crypto&q=chatgpt
https://gitlab/here_forawhile/torum
It's a HN clone, that syncs with HN that allows you to basically establish smaller private communities who can discuss anything that's on HN without actually being on HN.
It also indexes and let's you search through the DB which I find is really useful to find things that peak my interest.
'peak' refers to the top of a thing, commonly mountains
one subtle consistency bug that made it hard for me to interpret when I was clicking around: the small thumbnail plot vs the full plot often (always?) seem to use different colors.
The blue / orange gets assigned to the opposite labels in the A vs. B when you click, which made it confusing to understand.
Because in general we want to know the trend of categories more than of a word, asking for “auto pilot” for ex. should include “self driving”, FSD etc.
https://hackernewstrends.com/?q=Nim&q=Rust&q=Zig
This was a small project of mine after I've found out that I can simply the whole hackernews archive (~48GB) and play around with it.
You can compare terms just like in google trends and you can also see the exact posts & comments from that time.
I like that you can discover what went crazy in the timeline, they just come up as small burst of activity, it's quite fun to play around with it. https://hackernewstrends.com/?q=litecoin&q=dogecoin&q=solana...
I also have a seperate page for the "Who is Hiring?" posts, here is the distribution of programming languages over each monthly "Who is hiring?" post in HN ever. https://hackernewstrends.com/who-is-hiring
Any kind of feedback is welcome.
Where is this archive located you speak of?
A minor suggestion - I'd like to be able to render the current graph taller (full height of my browser window).
Also some sentiment analysis on the "people" graphs would be very insightful (particularly for the likes of Edward Snowdon, Julian Assange, Elon Musk and Sam Altman). Perhaps colour the area under the graph red-orange-green based on the sentiment?
The sentiment analysis is very interesting, I can do that easily. Could be a new page as well. Did you see this anywhere else or just your idea?
I was curious about Atom. According to the trend it’s still neck and neck with VS Code. But are people really talking about Atom the text editor that much still, or other types of atoms?
I'd love to have some sort of normalization option to separate more subtle positive trends from the general increase in number of posts.
I am really liking the trend for "linux": https://hackernewstrends.com/?q=linux
https://hackernewstrends.com/?q=linux&q=windows
Edit: Nvm seems like absolute count if you click the graph.
Hmm, did I break something?