Submitted 3 weeks ago by ueiqkkwhuwjw@lemmy.world to selfhosted@lemmy.world
https://github.com/binwiederhier/ntfy/releases/tag/v2.18.0
According to the release:
Adds experimental PostgreSQL support
The code was written by Cursor and Claude
14,997 added lines of code, and 10,202 lines removed
reviewed and heavily tested over 2-3 weeks
This makes me a bit uneasy, especially as ntfy is an internet facing service.
Am I overreacting or do you all share the same concern?
It looks like that tool is more or less built by a single developer (you already trust their judgment anyways!), and even though the code came through in a single PR it was a merge from a branch that had 79 separate commits: github.com/binwiederhier/ntfy/pull/1619
Also glancing through it a bit, huge portions of that are straightforward refactors or even just formatting changes caused by adding a new backend option.
I’m not going to say it’s fine, but they didn’t just throw Claude at a problem and let it rewrite 25k lines of code unnecessarily.
Any AI usage immediately discredits the software for me, because it calls into question all of their past and future work.
Wow a differentiated opinion on AI use :)
Yeah, I mean, with or without AI, I’ve always only had a big pull request for releases, from a stable release branch into the main branch, the release branch would be a merge of various branches or just be worked on directly on various stages.
One big pull request doesn’t really mean anything.
Something like graphite.com to create stacked PRs that are reviewable probably would have helped. Can be replicated with local LLMs or remote AI providers with locally configured agentic workflows. Never used graphite personally, but I’ve seen some open source maintainers use it to split up large PRs.
Huh, I was wondering how rrds would help…
@ueiqkkwhuwjw just this quote at the start of the release notes
> 14,997 added lines of code, and 10,202 lines removed, all from one pull request
This is already a major red flag even without the ai stuff right? Can't believe anyone would flaunt that like this.
The “single pull request” is a merge release from 79 separate commits.
Doesn’t matter, it’s entirely too much for one PR
Uh. I’d really prefer if people experimented with new technology a bit more cautiously and not directly jump to “the biggest release […] ever done”.
Upvote and comment on: github.com/binwiederhier/ntfy/issues/1645
They just replied:
What gave you the idea that this was a full rewrite? I moved things around with AI and added postgres support for the queries. Nobody has ever reviewed and tested anything more thoroughly than I did with this branch.
You are twisting what it actually is. You are assuming something that is not true.
This makes me think that they didn’t review or test it at all, lmao
Thanks for the link. As a short aside for the other people here: Try not to spam developers. That usually achieves the opposite and makes them miserable, when we want them to not burn out and write good software for us. A thumbs-up emoji is the correct reaction for the average person. Or a code-review highlighting specific issues in the code.
I’m a developer
I sometimes sometimes use AI for an answer to a complicated problem because normally I’d open up 20 pages , have to go through them all to find the right answer
AI gets me the answer right away, though it likely is completely wrong or at least partially wrong. Either way, it gives me a general direction and with that I only have to search through one or two pages to confirm, so the same process is just a little faster.
I laso have used AI on a couple of occasions to ask it to write code for a complicated problem. Again, you don’t copy the code, god no, it’s always the worst, and it is in 80% of the cases still at least riddled with bugs, or just complete bullshit. However, it might give me an alternative idea or a direction to take to implement or fix this complicated feature problem.
That’s the extent to which I’ve used AI and for the foreseeable future that won’t change because AI still can’t code. It’s still wildly flailing around and it might produce something that implements a certain functionality, but it’s a guarantee that that functionality will have more bugs and security holes than features
I am also a developer and agree entirely.
Asking for advice, examples or the occasional boilerplate is at most how I use AI and certainly not integrated directly into my IDE.
I understand this comment. AI sometimes saves a ton of mental power and time when I’m stuck on an issue. It can give some really good suggestions. Also, AI is a godsend for frontend shit. I don’t care what y’all say, I’m never touching CSS and HTML ever again. lmao.
Nah, wouldn’t do that. CSS needs to be well designed to function properly, you need actual developers for that or you’ll screw over your users.
But yeah, to give quick pointers and ideas to flesh out, it’s reasonably useful
If that is enough to warrant it’s extreme energy use, the spread of AI slop everywhere, the pollution, the uncontrolled datacenter expansions, the explosion in hardware costs it created, the countless death and suffering it caused through AI psychosis, the AI childporn bots (hello grok, are you still the world’s biggest child porn producer or did Elmo finally reign you in to again be mecha Hitler?), the…
Long story short, AI will likely end this world in a long list of fucked up ways, I don’t think it’s worth it
Until then, I’ll use it as a suggestion tool, not much more
Yeah, this is now inherently untrustworthy. Better to switch to an alternative.
Do you know any? I’ve never really looked beyond ntfy.sh until now
Gotify is supposedly a good alternative. Looking into it myself now.
I only know NextPush (Nextcloud App), but there is also something called Autopush I think?
There’s SunUp one F-droid, but I don’t know anything about them.
I recently switched to gotify. Push notifications to iOS aren’t as good but I’m happy with it.
If you use ntfy for UnifiedPush: unifiedpush.org/users/distributors/
I’m assuming this is some sort of canary message to indicate that the code base has been compromised, the author can’t talk about it, and everyone should immediately stop using the service. Surely no-one would be unwise enough to commit this otherwise?
Even ignoring the huge red LLM flag, a 25kLOC delta in a single PR should be cause for instant rejection as there’s no way to fully understand or test it, let alone in 2-3 weeks.
25kLOC delta in a single PR should be cause for instant rejection
Not to pick at nits, but it would be VERY different if it was 1k lines added and 24k lines removed. There’s something extremely satisfying about removing 10k+ lines of unnecessary code.
Sure, that would be a little different, but unless you could make a convincing argument, backed up with a solid set of unit tests, at the least, as to why and how you were able to remove that much code whilst only adding a comparatively small amount, I’d still be inclined to reject it and ask for it to be broken down into smaller units.
Now, that explaination might be something along the lines of it being dead code that is not called from anywhere, or even that it was a patched version of an upstream library, and the patch is now included in that upstream, in which case, fair enough, good work, and thanks very much. As a rewrite or refactor though, it’s too big to sensibly review and needs breaking down into separate features.
Definitely share your concern. Without strong review processes to ensure that every line of code follows the intent of the human developer, there’s no way of knowing what exactly is in there and the implications for the human users. And I’m not just talking about bugs. How do you know there isn’t malware?
They say it’s reviewed, but the temptation to blindly trust is there.
The size of that changeset means that it’s inherently unreviewable.
The commit history is something I’ve seen only in the PRs that even the most dysfunctional companies would demand a rewrite for.
Also, 2-3 weeks review? PostgreSQL support could be added in that time without the need for a damn „vibe check”. Hell, it would probably take less time than that.
To be fair they would have needed to spend time testing the manual implementation as well.
The problem I see mainly is that even if this rolls out perfectly, the erratic and changing nature if llms still make it pointless as a proof of concept. Next time Claude might fuck up in a fringe way that’s not covered by unit tests and is missed by manual tests.
On the other hand I guess I’ve been guilty myself on numerous occasions to implement fringe bugs into production code, but at least I learn from it.
Yeah, it could easily have added a couple of lines of code that sends everything to Northern Korean hackers because it found that in a bunch of repositories or just logging passwords to public logs or other things an experienced developer would never do. “AI” only replicates what it sees most often and as more spam and junk repos are added to its training data because “AI” companies are too concerned with profit to teach it properly, it could do tons of random stuff. It’s like training a developer by giving them random examples from the internet rather than specific ones. Of course they pick up bad habits. Even if it “works” it is almost never efficient or secure.
Definitely time to find an alternative. What the actual fuck is this
This doesn’t make me uneasy. It makes me resentful, a little angry, and a lot tired. Thanks for bringing it to attention, I will make sure that nothing of that project or from that author will ever cross my ecosystem again.
You’re gonna have a lot of hate in your blood if you go around acting like the most skilled engineers aren’t using AI to write code.
There’s a massive difference between “using AI to write code” and refactoring almost 15k lines in a single push.
The “best” uses of AI in coding are for small blocks. You don’t just tell it “I need a program that does X, Y, and Z” because that will (at best) result in horrible code. Instead, it’s best practice to use it for small blocks of code, where you tell it something more akin to “I need a function that takes {a} as a variable, does {thing}, and outputs {x}.” That way you’re not using it to generate giant swaths of code all at once, you’re just using it to generate individual functions that you can then use as needed.
But it also means that the “most skilled” (as you put it) programmers are basically putting themselves in a permanent debugging seat instead of working as a developer. And in many cases, debugging code can be just as (or more) difficult than writing the initial code. It’s also why senior devs exist to audit code from junior devs, because it’s assumed that junior devs will inevitably make mistakes that need debugging, or will make code that clashes with code from other junior devs. And it’s the senior dev’s job to ensure that the code is both functional and integrated properly.
And this “adding 15k lines of code and ripping out 10k lines” push smells a lot like the former “write me a program to do {thing}” usage.
Most skilled engineers, and even mildly skilled engineers don’t use slopgenerators to write code. Some of them use it sometimes to do some menial tasks, although I’m not convinced it actually saves them time. It sure doesn’t every time we measure it.
There is however a plague of low skilled people who convinced themselves that they’ve found a shortcut to being an engineer. Those people are producing bad things at a fast pace, and the only reason we’re not in an unsolvable crisis yet is that their slop isn’t hitting prod very often on account of being bad.
Look, if he wanted to introduce AI code, whatever, but doing it all at once in a 14k line change (biggest release ever by his words) is crazy.
there is this repo that lists some slopware : codeberg.org/small-hack/open-slopware maybe someone can add it
I think there’s room for a little bit of nuance that page doesn’t do a great job of describing. In my opinion there’s a huge difference between volunteer maintainers using AI PR checks as a screening measure to ease their review burden and focusing their actual reviews on PRs that pass the AI checks, and AI-deranged lone developers flooding the code with “AI features” and slopping out 10kloc PRs for no obvious reason.
Just because a project is using AI code reviews or has an AGENTS.md is not necessarily a red flag. A yellow flag, maybe, but the evidence that the Linux Kernel itself is on that list should serve as an example of why you can’t just kneejerk anti-AI here. If you know anything about Linus Torvalds you know he has zero tolerance for bad code, and the use of AI is not going to change that despite everyone’s fears. If it doesn’t work out, Linus will be the first one to throw it under the bus.
Lol my project has an AGENTS.md and its contents are basically, “Don’t use AI agents on this codebase.”
Upvote this guy
did not know that the serde developer tolnay is a military apologist. I’m disgusted. serde is a very good tool… I’ll think about what to do about this. such a shame…
the linux kernel is on that list, bro it’s time to switch!
Time to switch to Plan9!
Also Chrome, Firefox ans Ladybird!
oh no. not ladybird! You were supposed to save us!
Awesome page, thanks. Have bookmarked.
Harfbuzz though? That’s going to take some replacing. Hopefully someone will fork an earlier version. The thing that it does (accurate multi-script font shaping) is difficult to do; requires a lot of rule-of-thumb knowledge that’s unlikely to be possessed by a single person, needs a lot of collaboration.
we’re all so fucked
“but reviewed and heavily tested over 2-3 weeks by me. I created comparison documents, went through all queries multiple times and reviewed the logic over and over again. I also did load tests and manual regression tests, which took lots of evenings.”
This is the way.
that’s nowhere near enough testing for such a large change… special one written by the slop machine
Pretty much.
I’ve started using AI on a project last week and the first thing I do is write tests. Lots of tests.
With enough guardrails, you could actually get pretty decent quality output out of it and with enough regression tests, you can ensure that nothing’s actually breaking.
Similarly, reviewing its changes and actually reading the code that’s being generated to ensure correctness is necessary. However, I am finding ways to automate that and reduce the incident rate of problems to even lower than my co-workers.
At that point, I think: Why not just write the code yourself?
Writing the code is more fun that reviewing code, not to mention less error prone.
If you use ntfy mainly as a Unified Push distributor, then I highly recommend switching to a XMPP client that can do the same.
I was also using it for notifications but I’ll probably switch to E-Mail for that and find an alternative UP distributor.
Conversations is working very well on my phone for that.
Do you recommend an app?
The first three on this list can do it: joinjabber.org/docs/apps/android/
Explanation here: joinjabber.org/tutorials/service/unifiedpush/
I’d run for the hills
There are so many issues with AI
Like ppl thinking skilled engineers cannot vet AI output. AI is pretty good for programming.
I have a few decades programming experience, as a professional software engineer, an open source developer, and a DevOps engineer. There is no way in hell I would do a code review where 15k lines were added and a similar amount of lines removed without having a long discussion with the person who made those changes. I’d want to ask a lot of detailed questions about the changes, questions that an LLM isn’t likely to answer, and most definitely not questions I’d be inclined to try to type into an LLM to try to get an answer.
Over the years I’ve dealt with all manner of bugs, from overflows & underflows, to bad assumptions about logic flow, and much much more. The whole purpose of pointed questioning of the author is to be comfortable with decisions made in the code and to minimize the chances of all sorts of potential bugs.
And yet there are cases like the Huntarr debacle, where the dev simply thought “and make sure your code complies with best security practices” to their vibe code prompts actually made it secure.
They added 14k lines of code in a week, and ripped out 10k lines of existing code. That’s not something that a skilled programmer can reasonably vet in that amount of time. This is showing all the signs of AI slop, and none of the signs of debugged or vetted code.
It’s not. That’s the problem. It actually sucks ass. It’s super low quality for anything more complex they s very simple CRUD app or a simple function. I say this as someone who s a heavy LLM user. It’s just bad code. It makes all kinds of simple mistakes. Just because code compiles doesn’t mean it’s good or does what you need it to do
You’re absolutely right, and the vast majority of people on this platform seem to get offended by anything AI related. Software engineers have been reviewing code made by other people since the dawn of the craft. Guess what y’all, AI generated code looks exactly the same, if not better on the first pass at creating a thing.
Down vote me all you want homies. You’re living in a fantasy if you think all AI is slop. Sure, I can see how it’s ruining some content on the Internet, but for code related tasks, its going to dramatically change the world for the better.
I just set up a ntfy server for Unified Push earlier this week to use with Matrix. Now I have to turn around and immediately replace it…
Same here. Literally just set it up and now this.
I hope the author will roll this back or someone makes a fork. I don’t want to immediately switch technology to XMPP and do it all over again.
You could, in the meantime, simply not upgrade to the version that uses AI.
Since, from what I’m seeing around, people are having issues looking for an alternative.
Sigh. Time to switch to gotify
been using EMQX plus an MQTT client on my phone for a few months now, I like it better than gotify since the app was chewing through my battery like a vampire.
it might be better now since my issues happened three-ish years ago.
This EMQX?
Seems it’s no longer FOSS?
I’ve been using Gotify for a few notifications from Home Assistant and it doesn’t appear to be eating my battery.
It’s a little more responsive than ntfy - sometimes ntfy doesn’t alert for ages after the trigger (could be phone power saving the wifi…), but then I also get realerts from yesterday… not had that with Gotify.
Fuck, I love ntfy, it’s one of the best self hosted push notification systems I’ve used. It has been flawless so far.
Don’t like this.
Oh ffs…
Thanks for the heads-up
ts getting you pinned to 2.17 in the compose file 🥹🤞🥀
I’ll embrace the inevitable fork.
That’s concerning. If it was “I generated a function with an LLM and reviewed it myself” I’d be much less concerned, but 14k added lines and 10k removed lines is crazy. We already know that LLMs don’t generate up to scratch code quality…
I won’t use PostgreSQL with ntfy, and keep an eye on it to see if they continue down this path for other parts of ntfy. If so I’ll have to switch to another UP provider.
I’m so tired of that.
I’m using it for scripts notifications + unifiedpush. I don’t know where to start to find the fitting alternative.
The maintainer you and said that they tirelessly tested, reviewed and verified changes over the course of 3 weeks to make sure that things were running and operating correctly.
This is how it should be done. It’s not like they’re vibe coding this.
And the lead dev for Huntarr said they were following best practices, and had a heavy background in cybersecurity. And we’ve all seen how that turned out.
This change 100% smells like vibe code. They refactored nearly 15k lines of code in a single push. That’s not something you just do on a whim without a team of full time devs or vibe coding. And we know they don’t have the former, so it is almost certainly the latter.
If more people were contributing there wouldn’t be a need for AI.
Ai can be powerful and destructive at the same time.
Ai coding can help a lot in accelerating software development. In the right hands that is. Meaning the software engineer still reviews the code. Test it. And takes responsibility. In those cases there is nothing wrong with it.
The problem is that some programmers are using AI without even looking at the end results. Just approves everything, commits, push and release. That approach is wrong and especially inexperience engineers might fail in this trap. So in this case the code has most likely a lot of duplicated code, full with bugs and other issues. Some issues you encounter it for the first time, since it wasn't tested etc.
In the latter story, you feel the impact. And the downsides of Ai. And only see the negatives of Ai. You might say it's Ai slop even. Or vibe coded. Which is correct.
Tldr: Ai can be very powerful in the right hands. It still requires a lot of human time and effort to get it correct. And if the engineer is too lazy then you feel the consequences.
Uovote and comment on: github.com/binwiederhier/ntfy/issues/1645
Please add this to the post.
NOOOOOOOOO
Oh goddamn it, I’m using this and don’t have an alternative lined up
Lot of hate for a project maintained by a volunteer and offered for free here. Nobody forces this free stuff on you.
I’ve been meaning to put something like this in my setup for a while, but definitely not this now! List of alternatives in the Custom Communication section at awesome selfhosted
d15d@feddit.org 3 weeks ago
They are not even trusting it themselves. This is from the release notes
Fuck that.
MirrorGiraffe@piefed.social 3 weeks ago
Classic “test in production” strategy, very solid!
Railcar8095@lemmy.world 3 weeks ago
Test in production is the best. We spent months warning from data bugs and nobody bat an eye (upstream bug, not our responsibility but we noticed) When it was d launched in prod we just pointed out the bug that nobody fixed was still there and immediately a war room was formed and the bug fixed within an hour.
It honestly seems more efficient to let shit hit the fan than to fight everybody to do their job.
callmemagnus@lemmy.world 2 weeks ago
Consider a donation to help people providing you the open source software you seem to depend upon.
Usage of a helper tool to perform tasks on code whether it is AI or the IDE internal features can reduce the work load of benevolent developers who has not asked you to use their softwares.
Maybe the language was not appropriate but get real. With the little revenue generated by the usage of people complaining, the use of AI agentic coding might be the only way to being features without pushing benevolent devs to burnout.
november@piefed.blahaj.zone 3 weeks ago
Hmm, no, I think I’ll just uninstall.
ubergeek77@lemmy.ubergeek77.chat 2 weeks ago
What happened to “reviewed and heavily tested over 2-3 weeks” from the release notes? Maybe Claude wrote that too lol