This truly has grown past a JS problem. NPM was kind of the first time dependencies were installed by the project rather than through the OS. But nowadays this has become the norm, golang, rust, and to an extent python also work by installing dependies directly from git for the most part. This isn’t going to get any better unless with revert to OS based dependencies which noone wants to do because developers want the latest and greatest model.
The way I see it, there are two problems with NPM:
It can blindly run any shell command w/o the developers explicit permission.
Anyone can make an NPM module, and the community is so fractured - common tools/features are not built into the language (or a standard library or a “vetted” community library - like boost for C++)
The first issue might be solvable with things like WebAssembly. Then it’s the developer who gets to decide how far these pm-hooks will reach (both interns of filesystem, network, etc) on a per project basis.
The second will need a shift in community mindset… and all these supply chain attacks are the fuel for that. Unfortunately, it needs to get worse before it’ll get better.
The first issue is NPM specific sure, but the second is true of all the languages I mentioned. Even golang which originally had a goal of having a built in library so vast you didn’t need much depencies has devolved into a large and fractured community.
I completely agree with you on the second point. This is a problem for all languages, but maybe we (as a community) need to change the approval, reviewing process for adding new libraries and features to languages.
This isn’t going to get any better unless we revert to OS based dependencies which noone wants to do because developers want the latest and greatest.
You’re very succinct here: Developer do want the latest and greatest, even if the interface isn’t perfect, and they’ll need to refactor their code when the next revision comes out.
Languages often have much slower release cycles than 3rd party libraries. Maybe this is what needs to be improved.
There won’t be a silver bullet, but I kinda like how kubernetes handles it: release cycles are fixed to a calendar (4 times per year). New features are added and versioned as alpha, beta, release. This gives the feature itself time to evolve and mature, while the rest of the release features are still stable.
If you use an alpha/beta feature, you accept that bugs and interface changes will occur before it reaches a stable release. … and you get warning and errors, if you’re using an alpha feature, but it graduated to beta/release.
Unfortunately, many languages either make this unnatural/difficult (ie: from future import… ) or really only support it if you’re using 3rd party libraries (use whatever@v1.2.3-alpha1).
noone wants to do because developers want the latest and greatest model.
That’s not true at all, the OS doesn’t have, and shouldn’t have, everything that a package has…
The alternative isn’t for the OS to do it: its to implement everything yourself… Speaking previous from experience working at a company that did exactly that… It has its own set of problems… But it is at least secure 😅
There’s another alternative, which is manually adding libraries to your project yourself instead of doing it all automatically through a package manager.
Yes, it’s less convenient to download and import a package manually, especially if you need to do the same with a litany of dependencies, but I don’t feel like that’s a bad thing. Raising the barrier of entry for arbitrarily adding thousands of lines of other people’s code to your project would force people to think about how much of that they actually need.
Have you ever looked at the available packages in a Linux distribution like Debian or a BSD? There are thousands and thousands of library packaged to support software releases. Like I said, that had been the distribution model for the better of twenty+ years until this new, shittier, model.
Can’t they make dependencies something that get checked at launch time? The executable says “I have the following external dependencies pulled in. “ and then is a version is blacklisted, the executable should stop and throw an error saying exactly what component was blacklisted and stopped it from running.
Why can’t we have executable declare their dependencies at launch time to the OS?
That’s essentially how most distributions of Linux and Unix work. You package an app with a list of depencies like “libcaca >= 1.2.3” and that’s that. If that dependency isn’t available in the distro you need to have that packaged (and thus have a maintIner for said package) forst. The distro’s package maintainers are responsible for keeping an eye on the upstream sources and provide reviews. Then this sort of crap like NPM came along and it became popular for devs to package their own dependencies.
This isn’t going to get any better unless we revert to OS based dependencies which noone wants to do because developers want the latest and greatest model.
A few operating systems (e.g. OpenBSD) do actually (try to) enforce using pkg for Perl dependencies, due to Perl being “system Perl” instead of “packaged Perl”.
I’m not super familiar with Maven so I could be wrong, but doesn’t Maven still pull depencies from upstream? That doesn’t fix the problem. Having depencies packaged in the OS means there is in theory some level of overview and review by the package maintainer(s).
There are two reasons. One is the name spacing that is inherent in Maven and bolted on to npm, and the enforcement or lack of enforcement in the repository. You can read more about that here blog.sonatype.com/why-namespacing-matters-in-publ…
Then there’s the fact that npm runs “install” scripts when you download the component. This means if you can trick someone into grabbing your component by namespace confusion, typosquatting a name etc, you can get code run as soon as someone makes a mistake. Maven on the other hand only downloads the jars, it does not execute them.
Taken together, you have an easier path to tricking people to grabbing your component with npm and that trick leads directly to code execution.
I am on my phone, which is a bit too long to explain, but there are multiple facets to how NPM is worse than most packaging systems out there. There are enough on the web for you to browse and learn, if you are really interested to know more.
But, here, I quoted a little something from Brian from Sonatype.
wildbus8979@sh.itjust.works 3 weeks ago
This truly has grown past a JS problem. NPM was kind of the first time dependencies were installed by the project rather than through the OS. But nowadays this has become the norm, golang, rust, and to an extent python also work by installing dependies directly from git for the most part. This isn’t going to get any better unless with revert to OS based dependencies which noone wants to do because developers want the latest and greatest model.
LedgeDrop@lemmy.zip 3 weeks ago
The way I see it, there are two problems with NPM:
The first issue might be solvable with things like WebAssembly. Then it’s the developer who gets to decide how far these pm-hooks will reach (both interns of filesystem, network, etc) on a per project basis.
The second will need a shift in community mindset… and all these supply chain attacks are the fuel for that. Unfortunately, it needs to get worse before it’ll get better.
wildbus8979@sh.itjust.works 3 weeks ago
The first issue is NPM specific sure, but the second is true of all the languages I mentioned. Even golang which originally had a goal of having a built in library so vast you didn’t need much depencies has devolved into a large and fractured community.
LedgeDrop@lemmy.zip 3 weeks ago
I completely agree with you on the second point. This is a problem for all languages, but maybe we (as a community) need to change the approval, reviewing process for adding new libraries and features to languages.
You’re very succinct here: Developer do want the latest and greatest, even if the interface isn’t perfect, and they’ll need to refactor their code when the next revision comes out.
Languages often have much slower release cycles than 3rd party libraries. Maybe this is what needs to be improved.
There won’t be a silver bullet, but I kinda like how kubernetes handles it: release cycles are fixed to a calendar (4 times per year). New features are added and versioned as alpha, beta, release. This gives the feature itself time to evolve and mature, while the rest of the release features are still stable.
If you use an alpha/beta feature, you accept that bugs and interface changes will occur before it reaches a stable release. … and you get warning and errors, if you’re using an alpha feature, but it graduated to beta/release.
Unfortunately, many languages either make this unnatural/difficult (ie:
from future import…) or really only support it if you’re using 3rd party libraries (use whatever@v1.2.3-alpha1).Venator@lemmy.nz 2 weeks ago
That’s not true at all, the OS doesn’t have, and shouldn’t have, everything that a package has…
The alternative isn’t for the OS to do it: its to implement everything yourself… Speaking previous from experience working at a company that did exactly that… It has its own set of problems… But it is at least secure 😅
vithigar@lemmy.ca 2 weeks ago
There’s another alternative, which is manually adding libraries to your project yourself instead of doing it all automatically through a package manager.
Yes, it’s less convenient to download and import a package manually, especially if you need to do the same with a litany of dependencies, but I don’t feel like that’s a bad thing. Raising the barrier of entry for arbitrarily adding thousands of lines of other people’s code to your project would force people to think about how much of that they actually need.
Venator@lemmy.nz 2 weeks ago
Or you can just use git and review the changes to the packages when they change…
wildbus8979@sh.itjust.works 2 weeks ago
Have you ever looked at the available packages in a Linux distribution like Debian or a BSD? There are thousands and thousands of library packaged to support software releases. Like I said, that had been the distribution model for the better of twenty+ years until this new, shittier, model.
Venator@lemmy.nz 2 weeks ago
there are over 3.1 million packages available in the main public npm registry…
muusemuuse@sh.itjust.works 3 weeks ago
Can’t they make dependencies something that get checked at launch time? The executable says “I have the following external dependencies pulled in. “ and then is a version is blacklisted, the executable should stop and throw an error saying exactly what component was blacklisted and stopped it from running.
Why can’t we have executable declare their dependencies at launch time to the OS?
wildbus8979@sh.itjust.works 3 weeks ago
That’s essentially how most distributions of Linux and Unix work. You package an app with a list of depencies like “libcaca >= 1.2.3” and that’s that. If that dependency isn’t available in the distro you need to have that packaged (and thus have a maintIner for said package) forst. The distro’s package maintainers are responsible for keeping an eye on the upstream sources and provide reviews. Then this sort of crap like NPM came along and it became popular for devs to package their own dependencies.
tux0r@feddit.org 3 weeks ago
A few operating systems (e.g. OpenBSD) do actually (try to) enforce using
pkgfor Perl dependencies, due to Perl being “system Perl” instead of “packaged Perl”.wildbus8979@sh.itjust.works 3 weeks ago
Debian does as well for anything that is packaged. For python, golang, rust, etc as well.
PushButton@lemmy.world 3 weeks ago
Maven entered the room.
wildbus8979@sh.itjust.works 3 weeks ago
I’m not super familiar with Maven so I could be wrong, but doesn’t Maven still pull depencies from upstream? That doesn’t fix the problem. Having depencies packaged in the OS means there is in theory some level of overview and review by the package maintainer(s).
PushButton@lemmy.world 3 weeks ago
I am on my phone, which is a bit too long to explain, but there are multiple facets to how NPM is worse than most packaging systems out there. There are enough on the web for you to browse and learn, if you are really interested to know more.
But, here, I quoted a little something from Brian from Sonatype.