On C popularity vs python, size of stdlib, package managers in langs
I have a friend who is a ridiculously advanced programmer and he half-jokingly said that in C, there's an implicit slogan that 'You want a function? write it yourself!'. This is very much in the spirit of 'C', there's very little in the standard library and that's by design (when compared with other languages).
C++ went overboard with the STL later on. While still nothing compared to what would come later with interpreted languages.
This is how I experienced this evolution (I'm old).
From now on, I'm talking about Unixy workflows on terminal processing large amounts of text (not gui, not servers). This is what I was doing in the 90's.
There was a time where only 'things that had to be fast' were written in C. Hardcore cleaning, vector multiplications, Singular value decomposition... were in C. The rest were crappy bash scripts calling the C programs using pipes.
This was brittle as hell. When a tool gives you an error, it doesn't propagate 'up' if you called it with a pipe into a different programming language. Let alone that bash is not a real programming lang and errors are pretty bad.
Then Perl 5 came. Perl was wonderful for processing text. Regex was as fast as C. It has humor built into the language. The keyword to make something an object was 'bless' (terrible tackled-on OO in Perl, not worth doing). With Perl (perhaps with the web becoming mainstream beyond dialup modems), programmers started sharing code. There was a community (a mailing list that was very active). C didn't really have that, C was like programming alone, Perl was like programming with friends over beer. Compared to C + bash, Perl was a relief. Although Perl was still calling that C code using pipes, which I now consider a very bad practice. My friend (who is a very advanced programmer, where conversation started) showed me the pipefail bash variable array, that I completely missed then.
With code sharing, the first attempt at having a package manager on a language came about. It was CPAN, a website that collected perl packages (no install or update mechanism intitially). It was very primitive, but better than nothing. People experimented there. There were ordered hashes, a hybrid between a hash (dictionary in python terms) and an array. The man page started with 'it's alive!, it's alive!' :) There wase PDL, the perl data language, a very old ancestor of pandas that astronomers used. Packages increased in complexity and had dependencies (Fortran, C) that you needed to install with the OS package manager.
This was the first time that I, as a programmer, experienced that you could reuse someone else's solution rather than writing it from scratch (or copying it from a book :) ).
Then python came, with a much better OO story that made it possible to write bigger programs without going crazy. Perl 5 tried to be Perl 6 and failed, becoming extinct.
Python brought the slogan 'batteries included'. In modern times we have pushed that to an extreme, I think iOS had the idea 'there's an app for that'. Which means: you don't need to code anything to solve your problems, just install something from the app Store. Or the lang's package manager.
'Batteries included' became wildly successful. Now you had to decide: C's offering was still 'want a function? write it yourself!', and python was 'here's a bazaar of anything you can possibly need, just import it'. Programmers grumbled when they had to implement something in C.
R had tons of packages that did extremely niche, specific things in stats. People published papers with an R library. But R lived in a tiny corner of the world, whereas python was everywhere. Unixes started shipping python, the system python, to do glueing. It used to be perl, and before perl, bash. Python won. Python interface with C was good enough that anything computationally intensive was done calling the C library that the package shipped.
Code in those repositories was of varying quality, like everything on the web, but popularity and word of mouth usually got you to discriminate quickly.
Nowadays, even text editors have package managers. Gone are the days of downloading .vim files by hand to your home and calling them from .vimrc. When I started vim in the 90s, I had 2000 lines of .vimrc. Now I can replicate that with 20 (calling libs).
But the instinct looking into a package manager (or stack overflow!) became ingrained in programmers all over.
Not to mention everyone seemed to be lenient on trusting 3rd party code (more than what you could wipe out yourself) 'There's an app for that' thinking, but for programmers, who are used to build things.
The reckoning
I'm skipping steps in the History of package managers for programming languages here, but bear with me.
At this point, most 'popular, mainstream' programming langs had a package manager (go, rust, javascript, python etc). People relied on dozens of packages even for apps that don't really do all that much. Packages got updates, package managers would bring them to your app and you would find that things break. It's by no means a solved problem.
Then the cracks started to show. A security vulnerability here. A package author that turns his key package into 'protestware' (non-working code with a message, often social of political, to capitalize on the attention they gained), breaking code for everyone using it. You would worry about updating your packages and breaking your app. You would have no idea how many LOC your app was clocking, because those packages, if included, would make the number embarrasingly large. Not to mention authors would abandon a package, or a new, fancier one would come up and you would have to update your code to use the newer one. Not fun.
Or you would have a company taking ownership of the servers hosting the packages, or somehow of the process running the package manager (Javascript PPM and M$ come to mind but there are others)
Then we all looked at each other: do we need all this complexity?
The value prop of C 'want a function? write it yourself!' became somewhat suggestive.
The modern day: packages are less useful
C is like a shark. Still alive, and kicking ass, surrounded by creatures that are far more evolved and modern, that had added a lot of complex adaptations (mammals, birds). Still sharks are extremely successful in their ecosystem.
What if the ecosystem has changed, favoring the shark's adaptations? What if 'want a function? write it yourself!' has become not much harder than using a library with a ton of dependencies?
Enter 2023. Code generators (chatGPT, copilot) actually produce code from a text description. And they can write tests for it, and even debug it themselves (this is early days). It's another tool for a programmer that wants to implement things from scratch. It will be abused by junior programmers and probably the code quality of humanity on average WILL go down. But that doesn't mean that you (as someone who has written code for decades) cannot use it as an extra pair of hands, knowing full well of the limitations.
There's a clear alternative to 'import this' 'import that' and have > 2000 LOC before you started writing a single line. And this is new.
That is, code generators make it possible for lazy people to stop depending on libs. Although very likely the code generated by chatGPT will suggest libraries :) on languages with a package manager.
But C? You can actually perhaps write a solution in C as fast as one in python 'batteries included' now just because of those code generators. And there are advantages to doing that (beyond being faster):
1. It complies on lots of architectures. Behold what Justine Tuney is doing :) Portable C is actually real!
2. You don't have to audit code from a lib, nor trust it by default
3. You don't have to fear that updating packages in your system (or virtual env, or container) would break your 'batteries included' code
4. You can do formal verification (this I know nothing about, just learning now on your prompting!)
5. It's far easier to deploy
This is a very attractive value proposition!
/gemlog/