- At previous gig relocation of manufacturing to the UK stopped because it would be impossible to operate there due to severe delays in procurement and long delays in securing appropriate visas for employees. Manufacturing sent to different country. (for many projects I work on 1-2 days delivery is the expected norm. With 3-5 days for "slow" shipping)
- Stopped buying from UK companies due a) many UK companies no longer shipping to EU, b) long delays when ordering something from the UK.
Of course, this is what it looks like from my perspective. That doesn't represent the totality. But in my work (which spans a few different sectors), the UK sort of became a black hole that we avoid if we can. Find different locations and sources for products, move on.
Except for Germany, which of the major countries are you referring to? Show me numbers.
It has been 10 years. How long do you think this should take? 20 years? And how long before the lost decade (or perhaps decades) have been made up for and the UK goes net positive? 30 years? 50? (The area below the curve is important. You learned about integrals in school, right?).
Trust me: nobody I talk to is interested in doing business with UK companies or in the UK if they can avoid it. Which makes me curious: who are these companies that have seen a boost in foreign trade?
Well, that's the curse of machine learning: since everyone uses Python you have to deal with Python. Even though Python isn't very nice when things start to get serious and you don't want to spend your time fiddling with noise just to make something work at scale.
I'd wish the ML/AI/LLM crowd would see that it is in their interest to get better developer ergonomics at scale. (I don't want to have to turn to C++)
The ML/AI ecosystem is a minefield, and pure Rust rewrites (Candle, Burn, ...) are still immature and incomplete. But I'm pretty sure we're eventually going to see the same uptake that's already happening in the data processing world.
The performance is not the (only) issue. The issue is the death by a thousand cuts involved in distributing Python programs without a two page set of instructions that have to be followed to make it work. It is rarely "just works" unless you can make a lot of assumptions about the environment it runs in. It's why I generally steer away from any application written in Python. It is going to be painful.
However my experience might be a bit different since I actually have to deal with Python at scale and in a fairly dynamic environment.
I need to run hundreds of Python programs, written by dozens of programmers, over many years, that speaks to custom hardware, runs on a remote site, in a production environment that has to work and with new versions of things coming in all the time. Some of these Python programs not only link with C libraries, but run external binaries because developers didn't have time to integrate them as libraries because it takes forever to make it work on all the different os/arch combinations (easier to just run the C code in subprocesses).
This runs on three different CPU architectures (we're trying to eliminate one of them), two different operating systems and a pretty wide mix of hardware and system configurations I need to insulate Python from. Much of the hardware being custom built stuff. Because Python has a lot of exposed surface to the OS compared to a statically linked binary. (Roughly 100x the surface of statically linked binaries that don't link with libc -- which is evident by the insanely bloated OCI images that result from packaging what you need to run)
Modern compiled languages that have sorted toolchains makes it pretty easy to produce "production grade" os/arch specific binaries that can survive almost everywhere. You compile build a statically linked binary for each architecture to overcome the challenges of varied Linux runtime environments (see Linus T's frustrations with Linux and software distribution - it's not like it is easy to begin with). Go and Rust do this well.
So you end up having to containerize everything in ephemeral containers to lock down the execution environment while retaining some speed. But of course it isn't that simple, because if you depend on access to weird hardware and/or you run on custom built machines you have to detect this and ensure the application inside the container gets access to the things it needs from the container. So you have to fix that.
In a way that is almost completely invisible to the developer.
All of this has to be understandable and _reduce_ complexity for developers and operators so at the very least you don't follow the Python philosophy of "just throw another layer of complexity on it and make the instructions another page longer".
40-50kLOC later (in Go and Python, I have lost count) of code to try to make the problems go away, and I have something that is on the verge of actually being usable in a production environment for taming wayward Python code.
The easiest fix? If people could stop using Python because they don't want to learn a language that can produce something that is easier to distribute to users.
Believe me, I have spent months now trying to make Python work properly in a challenging environment. The only way this "worked" before was by just lowering standards to where the definition of "works" is flexible enough to count daily dumpster fires as "nominal". And of course people don't care. Python fosters a "it works for me" mentality where people don't know and don't care what it is like to be on the receiving end.
90% of problems I have because of Python would just disappear if people used languages that can produce robust binaries with limited exposure to system peculiarities. But that kind of requires people to understand why it is a problem in the first place. And people generally don't bother to know.
No, you are right about it not being limited to Python. But for python the common courtesies I am used to right out of the box tend to require extra effort on part of the programmer. And «extra» doesn’t usually happen.
Even C, with its ancient, haphazard, ugly, fragile, awkward toolchain, can often trivially produce binaries that will just work with very little effort.
I have spent decades of my life writing tooling, libraries and infrastructure. And no matter where you go, developers only do the bare minimum if they can get away with it. That doesn’t mean they are bad people. It means tools and infrastructure has to be designed with acute awareness of reality.
Python has been around for 35 years. And it still hasn’t evolved things we should take for granted today despite its increase in adoption. To me that’s pretty fucking awful project governance.
Historically (80s/90s) I started using Perl because I didn't have to write all those malloc and free I spent years writing in C and I could perform string operations much more easily. Then in the mid 90s because of that wonderful CGI.pm Perl module. But the plus of all those languages, and Java, was managed memory. Then in the mid 2000 I learned Rails, and after Rails I learned Ruby. It was like Perl but much easier to understand and again no types to type. Basically what I did in Java but in a fraction of the time and in a fraction of the lines of code. Then a customer asked me to work with Python on a Django app so I learned Python. It looks like a Ruby designed by Klingons but it's OKish.
All those bugs I constantly read about, they don't happen very often and are a good tradeoff. Maybe Rails and by Django are shielding me from some bug scenarios.
I'm not trying to be cute here, but it seems like you have mostly been using scripting languages without static typing. How do you know what it is like to develop in languages with strong typing that produce binaries and that these are unpleasant to use?
And dabbling doesn't really count. It takes time to actually learn a language. Much longer than most people are willing to admit.
Python has other, bigger problems that make it a constant headache. One of them being the dismissive attitude towards any and all of problems that come from versioning, dependencies and quirks that make it challenging to have robustness.
Criticisms are typically dismissed by suggesting heaping yet another "solution" onto the growing pile of "solutions" that you have to drag around with you. That people have to learn. That you have to install tooling for. That has to be vetted. That has to become part of the toolbox to get even seemingly simple things done. This attitude is a big part of the reason that I strongly advise people against using Python in production. On top of all the problems presented in a real-world setting.
Almost all of the time, people who are fond of Python are more interested in defending python, disparaging me, downvoting me etc that listen to why I make that recommendation.
(I get it. People like Python. What I think of Python as a language is irrelevant. In fact I don't have that much against it. But I do have a lot against it in a setting where you need reliability and repeatability)
I have spent the last month of my life building a system that can run Python tooling reliably in a business critical application. I knew this was going to be a pretty big job when I started, but for every problem I solve, a bunch of new problems arise. I am starting to see light at the end of the tunnel but it hasn't exactly been smooth sailing. I'm almost there for a first version, but there are a bunch of problems still to solve. Mostly because I care about developer ergonomics and that things should "just work". One important goal is that my solution shouldn't impose any significant cognitive burden on people who use it. That's really hard.
(I don't think the solution will be open source since my contract wouldn't allow for it. But I'll make the case at some point for why it should be open sourced)
And yes. There are statically typed languages available today that have decent tooling that provides superior developer ergonomics. I can understand that people don't want to learn new languages, but if you have the capacity to do so I would recommend trying to move on if the code you write has to run outside your own workstation. If an old fart like me can learn and adopt new languages, so can you.
Python the language is pretty nice. It has its warts, but I make my living in PHP which is practically made of warts. But the python ecosystem still seems to be trying to figure out this whole package management and project setup thing. In most languages I can do some form of `$blub install` where $blub is the language's official package manager or some close equivalent. It's just python that always screams at me that I have to set up and "enter" a virtualenv. I get what venv is for, but it's still a weird hack of hardlinks and relative paths that no other language seems to need, and a clumsy two-step dance of a UX that hasn't improved in like 20 years.
I am not sure you get what virtualenvs are: Python is never screaming at you to set up a virtualenv, it must be a particular package recommending use of virtualenv for easy set up without interfering with the rest of your system.
Virtualenv allows you to seamlessly run multiple Python ecosystems simultaneously, even within the same project directory. It's basically primitive containerisation mechanism that predates any actual containerisation systems on Linux.
You do not have to use it, but then you can easily slip into a sort of "DLL hell" (multiple incompatible library versions installed system-wide) with multiple projects — or need to bundle all dependencies within your project directly. None of this is specific to Python, really — any shared library system has the same challenge. How many other systems are there in active use making it as easy to use multiple incompatible versions of shared libraries per project or within the same project?
When in doubt, you can always retreat to the basics in Python world: put packages you need in a path of your choice, and point PYTHONPATH (sys.path) at it.
I thought I'd given sufficient clues that I actually do in fact know what virtualenv is. I was hip-deep into python when it was introduced, and thought it was a clever hack, but it's relying on python being compiled to find its libraries in a path relative to the python binary, so rather than use something sensible like a launcher that sets PYTHONPATH to a local dir like every other language does, it hardlinks the python install into the local dir in order to pretend everything is a system-wide global install. I always figured that hack was a temporary workaround, but it's going on a couple decades now.
As for the screaming, I'm talking about pip, which has the unique property of just refusing to work by default unless you use --user (what most other package managers now call "global") or are in one of these pretend-global environments. Ultimately it's just a paper cut, not a show-stopper, but it points to not just a failure to address the pain points of the package ecosystem, but a seeming refusal to do so.
Usually running pip as root overcame that problem — but it did cause all the other problems virtualenvs were introduced to work around (so I am not suggesting this as the solution).
You are right that there are many problems with packaging in Python, and yet I feel like virtualenvs are the smallest of those.
I believe we also need to compare tech this old to tech from the same era: obviously newer ecosystems had the benefit of hindsight, but how does managing dependencies compare between Perl and Python, for instance?
The biggest problem with Python packaging is — IMO at least — that it is actually attracting so many evolutions and proposals that ot is hard to stay on top of if you do not make Python packaging your core interest.
For managing dependencies in Perl, it was originally a similar story to Python: everything was system-installed, but many people would install things to their home dir and set PERL5LIB in their .bashrc. The cpan client was smart enough to detect and use the home install when writing its initial config, so you could call it a day. Later there was local::lib which fiddled @INC for the use of a project-local directory, and cpanminus defaulted to using it, and then Carton came around which is more or less a clone of Bundler from Ruby, also using local::lib under the covers.
What are the problems you have with tooling? Imo it's no worse than most other languages besides a very small handful of recent ones (rust, go) where everything is included
The easy approach is usually just throw it in an OCI container
There's not much concrete to go on here besides "I don't like the ergonomics"
Throwing it in an OCI container is not the "easy" approach. It is the beginning of opening a can of worms. I know, because I've been developing tooling to do that at scale for the last few months and behind every layer of worms there are uglier and bigger worms that you need to deal with.
And yeah, I have written a lot of code to insulate Python in containers while allowing meaningful access to hardware and services. While at the same time not heaping more complexity and cognitive surface at the developer. Including writing my own container software to actually understand what's involved at a more detailed level so I know what I'm doing when trying to make this work with existing container software. (No, I don't run any of my container managers in production since I don't want to maintain it -- but this also means a bit more complexity in using existing ones)
It may be "easy" in trivial cases, but it is very far from easy if you want to make something that can cater to a wide range of scenarios.
Do you have examples of issues? OCI is tarball of dependencies that doesn't fight with the OS userland.
Why do you need to write code to insulate Python in containers?
At the simplest level, you can add the flags to the container runtime (network host, host ipc, host process namespace) to turn off all the namespacing besides filesystem and the Python container runs just like a non containerized process.
And even there most of the custom code is just running a ton of combinations of inputs against docker build. The OCI container gets rid of "wide range of scenarios" for you standardizing the runtime environment
You are aware of the fact that you essentially have no rights at border crossings, right? Even if you are a US citizen entering the US.
This is why many companies have procedures for when employees visit certain countries, including the US. For instance that you are not allowed to bring your personal phone, your personal and work laptop or any medium that can hold sensitive or proprietary information.
Is this really the case? As far as I understand it, US citizens have an absolute right to enter the country. So they can sit you in a room and ask you questions all afternoon, but eventually they have to let you in.
File a lawsuit, and if that fails go to the media. Or do both at the same time. As bad as things are, we don’t live in 1984 or Soviet Russia yet, and this works for citizens. Maybe there’s one or two cases where this hasn’t worked, but that would be an extreme outlier. The courts still strongly support habeas corpus for citizens.
"At a time of sticky inflation and a softening job market, extra money in the bank can serve as a lifeline for many consumers. Yet just 47% of Americans indicate they have sufficient liquidity or access to funds to cover a $1,000 emergency expense, a Bankrate survey found. This can present challenges when dealing with job losses, medical issues or other problems that could exhaust one’s cash reserves."
So this raises the question for most Americans: file a lawsuit with what money? It also raises the question of whether your rights should depend on your bank balance. Now, we can pretend that anyone can start a successful gofundme to fund their efforts to have their basic rights protected, but you see where this is going?
As for the media: what media outlets? And why would they care unless it is a particularly egregious story or it involves someone famous? Thousands of people have their rights violated every day. Why would the media care about any "nobody"?
No you don't live in 1984 or Soviet Russia. You live in much worse circumstances because back in the Soviet times, the state, fortunately, had limited ability to surveil, track and control you. The capabilities a state gone bad have at their disposal today are much, much more invasive and effective. And the brutality you risk (eg CECOT) is arguably not materially better than getting shipped off to Siberia. At least around 90% of those who were shipped off to Siberia had fixed term sentences.
And perhaps you will never have the opportunity to litigate if you find yourself wronged. What if you die in custody due to a medical emergency that was not taken into account? Thousands die of medical neglect in custody every year in the US. (Look it up. State and federal custody, ICE, prison system etc). This happens to thousands of people every year and nobody cares. Least of all the media.
What if you find yourself in CECOT -- effectively isolated from the world and anyone in your personal network that could do something about it? Or in ICE detention where the very best case is that you disappear for days without anyone knowing where you are, and without the ability to contact anyone.
It frightens me that people can find their way to Hacker News yet be so ignorant of the reality the majority of Americans face. How can you not know basic statistical facts? How can you not ask yourself how someone who isn't rich, isn't well connected, doesn't have a strong network of people around them are supposed to defend their basic rights?
Know what you are talking about. Find out what reality is like for other people. Don't blab on about remedies that are purely theoretical for much of the citizenry. Know stuff!
Your hyperventilating about the current state of the US aside, there are many ways in which things could be worse. Are they great? No. But they have been worse, they are worse in many other countries, and likely they will get worse again here before long. If you think things are this bad now I can’t wait to see what you think about things a decade or so from now.
People currently do have ways to fight back, even those with limited resources. That may not be the case forever. But lying down and sighing about how hopeless things are helps nobody, least of all yourself.
Not sure if related but this is my story. I am Spanish. When leaving SF back to Europe from a Google IO, after control, waiting to scan my luggage, an officer stood besides me and started speaking in Spanish besides me.
The first minute my brain didn’t register because though Spanish is my mother tongue, I guess it was not ready for that. The police officer started to get irritated. Eventually my brain switched, I had a chat with him and he left.
I was totally freaked out the rest of the time, till the moment I boarded.
Only then I realized how frail our rights are when you are abroad.
reply