Hacker Newsnew | past | comments | ask | show | jobs | submit | more _Codemonkeyism's commentslogin

Key take away

"There is absolutely no reason to require your pilots to require a MAX simulator to begin flying the MAX,” the Boeing employee replied. "Once the engines are started, there is only one difference between NG and MAX procedurally, and that is that there is no OFF position of the gear handle. Boeing does not understand what is to be gained by a three-hour simulator session, when the procedures are essentially the same.”


I still think it's interesting to reflect on the mindset that leads to this conclusion. As far as I have been able to determine (although I'm not in the aerospace field), the 737 MAX is procedurally identical to the NG, except when something breaks. The failure modes are slightly different, with potentially lethal results. As a computer scientist, I'm not accustomed to thinking about functional equivalence in the presence of hardware failure, and maybe this Boeing employee was not sufficiently drilled on the need to consider such aspects for aeroplanes. It is of course the fault of Boeing corporate culture and internal procedures that this can be overlooked.


>As a computer scientist, I'm not accustomed to thinking about functional equivalence in the presence of hardware failure...

How are you not?

I mean, I get it 90% of the time we screw up the programming somehow, but as a computer scientist, I never ignore the possibility of hardware failure. Memory goes bad. Devices fail. Networks die. Semiconductors transiently in strange ways if you don't take the right precautions...

It's the entire impetus behind GIGO. If you shove garbage into a perfectly working software system; (corrupt data from a malfunctioning input source), you still get out garbage.

It's why life and safety critical automation is so fundamentally different from lower stakes programming tasks where "reboot the damn thing" is a viable option.

If your sensor goes bad, and you're in the air, you can't do squat to fix it. You have to detect the error, and fail the system gracefully by taking it out of the loop, informing the operator of the system failure, and most importantly, never allow that system to do anything that could jeopardize the ability of the operator to continue operating.

This is or at least I thought it was basic Control Systems 101...


> How are you not?

I research compilers and type systems. If the RAM dies while the compiler is running, you rerun the compiler on a new machine. A lot of computer science abstracts away the notion of hardware failure, because otherwise it becomes enormously cumbersome to talk about anything. This is fine as long as you don't actually build real high-reliability systems with the same approach.


>> How are you not?

> I research compilers and type systems.

I hope it's obvious that the software you work on is not supposed to be run during the flight.

The critical software is supposed to do as little as possible, and everything is expected to be in already compiled (and thoroughly verified) state.

And even for the product of yours, as soon as it is not used only for the research but as a production compiler which produces a firmware for the plane, it would have to be proven much more than what is expected from it while it is just an artifact of a research.

In short, even if you are lucky to just do the research, you should be aware (and thankful) that the critical software has other expectations. Including how it responds to failed sensors: different response to the external inputs is a fundamentally different software, even if you never thought about it before.


I think his main point was that for most of us, hardware failure is considered an adequate excuse for why something works -- most of us are not expected to have software that _continues working_ when things break.


The "failures" of the sensors are simply the "less common" inputs. The proper control software should simply be written for all possible inputs, which include inputs from faulty sensors, and the result of the processing should not have some catastrophic consequences.

Compare to the web app that awaits the username, but when the username is not the "most common" (e.g. contains some new unicode symbols, or is of zero lengh) it allows catastrophic security failure and intrusion.


A great deal of procedures focus on failure scenarios. If these are different, then the MAX is procedurally different from the NG.


Yes - this seems to me to be the same erroneous thinking that lead to the Ariane 5 maiden flight loss, caused by an integer overflow. There again the thinking was "this is effectively the same vehicle as its predecessor, therefore we do not need to test it thoroughly".


Similar thinking contributed to Therac-25 as well - the software worked safely on the old hardware, of course it will be safe on the updated hardware.


I suppose it depends on the field in which you work, but many safety-critical fields have an expectation that hardware failures are captured and mitigated and there are various tools to capture these design decisions and ensure they are tested. One example tool in this case would be a software fault analysis (FTA) or failure-mode-effects analysis (FMEA) that looks at a broken sensor input value as a failure mode.[1]

It's been my experience, however, that these sorts of design tools are more unfamiliar to software groups than hardware bubbas. It's not uncommon to simply see "software fails" as a failure mode which isn't very helpful. I'd be curious what the HN community's experience is with software as it relates to design tools like FTAs, FMEAs, hazard analyses, etc.

[1] https://standards.nasa.gov/standard/nasa/nasa-gb-871913


The other key take away I had was:

> "Boeing knew the approach might be questioned [Calling MCAS a simple addition to Speed Trim], so it sought input from its FAA-designated authorized representative (AR) "to ensure this strategy is acceptable” for certification.

> "After speaking with the [AR], concurrence was provided that we can continue to use the MCAS nomenclature internally...while still considering MCAS to be an addition to the Speed Trim function,” the memo said. "This will allow us to maintain the MCAS nomenclature while not driving additional work due to training impacts and maintenance manual expansions

I can imagine some Boeing employees being uncomfortable, but having it run past the FAA would have relieved that. Pretty shocking regulatory lapse. I know nothing about the AR system - is this a Boeing employee, or someone who works full-time for the FAA?


The FAA basically picks a Boeing employee and says "you represent and answer to us now."

The employee however, is still managed, and reports first to Boeing management. They're a glorified liaison/paperwork interface. This was different than before as I understand it, because the FAA used to become the direct report for their Designated Engineering Representatives under the old system. This meant there was no management layer running interference between the rep and the regulator.

I might be misremembering that though.


I believe with AR even the picking is done by Boeing. unlike DER. One huge downside of this is that the informal interactions between FAA and Boeing engineers are almost non-existent, leading to even greater lack of technical knowledge on the FAA side.


>I can imagine some Boeing employees being uncomfortable, but having it run past the FAA would have relieved that.

There were some emails released where a Boeing employee boosted about he used "jedi mind tricks" on the FAA people.


It's not that shocking except in hindsight. The speed trim system is, from what I can tell, incredibly similar to MCAS at least on paper. They both adjust the stabilizer trim in order to ensure the legally-required amount of force is needed to move the stick in certain scenarios, both activate 5 seconds after the last manual trim input when flaps are up and autopilot is off, and both rely on non-redundant sensors. The only difference is the exact combination of sensors used and the exact obscure scenario they're designed to protect against.


I read here previously, I think it was here, that the primary difference was that after repeated counteraction by the crew the speed-trim system would give up, whereas MCAS doubles down and reduces the time between its attempts to force trim "correction", literally wearing pilots down physically until the system wins and the plane crashes.

Another post showed how AoA sensors (IIRC) has systematic errors causing MCAS to operate when corrections weren't required. As you say, lack of redundant sensors.


Thank you for that explanation


The term for this is regulatory capture. That is why staff at the FAA share in the culpability of these tragic events.


After some years of FreeNAS at home and in the company I've decided to go with Unraid for my home storage, it's just easier to add and change drives and the UI is more focused on storage too. Adding how easy it is to use e.g. NextCloud with Unraid I'm much happier than with FreeNAS.


1. He says he has made regular backups, but now needs to restore all VPSs

2. The website says "Snapshots allow you to create a backup copy."

3. He says "No they do not allow snapshots download."


It sounds like snapshots are directly reachable from within FTP in a directory. Snapshots are a clean copy of the file system you can back up, but they are not backups.

He also states he has his backups, so he's mostly just whining because he's annoyed he has to reupload stuff. Which I get, but again, he should understand what snapshots are and aren't.


You do the same mistake as the other guy in the twitter thread, you mix Simple Hosting snapshots and the Cloud hosting volume snapshot.

Here the right one which state that they are backup: https://docs.gandi.net/en/cloud/volume_management/volume_sna...

Here's the one that you quote (which isn't the same service): https://docs.gandi.net/en/simple_hosting/common_operations/s...

Be careful next time judging with that little knowledge of the issue.


Interestingly, this page has been edited to add a warning: "A snapshot is a frozen version of your volume that allows you to restore it to a previous state. It should not be regarded as a backup of your volume. If your volume is deleted, all related snapshots will be deleted too."

Here's the page as of earlier today. https://web.archive.org/web/20200109194005/https://docs.gand...


As @dwild stated, that's the other type of snapshots used for web hosting (which I assume are copy-based backups due to how little storage sites usually use). These snapshots are never reachable directly to the customer; at best, they can restore a volume back to the snapshot's state or create a new volume at that state and attach it to a VM.

> He also states he has his backups, so he's mostly just whining because he's annoyed he has to reupload stuff.

Or they're annoyed that they paid for a service, at the very least billed as backup, only to be told "welp, it's gone".


How is this even possible?


not the same, but years ago i worked on a cctv/dvr system for a Giant Electronics company. we watched HOURS of video (most w/o audio) as part of the rma process. there was good reason for it, as a failure in a circuit is sometimes a function of heat and time. there was one case where the device's ambient temp would spike and the video encoders would produce garbage output every few minutes. replacing the encoders resolved the issued. we kept a locked cabinet where drives with potentially sensitive video was kept, for instance we had a device that had been at a bank during an attempted robbery and had been malfunctioning. we stored the drive until it was certain the recorded data was corrupted, and not the playback system on our device. eventually the authorities agreed to take custody so we could get it off our hands. i was on the tech end of things and not the business end, so i cannot say what sort of privacy expectations our customers had. most of these were installed in businesses...but we did have a few from private residences.


Well, people who are unaware, or uncaring of the fallout, decided to install peep holes in their homes, with the implicit belif that no creep would be so vile as to actully look in. Now we have a quarter billion of those holes installed. Neat huh?


But peep holes are one-way, and the manufacture does not have the ability to remotely view the peep-hole.


Question is more like "why wouldn't this be possible"


Google sends Google adwords ads with physical letters in Germany at least.


What does it mean to send an "adwords ad"? Has Google branched out from digital advertising and started distributing ads via the mail?


A solicitation to join adwords, often including an offer to run a certain amount (ex. $100 worth) of free ads for new users


I've received similar solicitations in the US.


Except the internet it seems.


A. Wow B. I don't want to put the creator in any way down, but always feel a little bit cheated by FPGA use.


It feels like the only mails I get from DigitalOcean are about Intel processors.


Intel feels so much like Boeing now.


I agree, I think airdrops with Stellar and a wallet on many developer computers could create the critical mass to more crypto applications.


As a keybase user, why should I want to be a part of that?


I don't know.


Then why did you write a post emphatically stating you agree with an opinion that the airdrops were a good idea?

Did you just get really excited by their enthusiasm?


You can downvote me as much as you want - I wish more people would say "I don't know." if they don't know - on the internet and in the office.


I don't think the downvotes are because of that. It's more that "creating a critical mass" via something people don't actually want is pretty much the definition of spam.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: