Expensive integer overflows, part N+1

Now the European Space Agency has published its preliminary report into what happened with the Schiaparelli lander, it confirms what many had suspected:

As Schiaparelli descended under its parachute, its radar Doppler altimeter functioned correctly and the measurements were included in the guidance, navigation and control system. However, saturation – maximum measurement – of the Inertial Measurement Unit (IMU) had occurred shortly after the parachute deployment. The IMU measures the rotation rates of the vehicle. Its output was generally as predicted except for this event, which persisted for about one second – longer than would be expected. [My italics]
This is a classic software mistake - of which more later - where a stored value becomes too large for its storage slot. The lander was spinning faster than its programmers had estimated, and the measured rotation speed exceeded the maximum value which the control software was designed to store and process.
When merged into the navigation system, the erroneous information generated an estimated altitude that was negative – that is, below ground level.
The stream of estimated altitude reading would have looked something like "4.0km... 3.9km... 3.8km... -200km". Since the most recent value was below the "cut off parachute, you're about to land" altitude, the lander obligingly cut off its parachute, gave a brief fire of the braking thrusters, and completed the rest of its descent under Mars' gravitational acceleration of 3.8m/s^2. That's a lot weaker than Earth's, but 3.7km of freefall gave the lander plenty of time to accelerate; a back-of-the-envelope calculation (v^2 = 2as) suggests a terminal velocity of 167 m/s, minus effects of drag.

Well, there goes $250M down the drain. How did the excessive rotation speed cause all this to happen?

When dealing with signed integers, if - for instance - you are using 16 bits to store a value then the classic two's-complement representation can store values between -32768 and +32767 in those bits. If you add 1 to the stored value 32767 then the effect is that the stored value "wraps around" to -32768; sometimes this is what you actually want to happen, but most of the time it isn't. As a result, everyone writing software knows about integer overflow, and is supposed to take account of it while writing code. Some programming languages (e.g. C, Java, Go) require you to manually check that this won't happen; code for this might look like:

/* Will not work if b is negative */
if (INT16_MAX - b >= a) {
   /* a + b will fit */
   result = a + b
} else {
   /* a + b will overflow, return the biggest
    * positive value we can
   result = INT16_MAX
Other languages (e.g. Ada) allow you to trap this in a run-time exception, such as Constraint_Error. When this exception arises, you know you've hit an overflow and can have some additional logic to handle it appropriately. The key point is that you need to consider that this situation may arise, and plan to detect it and handle it appropriately. Simply hoping that the situation won't arise is not enough.

This is why the "longer than would be expected" line in the ESA report particularly annoys me - the software authors shouldn't have been "expecting" anything, they should have had an actual plan to handle out-of-expected-value sensors. They could have capped the value at its expected max, they could have rejected the use of that particular sensor and used a less accurate calculation omitting that sensor's value, they could have bounded the calculation's result based on the last known good altitude and velocity - there are many options. But they should have done something.

Reading the technical specs of the Schiaparelli Mars Lander, the interesting bit is the Guidance, Navigation and Control system (GNC). There are several instruments used to collect navigational data: inertial navigation systems, accelerometers and a radar altimeter. The signals from these instruments are collected, processed through analogue-to-digital conversion and then sent to the spacecraft. The spec proudly announces:

Overall, EDM's GNC system achieves an altitude error of under 0.7 meters
Apparently, the altitude error margin is a teeny bit larger than that if you don't process the data robustly.

What's particularly tragic is that arithmetic overflow has been well established as a failure mode for ESA space flight for more than 20 years. The canonical example is the Ariane 5 failure of 4th June 1996 where ESA's new Ariane 5 rocket went out of control shortly after launch and had to be destroyed, sending $500M of rocket and payload up in smoke. The root cause was an overflow while converting a 64 bit floating point number to a 16 bit integer. In that case, the software authors had actually explicitly identified the risk of overflow in 7 places of the code, but for some reason only added error handling code for 4 of them. One of the remaining cases was triggered, and "foom!"

It's always easy in hindsight to criticise a software design after an accident, but in the case of Schiaparelli it seems reasonable to have expected a certain amount of foresight from the developers.

ESA's David Parker notes "...we will have learned much from Schiaparelli that will directly contribute to the second ExoMars mission being developed with our international partners for launch in 2020." I hope that's true, because they don't seem to have learned very much from Ariane 5.


Journalist ecomonic understanding makes me cry

The megalopolis of San Jose, CA has approved a rise in the minimum wage to $15 by January 1 2019. The usual suspects are weighing in approvingly, but my eye was drawn in fascinated horror to the way that the journalist (or press release author) expressed the financial changes expected:

Mayor Liccardo launched the effort last fall to follow the lead of five other cities in Santa Clara County and to come up with a regional approach to raise minimum wage throughout Silicon Valley.
City statistics show it would mean a $300,000 raise for 115,000 workers.
To which I can only say huh? Assuming they're on $12/hour now, they're working 100,000 hours per year?

What the author means, one assumes, is that each worker is going to benefit by just under $3 per hour, but that's a horrible way of expressing that statistic. And of course, the statistic itself is misleading. The workers are going to pay a varying amount of tax on that additional money, other benefits they are currently paid may change, and of course that assumes that otherwise their salary would not have risen at all by January 2019 despite the extra 2 years of experience and possible promotion they would have achieved by then.

But let's look at what the author believes is the downside of this measure - because they're trying to be even-handed, yes?

Some small business owners and non-profits worry raising the minimum wage would reduce their share of the economic pie. The result could either mean service reduction for non profits or price increases for mainstay businesses.
Or, you know, firings left and right for any worker whose skills aren't valued at $15/hour (plus additional costs) by the business they work at. Or businesses closing down because they're no longer economically viable. Or employers cutting existing worker benefits to offset the new costs. Heck, ask workers and business owners in Seattle how their new $15/hour minimum is working out.

You can just taste the disdain for business owners in the expression "reduce their share of the economic pie". Why exactly does the author think the owners have put in all the work and risk to create the businesses that create the jobs for these good people in the first place?

Always consider what happens when the shoe switches feet

The recent panic from the LGBT+ / Black / Hispanic communities about increased violence in the wake of Trump's victory has caused a sharp uptick in blogs and forum posts from various West Coast people, notably those of the transgender persuasion, claiming a new fear for the personal safety of them and their families. This seems to be based around the assumption that a Trump presidency will embolden the less savoury side of society prone to gay-bashing to perpetrate physical violence on them. Let's say, for arguments' sake, this is true: what should they do about it?

Larry Correia, author of the "Monster Hunter Nation" and related high-output high-sales fantasy book series, penned "A Handy Guide For Liberals Who Are Suddenly Interested In Gun Ownership" which is as sympathetic to the political gripes of Hillary/Bernie supporters as the title suggests, but does provide a lot of good practical advice about how you can go about getting armed and trained in effective self-defence. Correia owned a gun store and did a lot of concealed-carry training before his literary career properly started, so seems to know what he's talking about.

What he really nails is the ever-increasing squeeze on firearms possession, gun ranges and ammo purchase that has been happening in Democrat-controlled states over the past few years, and why it's relevant now:

When the already super powerful government wants to make you even more powerless, that scares the crap out of regular Americans, but you guys have been all in favor of it. Take those nasty guns! Guns are scary and bad. Don't you stupid rednecks know what's good for you? The people should live at the whim of the state!
But now that the shoe is on the other foot, and somebody you distrust and fear is in charge for a change, the government having all sorts of unchecked power seems like a really bad idea, huh?

It's hard enough owning a gun in California anyway, but cities like San Francisco have taken it to extremes. They have used local law changes to force all the gun shops to close down. In last week's voting, there was a strong San Francisco representation pushing state Proposition 63 to make ammunition purchases harder and more expensive. The net effect is that you can guarantee that no-one in San Francisco is carrying a gun unless they're a law enforcement officer or a criminal.

Gay bashing is far from a new crime in San Francisco. Despite the city's image as gay-friendly, there are enough unreconstructed citizens who are not keen on public displays of homosexuality or trans people for there to be a significant risk of violence. Since these folk know that their victims won't be armed, they have no disincentive to engage in these attacks. But if there were a few well-publicised self-defence shootings in reaction to gay bashing attempts, you can bet that the rate of gay bashing attempts would decline rapidly.

For now, California citizens have to deal with the laws as they stand - and as Correia notes, those laws make it hard for law-abiding citizens to be armed effectively:

See, traditionally Democrats don't like the 2nd Amendment and historically have done everything in their power to screw with it. Your gun laws are going to vary dramatically based upon where you live. It might be really difficult and expensive for you to exercise your 2nd Amendment rights, or it might be relatively easy.
But you’re scared right now! Well, that's too bad. Because for the most part Democrats have tried to make it so that citizens have to abdicate their responsibilities and instead entrust that only [the] state can defend everyone... That doesn't seem like such a bright idea now that you don't trust who is running the state, huh?
Perhaps San Francisco Mayor Ed Lee could take time out from his crusade against the gun industry to ensure that his vulnerable constituents can defend themselves against the increasing violence in his city. I'm not holding my breath for this to happen, but if the LGBT+ community wants to be able to protect themselves then Ed might be a good target for their lobbying. "Mayor Lee, why don't you want the gay community to be safe in your city?". They could recommend that Lee work with past SF Democrat mayoral candidate Leland Yee to draw on the latter's expertise in firearms supply.


Silicon Valley in the Time of Trump

The past few days have given me a great view into how the famously liberal population of the Bay Area has taken the election of Donald Trump. "Not well" is fair, but a yuuuuge understatement.

Do you know what California's principal export is? Whine.

The Bay Area is probably the most pro-Clinton anti-Trump group outside the island of Manhattan, and the residents were never going to be entirely happy with a Trump victory. I predicted butthurt-ness, and was I ever right. However even I, with my jaundiced view of human nature, never expected the level of rage and opprobrium directed at Trump and his voting enablers. So far I've seen - not heard but actually seen written on group emails and forums - the following:

  • claims of suicidal feelings, particularly from trans and gender-fluid folks;
  • assertions that anyone voting for Trump needs to publicly denounce Trump's perceived opinions about Black Lives Matter, Hispanics, gays (wut?) and immigrants;
  • statements that anyone voting for Trump needs to go work for another company;
  • room-sized group hugs to support each other post-election; and
  • claims that Trump and Pence wanted to electrocute people who were gay or trans.
Thank goodness Trump has elephant-thick skin, because there's probably enough libel in every Bay Area tech company's emails to pay for the building of another Trump Tower.

The straw that broke the camel's back for me was a bundle of complaints around the theme:

"I was hoping to teach my girls that, if you work hard and dream big, you can be anything you want to be. I would like to thank 2016 for putting me right."
It seems that a large number of people were going to use "Hillary as first woman president" as the totem for their children to show that the glass ceiling had been shattered. While I'm all in favour of showing children role models, is Hillary really the model you want to use?

I actually found it inspiring, in a way. The lesson I took from the election was that if you are a woman, even if you are a revolting and corrupt human being, you can make it to within a gnat's chuff of being the President of the United States, and your party organisation will happily screw over men to help you get its nomination. It wouldn't have taken much of a vote change in one or two swing states for Hillary to be elected, at which point I guarantee that no-one on the Dems side would be talking about upsetting the electoral college applecart.

Hillary is (of course) not happy and blames FBI Director Comey for her narrow defeat:

But our analysis is that [FBI Director James B.] Comey's letter raising doubts that were groundless, baseless, proven to be, stopped our momentum,” she said. “We dropped, and we had to keep really pushing ahead to regain our advantage — which going into the last weekend, we had."
She's right, of course. Comey's letter was quite possibly enough to cause Hillary voters in key states to stay home on polling day.

On the other hand, there were many other what-ifs, any one of which was probably enough to get her elected:

  • what if she had actually achieved something of note as Secretary of State?
  • what if she and Bill hadn't gone around the world soliciting hundreds of millions of dollars from various dubious countries and individuals?
  • what if she were actually personally likeable?
  • what if she'd not blown her chance to land a kill-shot on The Donald in the debates?
  • what if she'd insisted that the DNC not put its thumb on the scales, and instead beat Bernie fairly in the nomination?
All these were in her control, so to blame solely Comey for her loss seems rather obtuse.

And on the flip side, what if Comey had taken the - apparently quite reasonable - step to indict her for her recklessness in running her own email server and exposing any amount of State classified material to any intelligence service worth its name? Isn't she grateful to him for not doing that, at least?


Trump triumphant

Blimey, he actually did it. Just how poor a candidate must Hillary have been, with all the media, technical, organisational and financial advantages she had, to go down so badly to Trump? I'm guessing that Hillary 2020 is not going to be a thing.

I continue to feel very comfortable in my prediction of an unprecedent wave of butthurt about to appear from the Guardian opinion pages (and indeed all other articles) and the BBC US correspondents.


2016 US election prediction

It's less than 24 hours before we'll have a good idea whether Hillary Clinton has made it to the 270 electoral college votes needed to secure the presidency to which she clearly believes she's entitled. At this stage, although I wouldn't write off Trump, I'd have to say that Hillary is likely to make it. Her Get-Out-The-Vote ground game is much better organised than Trump's, Wikileaks and the FBI haven't landed a killer blow on her, and the media have carried water faithfully enough to keep most of her followers following. I'm sure a lot of Bernie supporters are extremely unhappy with the revelations of past weeks, but I suspect most of them will hold their noses and vote Hillary nevertheless.

Should The Donald continue his trend of confounding predictions and actually pull off an upset - winning Florida, Pennsylvania and such other states as needed to break 270 - I confidently predict the most ear- splitting snit of all times from 95% of the US media. Hillary herself might actually evaporate in a toxic plume of rage. It would be quite something to watch.