Software - Everything Is Broken

I don't agree with 100% of this article, but it's sufficiently true and well explained that it's worth reading the whole thing. Quinn Norton reports that "Everything is broken":

It was my exasperated acknowledgement that looking for good software to count on has been a losing battle. Written by people with either no time or no money, most software gets shipped the moment it works well enough to let someone go home and see their family. What we get is mostly terrible.

This near-perfectly expresses the problem with software. The only point I'd differ on is that it's not even that it "works well enough" - in reality it's shipped when it's perceived to work well enough by people who generally aren't able to tell how well it's actually working.

It's certainly true that people are awful users of software. This is generally because software is written and tested by people who are completely unrepresentative of the software user base. Here's an example from today. I try to connect, using Firebox, to a website which I happen to know has a problem with its security certificate (it's been revoked by the owner). Here's what I get:

OK, so let's suppose that I'm my mother. What the hell am I supposed to do with that information? It's good that Firebox has recognised that the site is broken and has stopped me connecting to this site - but "Please contact the website owners to inform them of this problem"? Seriously? How do I even know who the "website owners" are? Chrome is a little bit better - it warns that "if you try to visit [site] now you might share private information with an attacker" and suggests reloading the site in a few minutes or using a different wifi network, but it says that "something is interfering with your secure connection" when it would be better to say something like "I can't make a secure connection to this website - I've checked a couple of other websites and secure connections to them are OK, so it's probably just something wrong with this particular website". Chrome and Firefox's error messages in this situation are reasonably useful, but they're written for reasonably technically-savvy people - not for the vast majority of their user base.

As Quinn notes, for relatively non-technical people who don't generally have control over their computers, security is essentially impossible:

What's the best option for people who can't download new software to their machines? The answer was unanimous: nothing. They have no options. They are better off talking in plaintext I was told, "so they don’t have a false sense of security."

I think this is slightly pessimistic. Doing everything in plaintext makes it trivially easy for the intelligence agencies, crackers and other ne'er-do-wells to scoop up everything. Better is to ensure that the world uses such a diverse and changing ecology of software and hardware that even concerted efforts to compromise a security system will only yield a relatively small fraction of the world - we can't stop those people from compromising our security if they really want to, but at least we can make the bastards work hard for it.

SELECT, JOIN and a bit of Perl? $10K

Database nerds will like this one. The state of Nevada has a database system for the permanent records of children in the state education system. One parent, an opponent of the Common Core national curriculum which is sweeping the US public education, wanted to know what his childrens' permanent record said. He asked for a copy, but it wasn't available for free:

Because the SAIN system is not designed to create reports that display individual student data in a readable format, the parent was initially told that the requested reports do not exist and cannot be produced
[...]
[Nevada Department of Education] staff determined that it would take at least 3 weeks (120 hours) of dedicated programming time to fulfill the parent's request. At the applicable wage rate of $84.95/hour, the requested work resulted in a $10,194 price tag.

Not designed to create reports that "display individual student data in a readable format"? That's a new one on me. So when education officials threaten to put a student's trangression on their permanent record, that actually means "a place that no-one will look because it's not readable"?

The really shocking aspect is that Nevada is employing IT experts at $85/hour who can't knock up an appropriate couple of SQL statements like:

SELECT GradeReport.Grade, GradeReport.Text, Students.StudentName, GradeReport.ReportDate
FROM GradeReport INNER JOIN Students
ON GradeReport.StudentID=Students.StudentID
WHERE Students.StudentName = "Fred Eppolito" AND 
 Students.SchoolID = 12345;
 -- choose whatever fields are needed to uniquely identify a student

I'm fairly sure that writing the query, testing it, dumping that data out in CSV format, eyeballing it to check that nothing's bad and then sending on the file can be done in the space of an hour.

If the Department of Education IT staff can't do this for $85 per hour, which is about $175K per year, perhaps they shouldn't be paid so much. Otherwise, one might think that the state of Nevada can't be trusted to spend taxpayer money at all...

Healthcare.gov - it's not over yet

After the well-publicised last-minute save of the Healthcare.gov federal health insurance exchange in 2013, there was a general sighing of relief from the Affordable Care Act's supporters. The months of suspense and error codes had been very painful, but finally the federal exchange was working more or less, and millions were able to enroll in time for the deadline (even if they couldn't then find a doctor who would accept their plan).

It seems that such confidence may have been misplaced; the federal site was fixed but a number of states decided to implement their own exchanges and it hasn't turned out too well for many of them, and people with badges are looking at Oregon's exchange quite hard:

The various state exchanges were funded by the U.S. Center for Medicare and Medicaid Services. As of March 28, it had awarded $4.7 billion in grants nationally. Oregon has been awarded $303 million in five increments.
Typically in federally funded projects, states must show evidence they've met certain milestones before they get additional grants. In Oregon's case, the Oregon Health Authority at first, and later Cover Oregon, had several "gate reviews" where federal officials reviewed progress on the project.
Documents show Oregon may have presented a misleading picture to the federal government.

Oregon has a population of about 4 million people. Since ACA sign-up is about 2.5% of the US population, let's be generous and say 3% for Oregon; that's $303 million for 120K people, or about $2500 per sign-up - just for the website. They've now dropped the idea entirely and fallen back to use the federal site which at least kind of works.

Most of the money went to Covered Oregon's contractor Oracle, who probably won't be promoting this as a triumph because the website has not managed to enroll a single private customer:

After discovering a series of technical problems before the Oct. 1 launch -- including inaccurate calculations of the tax subsidies for which consumers would be eligible -- Cover Oregon decided to scratch its plans to go live along with the rest of the country and never managed to get online enrollment started.

This hasn't come from out of the blue: back in November Oracle were being grilled about missed deadlines. Their VP Tom Budnar reported that they were bringing in an Oracle "SWAT team" but it would appear their efforts were in vain. Oracle's stock price has steadily risen from $39.5 to $42 in the past month, so at least Mr. Budnar doesn't have to worry about loss of public confidence in the company. Mind you, a search of Oracle's website doesn't show any results with his surname, so maybe they're already planning to cut him loose - if they've not done so already. He's going to have a difficult round of interviews if that's the case; I'd advise him to take a year of sabbatical and hope most people have forgotten about Oregon...

Oregon is egregiously bad, but unfortunately not too far out of line with other states' exchanges. As NPR reports this week, costs per enrollee of the exchanges have made the federal effort look like a model project:

Even Covered California, the most efficient of the state-run exchanges at $758 per enrollee, still spent more than the average for the federal exchange. And California was the only state-run exchange with a per-person average under $1,000.

Okay, so why did this cost so much? We can look at a comparable solution in the private sector in the USA: TurboTax (web edition) which is delivered by the firm Intuit. Filing federal and state taxes in the USA is a substantial undertaking; most people either go to a tax preparer (H+R Block is a popular one for lower-income taxpayers) or use some tax prep software. Filling in Form 1099 and its many supplemental forms by hand is a great way to lose money, get audited, develop a drinking habit and hate of humanity, or all three. TurboTax is a popular web system for filing taxes; it does automatic imports of data from a number of sources (major employees and brokerages), has a user-friendly interface, lets you submit forms electronically, and generally makes tax preparation a breeze. My expat colleagues who have used TurboTax sing its praises as being slightly easier to use than the short-form UK tax form, performing substantially more complex calculations and optimizations.

US tax season is not dissimilar to healthcare signups - a short period (1-2 months) after you get the tax information from your employer and brokerages and before the April 15th deadline, usage spiking in evenings and on weekends. So lots (millions, I'd estimate) of users crowd onto the TurboTax site in a short time, with downtime or even slowness being commercially unacceptable - there are several alternatives to TurboTax. Turbotax covers its costs and generates income by serving basic users for free, simple uses for $30 per user and complex uses for $50 per user (state filing is an additional $20 or so). You can even cover your costs if you use Amazon heavily.

It's not an exaggeration to say that if the states and federal goverment had contracted TurboTax's Intuit to build and operate the site for $70 per user - one tenth of the current cost in reality - then they could probably have made it work. They already have the storage and serving systems capable of handling that level of traffic, the marginal cost is probably less than developing from scratch. So why didn't the states and feds do this? Why did Oregon contract Oracle? Because Oracle and other consulting firms are very good at bidding for federal and state contracts. They're pretty awful at making them work, but this doesn't matter - strange to relate, contracts are not issued and paid based on whether the delivered solution actually works. The incentives in government contracting have generated an ecosystem where actual ability to build working systems is one of the less important characteristics for which the environment selects.

The defence of the federal contracting system (instead of assigning the contract to a company who might actually be able to make the contract work) starts with "we have to protect the taxpayer from being exploited in no-bid contracts." Except that it seems that the taxpayer is already being exploited. If Intuit charged $400 per user, made out like a bandit, gave bonuses through the company right down to the mailroom boy's cat and sent the execs off on 1 year fact-finding trip to the Bahamas, the taxpayer would still come out ahead, and would have the bonus of a website that actually worked well and was user-friendly.

The next stage in the gentrification wars

This is going to be interesting. The "techie scum" who are moving into and gentrifying various parts of San Franciso and Oakland have had a few months of their buses being blocked and Google Glass ripped off their faces, but now the local residents have decided to up the ante and attack the businesses that serve the gentrifyers. Four high-end grocery stores were tagged with graffiti demanding that they clear out of town:

Milgrom and his workers spent the day cleaning graffiti from all four stores. He says this isn't the first time it's happened. Recently windows have been broken as well.

I suppose it's quite a clever strategy; don't go after individuals, because if they move out then others will just replace them. Instead target the aspects of the area that they like and that you don't like. I wonder who will be the next target: Chi-chi cocktail joints? Places that sell coats for small dogs?

Looking at the reviews for the local market part of the business you can see its target market: its provisions include lamb ragu, Japanese pumpkin and duck confit. This is clearly not catering to low-income locals. The fact that it's a group of four establishments means that you can amplify your message and make it clear that this isn't just some random graffiti - the owners are left in no doubt that they are being targeted. The Local's Corner restaurant part of the business has a negative review giving more clues:

Funny that this place is called Local's Corner, as it's a new restaurant that is part of the gentrification of the Mission and thus directly involved in displacing long-term residents (ie, locals)

Apparently the opposite of gentrification is "neighbourhood decay" but I'm sure there should be a snappier term for it; I'm plumping for "communirot".

Local communirotter Mary Magee of "Alliance of Californians for Community Empowerment" (who needs better branding) is not a big fan of the establishment, claiming that groups of "people of color" are not welcome at Local's:

Magee said ACEE [sic] is asking people to boycott all four stores, claiming people in the area have complained that they were told to leave or turned away because they didn't fit the restaurant's clientele. [...] Magee went on to say the Local brand also has a high price point, which ACEE feels is a form of gentrification because many of those living in the area can't afford it.

Judging by Mary's Facebook page she's as white as they come and painfully right-on. Her colleague Julien Ball is possibly even more of a cliche: anti-Israel, anti-death-penalty, active on change.org, anti-discriminatory-lending... I find it amusing that Mary and Julien chooses to speak for the "people of color" rather than, say, letting them speak for themselves. But it becomes clearer when the article reports that ACCE wants Local to hold "sensitivity training". How nice for poor bespectacled owner Yaron Milgrom:

Milgrom told KTVU he fell in love with the area because of its diversity. He can't believe he is now being accused of not embracing it.

Oh, believe it Yaron. If you're lucky, this is merely a shakedown operation to whip you into line and make it clear that the community activists in the area are in charge. If you're unlucky, they've taken a dislike to you and you're going to have to sell up and eat a huge loss (for what sane gentrifying business would want to take your place in the firing line?) or be run out of town on a rail.