2014-03-18

When protecting minorities screws them over

Fascinating. I came across this Dave Winer story of tech hiring and firing in 1985's Silicon Valley via the money quote:

...every time a company hires someone who is not a young male, they run the risk that the new hire isn't there to work, rather is there to scam you.
since from that quote I wondered "what the hell planet is this author on?"

Then I read the story. And blow me down if I didn't end up at least partially agreeing with the author. Go read the whole thing.

Commenter sep332 clarifies that the problem described (an older tech worker using his age to file a discrimination claim after being fired) isn't actually about age, sex or anything in particular:

The laws about protected classes are not about classes of people. Anyone can claim that they were discriminated against for gender reasons, not just women. White people can claim they were targeted because of their race, etc. So the people who couldn't realistically claim discrimination are the people who are most like the rest of the company. [my emphasis] I mean, if women are the majority of your company, then it would be hard for a woman to claim gender discrimination.
In the tech sector, the majority of your company are likely to be young, male, and (in Silicon Valley) a mix of white, Chinese and Indian in race; probably also straight although I have known a couple of small firms where that was not the case. If you're middle-aged, female, transgender, black, Inuit, Pacific Islander or Hispanic then you're almost certainly in a clear minority and hence possibly a "protected class".

As Winer notes, in a small struggling tech firm if someone comes at you with a discrimination lawsuit then you haven't the money or time to fight it. Unless it's a complete no-brainer (an 18 year old white male alleging discrimination on the grounds of inability to get out of bed) your best option is to pay up and move on. So what do you do when you have the risk of recruiting people who can sit back and do nothing while being practically un-fireable? Simply minimise the risk of recruiting them, by avoiding anyone who is in a good position to do this to you.

An ironic "well done" to everyone who has pushed through these employment laws, and a special raspberry to everyone who has filed (or backed) an abusive lawsuit exploiting these laws. You've screwed over everyone in the tech sector who's not a young male.

2014-03-15

Storing in the cloud

So this is interesting. Google is dropping its cloud storage rates to $10 per month per TB (though 100GB costs $2, twice that rate). Amazon S3 storage is currently $85 per TB per month and Microsoft Azure is $64 per TB per month for their cheapest option (Locally Redundant Storage). I'd expect these prices to be dropping fairly soon in response to Google's move.

How much does this actually cost to provide? Let's look at the cost of storing and accessing 1 TB of data. An internal SATA 1TB hard drive costs about $60 on Amazon - but a 2TB costs $85, and a 4TB costs $160 (retail). So we can figure on about $40 per TB of storage. How long will this drive last? Mean time between failures of hard drives is between 18 months and 3 years depending on manufacturer and usage; let's split the difference and say 2 years. Buying 1TB of storage over 2 years will cost the supplier about $40 in capital costs. Isn't this a rip-off?

Well, having a hard drive is one thing - being able to access it is another. You've got to get data into that hard drive, and probably you want to get it out again. Assuming that the entire volume of that drive is written once and read twice per 2 years (probably a lowball estimate) at a rate of about 5 Mbits/s, that means that in 1 day (86400 seconds) you could read (86400 * 5 / 8) MB or about 54 GB per day so it would take up about 10 days per year per user, and so you could support about 36 users on a 5 Mbit connection. Let's say we're using 4 TB drives so you need a 5 Mbit connection for each 9 computers in your storage.

It's not quite that straight forward though. Cloud storage is supposed to be reliable, and hard drives are manifestly not - they die all the time. Therefore you want at least a second copy of your data on a separate hard drive, and ideally you want that second copy to be in at least a separate building in case of a physical disaster (flooding, fire, tornado). Generally the further away the better, at least up to 100 miles or so, though distance tends to increase the expense of hosting because for every write to the data you need to send the write to your remote facility as well. Azure gives you the explicit option of how physically separate you want your data to be; Locally Redundant Storage vs Geographically Redundant Storage.

There's also the distinction between data loss and data unavailability; if the primary copy of the user's data is unavailable (e.g. because the data center has a planned or unplanned availability outage) cloud providers may give their customers the option of reading data from (or writing data to) the backup copy of the data. Customers can buy this kind of read access from Azure as an additional option (Read Access GRS).

If you as the cloud storage operator rack up 3600 users, then, you'll need 900 computers with a total of 3600 TB of storage in each of 2 sites. You'll need 500 Mbit/s of bandwidth on each site if you want to offer read redundancy, and about 170 Mbit/s of bandwidth between sites to replicate writes. You don't need customised hardware for this amount of traffic, but you do need to buy the bandwidth to get the data to and from the user. Azure quotes $120 for egressing 1TB of data; if we estimate that it actually costs them $100 then each user will cost you $200 (reading their data twice), so you will have $720,000 of bandwidth cost.

If the computers last about as long as the hard drives you'd expect them to cost you about $300 plus the storage ($160), say about $500 once you take into account rack and network switch hardware. I suspect power won't be too much of an issue since storage isn't a CPU intensive operation and user access is intermittent - idling computers without a display consume about 20W. So each site will cost you 900 x $500 over two years, and consume a steady 18 KW of power. Electricity costs about $100 per MWh to make in the cheaper parts of the USA, so power will cost you $2 per hour per site. So power is only about 7% of your equipment costs for a 2 year lifetime, and you end up paying about $1.8M in total in hardware, power and bandwidth to provide 1TB of cloud storage to 3600 users over 2 years. Each user pays $240 over that time, or $860K in total. So it would seem that $10 per hour is a massively losing proposition for the provider even before we take into account the human costs of designing, building and operating the system.

The real picture is more nuanced. We implicitly assumed that every user would use all the storage (and bandwidth) they paid for. In practice, they could conceivably be consuming only half of what they've paid for. As long as we can dynamically provision for users (having a small amount of storage headroom, and adding on more computers and drives as that headroom is threatened) we could provide maybe 60% of the maximum hardware needed, so instead of $1.8M our costs would be down to $1M or so. Still something of a loss.

I think the way cloud companies can make money - or at least avoid a loss - on this is to make use of the fact that the computers providing access to the drives are seldom even slightly busy. Instead of buying $300 of low power computing hardware to support each 4TB drive, just chain a few 4TB drives onto an existing computer that you're using for something else (say, Bing search, Google maps, Amazon website serving). When a user starts to access their data, temporarily reserve a core or two on the machines holding that data to serve it. That way you save nearly 60% of your hardware costs and might just be bringing your operation into slight profit.

You'll still need to pay the design and implementation costs for your system, not to mention the usual marketing and business operations, but these don't scale in proportion to your number of users. The more users you have, the better your business looks.

$10 per month per TB is a bit of a game changer. Suddenly storing in the cloud isn't massively more expensive than storing on your hard drive. I wonder what the next couple of years will bring?

2014-03-13

Piling on Piers Morgan

I was initially surprised that the news of Piers Morgan's fall from grace resulting in the cancellation of his CNN prime time talk show had been news in the UK, but I guess the prospect of Morgan returning to Blightly was sufficiently generally appalling that the Brits were quite concerned about the prospect.

Yesterday Piers had comedienne Chelsea Handler on his show. Noted for brutal honesty, such as discussing personal abortion and DUI stories, Chelsea didn't disappoint but possibly not in the way Piers expected. The discussion following the show's commercial break was quite enlightening:

Chelsea: I mean, in the middle of the commercial break – I want your viewers to know; they must know, because they're probably following you on Twitter. I mean you can't even pay attention for 60 seconds. You're a terrible interviewer.
Piers: Well, you just weren't keeping my attention. It's more of an issue with you than me.
Chelsea: That's not my problem. This is your show, you have to pay attention to the guest that you invited on your show.
Piers: If they're interesting enough ...[cut off]
Chelsea: Listen, it doesn't matter how interesting I am. You signed up for this job. [..] Well, maybe that's why your job is coming to an end.
Piers has - or had - the 9pm-10pm slot on CNN, which is prime time. (West Coast viewers, 3 hours behind the East Coast times quoted, can usually see the show live at 6pm Pacific or repeated 3 hours later). There's a constant battle for viewers between CNN and Fox in the evening, and Fox announced in August that newscaster Megyn [yes, really] Kelly would be taking over the Fox 9pm slot from previous incumbent Sean Hannity. Piers seemed keen on the challenge, tweeting "Bring it on, Megyn Kelly". Kelly duly brought it on, starting in early October and two months later was beating Piers 5 to 1 in viewers, up 10%-20% from Hannity.

Why did CNN viewers turn away from Piers in droves? Kelly is clearly easier on the eyes than Piers, but that can't be the whole story. I think Tim Stanley in the Telegraph gets closest to it:

But he acted as though no one had ever thought to discuss the subject[gun control] before. Like, ever. He tried to make gun control his own personal crusade, to "school" the Americans on law and order. And he displayed a crass insensitivity towards issues such as the importance of the Constitution or the American tradition of self-reliance. The scale of his ego was extraordinary. No US liberal has ever managed to challenge their country's fundamental respect for gun ownership. Why did he imagine that a guy with an English accent – the accent of George III no less – would succeed where Bill Clinton, Teddy Kennedy or Barack Obama had failed?
[...]
The embarrassing thing about the whole Piers Morgan affair is that it has turned the tables on us Brits. We always insisted that we were the courteous ones and the Americans were the boobs. In this case, it's been the other way around.
Americans really don't like being told, imperiously, what to do. Piers did this all the time, and worse wouldn't admit when he'd been beaten. He invited young Republican Ben Shapiro to debate guns with him after the Sandy Hook shooting, and had his clock unexpectedly cleaned - Shapiro tried to load the game by going for a "town hall" format which would have stuffed the audience with shooting victims and Shapiro wisely refused, telling him 1-on-1 or nothing. So, nothing.

Looking at interview styles, probably the best comparison on Fox with Piers is Bill O'Reilly who hosts the 8pm-9pm slot. O'Reilly has a similar format, inviting guests to discuss contentious issues 1-on-1, or sometimes 2-on-1, with him. One key difference, I think, is that O'Reilly doesn't try to lay traps - he relies on his (formidable) preparation and debating skills. Unlike Piers he has regular guests from a range of political views - liberal Juan Williams is one of the favourites - and the debate can get quite shouty at times but the guests generally know O'Reilly, know the position he'll take, and have wrestled with him often enough to give a good showing. More important, O'Reilly doesn't belittle them. He tries to reason, and even if you don't agree with his (often reactionary) position you can see that he's playing the man rather than the ball. Piers by contrast is a classic bully, trying to belittle opponents.

CNN's Anderson Cooper is a marked contrast to Piers; I've seen Anderson do interviews, and he can be tough - Greg Smith who resigned publicly from Goldman Sachs did an Anderson Cooper interview, and Cooper didn't go easy on him at all - but he's fair. You feel as if he's trying to make the interviewee explain what he's hiding from the audience, rather than browbeating the interviewee from a position of power just to establish a pecking order like a bully does. It would be hard to overstate how much most Americans look down on bullies. The War of Independence was essentially a reaction against perceived bullying (imposition by means of might) of the American colony by King George III. Piers is a classic bully, and Americans simply don't like that kind of person.

Interestingly it seems that even his own staff weren't keen on him:

"The makeup girls suffered the worst — he was rude and belligerent," says our source. "The general feeling is Morgan didn't show any respect to anyone working under him — the people who were trying to make him look good."
It would be uncharitable to note how hard a job that is. But this is Piers Morgan, so screw charity.

2014-03-11

The next step in the gentrification wars?

I'm going out on a limb here and saying that the massive fire in a new apartment building in San Francisco is not unrelated to the past year's increasingly violent struggle between long-term residents and new arrivals from the tech sector. According to local TV station KVTU:

Another Strata resident, 25-year-old Hisham Bajwa, said he could see the fire start to burn outside his window shortly before 5 p.m.
"There were two main points of fire, one on the left and one on the right," Bajwa said. "It got pretty big pretty fast."
An interesting observation; it would be surprising for an accidental conflagration to manifest in two separate points. Now eyewitness reports are famously inaccurate, and the fire could have spread internally before being visible in two external points, but it does make you wonder...

You can see the building under construction to the west of the intersection between 4th Street and China Basin Street. The implication of the size of the construction is that it was a new apartments block - opposite it is Strada Apartments which had to be evacuated, and just up the street is Channel Mission Bay Apartments. So why does a huge new apartment building start to burn down? A welding accident? A gas leak igniting? Or something more deliberate?

The Mission District in San Francisco next to where these apartments are located has been Ground Zero for the protests against the influx of tech and biotech workers from Apple, Google, Facebook, Genentech and others. Assaulting Google Glass wearers in bars, blockading shuttle buses or just generally protesting tech nerds has become an increasingly popular sport in central San Francisco. Since it's nearly impossible to increase rents significantly in San Francisco apartments or evict a renter who doesn't want to leave - even if their contract is at an end they can require the landlord to renew it at substantially the same rent and terms - the only way that most landlords can improve their rent income is to invoke the Ellis Act to "go out of business" and sell their properties to another company, which then changes the use of the building (often via a drastic knock-down and rebuild).

The organised protests to date have mostly been focused around the shuttle buses which take the tech workers to Cupertino, Mountain View, Sunnyvale and other places across the South Bay; they are a very visible and concentrated target. Perhaps now the anti-tech movement is changing tactics: instead of impeding the transport, make building luxury apartments a much more expensive and chancy business. If they cause enough disruption maybe they'll be able to slow the influx of tech money and keep their existing apartments.

The investigation into this (hundred millions of dollars?) fire could be very interesting.

Update: KTVU confirms a $220M+ project with 360 apartments and "Arson investigators were on the scene Tuesday and will return Wednesday morning. " I bet they will.