How Amazon is building substations, laying fiber and generally doing everything to keep cloud costs down
If there's anyone still left wondering how it is that large cloud providers can keep on rolling out new features and lowering their prices even when no one is complaining about them, Amazon Web Services Vice President and Distinguished Engineer James Hamilton spelled out the answer in one word during a presentation Thursday at the company's re:Invent conference: Scale.
Scale is the enabler of everything at AWS.
What would happen if we replaced those 16 disk-based V7000s with all-flash V7000s? Each of the disk-based ones delivered 32,502.7 IOPS. Let’s substitute them with 16 all-flash V7000s, like the one above, and, extrapolate linearly; we would get 1,927,877.4 SPC-1 IOPS – nearly 2 million IOPS. Come on IBM: go for it.
That’s right, IBM is understanding the Flash-based SSD SAN market and is making some benchmark systems to help market its disk arrays. Finally we’re seeing some best case scenarios for these high end throughput monsters. It’s entirely possible to create a 2Million IOPS storage SAN. You just have to assemble the correct components and optimize your storage controllers. What was once a theoretical maximum throughput (1M IOPs) is now achievable without anything more than a purchase order and an account representative from IBM Global Services. It’s not cheap, not by a longshot but your Big Data project or OLAP with Dashboard may just see orders of magnitude increases in speed. It’s all just a matter of money. And probably some tweaking via an IBM consultant as well (touche).
Granted that IBM doesn’t have this as a shipping product isn’t really the point. On paper what can be achieved by mixing matching enterprise storage appliances and disk arrays and software controllers is beyond what any other company is selling IS the point. There’s a goldmine to be had if anyone outside of a high frequency trading skunkworks just shares a little bit of in-house knowledge product familiarity. No doubt it’s not just the network connections that make things faster it is the IOPs that will out no matter what. Write vs. Read and latency will always trump the fastest access to an updated price in my book. But I don’t work for a high-frequency trading skunkworks either, I’m not privy to the demands made upon those engineers and consultants. But still we are now in the best, boldest time yet of nearly too much speed on the storage front. Only thing holding us back is the network access times.
- Extreme Blogging (ibm.com)
- IBM i Storage Options Overview (Which storage is right for me?) (ibm.com)
“I missed the mark with HyperCard,” Atkinson lamented. “I grew up in a box-centric culture at Apple. If I’d grown up in a network-centric culture, like Sun, HyperCard might have been the first Web browser.
Bill Atkinson‘s words on HyperCard and what could have been are kind of sad in a way. But Bill is a genius by any measure of Computer Science and programming ability. Without QuickDraw, the Mac would not have been much of a graphical experience for those attempting to write software for the Mac. Bill’s drawing routines took advantage of all the assembly language routines available on the old Motorola 68000 chip and eked out every last bit of performance to make the Mac what it was in the end; Insanely Great.
I write this in reference also to my experience of learning and working with HyperCard. It acts as the opening parenthesis to my last 16 years working for my current employer. Educational Technology has existed in various forms going all the way back to 1987 when Steve Jobs was attempting to get Universities to buy Macs and create great software to run on those same computers. There was an untapped well of creativity and energy that Higher Education represented and Jobs tried to get the Macintosh computer in any school that would listen.
The period is long since gone. The idea of educational software, interactive hypermedia, CD-ROMs all gone the way of the web and mobile devices. It’s a whole new world now, and the computer of choice is the mobile phone you pick-up on 2 year contract to some telecom carrier. That’s the reality. So now designers and technologists are having to change to a “mobile first” philosophy and let all other platforms and form factors follow that design philosophy. And it makes sense as desktop computer sales still erode a few percentage points each year. It’s just a matter of time before we reach peak Desktops. It’s likely already happened, we just haven’t accepted it as gospel.
Every technology is a stepping stone or shoulder to stand on leading to the next stepping stone. Evolutionary steps are the rule of the day. Revolution has passed us by. We’re in for the long slog, putting things into production making them do useful work. Who has time to play and discover when everyone has a pre-conceived notion of the brand device and use it will serve. I want X to do Y, no time to advise or consult to fit and match things based on their essential quality or essence of what they are good at accomplishing. This is the brand and this is how I’m going to use it. That’s what Educational Technology has become these days.
There is no end to the amount of stuff I get asked to do. I like the technical aspects and not so much the other bits. There is a lot of communications and expectation setting. And therein lies the rub. Read the rest of this entry »
The WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.
Here’s an excerpt:
4,329 films were submitted to the 2012 Cannes Film Festival. This blog had 17,000 views in 2012. If each view were a film, this blog would power 4 Film Festivals
Starting with this website tutorial I’m attempting to create a working config file that will allow me to install new Windows 7 Professional installs without having to interact or click any buttons.
Seems pretty useful so far as Sergey provides an example autounattend file that I’m using as a template for my own. I particularly like his RunOnce registry additions. This makes it so much more useful than just simply being an answer file to the base OS install. True it is annoying that questions that come up through successive reboots during the specialize pass on a Windows 7 fresh install. But this autounattend file does a whole lot of default presetting behind that scenes, and that’s what I want when I’m trying create a brand new WIM image for work. I’m going to borrow those most definitely.
I also discovered an interesting sub-section devoted to joining a new computer to a Domain. Ever heard of djoin.exe?
Very interesting stuff where you can join the computer without first having to login to the domain controller and create a new account in the correct OU (which is what I do currently) and save a little time putting the Computer on the Domain. Sweet. I’m a hafta check this out further and get the syntax down just so… Looks like there’s also a switch to ‘reuse’ an existing account which would be really handy for computers that I rebuild and add back using the same machine name. That would save time too. Looks like it might be Win7/Server2008 specific and may not be available widely where I work. We have not moved our Domains to Server 2008 as far as I know.
djoin /provision /domain to be joined> /machine /savefile blob.txt
http://technet.microsoft.com/en-us/library/dd391977(v=WS.10).aspx (What’s new in Active Directory Domain Services in Windows Server 2008 R2: Offline Domain provisioning.
There’s also a script you can download and run to get similar info that is Win 2000 era AD compliant: http://www.joeware.net/freetools/tools/adfind/index.htm
Random Thoughts just now: I could create a Generic WIM with a single folder added each time and Appended to the original WIM that included the Windows CAB file for that ‘make/model’ from Dell. Each folder then could have DPInst copied into it and run as a Synchronous command during OOBE pass for each time the WIM is applied with ImageX. Just need to remember which number to use for each model’s set of drivers. But the description field for each of those appended driver setups could be descriptive enough to make it user friendly. Or we could opt just to include the 960 drivers as a base set covering most bases and then provide links to the CAB files over \\fileshare\j\deviceDrivers\ and let DPInst recurse its way down the central store of drivers to do the cleanup phase.
OK, got a good autounattend.xml formulated. Should auto-activate and register the license key no problem-o. Can’t wait to try it out tomorrow when I get home on the test computer I got setup. It’s an Optiplex 960 and I’m going to persist all the Device Drivers after I run sysprep /generalize /shutdown /oobe and capture the WIM file. Got a ton of customizing yet to do on the Admin profile before it gets copied to the Default Profile on the sysprep step. So maybe this time round I’ll get it just right.
One big thing I have to remember is to set IE 8 to pass all logon information for the Trusted Sites Zone within the security settings. If I get that embedded into the thing once and for all I’ll have a halfway decent image that mirrors what we’re using now in Ghost. Next steps once this initial setup from a Win7 setup disk is perfected is to tweak the Administrator’s profile then set copy profile=true when I run Sysprep /generalize /oobe /config:unattend.xml (that config file is another attempt to filter the settings of what gets kept and what is auto-run before the final OOBE phase on the Windows setup). That will be the last step in the process.
- Hyper-V: Making Template Virtual Machines (cwing.wordpress.com)