The Next Generation of Dense-storage Instances for EC2

Perhaps you, like many other AWS users, store and process  huge amounts of data in the cloud. Today we are announcing a new generation of Dense-storage instances that will provide you additional options for processing multi-terabyte data sets.

New D2 Instances
The new D2 instances are designed to provide you with additional compute power and memory (when compared to the first-generation HS1 instances) and the ability to sustain a high rate of sequential disk I/O for access to extremely large data sets, all at a very affordable price. The instances are based on Intel Xeon E5-2676 v3 (code name Haswell) processors running at a base clock frequency of 2.4 GHz and come in four instance sizes as follows:

The prices listed above are for the US East ( Northern Virginia ) and US West ( Oregon ) regions . For more pricing information, take a look at the EC2 Pricing page.

You can also launch multiple D2 instances in a placement group for high bandwidth low latency networking between the instances.

Notes on Storage
The largest D2 instances (d2.8xlarge) are capable of providing up to 3,500 MB/second read and 3,100 MB/second write performance with a 2 MiB block size when launched with a Linux AMI.

In order to ensure the best disk throughput performance from your D2 instances on Linux, we recommend that you use the most recent version of the Amazon Linux AMI, or another Linux AMI with a kernel version of 3.8 or later. The D2 instances provide the best disk performance when you use a Linux kernel that supports Persistent Grants – an extension to the Xen block ring protocol that significantly improves disk throughput and scalability. The following Linux AMIs support this feature:

• Amazon Linux AMI 2015.03 (HVM)
• Ubuntu Server 14.04 LTS (HVM)
• Red Hat Enterprise Linux 7.1 (HVM)
• SUSE Linux Enterprise Server 12 (HVM)

For more information, read about Persistent Grants in the Xen Project  Blog.

The storage on this instance family is local, and has a lifetime equal to that of the instance. Therefore, you should think of these instances as building blocks that you can use to build a complete storage system. For example, you should build some redundancy in to your storage architecture (e.g. RAID 1, 5, or 6) and you should use a fault-tolerant file system such as HDFS or Gluster. You should also back up your data to Amazon Simple Storage Service (Amazon S3) or Amazon Elastic Block Store (Amazon EBS) for increased durability.

Enhanced Networking
With Enhanced Networking and extremely high sequential high I/O rates, these instances will chew through your Massively Parallel Processing (MPP) data warehouse, log processing, and MapReduce jobs. They will also make great hosts for your network file systems and data warehouses. In order to take advantage of Enhanced Networking, you need to use recent versions of the appropriate Windows or Linux AMIs and run inside of a VPC.

Amazon EBS–Optimized by Default
Each D2 instance type is EBS-optimized by default, and delivers dedicated block storage throughput ranging from 500 Mbps to 4,000 Mbps at no additional cost. EBS-optimized instances enable you to get consistently high performance for your Amazon EBS volumes by eliminating contention between Amazon EBS I/O and other network traffic from your D2 instance. For more information, see Amazon EBS-Optimized Instances.

Power to the People
Each virtual CPU (vCPU) is a hardware hyperthread on an Intel Xeon E5-2676 v3 (Haswell) processor.

The D2 instances take advantage of Intel Turbo for increased performance. As I have explained in the past, this technology allows the processor to run faster than its baseline speed (2.4 GHz) as long as it remains within predefined thermal and power limits, with an upper limit of 3.0 GHz.

The largest instance (d2.8xlarge) also gives you a pair of bonus features: NUMA support and CPU power management. NUMA (Non-Uniform Memory Access) allows you to specify an affinity between an application and a processor that will result in use of memory that is “closer” to the processor and therefore more rapidly accessed. CPU power management gives you control over the C-states and P-states to enable higher turbo frequencies and to lower performance variability, respectively.

Available Worldwide Now
You can launch D2 instances today in the US East (N. Virginia), US West (Oregon), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Asia Pacific (Sydney) regions as On-Demand, Reserved Instances, or Spot Instances.

— Jeff;