The prsentation brought you by Jared Rosoff, Director of Product Marketing and Technical Alliances at 10gen. The presentation includes an overview of MongoDB deployment on Amazon cloud service. You will also find some basic operational considerations with regards to deployment, backup and security management. Jared also published an interesting NoSQL database comparison on his blog, we strongly advice you to review.
The traffic of a typical web application directly linked to the amount of concurrent users that use the system. The variations in daily traffic volumes generated by a typical web application match our daily life schedule. Data Traffic transfer, running the application on Amazon cloud, can mean substantial costs. These cloud costs can even amount to more than half the overall charges for all instances utilized for the same amount of time.
The cost of traffic on Amazon AWS stems from two main factors:
1 - Outbound data transfer, between regions as well as out of the Amazon cloud.
2 - ELB data transfer fees across availability zones (AZ).
The Cloud changes add great complexity to how you track and maintain your data traffic. Controlling performance and cost in the Amazon cloud can be a great challenge for cloud newcomers. Even experienced early adopters sometimes struggle to manage their cloud right and find clear visibility into their ongoing real time traffic costs. This continues to be a challenge for cloud adopters, so I’d like to share a few brief points that can help improve and control.
Make sure you understand the different costs for data traffic usage:
1 - Incoming traffic is free all the time
2 - Traffic between instances in the same availability zone is free.
3 - Outgoing data is always charged, even between instances inside AWS if they are on different AZs or regions.
4 - If you are using ELB as your FE, you will have incoming data fees because traffic between ELB and your instances will always be charged as traffic between amazon cloud availability zones (AZs).
Add the data traffic parameters to your architecture – Try to minimize the amount of data traffic between different AZs; if you have a flow of instances that communicate with each other a lot, try to keep them in the same AZ (while considering HA as well….).
Use Private IPs - Always use EC2 private IP addresses when communicating between different instances. I have seen many setups where public IP addresses are used, which eventually are charged as outgoing traffic instead of inner-AZ traffic.
Depends on your application, you can’t always control data usage but you should have full visibility and the ability to analyze environment behavior.
1 - I recommend monitoring your data transfer levels using the AWS account activity (you see the monthly data usage there) to make sure your usage meets your expectations. If not, you should investigate your instances and understand who/what’s using your data.
2 - Use CloudWatch NetworkIn and NetworkOut metrics to determine whether there are instances that are serving more data than needed. The relevant metrics for Amazon AWS CloudWatch are DataIn and DataOut. These only show a specific point in time, so you will need to calculate totals in order to really understand traffic flow per hour.
Pricing for “In” an “Out” data transfer of S3 Storage and Amazon EC2:
Image Resource: http://aws.amazon.com/ec2/pricing/
For even more visibility, and so that you can better forecast your traffic costs, you need to analyze the growth of your data and its transfer rates. Knowing your application and becoming familiarity with how much traffic it can be expected to consume will help give you a better feel for your Cloud computing needs.
Learn More: 10 most common mistakes you should avoid operating your Amazon AWS environment
—————————————————————————
About the Author
Galed Friedmann
Galed is an IT and operations professional with over 10 years of experience in technology management and IT operations. Galed has held many IT management positions, working on both traditional dedicated hosting and Cloud environments. Today, Galed is Head of Operations at Onavo, a Sequoia-funded startup, which runs entirely on cloud technologies and on AWS platform.
Galed’s LinkedIn Profile
Keywords: amazon cloud services, cloud monitoring, amazon elb, amazon elastic load balancer, cloud cost, EC2, S3, data transfer, out-going traffic costs, amazon cloud usage, best practices,
A fact is that IT as a Service keeps taking market shares. The services are delivered as traditional SaaS, from a Managed- / Service Provider or as cloud services. We see growths in areas like IaaS and especially PaaS when application providers and vendors put applications in the cloud. But the speed is not what many expect. What’s causing this? I’ve been doing some thinking.
There is a common perception that cloud storage should not really worry you because it is very cheap and available at any time. Is that really true ? It is common to hear AWS consumers say that AWS storage = S3 – this is true but it is not the whole truth.
In the last year I have had the privilege of meeting lots of companies that take advantage of the public IaaS capabilities. While most of the companies are already using Amazon AWS, unfortunately, I see many IT leaders that are having bad experiences with Amazon AWS resources sprawling out across their organizations, resulting in increased uncertainty of the monthly cost. It’s a fact that cost is becoming a sore point in the cloud adoption phase, so instead of opting for a smart and planned process, the bad experience leads the IT leader to return to the “good old” “old and good” environment and simply buy new servers.
It seems that the decision makers have accepted that cloud computing leads to lower IT cost. Although many experience savings while switching to the cloud, the total cost of ownership for most is still too high. So before you go and spend more money on additional monitoring and management systems, here are some tips to help you regain control and reduce costs immediately.
According to Gartner’s report “Reimagining IT: The 2011 CIO Agenda”, almost half of all CIOs expect to adopt cloud technologies within the next five years. Not surprisingly Gartner’s analysts expect an extreme increase from 3% to 43% of the IT organizations that will run applications in the cloud. No doubt that most of the IT organizations already adopted SaaS, IaaS adoption is evolving rapidly and PaaS gain momentum.
“CIOs recognize that they need to reposition themselves and IT to support enterprise innovation and growth. However, two issues stand in their way: benefits realization (the achievement of business benefits) and IT skills. Skills are an issue because CIOs rely on bringing skills in from the outside whenever they need to get work done (see figure below). Both issues will prevent IT from reaching full potential unless the CIO addresses them”
A lot have already been said about the false cloud use where the IaaS platform utilized as an hosting extension of the IT organization’s data center and not taking advantage of the elasticity benefits to generate a cost effective and scalable IT operation. Using the public IaaS whether it is Amazon, Rackspace or any other vendor means using an highly dynamic environment which presents an increasing complexity hence loss of control. Checking the list below I can say that cloud (including all its layers IaaS, PaaS and SaaS) control basically contains the same aspects as the good old system management.
