-
Monday, May 11, 2009
You can download the source for my F# AWS application from here. The remaining posts start on April 13, 2009, and run through today (with a few interruptions from me on other topics).
Full story
Utility Computing
F#
Amazon Web Services
-
Friday, May 08, 2009
I wish I could say that their is something amazing that I did differently from everyone else who has hosted on EC2, but that’s just not so. Once you get an EC2 instance running, the rest of the process is identical to connecting to a virtual machine anywhere. If you need to add additional Windows components, the directions in this post are dynamite. If you use the management console, the directions on the Amazon article are pretty easy to map to actions in the UI without much thought. So, you are deploying to a virtual machine that is easily duplicated across the Amazon EC2 infrastructure. I would say that the strongest reason to use EC2 is that you already know how to use it. A second reason to use EC2 over your own data center is this: if you are using S3 and SimpleDB, you don’t pay for ...
Full story
EC2
Utility Computing
Amazon Web Services
-
Wednesday, May 06, 2009
(You can review my description of S3 here.) Because Amazon’s Simple Storage Service, S3, has been around for quite a while, the community has built a large number of libraries to access S3 programmatically. The C# space is pretty well crowded as well, and many folks wrote libraries that are good enough for their needs, then released those libraries to the world. The trick in picking an S3 library does not revolve around picking a best library. Instead, it involves finding one written by someone who had a similar set of needs. When looking for a library, think about how you want to add, update, and delete objects in S3. Write down the use cases. Then, download a set of libraries and keep messing with them until you find one that matches your needs. My planned usage involves: Create a bucket once, when the application first runs. (This step ...
Full story
Utility Computing
F#
S3
Amazon Web Services
-
Monday, May 04, 2009
This week, we close out our look at the Amazon Web Services implementation of the AppEngine Photo Application. S3 is perhaps one of the best known, most used services on AWS. Before we discuss using S3, I need to cover some basic terminology. The key words to know are bucket, object, and key. A bucket contains zero or more objects. An object is associated with one key. Buckets have names and are associated with an AWS account. Upon creating a bucket, you also create an addressable resource on the Internet. For example, I can create a bucket named MyS3Bucket. Upon creating that resource, S3 enables a new URI at http://MyS3Bucket.s3.amazonaws.com and at http://s3.amazonaws.com/MyS3Bucket. The bucket can be public or private. A public bucket can be accessed by anyone whereas a private bucket requires a token to access any contents. The owner of the bucket ...
Full story
Utility Computing
S3
Amazon Web Services
-
Friday, May 01, 2009
This post uses Amazon’s SimpleDB C# library. Deleting a record is really simple. Again, this is to delete an ASP.NET MembershipUser from SimpleDB. To do this, you simply pass the ItemName to a DeleteAttributesRequest(). Because that name is unique within the domain, the values will go away. override this.DeleteUser(username, deleteAllRelatedData) = PhotoWebInit.InitializeAWS let simpleDb = PhotoWebInit.SimpleDBClient let delAttr = new Model.DeleteAttributesRequest() delAttr.DomainName <- PhotoWebInit.domainName delAttr.ItemName <- username let delResponse = simpleDb.DeleteAttributes(delAttr) true Pretty easy, right? Next week, we’ll finish this up with a post on Simple Storage Service and a zipped copy of the source code (in case you want to see everything in one place).
Full story
F#
Amazon Web Services
SimpleDB
-
Wednesday, April 29, 2009
I recently talked about using SimpleDB to save or update a record. Today, we look at how to query against records in SimpleDB while using the Amazon SimpleDB C# Library. Each record in SimpleDB has an ItemName (unique, primary key) and a set of attributes (name-value pairs). Upon insert or update, all fields are indexed for easy querying. You can access the data using the Query API or Select API. Since I am already familiar with SQL, I picked the Select API as it closely resembles the standard SQL SELECT statement. Recall that SimpleDB stores data in domains. Any query goes against the stored domain. For example, I know that the MembershipUsers in the domain for my sample ASP.NET MembershipProvider all have a unique email address in a field named email. I also know that the email only exists on one record collection, so I shouldn’t be ...
Full story
Amazon Web Services
SimpleDB
-
Monday, April 27, 2009
This post is just a dump of a set of thoughts that have been running around in my brain. All three major cloud platforms, Google App Engine, Amazon Web Services and Microsoft Azure, offer a way to run worker tasks. Google recently introduced cron jobs. Amazon has Elastic Compute Cloud. Azure has the worker role. Each of these mechanisms works in a similar manner: a process looks somewhere for work to do (in the distributed data base, a work queue, or elsewhere) and then performs the task. I don’t have any issues here—this all makes sense. You need a headless process to take input and produce output. This is a staple of most systems I have worked on. I have another issue—how much is this going to cost me? While many people are busy climbing the Gartner hype cycle and are close to the “Peak ...
Full story
Azure
Google App Engine
Cloud Computing
Amazon Web Services
-
Friday, April 24, 2009
Last night (4-23), I saw that Amazon is now offering IBM applications by the hour. I thought “Cool!” Then I took a look at the pricing for these things. This pricing doesn’t take effect if you already own IBM licenses for the products and just want to host on EC2. If you own licenses, IBM has a table up to show you how to convert from Processor Value Units (PVUs) to EC2. These prices are for preconfigured Amazon Machine Instances (AMIs) with the IBM software ready to rock—no extra salesmen need to get involved. All that said, I have no idea how much a PVU costs for an application, but my guess is it costs “a lot”. A project I was on in 2007 required an IBM C Compiler to run on Z/OS (it was needed to interpret SQL statements into C programs that could run as stored procedures ...
Full story
EC2
Amazon Web Services
-
Wednesday, April 22, 2009
With SimpleDB, you need to create a domain to store any data. Domain creation is a unique event in the life of an application, happening at install or some other infrequent event (addition of a new product line, client, etc. depending on how you partition data). 99+% of the time, the code will need to use the domain to Create, Retrieve, Update, or Delete data. These are more commonly known as CRUD operations. AWS stores the data as name-value pairs called attributes. The attributes have an additional value indicating if the value is replaceable/updatable or if the value can only be set of creation. Each item in the database has a name and a set of 1 or more attributes. In this post, I walk through how MembershipUser records are created and updated. This post uses Amazon’s SimpleDB C# library. The example has the following ...
Full story
F#
Amazon Web Services
SimpleDB
.NET
-
Friday, April 17, 2009
Before you can access data in SimpleDB, you have to have a domain. Domain creation is fairly expensive in terms of time—up to 1/2 a second. Listing domains is really cheap—especially since you will normally have a maximum of 100 domains. When the application starts up, we want to check if the domains we need exist—if not, create them. Otherwise, mark an all clear so that the check doesn’t happen again. To handle all this work, we have a module named PhotoWebInit. The module instantiates a client capable of communicating with SimpleDB by reading the key and secret from configuration. Using that client, the code then checks to see if the domain we want, friseton_com, exists. OK—this code really looks to see if we have 0 domains or more. friseton_com is the first domain we need because if a user needs to be able to log in before any ...
Full story
F#
Amazon Web Services
SimpleDB
.NET
-
Wednesday, April 15, 2009
In writing the PhotoWeb application for Amazon Web Services, I took advantage of the fact that an EC2 instance is just a VM running on top of Xen. Because it is just a VM, the entire application can be developed and tested on your local dev box before deploying to EC2—I highly recommend doing as much development and testing on your local machine before deploying to the hosted environment. I decided to start out by handling the authentication piece first. Because hosted SQL Server on Amazon costs extra, I used SimpleDB as the datastore. Before covering how to use SimpleDB, I need to explain what SimpleDB is. As a database, SimpleDB has a lot more in common with old school ISAM databases than RDBMS systems like Oracle, DB2, and SQL Server. To store data, you first need a domain. A domain is a collection of data. You can add ...
Full story
Amazon Web Services
SimpleDB
-
Monday, April 13, 2009
Back in February, I walked through the development of a Photo storage application. The application originally comes from one of the examples in the REST book Kenn Scribner and I wrote, Effective REST Services via .NET. Photo sharing and uploads allow for me to present a well understood application without providing a lot of background. For a photo, you upload it somewhere and store metadata about the photo itself. We already covered Google App Engine in February. For this application, we will use Amazon Web Services, including SimpleDB, Simple Storage Service, and Elastic Compute Cloud. At the end, I’ll tell you what I thought of the experience. I’ll develop the application in F#. When I presented this code to the Midwest Cloud Computing Users Group for the April 2009 meeting, Amanda Laucher offered up that my use of F# used some idioms she hadn’t seen before. That’s a ...
Full story
EC2
Utility Computing
S3
Amazon Web Services
SimpleDB
-
Friday, April 10, 2009
The Amazon Web Services Management Console is SIGNIFICANTLY easier to use than the command line applications one must use to manage an EC2 instance. I had puzzled out all the commands and got things working. Tonight, I stumbled across the AWS Management Console: https://console.aws.amazon.com/ec2/. It’s a nice, point and click interface to setup an Amazon Machine Instance (AMI), bundle that instance into a bucket, figure out connection credentials, setup common security groups, etc. The interface has the advantage of telling me exactly how far along the system has gone with each step. This was particularly nice when bundling my AMI. Without the feedback, I wouldn’t have known what was going on for the 20 or so minutes where the instance was paused and packaged. The service even handles storing the image in your S3 account. I also had to install IIS tonight. The console made it easy to create ...
Full story
EC2
S3
Amazon Web Services
-
Wednesday, April 08, 2009
I received some great feedback on my survey of cloud computing platforms at the February Cloud Computing User Group in Downers Grove, so I’ve been asked to bring the show to the downtown Chicago meeting for the April meeting on Thursday, April 9 at 5:30 at Microsoft’s downtown location. If you saw the talk last month, I’ve been asked to beef it up with real world code running on all three major platforms: Google App Engine, Amazon Web Services, and Azure. It’s a simple photo sharing application based on the same code I showed in February for App Engine. Please sign up at https://www.clicktoattend.com/invitation.aspx?code=136727. The full meeting description: Join us for the fourth local meeting of the Cloud Computing User Group - this months being held at the Microsoft offices in Downtown Chicago. Note : this meeting will be a revisit and slight revamp of last month's content. As ...
Full story
Azure
Google App Engine
Cloud Computing
Utility Computing
Amazon Web Services
-
Monday, March 16, 2009
I was busy working on the Amazon Web Services version of my Photo Sharing application (the App Engine version appeared in February 2008). I was able to create a domain with the name friseton.com. I then wrote a simple query against this domain: select * from friseton.com where email=’dude@example.com’ SimpleDB responded and told me that the select query had an invalid format. On a wild guess, I thought it might be the dot (.) in friseton.com. Sure enough, it was. I deleted the friseton.com domain, added friseton_com, and the query started working. select * from friseton_com where email=’dude@example.com’ I don’t know if there is any syntax around allowing the dot in the domain name, but I found this, it works, and I’m moving on with the demo. I’ll learn the ins and outs as I go.
Full story
Amazon Web Services
SimpleDB
.NET
-
Friday, March 13, 2009
Amazon just announced a reserved instances plan for EC2. This new option significantly reduces costs for Linux/UNIX users. This option is not available for Windows users as of this time. The way it appears to work is this: User buys block of hours that is good for a 1 or 3 year term. User consumes hours and has this usage debited from the prepaid block of time. If the user fails to consume their hours within the 1 or 3 year term, the money is gone. The 1 year blocks sell 10833 hours. The 3 year blocks represent 16667 hours. For comparison, a week has 168 hours, a year has 8760 or 8784 hours (leap year dependent). Here's a cost comparison of the offerings: 1 yr Block of hours Pay as you go 3 yr Block of hours Pay as you go Standard/Small $325.00 $1,083.33 $500.00 $1,666.67 Standard/Medium...
Full story
EC2
Utility Computing
Elastic Compute Cloud
Amazon Web Services
-
Wednesday, March 11, 2009
I originally presented this talk to the Azure Cloud Computing User Group on February 25, 2009. Thanks to Bryce Calhoun for inviting me to present! The original meeting announcement had this summary: Scott Seely, Architect at MySpace, will kick off the meeting with a 20-30 minute overview of the top three cloud computing offerings available today: Google App Engine, Amazon EC3 and Azure Services. His discussion will be primarily focused on a compare/contrast of the functionality and features inherent to each platform. Enjoy!
Full story
CloudFront
Azure
Google App Engine
EC2
Simple Storage Service
Cloud Computing
Utility Computing
Elastic Compute Cloud
Cloud Storage
SQS
S3
Simple Queue Service
Amazon Web Services
SimpleDB
-
Tuesday, February 24, 2009
I'll be speaking Wednesday night at the Cloud Computing User Group in Downers Grove, IL. I have a short presentation on the main computing platforms. If you are a regular reader, you know that I've been spending some time going beneath the surface on major platforms. If folks like the high level overview, I'll do some more in depth talks in the future. Here are the details and the announcement that Bryce Calhoun sent out: Join us for the third local meeting of the Cloud Computing User Group – this month in Downers Grove. At this meeting, we will be learning about how Live ID integration works in the Azure cloud computing platform. We’ll demo and dig into the code of an application built in the cloud that integrates directly with the Live ID service and stores information specific to the individual associated with that ID. Also, Scott Seely, Architect ...
Full story
Azure
Google App Engine
EC2
Simple Storage Service
Cloud Computing
Utility Computing
SQS
S3
Simple Queue Service
Amazon Web Services
SimpleDB
-
Friday, February 20, 2009
Once upon a time, I worked for a company on a project to build a Web service stack. Prior to that, I wrote a number of articles for that company, including one that wound up being used as the basis for versioning efforts across several industry leaders (the latter based on conversations and personal e-mails). After all these years, people still come to me and ask how to inform consumers of the interface that the version they use is about to go away. Today, Amazon SimpleDB provided a great example of how to inform customers of a new interface. The announcement was well done, short, and was communicated to all users via e-mail as well as posted publicly on the Internet. Amazon provided this statement, which I think is a great way to communicate that things are about to change as well as why they are changing:...
Full story
Amazon Web Services
SimpleDB
-
Friday, February 06, 2009
At this point, I think I have a good idea about what GAE, AWS, and Azure offer in terms of potential. To force myself to learn things faster, I think that it's time to go out and build an application. For the sake of making apples to apples comparisons, I need an application that is fairly simple and allows me to exercise cloud storage as well as application hosting. I also want something that is super simple and that involves something I have already built. For my book, Effective REST Services via .NET, I put together an application that allowed a user to create and manage a photo album. I think it's appropriate to reuse that concept. For one thing, I already have the application coded, so I will be able to reuse a lot of ideas and code. The other thing that I like about this idea is ...
Full story
Azure
Google App Engine
Cloud Computing
Utility Computing
Amazon Web Services