EC2 on Rails
A Ruby on Rails EC2 “Appliance”
This is a public server image for Amazon’s EC2 hosting service that’s ready to run a Ruby on Rails application with little or no customization. Basically it’s a Ruby on Rails virtual appliance.
It includes a RubyGem with Capistrano tasks to manage server instances.
For details see the project website.
Matt Jaynes:
Wow! You rock! I built similar images for my custom lighttpd/PHP/sqlite setup, but then I switched to Rails and was dreading doing all that work again. Thanks so much for putting this together. Sending major karma and kudos your way!
21 June 2007, 2:10 amRoger:
This is awesome! I am sure a lot of people will benefit from this.
21 June 2007, 12:50 pmRG:
I’m really looking forward to seeing your benchmark / loadtesting results! Please keep us posted!
21 June 2007, 1:03 pmScott Raymond:
Nicely done. I spent a while this week getting your previous AMI release working for me; this one looks even easier to work with. May I suggest that you also release a script that can be used to build up this image from scratch — that way everyone can see precisely how this differs from the stock Ubuntu install, and we can submit patches to help improve it. Thanks again for the work you’re doing!
21 June 2007, 2:28 pmMarcos Neves:
Hi, Sorry my ignorance, but what usefull for would it be? Cause EC2 is very expensive ($0.10/hour/instance). That would be $72/month (data transfer not included). With that money, I can have a dedicated server!
21 June 2007, 3:28 pmOr there’s something I’m missing?
Shanti Braford:
Great writeup! Are you using EC2 or do you know of anyone who is using this for a production app?
It seems pretty risky if the server goes down — I guess the idea is you’d only lose 10 minutes worth of data at most?
And then if your database is 20-40GB, this option is probably off the table or you’d use a remote database or something then?
EC2 would be so killer if it came with persistent storage!!
21 June 2007, 6:37 pmSteve:
Hi,
How do you deal with losing your database is the server goes down? If you lose on average 5 mins worth of data if it goes down, the users will not be very happy!!!!
What the solution to this?
24 June 2007, 6:53 pmAdam Roth:
Shanti: Use S3 with EC2 for storage.
Paul: Get this goin’ with Nginx and I’d love to try it out. All the cool kids are running it now.
25 June 2007, 12:07 amlawrence sinclair:
I’ve managed to get your machine running. Thanks. But I’m stuck at the point where I need to mount S3 drives to store web content, log files, etc. Can you point me in the right direction? Thanks.
25 June 2007, 7:16 amNeil Wilson:
Don’t forget that you can happily *reboot* EC2 instances without losing storage. The storage is only ‘garbage collected’ when the instance is terminated - either explicitly via a shutdown, or via a catastrophic failure.
The only risk with the server is if it crashes completely. Given that it is on multi-reliable hardware, about the only possibility is a kernel bug or a network screw up.
Essentially the risk of it happening is very small. Losing 10 mins data over a very small risk isn’t worth worrying about. You’re more likely to get an Internet storm stopping connections.
25 June 2007, 10:41 amNeil Wilson:
EC2 isn’t at all expensive if you realise that
(a) it can be run directly by the end client.
(b) They can run the machine between 9-5 and switch it off the rest of the time.
(c) It has 1.75GB of RAM available!
Change your thinking. EC2 allows you to have machines that work like applications. You start them up and shut them down as required. They don’t have to be sat there all the time.
25 June 2007, 10:43 amNeil Wilson:
If you want to use PKI with sudo, can’t you just alter the appropriate PAM file. That gets away from requiring a password.
25 June 2007, 10:48 amPaul Dowman:
(Sorry for the slow replies, I’m on holiday at the moment and I’m out of the country with very sporadic internet access.)
Scott: That’s a good idea, I’d like to do that. I agree that’s a prerequisite to get any community input, which I’d really like.
Marcos: If you know that all you want now and in the future is a single server then a dedicated server might be a good option for you. EC2’s strength is being able to grow and shrink capacity on demand, and of course being able to take advantage of pre-built server images that other people have created. :-)
Shanti: I do know people who are using it for production apps at the moment and they are happy with it. As for the risk of the server going down, you have that anyway– when you build your own servers you can try to make each machine more failure-proof (raid drives, etc), or, assuming your app is big enough to require multiple servers, you can try to build redundancy at the data-center level. If you take that approach then EC2 is perfect. For database redundancy you can one or more MySQL slaves, and if your database is too large to dump on a regular basis you could even have one of those slaves off-site.
Steve: as Neil mentioned it’s pretty unlikely that your database virtual machine instance will spontaneously disappear but the solution is having a MySQL slave which is effectively a continuous hot backup.
Neil, re PKI with sudo: thanks, I’ll look into that!
25 June 2007, 12:52 pmPJ Hyett:
Very cool, thanks.
25 June 2007, 1:22 pmChris:
Paul,
Thanks for this … very, very cool stuff. Like Scott, I’m interested in seeing the script that put this together. There are a few things I’d change (s/mysql/postgresql and nginx) but this looks awesome.
I’m gonna have to blow the dust off the ol’ EC2 account.
25 June 2007, 2:34 pmSteve Odom:
Good work, Paul. You may check out the work I’ve done and see if there’s anything you can use. I created a wiki to post different Capistrano recipes and then incorporate them into an elasticrails plugin I wrote.
Check it out at:
http://www.elasticrails.com
Or the screencast I did:
http://s3.amazonaws.com/rc_files/elastic-rails.html
Hope it helps.
26 June 2007, 11:20 amEd:
Just wanted to second all the positive comments. This is something I’ve been meaning to get back round to (I got about halfway through setting up my own EC2 instance and gave up trying to get MySQL working), so you’ve saved me heaps of time and inevitable frustration. Great idea about backing the MySQL data to S3! Thanks again. I’ll try to contribute back any improvements if I can think of anything and get it working. Memcached is something I need to implement, so I’ll let you know how it goes. Also quite interested in Nginx (want to be one of the cool kids!)
26 June 2007, 5:18 pmRodrigo Couto:
I’ve seen that your image has more than 150 Gb of disk space, do I pay for this space on S3 also?
27 June 2007, 12:37 amJon:
Hi,
Would it be possible to also include sqlite + gems for smaller projects that don’t need MySQL?
Thanks, love it!
- Jon
27 June 2007, 2:57 pmMarcos Neves:
Paul Dowman: Can you share your iso image, so I can use it offline, with vmware?
Rodrigo Couto: yes, your image is charged as any other file on S3. But it´s just $0.18/GB/mo since it´s only 150MB, it´ll const only $0.02 month :)
29 June 2007, 7:58 amSam:
Thanks for creating this AMI! It saved me a lot of time in getting started. I second Scott’s request to post a script of how you configured the AMI so that folks can contribute to it.
29 June 2007, 12:22 pmNeal McBurnett:
Very nicely done - thanks!
I’m thinking that this sort of setup would also be useful for testing of development releases of Ubuntu. I don’t know if recent debootstraps can package up e.g. the Gutsy Gibbon Tribe 2 release, but if so, providing an AMI of that would make it easy for folks to test different server apps and configurations. I’m even picturing LDAP and Kerberos configs which would allow people to do regression testing, authtool testing and the like much more conveniently.
29 June 2007, 6:39 pmChris:
This looks great!
1 July 2007, 11:37 amAny chance of postgresql support too?
David:
Great stuff!
Now how do you think we could best share our efforts as an open source project? Do we send you the modifications directly or do you think there’s better a way to manage the revisions with SVN or something else.
Thanks a lot for your work.
3 July 2007, 11:23 amPaul Dowman:
David: I would love this to become an open source project, and so the first thing I’m going to do is a build script so that this can be repeatably built from source. I’ll set up a public subversion repository (or maybe use something like sourceforge or google code, any opinions?). In the meantime if you do want to contribute feel free to email me a patch!
Chris: I’m not planning postgres support at the moment. My priorities are a build script and then multiple server support.
3 July 2007, 9:04 pmAlex Reuter:
Thanks for sharing Paul -this is awesome.
Any chance RMagick is in its future?
6 July 2007, 11:21 pmEric Northam:
I noticed in /etc/init.d/ec2-every-startup you have a 15 second sleep
# this is an ugly hack to make sure DHCP initialization is finished:
sleep 15
Is there anyway around this sleep? Where did you run into a problem with DHCP not coming up in time?
7 July 2007, 11:28 amPaul Dowman:
Alex: Sure, RMagick is widely used. I don’t see a reason not to include it by default.
Eric: I’d like to find a better solution for that. In the first version I found that curl sometimes failed to download the public key and a little debugging determined that there was no valid IP address at that point. A little extra time solved the problem temporarily but I haven’t looked into it further. Networking is started by /etc/rcS.d/S40networking and the script that fetches the public key is called from /etc/rcS.d/S92ec2-every-startup.
7 July 2007, 1:07 pmEric Northam:
Maybe as a slightly more flexible workaround to the curl issue that doesn’t require investigation is to call curl and if it doesn’t return successfully continue to call it every second 15 times until it returns back successfully. If it doesn’t work after 15 times you could then just log an error.
7 July 2007, 3:06 pmPaul Dowman:
Eric: That’s smarter way to do it. I’ll do that. Thanks.
7 July 2007, 3:08 pmEric Northam:
How about creating a base feisty image without the mongrel cluster and custom configuration changes that were used to configure the cluster? There currently isn’t one listed in the public AMIs (actually there is one image listed but it’s not actually feisty). We could then have a standard feisty base image to use to bundle other configurations. It would help attract a larger user base and other interested maintainers if you do end up hosting the images on sourceforge or somewhere else.
7 July 2007, 3:25 pmJim Waterwash:
I’ve been hoping someone would tackle this. Thanks!
8 July 2007, 1:52 pmLooking forward to learning from your build script too.
piggybox:
Nice work.
Amazon EC2 definitely looks promising in terms of price and capacity, comparing to similar plans from Joyent and Media temple.
11 July 2007, 9:36 amEric Northam:
I looked over the rebundle.sh script and I think there might be a small bug. The line
ec2-bundle-vol -e “/root/.ssh,/home/app/.ssh,/tmp,/mnt” -d /mnt -k “$EC2_PRIVATE_KEY” -c “$EC2_CERT” -u “$AWS_ACCOUNT_
ID” || exit 3
expects the environment variables “$EC2_PRIVATE_KEY” and “$EC2_CERT” which aren’t set in the config file. I’m manually trying to do a rebundle and I noticed that ec2-register is looking for the same environment variables which on my image both default to /mnt/aws-config/. Maybe you missed a step in your docs above about setting these variables. If they were set then I think it would make KEY_FILE_NAME and CERT_FILE_NAME in the config file useless.
I also suggest that you don’t create a new bucket every time you rebundle the image. You only have a 100 buckets and they all have to be uniquely named. I think a better approach is to use the same bucket name every time and then customize the image names. You can easily do that with the -p flag for prefix when you call ec2-bundle-vol.
IMAGE_NAME_PREFIX=”$IMAGE_BASE_NAME-image-$TIMESTAMP”
11 July 2007, 10:46 amec2-bundle-vol -p “$IMAGE_NAME_PREFIX” …
Kengteh Min:
Awesome stuff!
Just want to let you know that the AMI needs ‘hpricot’ gem to be installed for capistrano migrate task to work. It would be even better if you could include gem in the AMI for the next release.
12 July 2007, 4:10 amPaul Dowman:
Eric: I think you’ve missed the second statement in the script, which is “. /usr/local/aws/config”. That includes the config file, which sets “$EC2_PRIVATE_KEY” and “$EC2_CERT”. If it’s not working for you make sure that you have created the directory /mnt/aws-config, and that the config file in there has all the correct settings.
Regarding the name of the backup S3 bucket: I think it’s a bad idea to base the bucket name on the image name. The reason is that you will likely start another instance of the same image (for testing), and then your test instance will write to the same bucket, overwriting your production data. I personally would rather clean up a few S3 buckets than risk losing my production data, but perhaps there’s another solution with neither of those problems? Opinions are welcome…
Kengteh: I hadn’t noticed that the capistrano migrate task requires hpricot. I’ll check it out, thanks.
13 July 2007, 8:21 amEric Northam:
Oops. I thought it was pointing to the other config file. Thanks.
I think by appending the timestamp to the image name with the -p flag you nearly rule out all possibility of overwriting the images. You would have to run rebundle.sh at exactly the same time on two instances running with the same image name set. The image parts also contain the timestamp. I guess it’s more a matter of preference since your probably not going to rebundle that often and your not likely to have conflicts because of the timestamp. It only gets really ugly with public shared images. Say two people use your image and decide to name their image ubuntu-rails. Now someone else calls ec2-describe-images for all the public images and gets several timestamped buckets for ubuntu-rails and chooses the latest because it’s the latest but might not necessarily the right owner or image. I also like to be able to list all my x-images by simply listing the contents of a single bucket. Amazon also groups images by bucket: “ec2-describe-images -a | grep amazon”
Eric
13 July 2007, 9:57 amsujal:
How does the mysql backup happen? I had a typo in the s3.yml file when I fired up my test instance, but even though I fixed it, I’m still not seeing backup buckets in my S3 account. I’m trying to debug it, but I’m at a loss for where the magic is happening. :-)
Sujal
18 July 2007, 6:31 pmsujal:
never mind, found it… got it working.
18 July 2007, 6:33 pmsujal:
Oddly enough, I had to set the cron job to run as root. Perhaps you need to add a cron.allow file with app in there? Otherwise, it wasn’t running.
18 July 2007, 6:56 pmPaul Dowman:
Sujal: I think there may be a bug in that image, I noticed that the cron job which does the backup didn’t seem to be working, and I think it was because there was no end-of-line character at the end of the file /etc/cron.d/backup_app_db_to_s3. Try editing that file and pressing enter at the end of the line. Let me know if that helps. I’m going to build a new image with that change in it.
18 July 2007, 8:08 pmsujal:
Paul, I’ll try that out. I was able to get it to work by changing the user to root instead of app (yes, yes, I know) but when I looked at the file just now, there is a new line at the end of the line… so maybe I added that without meaning to.
19 July 2007, 3:29 pmLeo Dirac:
I ran into a problem trying to install some gems and use them during development. (i.e. running with script/server) The gems by default install to /var/lib/gems/1.8/gems/… which isn’t in the default path. If you add this line to ~/.bash_profile then it works much better:
export RUBYLIB=”$RUBYLIB:/var/lib/gems/1.8/gems”
Also in debugging this, I noticed that something bizarre is going on with the environment variables for the “app” user. Launch a new instance and run “set” to see what I mean. 4000 lines come out, mostly code.
19 July 2007, 8:26 pmPaul Dix:
Paul, I just wanted to drop a line and say thanks for making this. I just got a node turned up with this image and my application. You rule!
20 July 2007, 2:17 amPaul Dowman:
Leo: Thanks for the feedback. I’ll add the gem path to RUBYLIB (/var/lib/gems/1.8/bin is in the PATH though). And you’re right, there is a bunch of crap in the ENV for the app user! I’m not sure where that came from. I’ll fix it asap!
24 July 2007, 8:18 amChristopher Ricca:
First of all, Paul: Thank you, thank you, thank you.
Second, I would normally avoid going off topic like this, but since we are all operating in the same environment in EC2, I thought someone might be able to shed some light on this:
I got my rails app up and running, and went through multiple (successful) deployments last night, when cap deploy started failing during the svn checkout, whining that:
svn: PROPFIND of ‘/myrepository/trunk’: could not connect to server (http://svn.myserver.com)
the strange thing is that ’svn co http://svn.myserver.com/myrepository/trunk‘ works from my development machine, and I can ping the server, but when I am logged in as app on EC2, ping gets 0 responses from svn.myserver.com. However, I can ping other domains (www.apple.com, http://www.myserver.com) from within EC2. The svn server is on dreamhost, but unless they are blocking my EC2 ip, I don’t the problem is there.
I wasn’t doing anything at the time of breakage server-side other than running cap deploy to see changes.
Thanks again
24 July 2007, 10:54 amLarry Kluger:
Hi Paul,
Excellent work, and excellent documentation, thanks. I now have your AMI running happily.
A couple of suggestions for your consideration:
1) I think a wiki of recipes for use with your image would be helpful. My first recipe contribution is the next post.
2) A version number would be very very helpful, you could display it as part of your current login message.
3) You’re essentially creating your own distribution–which is great. But you’ll want to avoid various theological discussions about which is the best way to create a Linux system, what should be included within it, why not add X, Y or Z, why have you added so much bloat of X, Y, and Z, et cetera ad nauseam.
Ideas about this:
a) Continue to recommend to people both how they can customize your pre-built image and how they can build their own new image with their changes. (Just as you’ve done above.)
b) Then, when people submit ideas that you think are not inline with your project’s goals, they can be referred to the “how to do it themselves” documentation.
c) Don’t add additional sw to the dist unless it really helps you achieve the goals of your project.
d) Add enablers, rather than sw. For example, a Capistrano script to add additional gems to the ones that you include in the dist. That way people can use the script rather than barrage you with “please include gem x” and “please update gem x to its newest version”
Hope this is of help,
Larry
25 July 2007, 7:47 amLarry Kluger:
Recipe: Enabling remote access to the MySQL db
WHY
You may want to enable remote access to the MySQL db to enable use of SQL tools such as the MySQL Query Browser, http://www.mysql.com/products/tools/query-browser/
SECURITY NOTES
1) This recipe uses the standard, unencrypted MySQL TCP transport. Your data and passwords will be visible to anyone who can peek. There are ways around this if it is a concern.
2) By opening the MySQL remote access, hackers could try brute force dictionary attacks. So pick a good password for your MySQL accounts!
3) You can mitigate item 2 by only allowing logins and access from specific remote IP addresses.
STEPS
1) Log in to your AMI image as root.
2) Log in to mysql as root. If you changed the root password from then you’ll need it.
mysql -u root # or
mysql -u root -p # you’ll be prompted for your root pw
3) Enable remote login from your app’s MySQL name and pw. Or create another login for remote access.
Note that you can create a login that has read-only access to the dbms for remote reporting and Excel access.
mysql> grant all on .* to ”@’%’ identified by ”;
where , and are replaced. Eg
mysql> grant all on app_production.* to ‘ror’@'%’ identified by ‘bigsecret’;
The % means from any IP address. Change it to a specific IP address if appropriate. Run additional grant commands to enable additional addresses. More help: see the MySQL docs for grant.
4) Change the MySQL config file to enable remote access:
Change /etc/mysql/my.conf by commenting out the line: bind-address = 127.0.0.1
5) Restart MySQL or reboot your AMI instance.
# reboot
6) On your development machine, tell Amazon that you want to change your EC2 security profile to enable remote access to the MySQL port, 3306
ec2-authorize default -p 3306 # changing the ‘default’ security profile. Use the one that you chose when you instantiated your AMI. You can specify which IP addresses or range of addresses will be granted access, see the Amazon docs.
7) That’s it, you can now remotely access your MySQL dbms from any remote SQL tool including Excel, MySQL Query Browser, etc. Watch out for people running queries that slow down your server! You can allow access only to stored procedures if that becomes an issue.
HTH,
Larry
25 July 2007, 8:13 amJaime:
Paul, thanks for the good work and great write up. This image is great place to start.
One thing to note is that the sample deploy.rb script doesn’t work with Capistrano v2.0. Several issues, for example namespace issues (e.g. “deploy”), and cold deploy (which calls migrate) before bootstrapping mysql.
These are not too difficult to work around, just thought it should be noted.
Another one is that (don’t know if this is a cap2 thing) is that it tries to start it with a script/spin, which I surmise is not what is wanted. So what exactly is the proper way to get mongrel_cluster supposed to be started, “cap start_mongrel_cluster”?
Thanks!
25 July 2007, 9:41 pmHon Dev:
Hi Paul thanks for the great image.
I haven’t used it for Rails stuff its just awesome to have a great Ubuntu Public Image set up available on ec2.
26 July 2007, 5:56 pmScott:
For some reason, I can’t install the mysql gem. Any ideas?
27 July 2007, 6:49 pmPaul Dowman:
Larry: Thanks! I think a wiki is a good idea, I’ll work towards getting that set up.
Jaime: I’m going to redo the sample recipes for Cap2 ASAP.
Scott: You don’t need to install the mysql gem. The libmysql-ruby ubuntu package is installed, which provides the same thing. I installed that (and rmagick) using the ubuntu package management instead of using rubygems so that they will stay in sync with the binary packages that they work with.
1 August 2007, 8:09 pmBrandon:
Is there something that is preventing PHP from being installed or working?
I’m setting up a server for a client who currently uses PHP heavily, but who I’m planning to do Rails work for, so I need something that can run both. Of course I’d love it if you added PHP to your image (it would allow PHPMySQL, etc. and there are lots of legacy apps that use PHP that lots of people are dependent on!) but barring that it would be great if there were an easy way to install it.
I installed Apache/MySQL/PHP on a Fedora 7 instance the other night and everything went incredibly smoothly. Just issue the yum commands and everything is taken care of. I figured it would be fairly similar on Ubuntu, just using apt-get instead (and a handy translation page found here) but when I try installing, I get messages that it failed or is already installed:
root@ec2-72-44-49-234:/etc/init.d# apt-get install php5
Reading package lists… Done
Building dependency tree
Reading state information… Done
php5 is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
root@ec2-72-44-49-234:/etc/init.d# apt-get install php
Reading package lists… Done
Building dependency tree
Reading state information… Done
E: Couldn’t find package php
root@ec2-72-44-49-234:/etc/init.d#
when I check to see if it is actually installed which php or which php5 I get no response. Any idea what’s going on? I’d prefer to start with your image so I can more easily start moving to Rails later.
3 August 2007, 12:37 amPaul Dowman:
Brandon: PHP5 should be installed and enabled, though I haven’t tried it. (BTW, I recommend ‘aptitude’ instead of ‘apt-get’).
3 August 2007, 9:18 pmBret Holstein:
Hi Paul,
Terrific AMI! My only problem is figuring out how you mounted the ‘/mnt/app’ (for the web folder) and ‘/mnt/log’ file systems. I am guessing that they are stored on S3.
A little help here and I’m all set. Thanks!
Bret
7 August 2007, 1:07 amOrlin Monad:
Have you tried elasticdrive.com for persistent storage? With this approach MySQL can write directly to an S3 block device - as if it is local. It would eliminate the need (drawback) to backup databases (every 10 minutes)… ElasticDrive is free for the time being. They say to have no plans for selling it in the near future. If the price is reasonable, I’d personally pay for it. Maybe one could choose either the current (backup) way or else turn on ElasticDrive with a configuration option / script?
Orlin
8 August 2007, 12:10 amAlex N:
Thanks for this! Probably saved me a lot of time. I have never installed a mongrel cluster before and got it working.
One additional change I had to make was to change the socket for production in database.yml to /var/run/mysqld/mysqld.sock
Others may not have capistrano installed so you might want to include that in your require gems on the development machine.
What is the preferred way to restart the mongrels without redeploying?
21 August 2007, 6:26 pmAlex N:
Ah, found it. It’s in the ~/aws/bin. For other’s sake:
~/aws/bin# ./mongrel_cluster_ctl_wrapper restart
To restart Mongrel instances. The other interesting scripts are there to.
21 August 2007, 8:29 pmAlex N:
Rebundle worked for me, but it left a ton of image.part.DD files and img-mnt in /mnt.
I think it should probably clean up after itself.
21 August 2007, 9:19 pmAlex N:
Sorry for so many comments, but I’m using it a lot :)
Just a warning to others - although the mongrel start script is in root’s home path, these scripts should NOT be run as root. This also applies to all other start scripts. You will need to chown or delete your log files and/or the indexes if you do this.
Perhaps these scripts should be in app’s home directory?
22 August 2007, 2:42 amPaul Dowman:
Alex: Thanks for the feedback. The recommended way to start/stop/restart mongrel is using /etc/init.d/mongrel. It ensures that mongrel_cluster_ctl runs as the app user. This is the script that is run automatically on startup and shutdown.
You raise a good point about the rebundle script cleaning up after itself. It just runs the Amazon tools which leave the image part files, but it might be nice to delete those after.
22 August 2007, 5:39 amEugen Martin:
First - thanks for the great job: your work is even MORE than most of open-source folks claim to have in mind!
Actually I am working on a clone of your image with Swiftiply + Mongel(in eventmode)+ Capistrano 2.0 + Capazon. It will be able to run multiple instances of the same Image in different roles and on the fly add/remove app-server instances. I think it will take some weeks, till it’s production ready..
I keep you posted if you are interested.
Greetz from Berlin
29 August 2007, 1:19 pmPaul Dowman:
Eugen: That’s great, I’m very interested to hear how it goes, and if you’re interested I’d love to work together on it.
30 August 2007, 7:58 pmEugen Martin:
Hey Paul!
I have been actually testing the performance of Swiftiply and some app servers full of mongrels - dynamically starting + stopping app servers and mongrels without balancer reconfiguration.
My first goal was to find out IF it is possible to add app-server instances under load. The result is: it IS possible. I really liked Swiftiply + swiftiplied_mogrels: good performance and easy to set up (though I lost some time then found + fixed a bug in the Swiftiply 0.6.1-gem which did not allow distributed mongrels connect to an other host than localhost). I think I will make some more tests with NginX + Swiftiply.
Though I must say: I am a bit dissapointed of the EC2-processor performance:
I tried to outperform a single hardware server (Opteron 1210, Dual 1,8 Ghz, 1GB RAM) with a bunch of EC2-Instances. I needed at least 4 of them to have a comparable performance (dynamic Rails requests, without DB load). If I benchmark requests with an excessive DB usage - the results are even more dissappointing. If you are interested, I can post some more details… Did you do any performance tests? Maybe its our application, that is so CPU-intensive….?
My further goal would be to write cap/capazon tasks to start/stop multiple app-server-instances all dynamically connecting to a given running load-balancer + DB-server. And another bunch of tasks setting up the DB-server + load balancer - which is not THAAAT complicated…
Unfortunately my company has decided NOT to take EC2 as primary deployment platform, because of the relatively high costs and the poor performance: but we might use EC2 as a supplementary and/or backup solution.
Therefore I think I will not work on the EC2-specific deployment part part any longer. But I will let you know, if I have any new details…
3 September 2007, 10:38 amJaime:
One big downside of not giving the app user sudo access is that how do you do things (such as overwriting image root files, etc.) that require root access? Paul suggests doing rsync, but then this is bypassing all the Capistrano goodness of roles, and ends up violating DRY-ness.
But try as I might i couldn’t get Capistrano to selectively with doing some things as one user (”app”) and some other things as another (”root”).
Is there a good way of doing this?
3 September 2007, 1:31 pmPaul Dowman:
Jaime: I don’t know how to get Capistrano to do different tasks as different users, but you can call rsync from a Capistrano task and have rsync log in as root, something like:
rsync -rlvzcC –rsh=”ssh -l root -i privkey” files/ “#{hostname}:/”
I know it’s a bit of a workaround but you could also have a Capistrano task that does something like this:
ssh -l root@hostname adduser app admin
Which will give the app user sudoers access, this only needs to be done once.
I admit these are workarounds. I’m open to further discussion about it, join the mailing list and bring this topic up there.
4 September 2007, 8:37 pmbeng:
why are gems in the non standard place ?
normal for ubuntu is here: /usr/lib/ruby/gems/1.8/gems/
how did you install rmagick, again “normal” is via: gem install -y rmagick
don’t get me wrong, love it, but just wished somethings were different…
you should have a base feisty up there with your bundling scripts, so others can have a going at rail enabling…
thanks for all the work
8 September 2007, 11:52 ambeng:
how to add this gem…
gem install -y libxml-ruby
with my “normal” vmware installation, that just works
8 September 2007, 11:59 amPaul Dowman:
beng:
Actually the normal location for gems on Ubuntu (when rubygems is installed using the Ubuntu/Debian package management) is /var/lib/gems/1.8/gems, you can verify that by looking at the list of files for the libgems-ruby1.8 package.
rmagick is installed using the Ubuntu/Debian package management system (which is also quite “normal”), and the reason it’s better to install it that way rather than using “gem install” is that it relies on native libraries (libmagick), and this way they will be kept in sync.
I’m not sure what you mean by “you should have a base feisty up there with your bundling scripts”.
8 September 2007, 12:23 pmbeng:
Thanks ! I have setup rails so many times following input from all over and gems always lands here /usr/lib/ruby/gems/1.8/gems/, but looks like your right !
Also, trying out the rmagick stuff now.
For the base feisty, there are quite a few feisty images out there, so never know which one to start with. I’d like a trustworthy feisty that
- feisty server
- bundling script
- tasksel install lamp
pretty much a snapshot of what you have but at an earlier stage of building up.
But now that you’ve explained my concerns, I’ll just stick with this…
still want/need my libxml-ruby gem though !!!!!!!!!!!!!
8 September 2007, 1:47 pmBrandon Z:
You’re right Paul, PHP is enabled. It does seem to be having problems. Some of the issues I’m having could be related to other things, and so may not be worth mentioning, but there are two things that seem clear indications of *something* wrong, even if these two things are not big deals on their own–they just seem like possible indications of deeper problems.
which php returns nothing.
php –version doesn’t give any info either.
and yet PHP pages are being processed (at least when specifically addressed with a full URL including the page name)… if anyone has any ideas I’m all ears.
12 September 2007, 7:47 pmPaul Dowman:
Brandon: The php command-line interpreter isn’t installed, only the php apache module is installed. You need to install the package php5-cli. “aptitude install php5-cli”.
Let me know what other unexplained behaviour you’re seeing and I’ll see if I can find an explanation for you! :-)
12 September 2007, 8:55 pmJeff:
This is great. Thanks for providing this.
4 October 2007, 1:17 amDan Farina:
Another vote to release a decent stand-alone “bare” Feisty (or Gutsy pretty soon…) image. Right now there aren’t any obviously good ones and I have quasi-stripped down your image + rebundled to fill this need. I’m extremely happy that the rebundling works so easily and that I get to use my GNU/Linux flavor of choice!
8 October 2007, 4:12 amFrederico Araujo:
Nice work. I wish I knew it before… :)
I wrote some nginx tasks
ec2onrails:server:nginx_start/stop
ec2onrails:server:nginx_configure
nginx to frontend 10 mongrel clusters
ps: there is a fair mode nginx version.
it uses round robin technique instead.
http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel
source:
28 November 2007, 6:32 amhttp://git.localdomain.pl/?p=nginx.git;a=tree;hb=upstream_fair-0.6
Martin Rehfeld:
Now that Amazon has annouced SimpleDB, I think it’s time to add that to the equation, some thoughts on http://inside.glnetworks.de/2007/12/15/amazon-simpledb-web-service-complementing-the-ec2-compute-cloud/
The Rails deployment stack just made a big leap ahead!
16 December 2007, 2:57 amdave:
Paul, thanks for the image and for making it public!
Very cool stuff ..
4 March 2008, 1:01 amSoren Burkhart:
Great work… this is awesome!
23 March 2008, 4:13 amYan:
Very cool! Just wanted to let everyone know we’ve launched a service (totally free and no signup required to try) that lets you custom configure a stack (whether rails or anything else) and build it to a variety of virtualization formats including vmware, xen, parallels, and of course Amazon EC2 - you can build and launch your stack into the cloud with just a couple clicks. Check it out: Elastic Server On Demand (http://es.cohesiveft.com). Feedback very welcome.
16 April 2008, 1:48 pm