HTML5, CSS3, jQuery, JSON, Responsive Design...

Setting up Domino on Amazon’s EC2 cloud servers

Michael Brown   March 18 2010 01:45:00 PM
A tutorial for how to set up a Domino server on Amazon's Elastic Cloud Computing (EC2) infrastructure.  Note that much of this is relevant to other types of application server too.  In fact, you can find a lot of pre-configured server images - known as AMIs in the EC2 world - for different types of application server.  But there's none for Domino yet.  These are coming though.  See Ed Brill's post for details.


Basics

I've chosen Linux as my server because:
  1. It's cheaper to run on EC2 than the equivalent Windows server
  2. Windows is a poor operating system in just about every regard that I can think of.  Windows' only (dubious) advantage is that there's a lot of so-called administrators around that claim to know how to run it properly.  (They don't.  Nobody does.  Nobody can, in fact.)  Thankfully, I've no need of any Windows server "expertise" for this particular project.
I'm going to pick Fedora 8 for my Linux server 8, merely because it's the the default Linux OS when you're setting up a new EC2 server.  Why Amazon has picked such a old version of Fedora for this, I've no idea, but they have.  Fedora in any version is not a supported platform for Domino, but I managed to get it run okay.


Before You Start

You will need:
  1. The Domino server installer for Linux.  I'm using 8.5.  Don't ask me where to get this from, let alone to post the code somewhere for you.
  2. A Notes Administrator client to do the Domino server post-setup configuration.  I'm using version 8.5, but an earlier version should suffice.  The Notes Administrator only runs in Windows, of course, so you'll need that too.  I ran XP in VirtualBox VM on a Ubuntu host for this purpose.  You'll also need to have the Remote Server Setup option installed.  This will appear as a separate option in your Lotus Applications menu folder in Windows.  If you don't see it there then you'll have to re-run the Notes installer and select it.  It's not installed by default.  (That's the Notes/Designer/Admin client installer.)
  3. Your credit card.  Sorry, EC2 servers are not free.  If you're an Amazon customer already - is there anybody that's not? - then they likely have your cc details already.  You can find EC2 pricing at http://aws.amazon.com/ec2/pricing/ and you'll immediately see how Linux is cheaper: 8.5 cents per hour Vs 12 cents per hour for Windows (that's US cents).  That's for the smallest on-demand instance, which is all we'll need here.
  4. A Linux workstation with SSH installed.  Most Linux workstation distributions will already have SSH installed.  If not, issue "sudo apt-get install openssh-client" (without the quotes) into a terminal to install  it (or "yum install" if you're using an RPM-based distribution).

If you want to use Windows as your workstation for this then time to get real.  The mechanisms for interacting with EC2 are heavily slanted towards *NIX-based systems and involve getting your hands dirty with the command line.  The server that I'm setting up in this example has no GUI, so you're not going to be pushing any shiny Windows 7 buttons to do anything with it.  Okay, if you really want to use Windows then you'll need to install Putty, which gives you SSH on Windows.  That's all the help you're getting on the Windows side though because, to be honest, that's about all I know!  (That's the last of the Windows bashing for this article, I promise!)



Signing Up

Get thee to http://aws.amazon.com/ec2/ and click the Sign up to Amazon EC2 button.  It should not take too long, especially if you're already an Amazon customer.  You will need to flash your credit card if Amazon's not already holding info on one for you already.

Once you've gone through the registration process, you'll need to link on the Sign in to the AWS Management Console link.  Here's how it looks

Main EC2 Console Screen

Or here's how it looks for me  I should say.   All the numbers on the My Resources section at the right hand side will read as zero for you because you've not set anything up yet!


Setting up an Instance
  1. To launch a new instance, you need to click on the (wait for it) Launch Instance button.  This will kick of the Request Instances Wizard, displayed below:

    Request EC2 Instance

    See what I mean about Fedora 8?  Oh yeah, that other OS is there too.  We don't need anything special on our Fedora server, so pick "Basic Fedora Core 8" and click its corresponding Select button.

  2. Take the defaults on the Instance Details dialog that comes up next, then click Continue.  Do the same for the second Instance Details dialog.

  3. On the Create Key Pair dialog that comes up next, click on the Create Key Pair radio button, if it's not already selected.  The name will default to "ec2-keypair" so let's take that and then click on the "Create & Download your Key Pair" link.  You'll be prompted for a save location; remember where you put it!!  And keep it safe.  Put a back up on another drive somewhere, such as a USB key.  If you lose this key then you'll be unable to access any instances that are associated with it!

  4. Create a new security group.  You should open up the following ports in order to access a Domino server:

    • SSH - Port 22
    • HTTP - Port 80
    • HTTPS - Port 443 (only necessary if you're using SSL)
    • Notes Client access - Port 1352
    • Notes remote setup access - Port 8585

    At this point, however, you'll only be able to enter the first three.  (SSH should be already done for you, in fact.)  This is because there's no "Custom..." option in the dialog.  We'll have to add the two Notes protocols later on.

  5. Click Continue to go to the Review dialog.  Here you can change anything that you need to, but you shouldn't need too.  Click on the Launch button and you're off!

Congratulations, your new instance should be up and running in a minute or two.  Warning: the clock is ticking against your credit card from this point onwards.


Post Config 1 - Finish off your security group

While you're waiting for your instance to come up, you can add the two Notes protocols to your security groups:
  1. Back in the AWS console, click on Security Groups on the left hand side. then click on your security group from the list.
  2. Drag the middle window up (just like it was a previewed email in Lotus Notes) so you can see it properly.  Notice that the drop-down on the left now has the "Custom..." option, which wasn't there when we were setting up on the Security Group in the Setup Wizard.
  3. First we'll do the Notes client protocol.  Change the the Protocol to "TCP" and both the From and To Ports to 1352.  The Source IP field should be set to 0.0.0.0/0, as it is on the three protocols that you already have.  When every field is completed, the Save button should become enabled, so go ahead and click it.
  4. Now we'll do the Notes Remote Setup protocol.  Repeat the process in the step above, only this time set your From and To Ports to 8585.  Click on the Refresh button at the top if not all of your protocols show in the list.  It should look as below:

    EC2 Security Groups

That's your security setup.  Time to connect to your instances.


Post Config 2 - Set permissions on your key pair file

SSH won't connect to your instances if the permissions on key file are not set correctly.  So, open a terminal console and type in the following:

chmod 400 /pathtoyourkeyfile/ec2_keypair.pem


Replace "pathtoyourkeyfile" with the ermm... path to your files!

If you don't do this step then you'll get the error "WARNING: UNPROTECTED PRIVATE KEY FILE!" when you try to SSH into your instance (see next) and your connection will be dropped.



SSH into your instance

Still in your terminal console, enter the code to connect to your instances.  The syntax will look like something like this:

ssh -i /pathtoyourkeyfile/ec2-keypair.pem root@ec2-174-129-63-93.compute-1.amazonaws.com


To get the exact string to connect to your instances is easy.  Find your instance in the instances list in the AWS Console and right-click on it, then pick "Connect" from the pop-up menu.  A dialog will appear containing the connection string for your instance.  Simply copy and paste it into an terminal console to make the connection.  Note though that the connection string that copy assumes that you've already changed (cd) to your keyfile's folder in the terminal before pasting in the string.  If you've not done that, then you need to prepend to the keyfile's path to its name, as I did in my example above.

When you hit enter into the console, you should see a message about "the authenticity of host" and the "RSA key fingerprint", and you'll be asked if you want to continue.  Just type "yes" (no "y" on its own) and hit enter.  You should now have connected to your instance.



Setting up an Elastic IP address for your instance

I was hoping to explain this part later as an optional (but very useful) tip.  But it seems that you need to assign an Elastic IP address to your instance in order to complete the Domino server's setup, so I'll explain it now.

Elastic IPs are a way of assigning apparently fixed IP addresses to your instances.  I say "apparently fixed" because you can assign them to different instances on the fly.  These re-assignments take effect almost immediately, and with no need to reconfigure any DNS anywhere.  You get five of them for free with your Amazon EC2 account.

To assign one to your instance:
  1. Log into the Amazon AWS Console and then click on Elastic IPs on the left hand side.
  2. Click on the Allocate New Address button at the top.  This should add a new IP address in the list.  (Click on the Refresh button if it doesn't.)
  3. Right-click on the IP address in the list and pick "Associate Address" from the pop-up.
  4. A dialog should appear with a drop-down field that shows you a list of all your running instance IDs.  Since you should only have one running at this point, select that one and click the Associate button.

You're done.  Painless, wasn't it?

You can now use the Elastic IP to refer to your instance in place of the full EC2 Public DNS address that we've been doing up to now.  E.g.:

ssh -i /pathtoyourkeyfile/ec2-keypair.pem root@1.2.3.4

assuming that 1.2.3.4 is your assigned Elastic IP address.

Note that when SSHing into your instance with the Elastic IP address, you'll get prompted about the "RSA key fingerprint" again.  Just answer "yes" when the prompt appears.




Setting up the Notes user on your instance

Initially, you'll only be to connect to your instance as the root user (hence "root@") because that's the only user that's setup by default.  You don't want to be running your Domino server as root though, so time set up a new user called "notes".

While connected to your instance, enter the following:

useradd notes
passwd notes


This will create a new user called "notes" and then prompt your to enter a password for it.

You still can't connect to your instance as "notes" until you've set up server-side SSH files for it.  While still connected to your instances as root, enter the following:

mkdir /home/notes/.ssh
cp /root/.ssh/authorized_keys /home/notes/.ssh


That's copied over the SSH keys to the correct place for the "notes" but the ownership of that folder and its files are still set to "root".   Typing the code below will make the "notes" user the owner of its files:

chown -R notes:notes /home/notes/.ssh


You can test it now.  Fire up another terminal console, and paste in your connection string again.  Before hitting enter though, replace "root@" with "notes@", so you have something like this:

ssh -i /pathtoyourkeyfile/ec2-keypair.pem notes@1.2.3.4




Getting the Domino Installer file onto your instance

Before you can install Domino, you need to get the Domino installer file on it.  We'll use SSHFS for this.  SSHFS is the file transfer equivalent of SSH.  It's a bit like FTP, only far more secure.  To install it on a Debian-based Linux system, such as Ubuntu, you'll need to type:

sudo apt-get install sshfs

into a local terminal console window to install it.

Mac OS X users will need to install the free Fuse for OS X package.  You'll also need their latest SSHFS package.

Also, before you can attach to your instance, you need to create a folder on your local workstation where you're going to mount the instance.  In a (local) terminal console, type the following:

mkdir ~/mountec2


That will create a folder called "mountec2" as a sub-folder of your current local user's home folder.  (The tilde character is a substitute for "/home/yourusername".)  Below is the syntax that I used to mount my instance's /home/notes folder at the new folder that I just created on my workstation:

sshfs -o IdentityFile=~/Desktop/ec2/ec2-keypair.pem root@1.2.3.4:/home/notes ~/mountec2


The ~/mountec2 folder on your workstation is now mapped to the /home/notes folder on your instances.  All you need to now is open the ~/mountec2 folder in Nautilus or Dolphin (or whatever your preferred file manger is) and drag the Domino installer file over to it.  Copying should now begin and it's time to put on a DVD... a long one.  On my ADSL2+ line, the Domino installer took two hours to copy to my instance.  That's from Australia though.  Your kilometrage may vary.



Installing Domino

While SSHed into your instance as the root user, change directory (cd) to the /home/notes folder and enter:

tar -xf *.tar


to extract the files from the .tar file installer.  When it's done - it will take a minute - type in the following to start the installer:

cd domino/linux
./install

and let the install routine do its thing.  You should take all the defaults that the installer offers.  It will take about five minutes to run though.  (See Installing Domino on Fedora Linux for more details on the running through the Domino installer's questions and answers.)



Configuring the Domino server

You'll need to use Windows (ugh!) for this next bit.  You'll need to be able to SSH into your instance at the same time.  This might be a bit awkward if you've only one machine to work on.  You'll either have to do everything Windows, using Putty for your SSH needs, or you can run Windows inside a Virtual Machine (which is my own preference).

Within Windows, launch the Remote Server Setup application, which is under the Lotus Applications menu group in Windows.  (Refer to the Before You Start section at the top of this article if you can't find the Remote Server Setup app.)  When the dialog pops up, paste in the Elastic IP address of your EC2 instance, but don't hit OK just yet.  You should leave it looking like so:

Domino's Remote Admin Setup tool

Now SSH into your instance as the notes users (that's "notes@" and not "root@").  We need to launch the Domino server in "listen mode", for which we'll need to append the parameter -listen.  Before that though, we'll need to switch into Domino's data folder, otherwise it will complain that it cannot find the notes.ini file.  Still logged into your instance as the notes user, type in the following:

cd /local/notesdata
/opt/ibm/lotus/bin/server -listen


The Domino server is now in listening mode.

Switch back to your Remote Serverr Setup app in Windows and hit the OK button.  All being well, the Server Setup routine should start.  I hope you've thought of a name for your Domain, Organisation and your first server!

When you get to the dialog that says "Make optional copies of ID files", make sure that you tick the checkbox of the bottom.  You can then save all your IDs to your local Windows machine or VM, rather than having to hunt around on your server instance for them.

When you get to the last dialog, answer "Yes" when it asks you if the server listener should be stopped.

You'll need to use your Elastic IP address when accessing your Domino server afterwards.




Starting and stopping your Domino server - Screen

You can start your Domino server from the instance's command line, as we did earlier.  Only now, we don't need to use the -listen parameter to start it in "listen mode", i.e.:

cd /local/notesdata
/opt/ibm/lotus/bin/server


When you start the Domino server in this way, you'll soon notice a problem though: whenever you logout of your SSH session, the Domino server stops!!! And you can't get around this problem by leaving yourself logged into it, even if that were desirable (which it isn't).  I tried several ways around this, including Daniel Nashed's startup script, but I could not persuade it to work on Fedora 8.  What did work is Screen.

Screen is a program that allows you launch programs in the background, which is what we want to do with our Domino server.  It's not installed by default on Fedora 8.  To install it, you need to SSH into your instances as the root user and then type:

yum install screen


(If you were using a Debian-based server, that would be "sudo apt-get install screen".)

Now open another local terminal window and SSH into your instance as the notes user.  (We don't want to launch Domino as the root user, remember.)  This is how to launch Domino using Screen:

screen
cd /local/notesdata
/opt/ibm/lotus/bin/server


This launches the Domino server but in a different Screen session.  To return to your original terminal session, simply hit CTRL->A then D on your keyboard.  You can now logout of your SSH session without shutting down the Domino server.

You can use the Notes Admin client, including the web version thereof, to stop your Domino server.  The syntax for the web version is 1.2.3.4/webadmin.nsf where 1.2.3.4 is your Elastic IP address.


Congrats!  You now have your very own Domino server in the cloud.  Enjoy.  Just remember that it's costing you money for every hour it's up there!

Comments

1Sean Cull  03/17/2010 5:27:12 PM  Setting up Domino on Amazon’s EC2 cloud servers

has anyone looked at hosting Domino in the Rackspace cloud ?

I suspect licensing is an issue but it does seem to have some advantages over EC2 although I am going by the marketing materials rather than any hard experience

{ Link }

Sean

2Mike Brown  03/17/2010 5:57:47 PM  Setting up Domino on Amazon’s EC2 cloud servers

@ Sean,

Never heard of them before, but I just looked at their site.

They certainly look cheaper for their smaller instances, but man, are they *small*!!! They start at 256 Meg of RAM and only 10 Gig of hard drive. Amazon's smallest instance has 1.7 Gig of RAM. By the time you scale up Rackspace to Amazon's equivalent sizes, the costs are also comparable.

A big Rackspace advantage - reading their marketing blurb - is the ability to scale up servers on the fly. EC2 doesn't do this. Instead, you're expected to scale up by adding extra instances and then splitting your load between them somehow.

3Sean Cull  03/17/2010 6:32:52 PM  Setting up Domino on Amazon’s EC2 cloud servers

I came across them as the hosts for the Posterous blogging platform.

Robert Scoble works for them too for ehat thats worth

4Mark Myers  03/17/2010 8:03:19 PM  Setting up Domino on Amazon’s EC2 cloud servers

very useful article, thanks (will go and use it now)

5Mike Brown  03/30/2010 9:01:13 PM  Setting up Domino on Amazon’s EC2 cloud servers

Since writing the article above, a Domino AMI image has made its appearance on Amazon EC2, see:

{ Link }

I've checked my own EC2 AWS Console and can confirm that the AMI image *is* there.

I've not tried to use it though. I will do so soon and then either update this article and/or write a new one.

6Sean Cull  04/01/2010 2:18:12 AM  Setting up Domino on Amazon’s EC2 cloud servers

Hi Michael, thanks for the inspiration and the guidance, I have blogged about setting up a persistent instance on ubuntu here

{ Link }

Sean

7Art  05/13/2013 11:13:27 PM  Setting up Domino on Amazon’s EC2 cloud servers

Great stuff, thanks! When I go to setup my server, the remote server setup connects OK, I enter all the info, but when I click Setup, it pauses for several seconds, the progress bar goes to 1% and then it puts up the error "Network communications error".

I don't see any errors reported in setuplog.txt on the server and the server did not crash. I tried enabling ALL TCP ports and disabling my firewall on Windows, same problem.

So it certainly connected to the server OK at the start, but somehow has a communications error along the way.

Did you ever run into this, and if so, any ideas? Thanks!

About