Just a quick note – I’m in the process of publishing a book, Clojure Recipes. You can see it for presale on Amazon here.

Introduction

This post shows you how to setup a Cassandra Cluster from scratch on MacOS Mavericks using Virtualbox VMs. This is non-trivial and if everything goes well, will probably take a couple of hours.

Note that this are my notes from going through this process, so there will definitely be room for improvement, either a simpler way to do things or a better way to explain what is going on.

Assumptions

• This has been written for a Macbook Pro Retina running MacOS X 10.9.3 (Mavericks) with 16GB of RAM. You’ll need to make a judgment as to whether your setup is similar enough for these steps to apply to your system.
• VirtualBox uses the language ‘guest’ to describe the client virtual machine running. This is not to be confused with the ‘guest login’ (which we won’t use anyway)

Pre-requisites

•  you need at least 9G free on your hard drive
•  you need at least 2.5G free RAM (running three virtual machines)

Outline

This has the following steps:

A. Downloads
B. Installing VirtualBox on a Mac
C. Getting Started with VirtualBox
D. Setting Up Ubuntu
E. Setting Up Java
F. Installing Cassandra
G. Run and Check the Cassandra Server
H. Duplicate the Virtual Machine for the Cluster
I. Setup Cluster Networking
J. Setting Up Cassandra

 

Process Steps

A. Downloads

1. Download VirtualBox – https://www.virtualbox.org/wiki/Downloads

2. Download The VirtualBox Extension Pack (Same URL as above)

3. Download a VirtualBox instance for Ubuntu 12.10

http://virtualboxes.org/images/ubuntu/#ubuntu1304

B. Installing VirtualBox on a Mac

4. Install VirtualBox (VirtualBox-4.3.6-91406-OSX.dmg)

5. Run VirtualBox to ensure it installed correctly, then exit the application.

6. Double-click the VirtualBox extension pack you downloaded earlier – follow the prompts to install the pack in Virtual Box

7. Expand the Ubuntu VirtualBox instance you downloaded earlier

C. Getting Started with VirtualBox

8. In VirtualBox start a new Machine with Machine -> Add and select the VirtualBox instance you downloaded earlier

9. Start the machine

10. If prompted for the Virtual Media Manager – remove the two mounted .iso virtual drives as you no longer have access

11. When logging in – use the password reverse for the ubuntu login, unless otherwise instructed at step 3

D. Setting up Ubuntu

12. Select the large cog in the bottom left hand corner for System Settings. Select Keyboard layout (not Keyboard). Use the plus button to add your keyboard, and the minus button to deselect Italian.

13. From the cog in the top right hand corner – select ‘about this computer’ – then wait for the ‘checking for updates’ button to change to ‘install updates’ – then install the updates. This may require a restart and re-login of the virtual machine

14. From the top left hand corner select Dash Home then Terminal

15. Update the keyboard from Italian to your local setting at the server level using the command

sudo dpkg-reconfigure keyboard-configuration

16. Install the virtual box guest additions on the ubuntu guest virtual instance

sudo apt-get install virtualbox-guest-additions

17. In your web browser, log onto the datomic site at http://my.datomic.com/ – request your keys under Account and download Datomic Pro from the Download tab. Also download the license key.

E. Setting Up Java

18. Install Java with the following commands:

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer


19. Check the installation with

java -version

(Ensure that you get Java 1.7 and not an OpenJDK variant)

20. Put JAVA_HOME into the environment using the following commands:

sudo nano /etc/environment
add the line:
JAVA_HOME=/usr/lib/jvm/java-7-oracle

21. Restart the machine, login and open a new terminal

sudo shutdown -r now

22. run the command

echo $JAVA_HOME

Ensure you get

/usr/lib/jvm/java-7-oracle

as a response.

F. Installing Cassandra

23. Locate the URL of the latest (post 2.0) install of Cassandra here http://cassandra.apache.org/download/

24. Create a Cassandra directory

sudo mkdir /home/cassandra

25. In a terminal window run

wget http://mirror.ventraip.net.au/apache/cassandra/2.0.3/apache-cassandra-2.0.3-bin.tar.gz
tar -xvzf apache-cassandra-2.0.3-bin.tar.gz
mv apache-casandra-2.0.3 /home/cassandra


26. Set up the folders:

sudo mkdir /var/lib/cassandra
sudo mkdir /var/log/cassandra
sudo chown -R $USER:$GROUP /var/lib/cassandra
sudo chown -R $USER:$GROUP /var/log/cassandra


27. Modify the environment

sudo nano /etc/environment
CASSANDRA_HOME=/home/cassandra/apache-cassandra-2.0.3

#add this to the end of the existing path variable – don’t add a new path entry.

PATH=…:$CASSANDRA_HOME/bin

28. Restart the virtual machine, login and start a terminal

sudo shutdown -r now

29. Check the settings with

echo %CASSANDRA_HOME

This should point to the directory just created.

G. Run and Check the Cassandra Server

30. Run

sudo sh /home/cassandra/apache-cassandra-2.0.3/bin/cassandra

31. Start a new terminal and run

sudo sh /home/cassandra/apache-cassandra-2.0.3/bin/cassandra-cli

Ensure that you get Connected to “Test Cluster” in the output

32. Exit the cli by running

exit;

33. In the other window, stop Cassandra by running

CTRL-C

If that doesn’t work – try this.

34. Clear the Cassandra Data

sudo rm -rf /var/lib/cassandra/*

35. Close and save the Virtual machine in VirtualBox

H. Duplicate the Virtual Machine for the Cluster

36. Right click on the virtual machine in the VirtualBox Manager and select Clone. Then ensure the checkbox Reinitialize the MAC address is checked. After this check Full Clone. You have now created the second node in the cluster.

37. Repeat this for the third node in the cluster.

I. Setup Cluster Networking

38. In Virtualbox select the menu VirtualBox -> Preferences -> Network -> Host Only Networks and select the plus button to create a new host-only network (take the default network name – probably vboxnet0)

39. Edit this setting and take note of of the values for
(a) Adapter-IPv4 Address
(b) DHCP Server – Server Address
(c) DHCP – Server Mask
(d) Lower address bound
(e) upper address bound

40. In the VirtualBox manager screen, control-click the second node and select Settings -> Network -> Adapter 1. Then change the drop down list value to Host Only Adapter. Then ensure that the name matches the network name set before (vboxnet0 for example).

41. In the terminal – run

ifconfig

and take note of the following values

(f) inet addr
(g) bcast
(h) mask

42. Now using terminal and nano – edit the /etc/network/interfaces file

sudo nano /etc/network/interfaces

and add the following:

iface  eth0 inet static
address 192.168.56.2
# i.e. below the dhcp range
netmask 255.255.255.0
network 192.168.56.1
broadcast 192.168.56.255
gateway 192.168.56.0

Ensure that you leave the entries for lo loopback

43. Now edit the file

sudo nano /etc/udev/rules.d/70-persistent-net.rules

to comment out all rows but the last one (and its comment) and change the eth# (eg eth1) to eth0 (at the end of the row)

44. Now restart the virtual machine:

sudo shutdown -r now

45. Confirm this has worked by ensuring your IP address you just set up comes up under eth0 when you run

ifconfig

If there is an eth1 entry – something has gone wrong – redo the steps above.

J. Setting Up Cassandra

46. Note the following Node Numbers

Node 0: 0
Node 1: 3074457345618258602
Node 2: 6148914691236517205


These were generated from here https://dl.dropboxusercontent.com/u/30184176/digitalocean/tokenCalc/index.html

47. On node 0 – edit the Cassandra settings file

nano /home/cassandra/apache-cassandra-2.0.3/conf/cassandra.yaml

and set the following settings:

cluster_name: ‘VirtualBox Cluster’

initial_token: 0

seed_provider:
- seeds:  "192.168.56.2"


listen_address: 192.168.56.2

rpc_address: 0.0.0.0

endpoint_snitch: RackInferringSnitch

48. On node 1 – edit the Cassandra settings file

nano /home/cassandra/apache-cassandra-2.0.3/conf/cassandra.yaml

and set the following settings:

cluster_name: ‘VirtualBox Cluster’

initial_token: 3074457345618258602

seed_provider:
- seeds:  "192.168.56.2"

listen_address: 192.168.56.3

rpc_address: 0.0.0.0

endpoint_snitch: RackInferringSnitch

49. On node 2 – edit the Cassandra settings file

nano /home/cassandra/apache-cassandra-2.0.3/conf/cassandra.yaml

and set the following settings:

cluster_name: ‘VirtualBox Cluster’

initial_token: 6148914691236517205

seed_provider:
- seeds:  "192.168.56.4"

listen_address: 192.168.56.4

rpc_address: 0.0.0.0

endpoint_snitch: RackInferringSnitch

50. Now start Cassandra on the first node (the seed node) using:

sudo sh /home/cassandra/apache-cassandra-2.0.3/bin/cassandra

(note that this will run as a background process – so if you hit enter after waiting for 30 seconds, you’ll get your command prompt back)

51. Now repeat this on each of the two secondary servers

sudo sh /home/cassandra/apache-cassandra-2.0.3/bin/cassandra

52. Fire up cqlsh and run a test:

/home/cassandra/apache-cassandra-2.0.3/bin/cqlsh -u cassandra -p cassandra

53. Run through the CQL commands here

References:

• https://www.virtualbox.org/wiki/Downloads
• https://www.virtualbox.org/manual/ch01.html#intro-installing
• http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html
  
• http://cassandra.apache.org/download/
• http://superuser.com/questions/357120/how-do-i-setup-a-virtualbox-server-with-a-static-ip
• https://www.digitalocean.com/community/articles/how-to-install-cassandra-and-run-a-single-node-cluster-on-a-ubuntu-vps
• https://help.ubuntu.com/10.04/serverguide/network-configuration.html
• http://draalin.com/setting-up-a-static-ip-address-in-ubuntu/
• http://askubuntu.com/questions/197628/how-do-i-find-my-network-ip-netmask-and-gateway-info
• https://forums.virtualbox.org/viewtopic.php?f=7&t=43090
• https://www.digitalocean.com/community/articles/how-to-configure-a-multi-node-cluster-with-cassandra-on-a-ubuntu-vps