Just a quick note – I’m in the process of publishing a book, Clojure Recipes. You can see it for presale on Amazon here.
Introduction
This post shows you how to setup a Cassandra Cluster from scratch on MacOS Mavericks using Docker running on Vagrant. This is non-trivial and if everything goes well, will probably take an hour.
Note that this are my notes from going through this process, so there will definitely be room for improvement, either a simpler way to do things or a better way to explain what is going on.
Assumptions
- This has been written for a Macbook Pro Retina running MacOS X 10.9.3 (Mavericks) with 16GB of RAM. You’ll need to make a judgment as to whether your setup is similar enough for these steps to apply to your system.
Outline
This has the following steps:
A. Install Homebrew
B. Install brew-cask
C. Install Vagrant
D. Set up a Vagrant instance running Ubuntu
E. Install Docker on Vagrant
F. Clone docker-cassandra on Vagrant
G. Customise the docker-cassandra instance
H. Start the Cassandra cluster
I. Test from your mac
J. Installing Cassandra on the mac
Process Steps
A. Install Homebrew
1. Run the following command to install Homebrew if you don’t already have it installed:
$ ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
B. Install brew-cask
2. Run the following commands to install brew-cask
$ brew tap caskroom/cask
$ brew install brew-cask
C. Install vagrant
3. Run the following command to install vagrant:
brew cask install vagrant
D. Set up a Vagrant instance running Ubuntu
4. To select a Ubuntu Image for Vagrant – run the following:
vagrant box add ubuntu-server-14.04 https://oss-binaries.phusionpassenger.com/vagrant/boxes/latest/ubuntu-14.04-amd64-vbox.box
vagrant init ubuntu-server-14.04
5. To configure networking for Vagrant – modify the file in /home/users/ to add at the end (after the End):
Vagrant::Config.run do |config|
config.vm.forward_port 9160, 9160
config.vm.forward_port 9042, 9042
end
(Note that docker-cassandra only supports forwarded ports – it doesn’t support vagrant bridges)
6. Start the Vagrant Ubuntu instance:
vagrant up
7. Test the vagrant instance by connecting:
vagrant ssh
You are now shelled into the vagrant terminal.
E. Install Docker on Vagrant
8. Run the following to install docker
sudo apt-get update
sudo apt-get install docker.io
sudo ln -sf /usr/bin/docker.io /usr/local/bin/docker
sudo sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io
9. Test docker works with
sudo docker run -i -t ubuntu /bin/bash
You are now shelled into the docker terminal
10. Exit from docker to vagrant
exit
F. Clone docker-cassandra on Vagrant
11. Install git on the vagrant instance (Vagrant terminal)
apt-get install git
12. Install nano on the vagrant instance (Vagrant terminal)
apt-get install nano
13. Clone docker-cassandra:
git clone https://github.com/nicolasff/docker-cassandra.git
G. Customise the docker-cassandra instance
14. Modify the docker install scripts to allow external password authentication
nano docker-cassandra/install/bin/install-cassandra
15. Add the lines
sed -i -e 's/authenticator: AllowAllAuthenticator/authenticator: PasswordAuthenticator/g' $CONFIG
sed -i -e 's/authorizer: AllowAllAuthorizer/authorizer: CassandraAuthorizer/g' $CONFIG
H. Start the Cassandra cluster
16. Create the docker image (vagrant terminal)
make image VERSION=2.0.3
17. Confirm the image has been created with
./list-images.sh
You should get
2.0.3
18. Start 3 nodes on the cluster with: (vagrant terminal)
./start-cluster.sh 2.0.3 3
19. List the nodes running: (vagrant terminal)
sudo docker ps
You should get a result similar to:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1b8062746694 cassandra:2.0.3 /usr/bin/start-cassa 11 minutes ago Up 11 minutes 9042/tcp, 9160/tcp sleepy_darwin
365c4df8fc65 cassandra:2.0.3 /usr/bin/start-cassa 11 minutes ago Up 11 minutes 9042/tcp, 9160/tcp angry_fermat
c36f05d87f9b cassandra:2.0.3 /usr/bin/start-cassa 11 minutes ago Up 11 minutes 0.0.0.0:9042->9042/tcp, 0.0.0.0:9160->9160/tcp determined_franklin
20. Check the setup of the nodes:
./client.sh 2.0.3 nodetool -h cass1 status
You should get a result similar to:
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 192.168.100.3 125.64 KB 256 33.4% 07d65d03-9e95-402c-8ebc-aad77fc22aef rack1
UN 192.168.100.2 116.96 KB 256 34.3% 1e525775-1393-49af-8e95-c7527a6b0e73 rack1
UN 192.168.100.1 114.91 KB 256 32.2% 5c5f239c-60b0-4960-8a89-21a3aafb13d7 rack1
I. Test from your mac
21. On a mac terminal run:
$ telnet 192.168.1.17 9160
This should give:
Trying 192.168.1.17...
Connected to 192.168.1.17.
Escape character is '^]'.
Then control-C to escape
^CConnection closed by foreign host.
This showed it worked.
J. Installing Cassandra on the mac
22. Ensure you have Java installed by running:
java -version
This should give the result similar to:
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b40)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b55, mixed mode)
If you don’t have Java installed – you can get it from here.
23. Download Cassandra from here to your Downloads directory.
24. From a terminal in the apache-cassandra-2.0.8 directory – run
csqlsh -u cassandra -p cassandra
You should get:
Connected to Test Cluster at localhost:9160.
[cqlsh 4.1.1 | Cassandra 2.0.3 | CQL spec 3.1.1 | Thrift protocol 19.38.0]
Use HELP for help.
cqlsh>
Congratulations – you have installed a Cassandra Cluster on your Mac using Vangrant and Docker.
Perfect timing, thank you! A couple of typos, I can’t recall them all, but one that sticks out at me in recent memory is the spelling of “csqlsh” at the end of the walkthrough.
Step H – precede with cd into docker-cassandra.
Step 18 gets:
vagrant@ubuntu-14:~/docker-cassandra$ ./start-cluster.sh 2.0.3 3
Starting node 1
sudo: pipework: command not found
Starting node 2
sudo: pipework: command not found
Starting node 3
sudo: pipework: command not found
But still appears to work.
Created /usr/bin/pipework, also needed apt-get install arping
vagrant@ubuntu-14:~/docker-cassandra$ ./client.sh 2.1.0 nodetool -h cass1 statussudo: pipework: command not found
fails, can you pls help
installed pipework – $ sudo cp install/bin/pipework /usr/bin/
But now, the instances are auto killed as soon as they are created, I dont know how.
vagrant@ubuntu-14:~/docker-cassandra$ ./start-cluster.sh 2.1.0 1
Starting node 1
vagrant@ubuntu-14:~/docker-cassandra$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c500700912f0 cassandra:2.1.0 /usr/bin/start-cassa 6 seconds ago Up 4 seconds 0.0.0.0:9042->9042/tcp, 0.0.0.0:9160->9160/tcp grave_thompson
vagrant@ubuntu-14:~/docker-cassandra$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
this works like a charm, but one question arises.
Why vagrant box? why not just use docker running on osx?
Docker is ultimately about Linux networking containers, and whilst you can use tools other than Vagrant, you never get away from the Linux virtual machine. http://stackoverflow.com/questions/17547112/why-cant-you-install-docker-natively-in-osx
Vagrant is probably the method with the best documentation and the most support. It’s a battle-tested, well-known path.