Private Datacenter

Citrusleaf is designed for ease of deployment and creating your own deployment is very simple.  A Citrus leaf server node has the following requirements:

• A 64-bit Intel or AMD processor
• Physical memory of 2G or more
• Gigabit Ethernet
• Linux distribution
• An interface open in the cluster's multicast IP domain
• Raw devices running on directly attached Intel X25-E SSD devices (optional).

A minimum cluster size of three server nodes is recommended for a production system.  Nodes communicate amongst themselves through a TCP-based internal cluster interconnect, which runs on top of IP (Gigabit Ethernet).  Note that you can preserve and leverage your existing datacenter investment by installing Citrusleaf on existing machines.

Once you have identified the hardware, it is a simple three step process to deploy: Install, Configure and Startup.

Install

Installing a Citrusleaf cluster is simple and involves installing just three packages on each server node in the cluster:

• citrusleaf-server - contains the server code
• citrusleaf-tools - contains administrative tools, and
• libcitrusleaf-python - contains the Python library required for running Citrusleaf admin tools.

Here is a sample output from running the installation script:

# dpkg -i citrusleaf-server*.deb citrusleaf-tools*.deb libcitrusleaf-python*.deb
(Reading database ... 36941 files and directories currently installed.)
Preparing to replace citrusleaf-server 27319c359b64f9036930ae55e277da10c43822e1-2 (using citrusleaf-server_27319c359b64f9036930ae55e277da10c43822e1-2_amd64.deb) ...
Adding group citrusleaf
Adding user citrusleaf
Password changed.
Unpacking citrusleaf-server ...
Preparing to replace citrusleaf-tools 27319c359b64f9036930ae55e277da10c43822e1-2 (using citrusleaf-tools_27319c359b64f9036930ae55e277da10c43822e1-2_amd64.deb) ...
Unpacking citrusleaf-tools ...
Preparing to replace libcitrusleaf-python 27319c359b64f9036930ae55e277da10c43822e1-2 (using libcitrusleaf-python_27319c359b64f9036930ae55e277da10c43822e1-2_amd64.deb) ...
Unpacking libcitrusleaf-python ...
Setting up citrusleaf-server (27319c359b64f9036930ae55e277da10c43822e1-2) ...

Setting up citrusleaf-tools (27319c359b64f9036930ae55e277da10c43822e1-2) ...
Setting up libcitrusleaf-python (27319c359b64f9036930ae55e277da10c43822e1-2) ...


Back to top

Configure

Before initial startup, Citrusleaf needs to be configured with settings for the service, logging, network, and namespaces.

Service

The service section allows settings to enable the server to run in one of three modes: as a specific user, as a specific group, as a daemon.

Any change to service configuration requires a server restart.

Logging

Every message produced by the server has a ''context'', which names the part of the code that generated the message, and a 'severity', which describes how acute the error is.  The severities are as follows:

Logs can be sent to upto 8 different output locations each location governed by one or more filters. For example,  here is a configuration that records all warning messages plus all info messages from the migration sub-system:

file /var/log/aerospike.log {
            context any warning
            context migrate info
}

Network

There are three configuration settings needed for the network, the heartbeat address, the fabric address, and the service address.

The heartbeat address is a multicast address and must be configured the same for all nodes in the same cluster. The heartbeat address (and port) is used for discovery of new members added to the cluster.

The fabric address (and port) is used by nodes within the cluster to directly communicate with each other.

The service address (and port) is used by clients to communicate with a node in the cluster. The service can be forced to reuse socket addresses by setting the option reuse-address.

Here is an example network configuration:

network {
            service {
                        address any
                        port 3000
                        reuse-address
            }
            heartbeat {
                        address 239.1.1.1
                        port 2999
            }
            fabric {
                        address any
                        port 4000
            }
}


Namespaces

Namespaces are policy containers for data and each namespace has a mandatory parameter (replication-factor) that determines how many concurrent copies of each data item will be maintained for that namespace. 

Storage for namespace can be defined as a collection of raw devices or as a directory on a filesystem. Here is an example setting for a namespace called Content that runs with two concurrent copies using a raw device:

namespace Content {

        storage-engine files {
                blocksize 4096
                device /dev/sdb
                device /dev/sdc
        }
        replication-factor 2
}


Citrusleaf is certified to run with Intel X25-E enterprise-class solid-state drives.

Back to top

Startup

The first server in a new cluster needs to undergo the ignition procedure. Here is an example of cluster ignition:

nodeA# cld --config-file /etc/citrusleaf/citrusleaf.conf --ignition
Apr 23 2009 17:42:17: INFO (as): aerospike starting
Apr 23 2009 17:42:17: INFO (as): CLUSTER IGNITION ENABLED
Apr 23 2009 17:42:17: INFO (as): initializing services...
Apr 23 2009 17:42:17: INFO (paxos): Paxos service ignited: bb69c1a78152200
Apr 23 2009 17:42:19: INFO (nsup): namespace supervisor started
Apr 23 2009 17:42:19: INFO (as): service ready: soon there will be cake!


Once the first server is up,  bringing up all subsequent nodes is straightforward (and identical):

nodeB# cld --config-file /etc/citrusleaf/citrusleaf.conf
Apr 23 2009 17:44:53: INFO (as): aerospike starting
Apr 23 2009 17:44:53: INFO (as): initializing services...
Apr 23 2009 17:44:55: INFO (paxos): waiting for cluster membership...
Apr 23 2009 17:44:55: INFO (paxos): received state synchronization message
Apr 23 2009 17:44:55: INFO (paxos): SUCCESSION [1.0]*: bb69c1a78152200 bb6d2ce4c81e000
Apr 23 2009 17:44:55: INFO (nsup): namespace supervisor started
Apr 23 2009 17:44:55: INFO (as): service ready: soon there will be cake!

That’s it! Note that the cluster is completely self-organizing.  When a node joins or leaves the cluster, the other nodes will describe the change:

Apr 23 2009 17:43:33: INFO (paxos): inserting node bb6d2ce4c81e000
Apr 23 2009 17:43:33: INFO (paxos): SUCCESSION [1.0]: bb69c1a78152200 bb6d2ce4c81e000


Once the first machine of the cluster comes up, Citrusleaf database is open for business and clients can start accessing the service even as the other servers come up.

Back to top