You can manage a Citus cluster manually on EC2 instances using CloudFormation. The CloudFormation template for Citus enables users to start a Citus cluster on AWS in just a few clicks, with also cstore_fdw extension for columnar storage is pre-installed. The template automates the installation and configuration process so that the users don’t need to do any extra configuration steps while installing Citus.
Please ensure that you have an active AWS account and an Amazon EC2 key pair before proceeding with the next steps.
CloudFormation lets you create a “stack” of AWS resources, such as EC2 instances and security groups, from a template defined in a JSON file. You can create multiple stacks from the same template without conflict, as long as they have unique stack names.
Below, we explain in detail the steps required to setup a multi-node Citus cluster on AWS.
1. Start a Citus cluster
To begin, you can start a Citus cluster using CloudFormation by clicking here. This will take you directly to the AWS CloudFormation console.
You might need to login to AWS at this step if you aren’t already logged in.
2. Select Citus template
You will see select template screen. Citus template is already selected, just click Next.
3. Fill the form with details about your cluster
In the form, pick a unique name for your stack. You can customize your cluster setup by setting availability zones, number of workers and the instance types. You also need to fill in the AWS keypair which you will use to access the cluster.
Please ensure that you choose unique names for all your clusters. Otherwise, the cluster creation may fail with the error “Template_name template already created”.
If you want to launch a cluster in a region other than US East, you can update the region in the top-right corner of the AWS console as shown below.
4. Acknowledge IAM capabilities
The next screen displays Tags and a few advanced options. For simplicity, we use the default options and click Next.
Finally, you need to acknowledge IAM capabilities, which give the coordinator node limited access to the EC2 APIs to obtain the list of worker IPs. Your AWS account needs to have IAM access to perform this step. After ticking the checkbox, you can click on Create.
5. Cluster launching
After the above steps, you will be redirected to the CloudFormation console. You can click the refresh button on the top-right to view your stack. In about 10 minutes, stack creation will complete and the hostname of the coordinator node will appear in the Outputs tab.
Sometimes, you might not see the outputs tab on your screen by default. In that case, you should click on “restore” from the menu on the bottom right of your screen.
You can use the cloudformation console shown above to monitor your cluster.
If something goes wrong during set-up, the stack will be rolled back but not deleted. In that case, you can either use a different stack name or delete the old stack before creating a new one with the same name.
6. Login to the cluster
Once the cluster creation completes, you can immediately connect to the coordinator node using SSH with username ec2-user and the keypair you filled in. For example:
ssh -i your-keypair.pem email@example.com
7. Ready to use the cluster
At this step, you have completed the installation process and are ready to use the Citus cluster. You can now login to the coordinator node and start executing commands. The command below, when run in the psql shell, should output the worker nodes mentioned in the pg_dist_node.
/usr/pgsql-9.6/bin/psql -h localhost -d postgres select * from master_get_active_worker_nodes();
8. Cluster notes
The template automatically tunes the system configuration for Citus and sets up RAID on the SSD drives where appropriate, making it a great starting point even for production systems.
The database and its configuration files are stored in /data/base. So, to change any configuration parameters, you need to update the postgresql.conf file at /data/base/postgresql.conf.
Similarly to restart the database, you can use the command:
/usr/pgsql-9.6/bin/pg_ctl -D /data/base -l logfile restart
You typically want to avoid making changes to resources created by CloudFormation, such as terminating EC2 instances. To shut the cluster down, you can simply delete the stack in the CloudFormation console.