Scaling guide

In an Elasticsearch instance for the ObjectRocket Dedicated platform, the data is stored in dedicated data nodes that can be scaled up or out. Choosing to scale out (add data nodes), or scale up (change plan size) depends on the current state of the cluster and your specific needs.

Adding data nodes is the easiest way to grow your cluster. For example, if you’re using the 8 GB plan size, each of your data nodes has 8 GB of storage. Whenever you need additional capacity, you can simply add a new data node, which adds another 8 GB of capacity. On the ObjectRocket platform, data nodes can be automatically added with RocketScale ® technology, or manually via the UI.

If you already have a lot of data nodes, have large shards, or an application where fewer, larger nodes provide a performance advantage, then a plan resize might be better. A plan resize takes every data node in your cluster and moves it to a different plan size (such as from 8 GB to 16 GB) and then adds or removes data nodes as necessary. To request a plan resize, send a ticket to support. ObjectRocket can usually complete a plan resize without downtime.

Adding a data node

A new instance initially has two data nodes. You can more add more data nodes after creating the new instance.

To add more data nodes, use the following steps:

  1. Navigate to the Elasticsearch instance in the Control Panel.
  2. Select Data Nodes.
  3. Click Add Data Node and follow the prompts to add a new data node.

Additional charges apply. Review the pricing page for details.

_images/add_datanode.png

Removing a data node

To ensure that data nodes are always removed safely, remove a data node by submitting a support ticket.

RocketScale

RocketScale ® is an ObjectRocket technology that automatically adds data nodes as necessary. This service monitors the space utilization of each data node,

and when it detects a node above a pre-set threshold for a set amount of time, it adds a data node to ensure your cluster doesn’t run out of space.

By default, all clusters have this feature enabled and add a new node when storage usage stays above 87% for 10 minutes. Some exceptions are addressed below.

You can disable or modify the thresholds at any time. However, disabling this feature might affect ObjectRocket’s ability to guarantee that your cluster never goes red due to a lack of capacity.

How to configure RocketScale

  1. In the ObjectRocket UI, select **RocketScale ® ** for the instance in question.
_images/esrocketscalebutton.png

You have the option to:

  • Enable or disable RocketScale on the instance.
    • Disabling Rocketscale stops the ObjectRocket service from automatically adding data nodes when needed.
  • Set the RocketScale Disk Usage Threshold for the instance.
    • This threshold determines the level of storage utilization that must be exceeded on a data node before RocketScale activates.
    • The default is 87%, but you can adjust this setting based on your needs. Setting this threshold above 90% might put your cluster at risk of going red as it fills up.
    • This setting is limited to a range of 70-95%.
  • Set advanced Rocketscale™ settings.
    • Rocketscale has extra built-in settings that let you control the behavior of Rocketscale for your instance.
    • Number of Minutes in Violation (default = 10 minutes, range = 1 - 1000): This number is the window of time for which the utlization must be above the threshold before RocketScale adds a new data node.
    • Max Node Limit (default = 12, range = 3 - 20): The maximum number of nodes that your cluster can have before Rocketscale stops adding more. Even if you cross the Disk Usage Threshold, RocketScale doesn’t add any more data nodes after your cluster has reached this size.
    • Cooldown Time (default = 600 seconds, range = 300 - 10000): This number is the amount of time in seconds that the service waits between checks of node storage utilization.
_images/esrocketscalewadvanced.png

Conditions that prevent RocketScale from adding nodes

RocketScale ® is designed to stop adding nodes under the following conditions:

  • If it determines that adding data nodes does’t help reduce utilization.
  • When a cluster is in a bad state.
  • When a plan resize might be a better option.

Currently, the conditions that stop RocketScale from adding a node are:

  • RocketScale is disabled: If RocketScale is disabled, it does not attempt to add nodes. The support team might still add a node if your cluster fills up and is in danger of failing.
  • Cluster has more data nodes than shards: Elasticsearch can’t split a shard, so any data nodes beyond the number of total shards isn’t helpful, and Rocketscal™ stops adding nodes.
  • Cluster has more nodes than the max node limit setting: In most scenarios, clusters with more than 12 data nodes function more efficiently with a larger plan size and less data nodes. However, you can modify this setting for larger clusters where more data nodes are needed.
  • Cluster has nodes with low utilization: If your cluster has nodes with less than 40% utilization, but you also have a node with high utilization, it might be a sign that something is wrong with the cluster that requires the attention of the Support team. If RocketScale detects this scenario, it turns scaling off for the instance and notifies the Support team to look at your cluster.