Elasticsearch and Kibana 5+¶
Elasticsearch and Kibana 5 brought huge new features and capabilities to the Elastic Stack that have mostly carried forward to 6 as well, but with them a number of modifications to the ObjectRocket for Elasticsearch offering.
Ingest Nodes¶
One of the biggest new features in Elasticsearch 5 was the new ingest node. The ingest node role is similar to Logstash in that it’s meant to operate as a preprocessor to manipulate and transform data before it enters an Elasticsearch index. Along with this functionality, comes new functions, plugins, and scripting requirements.
Ingest Nodes on the ObjectRocket Platform¶
Currently, client nodes will share double-duty as both client and ingest nodes on the ObjectRocket platform. The dedicated master and data nodes’ roles will not change. This will give you four nodes to use as you wish for search and ingest traffic. Here are a few options on how you can use them on the ObjectRocket platform:
- Don’t use ingest at all: You can use the four provided client addresses for standard Elasticsearch traffic just as you did before. You don’t have to modify any settings or configuration. The ingest functionality will still be there when you need it.
- Spread search and ingest across all nodes: You can still point your application at all client nodes just like earlier Elasticsearch versions and also send ingest traffic to one or all of them at the same time. This is easy to set up and requires minimal planning, but be aware that the additional load of both ingest and search traffic could slow down client access during load spikes.
- Isolate search traffic and ingest traffic to different client nodes: In this scenario, you only point your application to a subset of the cluster’s client nodes for search traffic and use the remaining client nodes for ingest traffic. You’ll need to plan this out in how you to connect from your application and remember how you’re using each node, but this adds the benefit of isolating a high ingest workload from impacting the performance of search and vice-versa.
This is the current architecture, but we’re flexible. If you have any special needs or requirements, feel free to contact our support team.
Ingest Plugins¶
There are also a new set of plugins available specifically for ingest nodes. By default we will install:
- ingest-attachment
- ingest-user-agent
- ingest-geoip
If you need anything beyond the default plugins, feel free to contact our support team if there are any you’d like added to your ingest nodes.
Ingest scripts¶
The ingest nodes also support their own scripts, which can be inline, stored, or file scripts. As with the rest of the cluster, inline and stored scripts are limited to sandboxed languages, but file based scripts can use the full list of supported languages. However, we don’t allow users to upload their own scripts, so all requests for script uploads must go through our support team.
Site Plugins¶
Elasticsearch no longer directly supports site plugins with version 5.0 and beyond, but we’ve been able to continue offering them on the ObjectRocket platform. The biggest change with 5.0 was to remove the ability to run the site from the elasticsearch host itself, but since we run multiple dedicated containers for each Elasticsearch cluster, we were able to move the site portion of the plugin to another host.
The site plugins will still be behind your cluster’s firewall, so you’ll need to add ACLs and the appropriate users to access them. The “ACL Role” that will control access to the site plugins is the “Elasticsearch” role.
For the full list of plugins that we support, see the features page.