Almost 5 month ago AWS exposed a new service – “Elasticsearch service”. So we can now have a managed elasticsearch cluster and we can have Kibana on it without the hassle of managing it. But then we need to work with it and discover the bleeding edges –
- Old version – current Elasticsearch version is 2.2.0 and 1.7.* is maintained. Elasticsearch version in AWS is 1.5.2. It is 8 months old version, it is ages in this industry. Kibana current version is 4.4.0 while Kibana version offered on AWS is 4.0.3. Many enhancements, bug fixes, breaking changes and AWS is behind. This is specially interesting comparing to the recent velocity on EMR images. As there are some breaking changes between 1.* versions and 2.* I would at least expected the option to choose a version. Even more annoying there is currently no data about when and how AWS going to upgrade the versions.
- Plugins not supported – adding plugins is relevant for expanding Elasticsearch usability, monitoring and for adding customized Kibana visualisations (e.g for KPI’s). I believe every company which uses Elasticsearch will have this need in some point. AWS currently support only a close set of plugins and don’t enable installing your own plugin and there is no data about future plans.
- Scripting not supported – scripting is relevant for adding data, processing data, interactions between fields, etc. Again a need that I believes that raises in every company after a while. This features also is not enabled in AWS Elasticsearch Service. There are ways to by-pass some of it but why work hard if Elastic already created this functionality? Here too, not clear if there are any future plans to enable it.
- No VPC support – this means that the Elasticsearch endpoint is publicly-accessible. There are few solutions but it is a security issues and more over it does not comply with the other AWS products logic.
- Backup issues – no easy way to export data and to control how frequent and when your data is backed up. AWS cloud plugin for Elasticsearch can solved it, but it is not supported..
Conclusion – Elasticsearch is a very strong tool and even stronger when combined with Kibana. On the other hand, AWS usually produce very good products.But this time, the combination of those two is not there yet. It seems to me like they might jumped to the water too early (possibly in order to compete with Found \ Elastic Cloud) with a product which is not mature enough.
Most of the services AWS offer are based on their own innovation and \ or development and are not coupled so strong with external product (the other product I can think of is elasticcache which I have not worked with yet). From a business point of view, it is weird for me to see such a dependency on external product. If there is a high demand for Elasticsearch on AWS I would prefer to have a tutorial explaining how to set a cluster rather than a doubtful product.