We used to use ElasticSearch a lot for log aggregations. Its a beast of its own. you still need a dedicated team to handle this. Eventually moved to Splunk, wavefront like solutions. It ll costs a lot lesser and frees up engineering time to build a better product.
Genuine questions as someone with no experience in dealing with large sclae log aggregation: Can you share some details on what kind of issues you ran into in production with Elastic Search that needed a dedicated team to manage ?
related to this: anyone can share practical advice when it comes to running Elasticsearch for application and infrastructure log aggregation?
We started from using Elastic Cloud which is nice enough and saves us the time to initial configuration. However I'm still unsure if the choices I made are the right way to go when designing the indexes.
