{"id":495601,"date":"2023-08-31T14:32:41","date_gmt":"2023-08-31T12:32:41","guid":{"rendered":"https:\/\/www.devoteam.com\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/"},"modified":"2023-08-31T14:32:41","modified_gmt":"2023-08-31T12:32:41","slug":"reviving-a-struggling-elastic-cluster-for-fixami","status":"publish","type":"success-story","link":"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/","title":{"rendered":"Reviving a struggling Elastic cluster for Fixami"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">In today&#8217;s data-driven landscape, efficient data retrieval is the backbone of success. Imagine a situation where a company&#8217;s <a href=\"https:\/\/nl.devoteam.com\/local-partner\/elastic\/\">Elastic<\/a> cluster, which used to work smoothly, suddenly starts running very slowly. Searches take forever and finding important documents becomes a real challenge. This customer success story is about a consulting session that turned things around, fixing the struggling Elastic cluster and making document searches seamless again.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-about-fixami\">About Fixami<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Fixami is an e-commerce organization focused on selling power tools to professionals and home handymen. Their headquarters and adjacent warehouse is located in a new building, with top facilities, in Tilburg.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">They have been one of the fastest growing companies in the south of the Netherlands for over five years. Continuous focus on innovation and optimization ensures that Fixami continues to grow and is partly responsible for the transition from &#8216;bricks to clicks&#8217; of the tool market.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-figuring-out-the-issue\">Figuring out the issue<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">When Fixami came to us, they were dealing with a big problem. Their Elastic cluster, which was the heart of their data system, was not performing well. Searches that used to be fast were now incredibly slow and they couldn&#8217;t find the documents needed. It was clear that something was seriously wrong and the client was getting frustrated.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-the-issues\">The issues<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The first step in reviving the Elastic cluster was to figure out what was causing the slowdown. We began by looking at the Fixami&#8217;s system setup, cluster configuration, and use of data. Here are the main problems we found:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Hardware and Resources<\/strong>: The system wasn&#8217;t using its resources efficiently. This was causing the slow performance. The memory and processing power allocated to the cluster didn&#8217;t match the workload, creating bottlenecks. It was hosted on GCP and using Kubernetes (ECK) setup.&nbsp;<\/li>\n\n\n\n<li><strong>Cluster size and Data Volume<\/strong>: The cluster size was quite large for the volume of data entering the cluster.&nbsp;<\/li>\n\n\n\n<li><strong>Shard Overload<\/strong>: The number of shards in the cluster was much higher than the number of nodes. This imbalance was causing a strain on the system and hampering performance.<\/li>\n\n\n\n<li><strong>Node Configuration<\/strong>: The Elastic nodes, the individual parts of the cluster, were not set up properly. This was causing them to not communicate efficiently, slowing down the entire system. Node roles were set to Default i.e. all roles were enabled on all the nodes.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-the-solutions-to-get-back-on-track\">The solutions to get back on track<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To fix these issues, we had to make some changes to how the cluster was set up and how it worked. Below are the main things Devoteam did to make the Elastic cluster work better.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-text-color\" id=\"h-making-better-use-of-the-resources\" style=\"color:#f8485e\">Making better use of the resources<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">We recommend giving the cluster more memory and processing power. This would help it handle searches faster and more efficiently.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-text-color\" id=\"h-node-configuration\" style=\"color:#f8485e\">Node configuration<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">We reconfigured the Elastic nodes so that they communicated more efficiently. Updated existing nodes to create 1 dedicated Master node and other remaining 2 nodes as Data and Master node. Removed ML, Coordinator and other irrelevant node roles.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-text-color\" id=\"h-balancing-shards\" style=\"color:#f8485e\">Balancing shards<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">We adjusted the number of shards in the cluster to match the number of nodes. This balanced setup eased the strain on the system and improved performance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The number of shards you should allocate in an Elasticsearch cluster depends on various factors, including your data volume, use case, hardware resources, and anticipated growth. However, a general guideline is to aim for an even distribution of shards across your data nodes while considering factors like hardware capacity and query performance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here&#8217;s a simple formula to calculate the number of primary shards based on the number of data nodes:<\/p>\n\n\n\n<p class=\"has-text-color wp-block-paragraph\" style=\"color:#f8485e\"><strong>Number of Primary Shards = Total Number of Data Nodes * Desired Shards per Node<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, if you have 3 data nodes and you want to allocate 1 shard per node:<\/p>\n\n\n\n<p class=\"has-text-color wp-block-paragraph\" style=\"color:#f8485e\"><strong>Number of Primary Shards = 3 * 1 = 3<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This approach aims to evenly distribute the data and workload across the nodes. Each primary shard is stored on a separate node, allowing Elasticsearch to parallelize search and indexing operations.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-result\">Result<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">With these changes in place, the Elastic cluster&#8217;s performance improved significantly. Slow searches are no longer a problem, and finding documents became quick and easy.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-elastic-documentation-references\">Elastic documentation references<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Our solutions were in line with the recommendations given in Elastic&#8217;s official documentation.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Elastic Documentation: <a href=\"https:\/\/www.elastic.co\/guide\/index.html\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/www.elastic.co\/guide\/index.html<\/a><\/li>\n\n\n\n<li>Hardware Optimization: <a href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/hardware.html\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/hardware.html<\/a><\/li>\n\n\n\n<li>Index Configuration: <a href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/index-modules.html\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/index-modules.html<\/a><\/li>\n\n\n\n<li>Shard Management: <a href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/shards.html\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/shards.html<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>In today&#8217;s data-driven landscape, efficient data retrieval is the backbone of success. Imagine a situation where a company&#8217;s Elastic cluster, which used to work smoothly, suddenly starts running very slowly. Searches take forever and finding important documents becomes a real challenge. This customer success story is about a consulting session that turned things around, fixing [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","categories":[],"tags":[3262,3263,2717],"industry":[],"class_list":["post-495601","success-story","type-success-story","status-publish","hentry","tag-elastic-en-nl","tag-eslatic-en-nl","tag-netherlands-en-nl"],"acf":[],"cards":"\n\t<div class=\"single-post-card\">\n\n\t\t\n\n\t\t\n\t\t<div class=\"wp-block-group is-vertical is-layout-flex wp-container-core-group-is-layout-43282307 wp-block-group-is-layout-flex\">\n\t<p style=\"font-style:normal;font-weight:700\" class=\"has-link-color wp-elements-1 wp-block-lp-post-type has-text-color has-primary-color has-small-font-size\">Success story<\/p>\n\n\t\t\n\t\t<h3 style=\"font-style:normal;font-weight:400\" class=\"wp-block-post-title has-base-font-size\"><a href=\"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/\" target=\"_self\" >Reviving a struggling Elastic cluster for Fixami<\/a><\/h3><\/div>\n\t\t\n\t<\/div>\n\n","yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v28.4 (Yoast SEO v28.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Reviving a struggling Elastic cluster for Fixami | Devoteam<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Reviving a struggling Elastic cluster for Fixami\" \/>\n<meta property=\"og:description\" content=\"In today&#8217;s data-driven landscape, efficient data retrieval is the backbone of success. Imagine a situation where a company&#8217;s Elastic cluster, which used to work smoothly, suddenly starts running very slowly. Searches take forever and finding important documents becomes a real challenge. This customer success story is about a consulting session that turned things around, fixing [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/\" \/>\n<meta property=\"og:site_name\" content=\"Devoteam\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/success-story\\\/reviving-a-struggling-elastic-cluster-for-fixami\\\/\",\"url\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/success-story\\\/reviving-a-struggling-elastic-cluster-for-fixami\\\/\",\"name\":\"Reviving a struggling Elastic cluster for Fixami | Devoteam\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/#website\"},\"datePublished\":\"2023-08-31T12:32:41+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/success-story\\\/reviving-a-struggling-elastic-cluster-for-fixami\\\/#breadcrumb\"},\"inLanguage\":\"en-NL\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/success-story\\\/reviving-a-struggling-elastic-cluster-for-fixami\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/success-story\\\/reviving-a-struggling-elastic-cluster-for-fixami\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Success stories\",\"item\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/success-story\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Reviving a struggling Elastic cluster for Fixami\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/#website\",\"url\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/\",\"name\":\"Devoteam\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-NL\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Reviving a struggling Elastic cluster for Fixami | Devoteam","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/","og_locale":"en_US","og_type":"article","og_title":"Reviving a struggling Elastic cluster for Fixami","og_description":"In today&#8217;s data-driven landscape, efficient data retrieval is the backbone of success. Imagine a situation where a company&#8217;s Elastic cluster, which used to work smoothly, suddenly starts running very slowly. Searches take forever and finding important documents becomes a real challenge. This customer success story is about a consulting session that turned things around, fixing [&hellip;]","og_url":"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/","og_site_name":"Devoteam","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/","url":"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/","name":"Reviving a struggling Elastic cluster for Fixami | Devoteam","isPartOf":{"@id":"https:\/\/devoteam.info\/en-nl\/#website"},"datePublished":"2023-08-31T12:32:41+00:00","breadcrumb":{"@id":"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/#breadcrumb"},"inLanguage":"en-NL","potentialAction":[{"@type":"ReadAction","target":["https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/devoteam.info\/en-nl\/success-story\/reviving-a-struggling-elastic-cluster-for-fixami\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/devoteam.info\/en-nl\/"},{"@type":"ListItem","position":2,"name":"Success stories","item":"https:\/\/devoteam.info\/en-nl\/success-story\/"},{"@type":"ListItem","position":3,"name":"Reviving a struggling Elastic cluster for Fixami"}]},{"@type":"WebSite","@id":"https:\/\/devoteam.info\/en-nl\/#website","url":"https:\/\/devoteam.info\/en-nl\/","name":"Devoteam","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/devoteam.info\/en-nl\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-NL"}]}},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"Julien Pawlowski","author_link":"https:\/\/devoteam.info\/en-nl\/author\/"},"uagb_comment_info":0,"uagb_excerpt":"In today&#8217;s data-driven landscape, efficient data retrieval is the backbone of success. Imagine a situation where a company&#8217;s Elastic cluster, which used to work smoothly, suddenly starts running very slowly. Searches take forever and finding important documents becomes a real challenge. This customer success story is about a consulting session that turned things around, fixing&hellip;","_links":{"self":[{"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/success-story\/495601","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/success-story"}],"about":[{"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/types\/success-story"}],"version-history":[{"count":0,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/success-story\/495601\/revisions"}],"wp:attachment":[{"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/media?parent=495601"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/categories?post=495601"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/tags?post=495601"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/industry?post=495601"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}