{"id":514392,"date":"2020-11-26T14:32:55","date_gmt":"2020-11-26T13:32:55","guid":{"rendered":"https:\/\/www.devoteam.com\/expert-view\/azure-data-lake-grow-fearless-part-2\/"},"modified":"2020-11-26T14:32:55","modified_gmt":"2020-11-26T13:32:55","slug":"azure-data-lake-grow-fearless-part-2","status":"publish","type":"expert-view","link":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/","title":{"rendered":"Azure Data Lake: Grow Fearless &#8211; Part 2"},"content":{"rendered":"<p>&nbsp;<\/p>\n<p><strong>In my previous blog \u201c<a href=\"https:\/\/nl.devoteam.com\/en\/blog-post\/azure-data-lake-grow-fearless-part-1\/\" target=\"_blank\" rel=\"noopener noreferrer\">Azure Data Lake: Grow Fearless \u2013 Part 1<\/a>\u201d I described the business benefits of data growth, but also the challenging parts of fast-growing data. In this blog, I\u2019ll dive deeper and show you the promising parts of Azure Data Lake and why it&#8217;s worth considering.<\/strong><\/p>\n<h2>Why should you consider Azure Data Lake?<\/h2>\n<h4><em>Scalability<\/em><\/h4>\n<ul>\n<li>Unlimited storage empowers rapid business growth.<\/li>\n<li>Easy-to-scale prepare business for any big leap in development.<\/li>\n<\/ul>\n<h4><em>Flexibility<\/em><\/h4>\n<ul>\n<li>ADLS Gen2 can adapt to any data format, structured, semi-structured, and unstructured.<\/li>\n<li>Data can be stored in raw format and can be explored in its native feature without the traditional ETL process.<\/li>\n<li>ADLS Gen2 can easily connect with other Azure components. It can serve source data to data warehouses or databases and act as a landing area for transformed data from Azure Data Factory and Azure Databricks. Additionally, PowerBI can connect directly to it for analysis.<\/li>\n<\/ul>\n<h4><em>High availability<\/em><\/h4>\n<p>There are always multiple copies of data in the ADLS Gen2 to be fully prepared for any unexpected failures and disasters. There are always three copies of data within a single region. Azure storage redundancy provides local to global solutions in the protection of local, zone, or regional outages or disasters. Besides disaster recovery, those replicates can also be used to repair corrupted data to guarantee data integrity.<\/p>\n<h4><em>Security <\/em><\/h4>\n<ul>\n<li>Auditing: diagnostic logging can be enabled to record data access traces.<\/li>\n<li>Access control: ADLS Gen2 provides access control on individual files and folders and integrates with Azure Active Directory for identity and access management. Permissions can be given specifically to someone on certain files or folders. It is also possible to have multi-factor authentication, role-based access control (RBAC), monitoring, and alerting.<\/li>\n<li>Encryption: data stored in ADLS Gen2 are encrypted by default (encryption-at-rest). Microsoft can manage the encryption key for you and you have the choice to manage it yourself. It is also possible to configure a secure transfer and enable encryption-in-transit.<\/li>\n<\/ul>\n<h4><em>Better Performance<\/em><\/h4>\n<ul>\n<li>Query performance: ADLS Gen2 has a hierarchical file system. This feature not only enables granular security mentioned above but also partitions. Therefore, if you connect with any software that can perform partition scan, the query performance will be highly improved.<\/li>\n<li>Data load: ADLS Gen2 is able to relocate data through metadata-only operation, which is easy and <a href=\"https:\/\/nl.devoteam.com\/en\/microsoft-azure-platform-cost-optimization\/\" target=\"_blank\" rel=\"noopener noreferrer\">cost-efficient.<\/a><\/li>\n<\/ul>\n<div class=\"block-fullwidth-dark\">\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/Cover-1-click-brochure-300x224.png\" alt=\"1-click data platform brochure cover\" width=\"300\" height=\"224\" class=\"alignright size-large wp-image-81528\" \/><\/p>\n<p><span>Tired of big investments and complex timelines when it comes to setting up a data platform? We will set up your complete data platform infrastructure fully automated in just one click. <\/span><strong>4x faster<\/strong><span>\u00a0than usual and\u00a0<\/span><strong>3x lower<\/strong><span> platform enablement costs.<\/span><\/p>\n<div class=\"row-more clearfix\"><a href=\"https:\/\/nl.devoteam.com\/en\/1-click-data-platform\/\" class=\"bt-more\" target=\"_blank\" rel=\"noopener noreferrer\">Learn More<\/a><\/div>\n<\/div>\n<h2><strong>How to Load Data to Azure Data Lake<\/strong><\/h2>\n<p>For loading data into Azure Data Lake, there are different ways for different situations. The following table shows some use cases and tools that can be used.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/how-to-load-data-to-azure-data-lake.png\" alt=\"how to load data to azure data lake\" width=\"798\" height=\"145\" class=\"alignleft size-full wp-image-81543\" \/><\/p>\n<h4><em>Data quality<\/em><\/h4>\n<p>It is one of our main priorities to guarantee accurate and relevant data to be provided to the business on time for decision-making. Together with other services, such as Azure Synapse Analytics, Azure Databricks, and Azure Data Catalog, ADLS Gen2 supports the realization of better data quality, including accuracy, timeliness, integrity, and relevance.<\/p>\n<p>Although unlimited storage of different types of data is impressive, unorganized storage of a large amount of data can be a disaster. It is recommended to follow the best practices to maintain a good data lake instead of turning it into a data dump.<\/p>\n<ul>\n<li>Have a data storage plan before data loading can be a good start. We can organize Data Lake into layers and folders to serve different use cases. For example, we can organize data into the raw data layer, ETL layer, and consumption layer. Data can further be organized per client, source channel, year, and month.<\/li>\n<li>Using Azure Data Catalog to make data easy to search and recognize.<\/li>\n<li>Purge duplicated and unneeded data. Periodically examine if data need to be reorganized to fit business development and adjust if necessary.<\/li>\n<\/ul>\n<h4><em>Security<\/em><\/h4>\n<p>As above-mentioned<em>, <\/em>ADLS Gen2 supports auditing, access control, and encryption. Data professionals can utilize those features by following best practices.<\/p>\n<ul>\n<li>Analyze audit logs periodically to control quality and identify risks.<\/li>\n<li>Apply granular access control per folder\/file and role. Update access control on time in case of personnel or position changes.<\/li>\n<li>Rotate the encryption key periodically.<\/li>\n<\/ul>\n<h4><em>Lifecycle management<\/em><\/h4>\n<p>Manage and consume data in an efficient way is what we want to achieve throughout the whole life cycle. ADLS Gen2 provides different access tiers including hot, cool, and archive with decreasing cost rates and increasing retrieving time. Therefore, we can store data more efficiently based on access frequency. ADLS Gen2 also has a lifecycle management feature itself which gives you the possibility to schedule rules to move data to the right tier automatically.<\/p>\n<div class=\"block-fullwidth-dark\">\n<p>Do you want to learn more about Azure Data Lake? Click on the button below and watch my recorded webinar &#8220;Data Lake &#8211; Grow Fearless&#8221;. Please contact me if you have any questions.<\/p>\n<div class=\"row-more clearfix\"><a href=\"https:\/\/www.nl.info.devoteam.com\/webinar-azure-data-lake-grow-fearless\" class=\"bt-more\" target=\"_blank\" rel=\"noopener noreferrer\">Watch recorded webinar<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>&nbsp; In my previous blog \u201cAzure Data Lake: Grow Fearless \u2013 Part 1\u201d I described the business benefits of data growth, but also the challenging parts of fast-growing data. In this blog, I\u2019ll dive deeper and show you the promising parts of Azure Data Lake and why it&#8217;s worth considering. Why should you consider Azure [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","categories":[],"tags":[2308,2774,2747],"industry":[],"class_list":["post-514392","expert-view","type-expert-view","status-publish","hentry","tag-data-en-se","tag-microsoft-en-se","tag-netherlands-en-se"],"acf":[],"cards":"\n\t<div class=\"single-post-card\">\n\n\t\t\n\n\t\t\n\t\t<div class=\"wp-block-group is-vertical is-layout-flex wp-container-core-group-is-layout-43282307 wp-block-group-is-layout-flex\">\n\t<p style=\"font-style:normal;font-weight:700\" class=\"has-link-color wp-elements-1 wp-block-lp-post-type has-text-color has-primary-color has-small-font-size\">Expert View<\/p>\n\n\t\t\n\t\t<h3 style=\"font-style:normal;font-weight:400\" class=\"wp-block-post-title has-base-font-size\"><a href=\"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/\" target=\"_self\" >Azure Data Lake: Grow Fearless &#8211; Part 2<\/a><\/h3><\/div>\n\t\t\n\t<\/div>\n\n","yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v28.4 (Yoast SEO v28.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Azure Data Lake: Grow Fearless - Part 2 | Devoteam<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Azure Data Lake: Grow Fearless - Part 2\" \/>\n<meta property=\"og:description\" content=\"&nbsp; In my previous blog \u201cAzure Data Lake: Grow Fearless \u2013 Part 1\u201d I described the business benefits of data growth, but also the challenging parts of fast-growing data. In this blog, I\u2019ll dive deeper and show you the promising parts of Azure Data Lake and why it&#8217;s worth considering. Why should you consider Azure [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/\" \/>\n<meta property=\"og:site_name\" content=\"Devoteam\" \/>\n<meta property=\"og:image\" content=\"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/11\/Industries-Wordpress-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1280\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/azure-data-lake-grow-fearless-part-2\\\/\",\"url\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/azure-data-lake-grow-fearless-part-2\\\/\",\"name\":\"Azure Data Lake: Grow Fearless - Part 2 | Devoteam\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/azure-data-lake-grow-fearless-part-2\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/azure-data-lake-grow-fearless-part-2\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/devoteam.info\\\/wp-content\\\/uploads\\\/2024\\\/12\\\/Cover-1-click-brochure-300x224.png\",\"datePublished\":\"2020-11-26T13:32:55+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/azure-data-lake-grow-fearless-part-2\\\/#breadcrumb\"},\"inLanguage\":\"en-SE\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/azure-data-lake-grow-fearless-part-2\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-SE\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/azure-data-lake-grow-fearless-part-2\\\/#primaryimage\",\"url\":\"https:\\\/\\\/devoteam.info\\\/wp-content\\\/uploads\\\/2024\\\/12\\\/Cover-1-click-brochure-300x224.png\",\"contentUrl\":\"https:\\\/\\\/devoteam.info\\\/wp-content\\\/uploads\\\/2024\\\/12\\\/Cover-1-click-brochure-300x224.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/azure-data-lake-grow-fearless-part-2\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Expert View\",\"item\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/expert-view\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Azure Data Lake: Grow Fearless &#8211; Part 2\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/#website\",\"url\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/\",\"name\":\"Devoteam\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/devoteam.info\\\/en-se\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-SE\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Azure Data Lake: Grow Fearless - Part 2 | Devoteam","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/","og_locale":"en_US","og_type":"article","og_title":"Azure Data Lake: Grow Fearless - Part 2","og_description":"&nbsp; In my previous blog \u201cAzure Data Lake: Grow Fearless \u2013 Part 1\u201d I described the business benefits of data growth, but also the challenging parts of fast-growing data. In this blog, I\u2019ll dive deeper and show you the promising parts of Azure Data Lake and why it&#8217;s worth considering. Why should you consider Azure [&hellip;]","og_url":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/","og_site_name":"Devoteam","og_image":[{"width":1920,"height":1280,"url":"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/11\/Industries-Wordpress-1.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/","url":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/","name":"Azure Data Lake: Grow Fearless - Part 2 | Devoteam","isPartOf":{"@id":"https:\/\/devoteam.info\/en-se\/#website"},"primaryImageOfPage":{"@id":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/#primaryimage"},"image":{"@id":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/#primaryimage"},"thumbnailUrl":"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/Cover-1-click-brochure-300x224.png","datePublished":"2020-11-26T13:32:55+00:00","breadcrumb":{"@id":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/#breadcrumb"},"inLanguage":"en-SE","potentialAction":[{"@type":"ReadAction","target":["https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/"]}]},{"@type":"ImageObject","inLanguage":"en-SE","@id":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/#primaryimage","url":"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/Cover-1-click-brochure-300x224.png","contentUrl":"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/Cover-1-click-brochure-300x224.png"},{"@type":"BreadcrumbList","@id":"https:\/\/devoteam.info\/en-se\/expert-view\/azure-data-lake-grow-fearless-part-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/devoteam.info\/en-se\/"},{"@type":"ListItem","position":2,"name":"Expert View","item":"https:\/\/devoteam.info\/en-se\/expert-view\/"},{"@type":"ListItem","position":3,"name":"Azure Data Lake: Grow Fearless &#8211; Part 2"}]},{"@type":"WebSite","@id":"https:\/\/devoteam.info\/en-se\/#website","url":"https:\/\/devoteam.info\/en-se\/","name":"Devoteam","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/devoteam.info\/en-se\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-SE"}]}},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"group.it.system","author_link":"https:\/\/devoteam.info\/en-se\/author\/"},"uagb_comment_info":0,"uagb_excerpt":"&nbsp; In my previous blog \u201cAzure Data Lake: Grow Fearless \u2013 Part 1\u201d I described the business benefits of data growth, but also the challenging parts of fast-growing data. In this blog, I\u2019ll dive deeper and show you the promising parts of Azure Data Lake and why it&#8217;s worth considering. Why should you consider Azure&hellip;","_links":{"self":[{"href":"https:\/\/devoteam.info\/en-se\/wp-json\/wp\/v2\/expert-view\/514392","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devoteam.info\/en-se\/wp-json\/wp\/v2\/expert-view"}],"about":[{"href":"https:\/\/devoteam.info\/en-se\/wp-json\/wp\/v2\/types\/expert-view"}],"version-history":[{"count":0,"href":"https:\/\/devoteam.info\/en-se\/wp-json\/wp\/v2\/expert-view\/514392\/revisions"}],"wp:attachment":[{"href":"https:\/\/devoteam.info\/en-se\/wp-json\/wp\/v2\/media?parent=514392"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devoteam.info\/en-se\/wp-json\/wp\/v2\/categories?post=514392"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devoteam.info\/en-se\/wp-json\/wp\/v2\/tags?post=514392"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/devoteam.info\/en-se\/wp-json\/wp\/v2\/industry?post=514392"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}