{"id":601877,"date":"2020-02-24T13:54:02","date_gmt":"2020-02-24T12:54:02","guid":{"rendered":"https:\/\/www.devoteam.com\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/"},"modified":"2020-02-24T13:54:02","modified_gmt":"2020-02-24T12:54:02","slug":"how-to-achieve-a-high-standard-of-data-quality","status":"publish","type":"expert-view","link":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/","title":{"rendered":"How to achieve a High Standard of Data Quality"},"content":{"rendered":"

 <\/p>\n

Ensuring the quality of data and information is vital to any project, whether it is a doctor analyzing test-results to reach a diagnosis, or an economist reading the balance sheets of large companies to measure a country’s GDP.<\/span><\/p>\n

 <\/p>\n

\n

Have you read the previous blog-post “Data Engineering in a Mad Data World”?<\/h3>\n

We recommend you to read this 1st blog-post to understand the complete environment and responsibilities of a data engineer in today’s mad data world.<\/p>\n

Read More<\/a><\/div>\n<\/div>\n

In recent years, the norms, concepts, and frameworks to assist us in the implementation, monitoring and management of data quality have been updated frequently. The main reason for this is the infinite increase of data volume, and with that, ways to measure data quality.<\/span><\/p>\n

In 1986, we measured data in up to 8 dimensions <\/span><\/h2>\n
    \n
  1. Performance <\/span><\/li>\n
  2. Features<\/span><\/li>\n
  3. Reliability<\/span><\/li>\n
  4. Conformance<\/span><\/li>\n
  5. Durability<\/span><\/li>\n
  6. Serviceability<\/span><\/li>\n
  7. Aesthetics<\/span><\/li>\n
  8. Perceived Quality<\/span><\/li>\n<\/ol>\n

    \"\"<\/a><\/p>\n

    Today, that number has risen to 11<\/span><\/h2>\n

    Today, that number has risen to 11 and several definitions have been changed <\/span><\/p>\n

      \n
    1. Accuracy<\/span><\/li>\n
    2. Completeness<\/span><\/li>\n
    3. Consistency<\/span><\/li>\n
    4. Precision<\/span><\/li>\n
    5. Privacy<\/span><\/li>\n
    6. Reasonableness<\/span><\/li>\n
    7. Referential Integrity<\/span><\/li>\n
    8. Timeliness<\/span><\/li>\n
    9. Uniqueness<\/span><\/li>\n
    10. Validity<\/span><\/li>\n<\/ol>\n

      There are some authors and experts who believe that Granularity (11<\/strong>) is another dimension that should be used, as it becomes vital in cases where data from telephone calls, e-commerce, and financial transactions is used.<\/span><\/p>\n

      How to ensure data quality<\/h2>\n

      A good engineering and data quality practice is to keep the raw data copy and storage in the data lake, where this copy should always be the same as the source data. <\/span><\/p>\n

      To guarantee the integrity of the data and information in these two processes (copy and storage), the data engineer will need to measure the data according to its type, format, business requirements, technologies used for transport, and its storage. With this, he or she will practically make use of all the 11 dimensions. To ensure a high-quality implementation, we at Devoteam recommend End-to-End management with the involvement of all those interested in this chain. Both users and those responsible for the use case.<\/span><\/p>\n

      With Azure data platform products and solutions<\/a>, such management is easy to implement. It also provides a 360 view of the entire process. The data monitoring is collected and consolidated by Azure Metrics<\/a> and made available for analysis to the users through Spotfire<\/a> dashboards.<\/span><\/p>\n

      The data will be collected from more than 90 different types of connectors. The data is then analyzed, resulting in the reporting of inconsistencies or anomalies, which will be fixed before the data is stored thanks to different features delivered by Azure Data Bricks<\/a>.<\/span><\/p>\n

      Considering measuring data quality in real-time transactions, our consultants have experience in the process of data ingestion with Azure EventHub integrated with Azure Monitor, to monitor its integrity, consistency, and accuracy. Using Spotfire dashboards<\/a>, this extracts the information needed for a specific use case.\u00a0<\/span><\/p>\n

      In the next article<\/span> we will talk about privacy, where we will see that Azure Data Factory also helps to ensure data masking or its disposal.<\/em><\/p>\n

      Our Data as a Service offer<\/a><\/div>\n

      <\/h3>\n

      For further reading:<\/h3>\n

      MUST-READ:<\/strong> Implementation and roll-out of a Pan European Data Platform for Supply Chain<\/a><\/p>\n

      BLOG-POST: <\/strong>The role of data analytics in the Sales and Operational Planning<\/a><\/p>\n

      REFERENCE CASE:<\/strong>\u00a0Building a BigQuery Data Warehouse for a disruptive employment agency<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

        Ensuring the quality of data and information is vital to any project, whether it is a doctor analyzing test-results to reach a diagnosis, or an economist reading the balance sheets of large companies to measure a country’s GDP.   Have you read the previous blog-post “Data Engineering in a Mad Data World”? We recommend […]<\/p>\n","protected":false},"featured_media":351586,"template":"","categories":[],"tags":[2297,2717],"industry":[],"class_list":["post-601877","expert-view","type-expert-view","status-publish","has-post-thumbnail","hentry","tag-data-en-nl","tag-netherlands-en-nl"],"acf":[],"cards":"\n\t

      \n\n\t\t
      \"How<\/a><\/figure>\n\n\t\t\n\t\t
      \n\t

      Expert View<\/p>\n\n\t\t\n\t\t

      How to achieve a High Standard of Data Quality<\/a><\/h3><\/div>\n\t\t\n\t<\/div>\n\n","yoast_head":"\nHow to achieve a High Standard of Data Quality | Devoteam<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to achieve a High Standard of Data Quality\" \/>\n<meta property=\"og:description\" content=\"  Ensuring the quality of data and information is vital to any project, whether it is a doctor analyzing test-results to reach a diagnosis, or an economist reading the balance sheets of large companies to measure a country’s GDP.   Have you read the previous blog-post “Data Engineering in a Mad Data World”? We recommend […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/\" \/>\n<meta property=\"og:site_name\" content=\"Devoteam\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/how-to-achieve-a-high-standard-of-data-quality\\\/\",\"url\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/how-to-achieve-a-high-standard-of-data-quality\\\/\",\"name\":\"How to achieve a High Standard of Data Quality | Devoteam\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/how-to-achieve-a-high-standard-of-data-quality\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/how-to-achieve-a-high-standard-of-data-quality\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/devoteam.info\\\/wp-content\\\/uploads\\\/2024\\\/12\\\/structure-light-led-movement-158826.jpeg\",\"datePublished\":\"2020-02-24T12:54:02+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/how-to-achieve-a-high-standard-of-data-quality\\\/#breadcrumb\"},\"inLanguage\":\"en-NL\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/how-to-achieve-a-high-standard-of-data-quality\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-NL\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/how-to-achieve-a-high-standard-of-data-quality\\\/#primaryimage\",\"url\":\"https:\\\/\\\/devoteam.info\\\/wp-content\\\/uploads\\\/2024\\\/12\\\/structure-light-led-movement-158826.jpeg\",\"contentUrl\":\"https:\\\/\\\/devoteam.info\\\/wp-content\\\/uploads\\\/2024\\\/12\\\/structure-light-led-movement-158826.jpeg\",\"width\":2241,\"height\":1500},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/how-to-achieve-a-high-standard-of-data-quality\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Expert View\",\"item\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/expert-view\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"How to achieve a High Standard of Data Quality\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/#website\",\"url\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/\",\"name\":\"Devoteam\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/devoteam.info\\\/en-nl\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-NL\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to achieve a High Standard of Data Quality | Devoteam","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/","og_locale":"en_US","og_type":"article","og_title":"How to achieve a High Standard of Data Quality","og_description":"  Ensuring the quality of data and information is vital to any project, whether it is a doctor analyzing test-results to reach a diagnosis, or an economist reading the balance sheets of large companies to measure a country’s GDP.   Have you read the previous blog-post “Data Engineering in a Mad Data World”? We recommend […]","og_url":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/","og_site_name":"Devoteam","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/","url":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/","name":"How to achieve a High Standard of Data Quality | Devoteam","isPartOf":{"@id":"https:\/\/devoteam.info\/en-nl\/#website"},"primaryImageOfPage":{"@id":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/#primaryimage"},"image":{"@id":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/#primaryimage"},"thumbnailUrl":"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826.jpeg","datePublished":"2020-02-24T12:54:02+00:00","breadcrumb":{"@id":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/#breadcrumb"},"inLanguage":"en-NL","potentialAction":[{"@type":"ReadAction","target":["https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/"]}]},{"@type":"ImageObject","inLanguage":"en-NL","@id":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/#primaryimage","url":"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826.jpeg","contentUrl":"https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826.jpeg","width":2241,"height":1500},{"@type":"BreadcrumbList","@id":"https:\/\/devoteam.info\/en-nl\/expert-view\/how-to-achieve-a-high-standard-of-data-quality\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/devoteam.info\/en-nl\/"},{"@type":"ListItem","position":2,"name":"Expert View","item":"https:\/\/devoteam.info\/en-nl\/expert-view\/"},{"@type":"ListItem","position":3,"name":"How to achieve a High Standard of Data Quality"}]},{"@type":"WebSite","@id":"https:\/\/devoteam.info\/en-nl\/#website","url":"https:\/\/devoteam.info\/en-nl\/","name":"Devoteam","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/devoteam.info\/en-nl\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-NL"}]}},"uagb_featured_image_src":{"full":["https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826.jpeg",2241,1500,false],"thumbnail":["https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826-150x150.jpeg",150,150,true],"medium":["https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826-300x201.jpeg",300,201,true],"medium_large":["https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826-768x514.jpeg",768,514,true],"large":["https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826-1024x685.jpeg",1024,685,true],"1536x1536":["https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826-1536x1028.jpeg",1536,1028,true],"2048x2048":["https:\/\/devoteam.info\/wp-content\/uploads\/2024\/12\/structure-light-led-movement-158826-2048x1371.jpeg",2048,1371,true]},"uagb_author_info":{"display_name":"group.it.system","author_link":"https:\/\/devoteam.info\/en-nl\/author\/"},"uagb_comment_info":0,"uagb_excerpt":"  Ensuring the quality of data and information is vital to any project, whether it is a doctor analyzing test-results to reach a diagnosis, or an economist reading the balance sheets of large companies to measure a country’s GDP.   Have you read the previous blog-post “Data Engineering in a Mad Data World”? We recommend…","_links":{"self":[{"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/expert-view\/601877","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/expert-view"}],"about":[{"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/types\/expert-view"}],"version-history":[{"count":0,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/expert-view\/601877\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/media\/351586"}],"wp:attachment":[{"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/media?parent=601877"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/categories?post=601877"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/tags?post=601877"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/devoteam.info\/en-nl\/wp-json\/wp\/v2\/industry?post=601877"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}