{"id":35833,"date":"2020-02-27T07:45:05","date_gmt":"2020-02-27T15:45:05","guid":{"rendered":"https:\/\/joeydantoni.com\/?p=2295"},"modified":"2020-02-27T07:45:05","modified_gmt":"2020-02-27T15:45:05","slug":"storage-field-day-19-minio-sfd19","status":"publish","type":"post","link":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/","title":{"rendered":"Storage Field Day 19 MinIO #SFD19"},"content":{"rendered":"

In January I had the chance to attend Storage Field Day 19 in Santa Clara, where we got to meet with a wide variety of startups and large enterprise storage companies. One of the more interesting companies we meet with was MinIO<\/a> which has a really interesting and compelling object-based storage product.<\/p>\n

\"IMG_3141\"
Some of MinIOs Customers and Use Cases<\/figcaption><\/figure>\n

I’ve talked about object storage here before, but it’s a very different paradigm than the traditional block based storage you may currently be using. With block storage files are split into evenly sized blocks of data (typically somewhere between 64 KB and 1 MB depending on your vendor). Data protection is provided by traditional RAID options.<\/p>\n

Object storage on the other hand doesn’t split files into blocks. Files are stored as objects which contain the file data, metadata, and a unique identifier. There is no limit on the size or amount of the metadata associated with the file. If you have ever created a managed disk in Azure, taken a backup to URL, or used an Azure SQL Database you’ve used object based storage. In object based storage, redundancy is generally provided by maintaining three copies of the object (e.g. a write isn’t considered complete until it writes to all three copies).<\/p>\n

Object storage is designed to solve problems of scale. One of the things I learned at Comcast was that the cost of SAN storage didn’t scale to some of the massive petabyte scale data problems we had. The management overhead, the cost, and sometimes even the storage itself does not scale. This is a problem largely for companies like Microsoft, Amazon, Google, Facebook, etc, who have massive amounts of data to store.\u00a0 But as data volumes grow there are lots of other firms who have very large volumes that they need to manage.<\/p>\n

MinIO is a firm that offers such a solution. MinIO offers open source storage management software that offers extremely fast (183 GB\/s reads and 171 GB\/s writes). It is fully compatible with Amazon’s S3 API, which has somewhat become the de facto standard for object storage. They were working on Azure Blob Storage support when we visited.<\/p>\n

One of the ways MinIO is able to get such good performance out of pretty standard hardware is by taking advantage of SIMD processor instructions, which all more text and number crunching to be performed per CPU instruction which dramatically increases performance. SQL Server uses this through the query processor’s use of batch mode.<\/p>\n

MinIO’s storage can also be used as a persistent store for Kubernetes (drink), or used for systems like Spark, TensorFlow, and a replacement for Hadoop HDFS. Where you would probably use this in your environment would be to replace your file servers, or as a target for container storage, or maybe even an analytic store. Or you want to become a cloud storage provider and you need to host 50 PB of data in your data center.<\/p>\n","protected":false},"excerpt":{"rendered":"

In January I had the chance to attend Storage Field Day 19 in Santa Clara, where we got to meet with a wide variety of startups and large enterprise storage companies. One of the more interesting companies we meet with was MinIO which has a really interesting and compelling object-based storage product. I’ve talked about […]<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[551,248,141],"tags":[],"class_list":["post-35833","post","type-post","status-publish","format-standard","hentry","category-blog","category-clustering","category-storage"],"yoast_head":"\nStorage Field Day 19 MinIO #SFD19 – Denny Cherry & Associates Consulting<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Storage Field Day 19 MinIO #SFD19 – Denny Cherry & Associates Consulting\" \/>\n<meta property=\"og:description\" content=\"In January I had the chance to attend Storage Field Day 19 in Santa Clara, where we got to meet with a wide variety of startups and large enterprise storage companies. One of the more interesting companies we meet with was MinIO which has a really interesting and compelling object-based storage product. I’ve talked about […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/\" \/>\n<meta property=\"og:site_name\" content=\"Denny Cherry & Associates Consulting\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DennyCherryAssociatesConsulting\/\" \/>\n<meta property=\"article:published_time\" content=\"2020-02-27T15:45:05+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/joeydantoni.com\/wp-content\/uploads\/2020\/02\/IMG_3141.png?resize=660%2C495&ssl=1\" \/>\n<meta name=\"author\" content=\"Joey D'Antoni\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@dcacco\" \/>\n<meta name=\"twitter:site\" content=\"@dcacco\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Joey D'Antoni\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/\",\"url\":\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/\",\"name\":\"Storage Field Day 19 MinIO #SFD19 – Denny Cherry & Associates Consulting\",\"isPartOf\":{\"@id\":\"https:\/\/www.dcac.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/joeydantoni.com\/wp-content\/uploads\/2020\/02\/IMG_3141.png?resize=660%2C495&ssl=1\",\"datePublished\":\"2020-02-27T15:45:05+00:00\",\"dateModified\":\"2020-02-27T15:45:05+00:00\",\"author\":{\"@id\":\"https:\/\/www.dcac.com\/#\/schema\/person\/ac58f2207d3f180aa3e4ee0ce6af0885\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/joeydantoni.com\/wp-content\/uploads\/2020\/02\/IMG_3141.png?resize=660%2C495&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/joeydantoni.com\/wp-content\/uploads\/2020\/02\/IMG_3141.png?resize=660%2C495&ssl=1\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.dcac.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Storage Field Day 19 MinIO #SFD19\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dcac.com\/#website\",\"url\":\"https:\/\/www.dcac.com\/\",\"name\":\"Denny Cherry & Associates Consulting\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dcac.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dcac.com\/#\/schema\/person\/ac58f2207d3f180aa3e4ee0ce6af0885\",\"name\":\"Joey D'Antoni\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.dcac.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/3fede03569772a55a091b587d76f78d4?s=96&r=x\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/3fede03569772a55a091b587d76f78d4?s=96&r=x\",\"caption\":\"Joey D'Antoni\"},\"url\":\"https:\/\/www.dcac.com\/author\/joeydcac-co\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Storage Field Day 19 MinIO #SFD19 – Denny Cherry & Associates Consulting","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/","og_locale":"en_US","og_type":"article","og_title":"Storage Field Day 19 MinIO #SFD19 – Denny Cherry & Associates Consulting","og_description":"In January I had the chance to attend Storage Field Day 19 in Santa Clara, where we got to meet with a wide variety of startups and large enterprise storage companies. One of the more interesting companies we meet with was MinIO which has a really interesting and compelling object-based storage product. I’ve talked about […]","og_url":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/","og_site_name":"Denny Cherry & Associates Consulting","article_publisher":"https:\/\/www.facebook.com\/DennyCherryAssociatesConsulting\/","article_published_time":"2020-02-27T15:45:05+00:00","og_image":[{"url":"https:\/\/i0.wp.com\/joeydantoni.com\/wp-content\/uploads\/2020\/02\/IMG_3141.png?resize=660%2C495&ssl=1","type":"","width":"","height":""}],"author":"Joey D'Antoni","twitter_card":"summary_large_image","twitter_creator":"@dcacco","twitter_site":"@dcacco","twitter_misc":{"Written by":"Joey D'Antoni","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/","url":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/","name":"Storage Field Day 19 MinIO #SFD19 – Denny Cherry & Associates Consulting","isPartOf":{"@id":"https:\/\/www.dcac.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#primaryimage"},"image":{"@id":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/joeydantoni.com\/wp-content\/uploads\/2020\/02\/IMG_3141.png?resize=660%2C495&ssl=1","datePublished":"2020-02-27T15:45:05+00:00","dateModified":"2020-02-27T15:45:05+00:00","author":{"@id":"https:\/\/www.dcac.com\/#\/schema\/person\/ac58f2207d3f180aa3e4ee0ce6af0885"},"breadcrumb":{"@id":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#primaryimage","url":"https:\/\/i0.wp.com\/joeydantoni.com\/wp-content\/uploads\/2020\/02\/IMG_3141.png?resize=660%2C495&ssl=1","contentUrl":"https:\/\/i0.wp.com\/joeydantoni.com\/wp-content\/uploads\/2020\/02\/IMG_3141.png?resize=660%2C495&ssl=1"},{"@type":"BreadcrumbList","@id":"https:\/\/www.dcac.com\/2020\/02\/27\/storage-field-day-19-minio-sfd19\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.dcac.com\/"},{"@type":"ListItem","position":2,"name":"Storage Field Day 19 MinIO #SFD19"}]},{"@type":"WebSite","@id":"https:\/\/www.dcac.com\/#website","url":"https:\/\/www.dcac.com\/","name":"Denny Cherry & Associates Consulting","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dcac.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dcac.com\/#\/schema\/person\/ac58f2207d3f180aa3e4ee0ce6af0885","name":"Joey D'Antoni","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dcac.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/3fede03569772a55a091b587d76f78d4?s=96&r=x","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/3fede03569772a55a091b587d76f78d4?s=96&r=x","caption":"Joey D'Antoni"},"url":"https:\/\/www.dcac.com\/author\/joeydcac-co\/"}]}},"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/posts\/35833","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/comments?post=35833"}],"version-history":[{"count":31,"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/posts\/35833\/revisions"}],"predecessor-version":[{"id":37795,"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/posts\/35833\/revisions\/37795"}],"wp:attachment":[{"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/media?parent=35833"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/categories?post=35833"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dcac.com\/wp-json\/wp\/v2\/tags?post=35833"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}