问:
我们公司主要从事邮件系统方面的业务,所以,这是一个关于邮件系统存储方面的一个问题。
邮件系统数据的特点就是小文件多,并且目录多,一部分数据访问频繁,读写删除操作也频繁(活跃用户的数据),还有一部分数据基本不访问(非活跃用户),很少有读写删除操作。
当一个有40万用户的邮件系统,那么存放邮件数据的存储目录下面会有非常多的目录和文件(一个用户一个HomeDir,用户HomeDir下面存放用户的邮件),HomeDir至少40万个,每个HomeDir下面可能有用户的自定义文件夹。HomeDir下面存放的用户邮件数量多少不等,最多的可能有800封以上的邮件,平均每人30封邮件左右。大部分邮件大小在5-100K左右,也存在一部分带有较大附件的邮件,大小在1M-10M之间。
以上是应用特点的简单介绍,针对这样的应用环境,应该如何选择存储设备?做什么样的RAID能达到最好的性能?如何部署备份系统?
像类似这样的应用应该还有很多,例如:flickr、bababian等等这样的网络相册应用,应该和邮件应用类似,都是小文件,访问频繁。也很想了解他们是如何选择和部署存储系统的。
目前,我们的存储在使用一段时间后,只能使用df命令来查看磁盘空间,使用du命令查看存储目录占用空间的时候将会用掉非常非常长的时间,这个时间是绝对不能忍受的长。
使用备份方法是每天用rsync,将发生变化的文件同步到另外一个存储空间上,我认为这不是一个较为理想的备份方式,希望能再进行改进。
在RAID方面,我们基本上选择的都是用RAID5,也有使用RAID10,但是感觉这两者在性能上没有什么明显区别。不知您是否有测试存储性能方法及测试模型,我也想对我们系统中这种环境下的存储进行测试和分析,希望能找到最佳的方案。
sansky答:
你所说的几个系统具有频繁访问、随机访问、文件数量大等特点,对服务器和存储的第一个要求是高IOPS。我个人对存储系统的建议如下:
1、存储设备的的cache IOPS最好不低于12W,磁盘IOPS不低于7w,也就是说必须是中端以上存储设备,低端设备的性能不足以满足系统要求;
2、建议采用全光纤存储设备;由于总容量不大,可以选择146GB磁盘。如果考虑到价格,也可以选择高性能的FC-SAS存储。
3、磁盘访问的性能将会是整个系统的瓶颈。建议存储设备的磁盘端口不少于4个,配置不少于3个JBOD,SBOD最好。磁盘总数量48块磁盘最好,建议少于32块。否则磁盘太少,存储设备的性能发挥不出来。
4、每个RAID组中的磁盘数量8-12个,优选RAID5。
5、40个homedir最好分散在4个以上的RAID组中,也就是不同的LUN中。无论是windows还是linux系统,当一个lun中的文件数量过大时,其文件系统检索和访问能力多会大幅度减小。homedir分散在多个lun中,减少每一个文件系统中的文件数量。
6、多个RAID组的优先级高于每个RAID组中的磁盘数量。
7、对于备份来讲,如果考虑价格,可以选择用备份软件。如果投资允许,可以考虑CDP软件。
一点拙见,仅供参考。
57 条回复
灯火兄, 我转载一下你的这篇博文可以否? 注明作者是Sansky 还是写你真名?带上公司名称和职位的
写上sansky和存储部落就可以了。
小声嘀咕一下,问题是我问的,哈哈……
以后还得跟sansky多多学习
磁盘IOPS不低于7w?
不知道你对存储的理解是怎样的?
关于邮件,我也随便说两句,正好今年做了几个邮件系统的存储项目。
首先,邮件系统的IO问题,往往是由服务器和存储共同承担的,所以设计时,只需要能够匹配服务器的IO处理能力就可以了,这就要看具体采用的是什么邮件系统,以及前段服务器的IO负载能力总合了。所以其实在这里ISCSI和FC的选择上,差距不大。诸如263的邮件系统已经有很大一部分转由ISCSI来承担了。
其次,邮件的备份问题。邮件的备份如果按照传统的模式加载备份软件的话,其实很多数据是无法保护的。原因有二。1、很多邮件系统为了降低读写负载,都采用在闲时集中删除用户自删除的邮件(也就是说你删了你的邮件,其实并不一定就删除了),这也就造成了留给备份的时间较短,有时候大系统中甚至无法完成一次全备份。
2、很多时候,用户需要找到的是它删除过的邮件,而由于备份很难去保留这部分数据,而直接导致备份的逻辑无效。
所以在邮件系统领域,备份往往才有邮件归档的方式进行,然后再对归档的内容进行备份,但是这样所需要的存储容量就相当可观,这虽然是很多存储厂商希望看到的,却不是用户希望的。呵呵
最后,就是对于架构的一点小小补充,一般邮件系统处于成本考虑,全部采用IO节点和计算节点的方式进行设计,所以性能主要局限在IO节点上,后端的存储真的没必要用高性能、价格昂贵的全FC存储。如果真要提升性能,倒是更加建议采用高性能SAN文件共享系统+低端盘阵的方式,更加的高效、稳定。
个人的一点经验,可能很多地方不对,欢迎拍转,但别问候偶父母,呵呵
多谢回复,学到不少东西。
相互学习,也希望能看到更多关于行业应用的文章,大家共同探讨,这样感觉会成长更快
好看的主题
用SSD的解决方案,很简单
想问下下,
1、您说的磁盘性能不低于7w
3、建议48块盘
据我所知,磁盘48块15k,也就9K多的iops,不知道方案是否有误?
Wow, that’s what I was seeking for, what a stuff! present here at this webpage, thanks admin of this web site.
Pretty section of content. I just stumbled upon your web site and in accession capital to assert that I acquire in fact enjoyed account your blog posts. Any way I’ll be subscribing to your augment and even I achievement you access consistently fast.
If you wish for to improve your familiarity only keep visiting this site and be updated with the most recent news posted here.
This is a very good tip especially to those new to the blogosphere. Short but very accurate information Thanks for sharing this one. A must read article!
I was wondering if you ever considered changing the page layout of your site? Its very well written; I love what youve got to say. But maybe you could a little more in the way of content so people could connect with it better. Youve got an awful lot of text for only having one or two images. Maybe you could space it out better?
Its like you read my mind! You seem to know so much approximately this, like you wrote the ebook in it or something. I feel that you could do with some % to force the message house a bit, however other than that, this is great blog. An excellent read. I’ll definitely be back.
Great goods from you, man. I’ve understand your stuff previous to and you’re just too great. I really like what you’ve acquired here, really like what you’re stating and the way in which you say it. You make it entertaining and you still take care of to keep it sensible. I cant wait to read far more from you. This is actually a great site.
Im thankful for the blog article.Really looking forward to read more. Really Great.
After I originally commented I seem to have clicked on the -Notify me when new comments are added- checkbox and now each time a comment is added I get 4 emails with the same comment. Is there a way you can remove me from that service? Kudos!
Asking questions are really good thing if you are not understanding anything completely, but this article provides good understanding even.
Keep this going please, great job!
Nice post. I learn something new and challenging on sites I stumbleupon everyday. It will always be interesting to read content from other writers and practice a little something from their sites.
I don’t even understand how I stopped up here, however I thought this submit was good. I don’t understand who you are however definitely you are going to a famous blogger when you are not already. Cheers!
thesis examples essaymla essay headingintroduction essay
Wow that was strange. I just wrote an really long comment but after Iclicked submit my comment didn’t appear. Grrrr… well I’m not writing all that over again. Anyways, just wanted tosay superb blog!
What’s Happening i’m new to this, I stumbled upon this I have found It positively helpful and it has helped me out loads. I hope to give a contribution & assist other users like its helped me. Good job.
Greetings! This is my first visit to your blog!We are a collection of volunteers and starting a new initiativein a community in the same niche. Your blog provided usbeneficial information to work on. You have done a outstanding job!
Wow, this post is pleasant, my sister is analyzing such things, so I am going to inform her.
Aw, this was an exceptionally good post. Spending some time and actual effort to create a really good articleÖ but what can I sayÖ I hesitate a whole lot and never seem to get anything done.
Pylkoa rquhyj walmart pharmacy canada pharmacy
Wow, great blog post.Really looking forward to read more. Want more.
Лучший частный эромассаж Москва в боди салоне
Way too many bloggers nowadays however very few have articles worth spending time on browsing.
ivermectin cream 5 ivermectin for sale – ivermectin topical
I have to thank you for the efforts you have put in penning this blog. I’m hoping to check out the same high-grade content from you in the future as well. In truth, your creative writing abilities has encouraged me to get my own blog now 😉
what is chloroquine used for shortage of hydroxychloroquine chloroquine phosphate medicine
I don’t even know how I ended up here, but I thought this post was good. I don’t know who you are but definitely you are going to a famous blogger if you are not already ;) Cheers!
Pretty! This was a really wonderful post. Thanks for providing this information.
Your style is very unique compared to other people I have read stuff from. Many thanks for posting when you have the opportunity, Guess I will just bookmark this web site.
Thanks in support of sharing such a good thinking, article isfastidious, thats why i have read it fully
Sweet blog! I found it while surfing around on Yahoo News. Do you have any tips on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Many thanks
Great, thanks for sharing this blog.Really thank you! Keep writing.
ciprofloxacin dangerous erythromycin uptodate ciprofloxacin gastritis
ivermectin lice – ivermectin new zealand ivermectin uk
do my coursework online coursework moderation
Hello Тhere. I found your Ƅlog using msn. Thisis a really well written article. Iwill be sᥙre to ƅookmark it and cߋme back tto redad more of your userulinformation. Τhanks ffor tһe post. I’ll definitely comеback.Herre is mmy wwbsitе – lihat disini
Great info and straight to the point. I don’t know if this is really the best place to ask but do you guys have any thoughts on where to hire some professional writers? Thanks 🙂
Merely a smiling visitor here to share the love (:, btw great design . “Audacity, more audacity and always audacity.” by Georges Jacques Danton.
It’s hard to find well-informed people for this topic, but you sound like you know what you’re talking about! Thanks