From ea31be602e8263382d4577064e919e34ac5462e1 Mon Sep 17 00:00:00 2001 From: Tom <tom@tom-librem> Date: Tue, 8 Nov 2022 10:18:56 -0600 Subject: [PATCH] add ATTIC guidelines and best practices document --- content/good_hcc_practices/._index.md.swp | Bin 0 -> 16384 bytes .../attic_guidelines_and_best_practices.md | 45 ++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 content/good_hcc_practices/._index.md.swp create mode 100644 content/good_hcc_practices/attic_guidelines_and_best_practices.md diff --git a/content/good_hcc_practices/._index.md.swp b/content/good_hcc_practices/._index.md.swp new file mode 100644 index 0000000000000000000000000000000000000000..45769c1032ad5c242e41d59c17632b65ad1883bd GIT binary patch literal 16384 zcmYc?2=nw+u+TGPU|?VnU|^^%$V#3l6~mZX%)n5Rp9>Nth)eQwb#pS4ic$$EuPe#V z)h|gcF450OPS#DyPcGI^&d)1J%`4GQ&(BYZ&qz*=FDOb(F3C(zE!L0E%u7kF(92CB zX#c2!(GVC70b)Xcm%-S`&;X=TSxHerSSS?49L1v{Fd71*Aut*OqaiRF0;3@?8Umvs zFd70QBm_zdm>B9A7#NtK{xyTrjA%47lrImZ&7m|*9h5stjfTKz2#kinXb6mkz-S1J zhQMeDjE2By2#kinXb6mkz-R~z!VpMIVPFtuU|?Veb>syY7-0SXKl}^~_xTwZ*77qj z%;#rdn9a|?FpHmo!Hu7R;Ugad!!kYwhNXNA4Ac1-7+m=n7+&%+F!b><FzE9#FkIwe zU})oEV6f$3VEDq#z_6a1fx(ZPf#Dq&1H)S`28O#_3=CCV3=Fnh3=H=;85pWK85sOI z85rJkFfbh8U|_K0U|_h<&cM*f&cL9`&cJYyje((pje$Xqje%hwD+7Z&D+9w576yh1 zEDQ_=EDQ`6nHd;rnHd=5m>C#knHd;lm>C#;F)=XAVPasgVq##p#mK-=#t88v8_1uS z@sL_k0$zBKpQn(YrjVGEqmY)FlUl5hU!;(zP+Xc+oLT~s01N6Vgk+>DXq6-uX(^=T z7v(0F6e}bar79Gc6s0ESD&!YsrskETrsyan=A|ekDx{U><S3-1Cg<lC6r~myXXfWA zFo2gffDA0jNQKy>r{J4kl$u|bTBM^;m0180R7g(DQ%FixC@jrP&d#Y+0BcK4QAny( z0BI~sEXvF;Ee5ME$<NOzhFY1ElV7gDP?TDblb8%qov2WpnU|geu}e=OJXN6x<e=Qt zycC5}kd%T}Rc3*fLSix4;KZWjjLfoBsP90c2q$HyrWPn9<|#1bCgxQt6z9UjN=Km} zCpED+RiU7?1Y}Av#I|CE%sh~KxS5$CXMlrEN1?bhIYXf&BN41Y0c;{Th{55>prxRt z6<VCC0M%NoP?E1uQk0lioR(Uokds)Ho~n>nnwykb1PV#8K0Pfh1<y2v%KTEW2?eP| zpa{y$O9zK1LUVqa0)w)$f@fYyYEfQd4mhS<E1(>|)ROZ2qU>S@28Pm-%$&?BkpA5K zqEv;%l(NJ;P>>cR7A5B7q~<7i>ia8zmcAqwWu_KG{gYU%paoZuk&<Sr1vVccmX}(R zoRX%c$KaV)T#}lYqN9+UpI4ljl3Jutl$w|VvJRAV%8N2fKx~LllXCKtvx`AKO-U?C z1O;ktYHof}r2;6;D3oO8rs{xFD?>_VakdUfx-!4CNTHx8KfNe17Zffb8JWcji3J5k z`H9IH;KW*7oC=9=P}nIHR~DC~=7OD?n^>ukoLE|%s*s+Vms*sVqmWr#T$);}rvQ!< z$Fls)6osP1g3OehN`?G_)I5+gK~7H2$p@z<L`>>{QcrnmP7a7pD@rXaP0cIGsl@8h zl6(aQP;vsL?Yz|F)Z*gAqDpYg6)WVWrlx@6KPgoqHM1lmwMZd1zYHRjs8Envl$)5B znpdJwT#{dum<|bwl+>KmlGGGEkel^EF{-7I2}+G6sd*_7ok^(*T3RKkxdr(}Ae(aX zlM_oo`7<rQ2y7C_TUq%@3i+ia1*M=|o|gh~j+U05f@4}qY7sc`>L@Ve=I3RW<b$}W z#i0C}Uy@j?P?VZpnv+<RQwhnJ5chx!7myD@fes4(#1e2?&C4$Vdoncz6phLG1>iiM z3-U8K?IHXKbEcjGLr`jBat6qiB^gNef}$@mB_$K&m&BZ$N@!$*G9}pg(5MENV@0WH z`9-PlyqlYtovKh=T9gXTLr5W)0V*2G^A!q9!Cuo<$j<|p4+>gZAde^{=a=S{XhBjs z$alr5$@zIHNUA{TQlYpYF*y|!cnr}&rFr0F<K*KQ9HQWulbN2Eo0?Y=t5IETqfnHZ zQ<R#fpi~4=6rYrnSX>gH2vek_U{_nKsZfxZ4sk<bo<eFxVs1fBsvZL<p@6~=9A#kd z7AvF`<>xAB=@%y#C6**-Xo12MnuxTZ2_wHqK}$b5KQ}i&PfJfB08)?@r>26#i~&@- zLt-yQp*$ls4=f5w7-^Y#nZ+5Pj1iJiTC4*W2RT)txFodzlvhDPkp!|-p*XoHv!Fzw zI3vF_Cq)4kzX}ZC(1YeZP|W6lT?fjdpd_1Gq>z`Nl3EN3GjOeg@I*>xQEGAtsJ4Mr zKc#t@g{7&Wyr7Vkp9D#A3|b1o`MIg!Vk#4~(yka(tY_sX6)S)eNm6QFYFcIqG<>u` zj)Ep81#lq)DsMp*RAO<7jshZ|z}x^S4V?3H3rfKS70AJ9Ir-(quzD*mH9fH;vn&-< zb>$W$CYL~JqtZNB?#|4qL@CI@<r_Fe@(WUnz?rdFOAAz-CFhia3r4UctP&{JLk_-T zg_P1Fko&=I&&^DS7yv4uAgp3gQQ(7AZ@{gCxVQwA_!ILKGV}6_QZn-rOHvgQb3y4E z9Fv*3si3HV*$paU^NT=514uRxQfDwgX>huOX0u`)h1BAL)Z|QXR!l5WC{9iUB~-9a zKy{-6sBr-<aw_wZLGDNRT|px$F*&=m093P=mgMIqf<h1)O`6b9P6kDAY6Ymcgama- zVg<;QwEQ9lEu>Zj%xU1zf%^i<E^s}ZnOByd1FHI=MK#E^kc_HPo|u!JnU}5!@^5Nc zYEdNvNKIxDsF{RW2k0o2XJjU4fQsCr)ErncNUbPH%>yMuP*iAX`Q<AF_y>DNAcq$y z1%qnsVg+!uQJz_n0ct<MeTZJ?gVIN0u>z<hgct+&5~zxI12s2-!Btx^14B-J2{_J6 zGC&n=F{pu~0BV?J=72&hv7jI)6H)~fgDcbIjMQY1m%;fOl;4X>5=%;p6>>{+N-_&_ zQo+@3u|lFkZf0I-Nh-uoklHaPu`~}-G{Y4aXM&iCd8wc_TR~AiD8@ix2PsZ+^3yZ( zz?mIhS%b1@W*#Wzr$Sq44AI4<Nx7LN3W=aR4o;34sS1z+rm!@%G!?b@geV5(&3I6r zhZLK74AG#X0NmUyOI0X_v`JAlCKsh9g0&^)#Um*OnF28c96Sv0kVb0yfY$ed12Zp0 z0a|>6S{=v%o0keJ20={>P+Jldxu7Z;T#FT@7U!21C8rjH+agJ+3MHvUx!~-WqNm^n zYLVwvDmVv(g6c_V8wlh^aIyk5dttis(F0ZiRF3Cl=4OIh=f(L7pcZUSYF>ItMkUBF zgefJ7#n}qwnK?NM8JX!Bph7A$8&rxhV3-Li^Ak%TYB7pCaJqpuLO~5jaC*xJw_Zw% zQ;Uj`8~vb~f&tWqLPQ$a|In5NC`dtZs0V7_CMp!A7N>%m#h@l*X&xvEf|7A*3Ao4v z7c^*|VSrXVNuZ<zsvq-mD#6al$xP2EQ2==uDH3z@%Rm-_oRbP_M?f3U#h{jIZUH2> zK%17uI*@9>DODjmGY{NG19=5KIj1ThL_lRiaYkZMY6_@)%}WP2iNFB@_HkwjxN+_S z&Jce2DX9z$A^8f4;6`UjMrN^&LU}5vVOy4&0;!+V@<BZwkb~1pGgDG?GV@X)SsS#f zT*1THSpmMPT#o@%AA(xTAm4&YPO!b|3ZPZzWtk<JsgRlxdsHUplopqy7C{mw$d_Qv zdI}8AMTvQ-V8;iSC+6uWfSTSV<@w;$ql=sXU?%8*(;`M7fb_x4WYE^uW++buM~|(7 zlBKB<LrN;76#;6a+A1iy6Y{MRLrG>yPO1V(ZGL_VI4A<Z89zC-SP5(asQ-VOiGd** zG{G;xzyO=y-^S0tu$7;IVJ<%dgC2;@$H1V$$H1_jmw~~Imx19j4+BFi4+FysZU%-K z+zbrT+zbr!xfmD{xEL7TaWXI@a56Bw<X~V3<X~WU#LmEw!p^|(2-=VLU}Io7!pgv) z$I8I4frWt~goS}2n1zAi4l~3ZNg%gi_R&X0MnhmIhCpI*ab|G|sQia_vq0Nk5|c~7 z88W!Epdi1fL?N+60o3)$OwP|!$V<#kwN*;ZD@)DEFGvMfxzV6KGsO_K4yAcHdZ{U; zu^PFFnK>o-R#@aT^%%mzV^aBf>EQl0sCNSz?*R>tl;ndu;NTJqQhF+YN>&9>`3Z6; zxRwDa%t$N)m1~ut5sKv0B2Z0}lwVo`>Kzs-Alm!+Mc}?GxI$%MC@oG+E6q{J%mX#! zz%>%MB1EdQK;v4Vx*t@A=A~AYfE!ZazO+I?Vo^zEGN|8-+-v|<8AYXedf*x|IyW^p zz9<zGYN*W?L|qALsT705O;b-HJOk7SgmjU?<7D8LGpNH|P?VaInG6|yf_5C?tq)Mm zo(dWV$%9um3=rMmQLDsag=ohdP+gO#5C9ox0)?DH5ZGPCs1DOdZd!sk@!%doT4u6- zBE*dNf};F1aK##*n3octlA2VSo}QVPj*v�rfl7f1v6Y)HN(B&4UCZsMdzIi%N?@ zUF4M10#L+fq$ZYuI$Y2;43d+;=>QZL;J!g=acYX5LS%j^IGCVe3W;$fJ?fA#nPLTS z1Dye+8)_-Yeozgo0PP-ty7kbJr-Gu)#FEsUN<D_mVx(prIH-$KL6MvYt}_*qGr)lf z?vj9vE=o--&d&oWfHW0TQ$S51NHbAKp*SBl_J}%u0qR&W7`i)wdMrVXzB&qq3K^g( zHa`vCGzB>n(pQF$k%F3*$@xX8T3R4G3R06HU06K@|1{8Ewai@5NFO9M!4nI(ItQhw z)V%!C^bBzR0l6F0X<z`4N#^A%K*C1{97_;8LDrxJ6KEhA)LTr;OicmxF~P%lkT#)0 zuuo`^FLWdU9A*kyS}Cb%iKRIuAg4lt1Js%V4P=2@S4F7`4E~_@W_f0DstzO$K~4ho zERs?c$`c_YjG)dZ!Yd{D3i+VcVP<Y_Y6>L26w(qia}<(NA%lYXX$mEk1^J*Rb8coa zc)%~QD7_RE&w306pyDns9g^e0jTe|4s1uu)0&;{xN`9Vti2}HA0ht6I-hd^zlzd28 z1R0l3%u}F3cLh9f=>#6?0*&V9W|n|@H;`VGjzVS$C}F`z#6caAd<8^G0w+Q(El_a? z>OiDcq$WegJ3tYPlp_?9^NT=3GNAE3m|}*^+=3!VV;ZD7IX@*;N1?bN2kHQj7$`n6 z^Gfm|W9gtF#l+%lNdFTO1mIq1L4IZ)*v)7~kc)4Ka{y{#lagDKTo8|<3tAF_#`Cm5 zWkqgcMP_bkE=n#?0L^`r7Ae4*sGu-0S4c^$ECzK*A*1R=;BF_Vq%2kdk4l2F52)M* zb+r;pkn1<dcs62)JH05sv;dqciZejf9k|D#0QNYjUjs4(+KmDgWI6eH>8V9}3}Ev? z1qvkZfHDNM=L~A@mt^D@r-GaW>JxxR8$bygl1K}R@{@8>b0KZ@qSTbqWJqVBI5{IV zr8FnCNF6jlTU-Ddc`r-NOD%?U8NdS@`FW|}6r-gD>C8j2A2|3SgYk%I2Z(1hazOe^ zG7|F?G7@vrK;sqI3K~sl4hDG`6sVx6fV)Sb7*t9_!xdbf7J<i&K&1~<Uw#2-NVgc8 zk&%-gxUUMTQNcOa5R?>CLDNLg(m{^_G{C8(tD6T7b|ozZs8OJj5nQT(ODs^T1sMVw zr+|!aK*ve)i@+vA3wv;pgk-ypLK37&0L|Bsx~&paF?oW+0!ghxDQMCGrv;!=EwKbP z!~;oaCHbH^Fo>a`LJ?fyfO3u&+(V!~mM&<ZR7p#L0amJk(++CUmYJsj9wxzEWq|6e z)MC&83#7_POv*u?Wdar0#d-|TE}cSgW_n&`T4r)0cupfJwFERJ03IQR<PK1RFf@aP z9jNdC)vchhTz^o8NmYQ11!N@VrRadGfz)DfA_Hp%Tcwbh2O5h3g&D}l3|a~y;3xqV z?s+K+{spOdz5&P$CCEr7WZo(jF;E3D3>0$U5l2u^=_z>RmxGeKjzVsJF?f0tWCc=O z6@#jmd<9Uo#sD2Q0S!69D@uqdzL~|4athpnL~itfGGj(!Q7$;wBU*pz#qf#;97Tvo zNraC1f#w=AQjxs{8x#b!3iKEt_CXuNkU11kcJdAIMAm~NC_t`;lnUTL0gqFHhVo$H znqP!9SV6G|E<d5w2q>?>5(_*zqnQmVuE3cEQh>l~ZnU}uE!e>|B-lD-Wd+zYi6he3 zBm=1R1hP95G=u_5;UE(r6+=nBLUvw$xk6?dG}H@AQ;R{P#-QQmVn~&nl$x5Skd|MX Im!ii20B-Ow$N&HU literal 0 HcmV?d00001 diff --git a/content/good_hcc_practices/attic_guidelines_and_best_practices.md b/content/good_hcc_practices/attic_guidelines_and_best_practices.md new file mode 100644 index 00000000..084fe6db --- /dev/null +++ b/content/good_hcc_practices/attic_guidelines_and_best_practices.md @@ -0,0 +1,45 @@ ++++ +title = "ATTIC Guidelines and Best Practices" +description = "ATTIC Guidelines and Best Practices" +weight = "100" ++++ + +## Introduction +ATTIC is a near-line digital archival storage service for research, scholarly, and creative activity data, provided to those affiliated with the University of Nebraska. A [near-line archival storage system](https://en.wikipedia.org/wiki/Nearline_storage) is not typically directly accessible (i.e., mounted), and uses storage technology that can be accessed more rapidly than traditional data archival media (e.g., tape). HCC designed ATTIC to meet the requirements of a near-line archival storage system and be as affordable as possible. ATTIC is not designed for active use. Digital storage systems that provide online direct access are designed to be performant, can require specialized hardware, networking, and/or software, and are comparably more expensive to implement and maintain. + +ATTIC uses the [Zettabyte File System (ZFS)](https://en.wikipedia.org/wiki/ZFS), an open-source, widely used, flexible, and resilient filesystem. ATTIC's storage configuration is designed to follow modern best practices and has multiple redundancies built-in for different types of potential failures. ATTIC is installed in climate-controlled facilities with restricted access and generator-backed redundant power. Copies of every file exist at two distinct geographic locations (Lincoln and Omaha), connected by a high-speed redundant network. + +### Expectations + +- HCC is committed to reducing time-to-science and maximizing value for researchers at the University of Nebraska in the areas of computing, networking, and storage. +- ATTIC is not free; it is a community-supported resource. +- HCC depends on the good citizenship of researchers to collectively provide a positive user experience on shared resources, including ATTIC. +- Users are expected to follow [best practices](#guidelines-and-best-practices) when using ATTIC to collectively help prevent excessive load and degraded performance which can be harmful to the larger community. +- HCC reserves the right to curtail problematic use of ATTIC (and other resources) to protect the integrity of the systems. + +### Guidelines and Best Practices + +- Allocations and Billing + - ATTIC storage allocations are leased to single HCC groups in one terabyte incremental units. Suballocations are not possible. + - HCC is generally not allowed to lease ATTIC storage allocations outside of the current fiscal year[^1]. ATTIC users normally lease for a full year with one payment at the beginning of the fiscal year. If an HCC group begins to use ATTIC in the middle of a fiscal year, leasing charges can be prorated to the end of the current fiscal year. + +- Data Transfer and Sharing + - [Globus](https://globus.org/) is the preferred tool for transferring and sharing data. + - It is the responsibility of the researcher/PI to ensure that data is shared only with authorized persons. Public sharing should be used with care. + - ATTIC should not be relied upon for a permanent archive (referenced in publications). Campus research data librarians should be consulted with for options related to data publication. + - Archiving software tools such as [ZIP](https://en.wikipedia.org/wiki/ZIP_(file_format)) and [tar](https://en.wikipedia.org/wiki/Tar_(computing)) should be used, when possible, to combine many small files into one to reduce the total number of files[^2]. This has the beneficial effect of increasing computing workflow efficiencies and reducing the burden on ATTIC. + +- Backups and Retention + - The ideal ATTIC data storage use-case is a set of well-curated, archived (written once, read seldom) data files. Automated, frequently synced, rapidly changing, remote system ‘backups' are generally discouraged as they can produce excessive system load and are outside the scope of ATTIC[^3]. + - On HCC compute clusters, home directories ($HOME) are backed up automatically as a provided service. The $WORK and $COMMON filesystems are not backed up. Any precious files should be stored on $HOME, ATTIC, or other trusted storage systems. + - If a researcher unintentionally deletes a file on ATTIC, they can [contact HCC support](mailto:hcc-support@unl.edu) for help in restoring it. Note, the window for recovery is limited so HCC must be informed as soon as possible. + - When an ATTIC allocation lease is not renewed the researcher must move any files to other storage systems if the researcher wants to keep them. + - If an ATTIC allocation lease expires, HCC will remove any remaining files ninety (90) calendar days from the expiration date. HCC will make a good faith effort to contact the researcher and/or their department of record before removing any data. + +- Sensitive and Protected Data + - All researchers using ATTIC must adhere to the policies described in the [University of Nebraska Executive Memorandum 42](https://nebraska.edu/offices-policies/policies/no-42-policy-on-risk-classification-and-minimum-security-standards). + - It is the responsibility of the researcher/PI to ensure that no ‘high risk' data (e.g., HIPAA or other PID data sets) as defined in the [University of Nebraska Executive Memorandum 41](https://nebraska.edu/offices-policies/policies/no-41-policy-on-research-data-and-security) are stored on ATTIC. + +[^1]: In some circumstances multi-year leasing may be possible, please [contact HCC](mailto:hcc-support@unl.edu) for details. +[^2]: [https://hcc.unl.edu/docs/good_hcc_practices/#file-systems](https://hcc.unl.edu/docs/good_hcc_practices/#file-systems) +[^3]: Please consult with HCC before engaging in ongoing frequent network data transfers to or from ATTIC. -- GitLab