Open Net Zero logo
Current and projected research data storage needs of Agricultural Research Service researchers in 2016
L o a d i n g
Owner
United States Department of Agriculture - view all
Update frequencyunknown
Last updatedover 1 year ago
Overview

The USDA Agricultural Research Service (ARS) recently established SCINet , which consists of a shared high performance computing resource, Ceres, and the dedicated high-speed Internet2 network used to access Ceres. Current and potential SCINet users are using and generating very large datasets so SCINet needs to be provisioned with adequate data storage for their active computing. It is not designed to hold data beyond active research phases. At the same time, the National Agricultural Library has been developing the Ag Data Commons, a research data catalog and repository designed for public data release and professional data curation. Ag Data Commons needs to anticipate the size and nature of data it will be tasked with handling. The ARS Web-enabled Databases Working Group, organized under the SCINet initiative, conducted a study to establish baseline data storage needs and practices, and to make projections that could inform future infrastructure design, purchases, and policies. The SCINet Web-enabled Databases Working Group helped develop the survey which is the basis for an internal report. While the report was for internal use, the survey and resulting data may be generally useful and are being released publicly. From October 24 to November 8, 2016 we administered a 17-question survey (Appendix A) by emailing a Survey Monkey link to all ARS Research Leaders, intending to cover data storage needs of all 1,675 SY (Category 1 and Category 4) scientists. We designed the survey to accommodate either individual researcher responses or group responses. Research Leaders could decide, based on their unit's practices or their management preferences, whether to delegate response to a data management expert in their unit, to all members of their unit, or to themselves collate responses from their unit before reporting in the survey. Larger storage ranges cover vastly different amounts of data so the implications here could be significant depending on whether the true amount is at the lower or higher end of the range. Therefore, we requested more detail from "Big Data users," those 47 respondents who indicated they had more than 10 to 100 TB or over 100 TB total current data (Q5). All other respondents are called "Small Data users." Because not all of these follow-up requests were successful, we used actual follow-up responses to estimate likely responses for those who did not respond. We defined active data as data that would be used within the next six months. All other data would be considered inactive, or archival. To calculate per person storage needs we used the high end of the reported range divided by 1 for an individual response, or by G, the number of individuals in a group response. For Big Data users we used the actual reported values or estimated likely values.

NAL-KSDdata scienceinformatics
Additional Information
KeyValue
dcat_modified2023-09-26
dcat_publisher_nameAgricultural Research Service
guid88482aa1-9369-4b36-a6dd-a6f6b3b24306
language
harvest_object_id0d4a5af6-8f3b-44f1-8b14-10a73562adb6
harvest_source_id2c0b1e04-ba48-4488-9de5-0dab41f9913f
harvest_source_titleUSDA Open Data Catalog
Share this Dataset
Trust Signals
Trust Framework(s)None
Assuranceunknown
Data Sensitivity Classunknown
Licenceunknown
Files
  • Appendix A: ARS data storage survey questions

  • Responses from ARS Researcher Data Storage Survey

  • CSV of Responses from ARS Researcher Data Storage Survey