Storage is big at USCF. Research requires the use of large data sets. Library needs to save archives. Every system in operation needs some form of backup. Disaster recovery. Analytics. Etc.
As time goes by the demand for storage continues to increase and cost pressure continues to be a major motivator for target storage services. We are actively working to reduce the cost of storage and provide easy access facilities for data transfer to and from the cloud, lifecycle management of data reduce overall costs and versioning and replication for critical data sets.
Currently in process is our Basic Storage Service MVP. This MVP will be the precursor to a suite of storage services offered by the central ITS Cloud Services team. In its initial implementation an individual will be able to provide basic recharge information and receive an auto provisioned AWS account preconfigured under our master payer account organization in AWS with the appropriate level of security controls to qualify for coverage under our BAA with AWS. Inside this account a single bucket will be auto provisioned and made available for access with our campus authentication tools and single sign on access using myaccess.ucsf.edu.
On the roadmap is the ability to connect one of the S3 buckets in your AWS account to an on-prem ITS hosted file server that you can access using standard Windows File Sharing or via NFS on Linux.
We would love to hear any further suggestions. Please stay tuned for more details!
We had a great turnout for our first cloud community learning event. Kevin Murakoshi presented an engaging overview and demostration of using serverless technolgy in support of student data analytics. There were a number of great questions and I recieved several comments from the audience asking for more presentations on the same lines. We are working with AWS as well as other cloud vendors to develop external presentations and welcome anyone from the community here at UCSF who would like to present on their architectures and solutions.
I have the recording of the presentation available for download at the following Box location. Please pass along to those who might have missed the presentation.
Serverless Student Data Analytics on AWS
After the presentation I recieved a follow up from Kevin:
Hi Ryan,
Thanks for hosting me on the call yesterday. There was a question at the end about creating glue jobs via cli/api calls and how that compared with uploading code to lambda.
It looks like CLI/API control of glue jobs is considerably simpler, you don’t have to build zip files the same way that lambda requests them. You can upload code to S3 and then point the glue job at the code in S3. Libraries can be added in the same way.
https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html
h...
Cloudy with a chance of digital transformation... (cio.com)
We all need information to do our work and research. The ability to consume, process and disseminate information at the speed of demand and at the scale of enterprise is what cloud technology makes possible. We are all familiar with mobile apps, web apps, even desktop-based client server apps. All these ways of accessing information are either backed by cloud technology or in the process of being transformed to do so.
There is a lot of ambiguity about the word cloud and what it means in general, and what it means to each of our ways of working. Cloud is not really a vendor, nor is it a specific technology. Cloud is a mindset and a way of causing systems to interact with each other. This mindset is what allows systems to communicate over the internet delivering computing resources on-demand, redundantly in a highly available way with nearly unlimited ability to scale all without needed to understand the underlying mechanics of how those computing resources are provisioned and managed.
The thing that is interesting about cloud and how it can support digital transformation is that since it is a mindset at its root, that same mindset can be applied to our internal private technology stacks, and it can also be applied to the very way that we work together across teams to empower research, accelerate education and deliver a high touch experience in our patient care.
...