We have rolled out quite a few security enhancements for the Secure Enterprise Cloud environment.  There are probably too many to cover in an email, but the big-ticket items that are already GA are as follows.  The whole goal of the platform was to ensure we're P3/P4/HIPAA compliant, so security is our main focus:

Core Secure Enterprise Cloud Platform Features:

SSM Fully-Automated Patching (managed) - IT deploys and manages a scalable, multi-OS solution for fully-automated patching and maintenance, supporting Ubuntu, CentOS, and Windows. Client also benefits from IT-managed, centralized storage and auditing of instance access logs, patch logs, and software inventory databases.  Client immediately realizes dramatically reduced administrative overhead via automated OS patching and agent installation.

SSO Integration (ADFS) + DUO MFA for AWS Console and CLI (managed): Clients benefit from IT-managed deployment of SAML IDP, IAM resources, and DUO MFA integration required to make seamless access to AWS from both the console (GUI) and/or CLI.  Client AD accounts are managed via existing IT Identity and Access processes. Users can sign-in with existing UCSF domain credentials, using role-based access only - legacy (long-lived access/secret key) access is prohibited, and no client credentials live longer than 12 hours.

Bastionless, SSO-based Access to EC2 with MFA (managed) - Client directly benefits from the elimination of administrator overhead for patching / management tasks, ease-of-use, and enhanced security via SSO-integrated access to EC2 resources with DUO multi-factor authentication (MFA) and endpoint profiling.  Client realizes direct benefit of passwordless, SSH/RDP-less access to resources for streamlined connectivity with enhanced security and double encryption (TLS/SSL + KMS).  IT handles deployment of SAML IDP, IAM, KMS, SSM, and DUO MFA resources required to make single-click access to EC2 possible.  All session logs are shipped to IT-manged security account for long-term centralized storage and forensic investigation.

Vulnerability scanning (managed) - IT deploys and manages Inspector templates and Inspector agents to all instances within the entire SEC platform to provide native native security assessment services for EC2 instances, including vulnerability scanning, CVE assessments, CIS benchmarks, security best-practices, runtime behavior analysis, and network reachability.  IT also offers centrally-hosted Tenable Scanners local to the SEC cloud region.  Client directly benefits from native vulnerability scanning, and indirectly benefits from additional integrated 3rd-party security services, including Prisma Cloud (Redlock, Evident.io, and Twistlock), Palo Alto Networks Next-Generation firewalls, Symantec, ForeScout, IBM QRadar, Tenable.io, Tenable SC, and NetSparker.  All SEC platform instances are periodically scanned by multiple industry-leading tools (Tenable SC, Tenable.io, etc)

CloudWatch Logging (managed) and CloudWatch Alarms (managed) - Standard cloud operating best-practice for alerting.  Required for both UCSF and HIPAA/NIST/HITRUST compliance minimums.  Client benefits directly from pre-configured best-practice alerts and centralized IT operations monitoring, resulting in less administrator overhead, quicker response times, and integrated 3rd-party tools.  Client also benefits from pre-installed, pre-configured CloudWatch logging agent

Centralized CloudTrail (managed) - Standard cloud operating best-practice for logging and monitoring.  Required for both UCSF and HIPAA/NIST/HITRUST compliance minimums. Client benefits directly from pre-configured best-practice logging configuration and centralized IT operations support, resulting in less administrator overhead.  IT absorbs cost of long-term storage, processing, and operational review of CloudTrail logs in a dedicated central-logging account

Centralized VPC Flow Logs (managed) - During normal operation, log-processing Lambdas work in conjunction with Kinesis Firehose to process and centrally-store VPC flow logs. Clients directly benefit from reduced management overhead, and elimination of local VPC flow logs storage costs.  IT absorbs the long-term storage of VPC flow logs (in a separate, dedicated security account), including operational costs associated with log monitoring, alerting, and incident response.

KMS (managed) - IT deploys and manages KMS Customer-Managed Keys (CMKs) and all key policies required for standard operation, including cross-account CMK permissions.  Client directly benefits from strongest available encryption enabled and configured on SEC services by default, resulting in reductions to administrator overhead.  IT takes care of key rotation and (baseline) policy management.  Because KMS encryption is mandatory for HIPAA/HITRUST/NIST compliance, and it is virtually impossible to distinguish Client-deployed key requests from IT-deployed key requests, KMS costs will always be recharged to Client.

Config (managed) - IT deploys and manages standardized set of Config rules for enabled services, including SSM and Lambda-based auto-remediations for common configuration errors and compliance violations.  Client directly benefits from best-practice Config rules, automated configuration monitoring, centralized configuration history storage, and minimized time-to-remediation (where auto-remediation applies), resulting in less administrator overhead.  Clients indirectly benefit from a number of IT-funded 3rd party services integrated with Config, including ForeScout and Prisma Cloud (Redlock, Evident.io, and Twistlock)

Lambda and SSM Auto-remediation (managed) - During normal operation, auto-remediation Lambdas are executed when non-conforming resources and events are detected.  UCSF IT also deploys Lambdas responsible for aggregating and processing VPC flow logs, which ensure clients do not need to setup and store VPC flow logs locally.  IT absorbs the long-term storage of VPC flow logs, including operational costs associated with log monitoring, alerting, and incident response.  Clients directly benefit from enhanced security and administrative safeguards provided by IT-provisioned auto-remediation functions.  Client indirectly benefits from 3rd party integrations with industry-leading SIEM tools including IBM QRadar, Plixer, and S

GuardDuty (managed) - IT deploys and manages centralized GuardDuty master-member architecture delivering native threat intelligence to the entire SEC platform.  Client indirectly benefits from additional integrated 3rd-party security services, including DUO, Prisma Cloud Prisma Cloud (Redlock, Evident.io, and Twistlock), Palo Alto Networks Next-Generation firewalls, Symantec Endpoint Protection, FireEye, ForeScout, IBM QRadar, Tenable.io, Tenable SC, NetSparker, Pulse Secure, and Infoblox.  IT ensures integration between GD and all native services (CloudTrail, VPC Flow Logs, and DNS logs) are configured to best-practice.

S3 (managed bucket policies, encryption, etc) - Clients directly benefit from IT-managed best-practice access and load-balancer logging, bucket policies, and CMK-backed KMS-enabled bucket encryption. IT has configured lifecycle policies for IT-provisioned buckets to ensure Clients do not pay for long-term log storage costs.

SecurityHub (managed) - IT deploys and manages a centralized SecurityHub master-member architecture providing security and compliance visibility to all accounts within the SEC platform.  Client directly benefits from native security compliance checks, including PCI DSS, CIS, and AWS foundational best-practices. Clients indirectly benefit from additional integrated 3rd-party security services, including Prisma Cloud (Redlock, Evident.io, and Twistlock), Palo Alto Networks Next-Generation firewalls, Symantec, IBM QRadar, Tenable.io, and Tenable SC. 

VPC + Network Security (managed) - IT deploys and manages redundant, high-throughput network capacity (50Gb/s +).  Client directly benefits from highest throughput available on AWS, over 35+ centralized VPC endpoints (EC2, SSM, Logs, KMS, STS, EMR, etc), industry-leading Infoblox DNS systems, and IT-provided management of Security Groups and NACLs. Client also directly benefits from IT's resilient, high-capacity Direct Connect circuits for low-latency, high-bandwidth hybrid traffic.  Client realizes reduced egress fees (no need for VPNs, NAT GWs or IGWs), negotiated egress traffic discounts, and reduced administrator overhead.  Client indirectly benefits from strong East-West and North-South traffic security, including integration with numerous industry-leading network-security toolsets including Palo Alto Networks NGFWs, FireEye, ForeScout, Gigamon, Symantec, Infoblox and more.  IT absorbs all TGW processing fees for traffic traversing inline-service tools, VPC endpoints, and to/from UCSF premises

BigFix (managed) - IT has deployed a local BigFix relay in the SEC shared-services environment, and handles pre-installation of BigFix agents on Client AMIs via the Golden AMI provisioning/sharing process.  Agents are automatically provisioned, patched, and updated with zero Client interraction.

Endpoint Protection - IT has automated the configuration and deployment process Symantec Endpoint Protection (SEP) agents for both Linux and Windows.  The Secure Enterprise Cloud is currently one of the first to support SEP for Linux.  IT also pre-configured host-based firewalling on Golden AMI images, ensuring AMIs comply with IT Security regulations by-default. Clients immediately benefit from enhanced endpoint protection posture with minimal administrative and/or user overhead.

The rough cost of the provisioned services is $52/mo (baseline), 36.60 of which comes from the VPC attachments.

Additional Platform Features:

100% Infrastructure-as-code - IT manages resilient Terraform Enterprise cluster from which all SEC client services and the SEC core infrastructure are provisioned/configured.  Not a single resource deployed by IT is configured manually!  SEC Administrators have very few additional privileges over SEC clients - all infrastructure configuration management occurs from automation tools (Jenkins, Ansible, Terraform, etc) with no human interaction (outside process approval).  The automation is currently capable of fully deploying a new account and configure all services in under 30min!  We have even tested a complete regional rebuild successfully.

Golden AMI + Hardening Pipeline - IT provides hardened AMIs (currently 5 OS flavors) to clients, complete with integrated central IT tools like FireEye, Inspector, CloudWatch, SEP, BigFix, etc.  Fully-automated pipeline is made up of a collection of open-source tools including Ansible, Packer, Jenkins, and Terraform which can very quickly and easily be adapted to address other cloud providers (Azure, VMware, GCP, etc).  Images are periodically patched, updated, and released to clients transparently.

VCS-backed-everything - All Secure Enterprise Cloud services and pipelines are backed by GitHub (Enterprise) VCS hosted internally within UCSF's datacenters for HIPAA compliance.  This includes everything from the TF IaC defining both client and platform service configuration to the config management roles and CI/CD playbooks that tie all of our functionality together. 

Integration with other central IT teams - The UCSF Cloud team has worked closely with many other IT teams throughout this process, particularly IT Security, Networking, Storage, and Systems.