Hybrid storage and data transfer
Many companies keep data on-premises while using the cloud, or need to move very large amounts of data into AWS. These services bridge that gap. Task statements 3.1, 3.5 and 4.1 all touch them.
AWS Storage Gateway
A hybrid service that gives on-premises applications access to cloud storage through standard protocols, with local caching.
| Gateway type | Protocol | Backed by | Use |
|---|---|---|---|
| S3 File Gateway | NFS / SMB | S3 (objects map to files) | File shares backed by S3; data lands in S3 for analytics, lifecycle, backup |
| FSx File Gateway | SMB | FSx for Windows File Server | Low-latency local access to FSx shares (AWS has limited availability of this type for new customers — check current status) |
| Volume Gateway — cached | iSCSI | S3, with frequently used data cached locally | Primary data in AWS, low-latency local cache |
| Volume Gateway — stored | iSCSI | Full dataset local, asynchronously backed up to S3 as EBS snapshots | Primary data on-premises, backups in AWS |
| Tape Gateway | iSCSI virtual tape library | S3 / Glacier archive tiers | Replace physical tape backups without changing backup software |
"On-premises apps need file access to data that should live in S3" → S3 File Gateway. "Replace physical tapes, keep existing backup software" → Tape Gateway.
AWS DataSync
Online data transfer service that moves data between on-premises storage (NFS, SMB, HDFS, object storage) and AWS storage (S3, EFS, FSx), and between AWS storage services.
- Uses an agent on-premises; encrypts in transit; verifies integrity.
- Much faster than ordinary copy tools; schedule recurring transfers; bandwidth throttling.
- Cue: "migrate/sync file data to S3 or EFS over the network, on a schedule".
AWS Transfer Family
Fully managed SFTP, FTPS, FTP and AS2 endpoints that store files in S3 or EFS. Keep partners' existing file-transfer workflows unchanged while the data lands in AWS. Cue: "partners upload files via SFTP".
The AWS Snow Family (offline transfer and edge computing)
Physical, rugged devices that AWS ships to you:
- Offline data transfer when the network is too slow or expensive for the data volume — load data locally, ship the device back, AWS imports it into S3.
- Edge computing in disconnected or harsh locations (ships, mines, field sites) with on-board compute.
AWS has been narrowing the Snow Family over time (some devices discontinued or restricted to existing customers). Check current availability — but know the concept for the exam: physical device → offline transfer of tens of TB to PB, or compute at disconnected edge sites.
Choosing a transfer method
Rough transfer time over a network:
time ≈ data size ÷ effective bandwidth
100 TB over a 1 Gbps link at ~80% utilisation ≈ 100 × 8,000 Gb ÷ 0.8 Gbps ≈ 1,000,000 s ≈ 11.6 days
| Situation | Best fit |
|---|---|
| One-off or recurring file migration over the network | DataSync |
| Ongoing hybrid access to cloud storage from on-premises apps | Storage Gateway |
| Partners using SFTP/FTPS | Transfer Family |
| Huge dataset, slow/limited network, deadline | Snow Family device(s) (offline) |
| Fast uploads to S3 from far away | S3 Transfer Acceleration |
| Consistent, high-bandwidth, ongoing transfer | Direct Connect (+ DataSync over it) |
| Database migration | AWS DMS (see migration) |
Exam patterns
- "Migrate 50 TB of NFS data to EFS, then keep them in sync during cutover" → DataSync.
- "Office users need SMB shares whose data must end up in S3" → S3 File Gateway.
- "Petabytes to move, 100 Mbps internet, two weeks to finish" → Snow Family devices.
- "External vendors deliver files by SFTP; store them in S3" → Transfer Family.
Senior Full-Stack Engineer · AI & AWS. I design and run production systems on AWS — serverless, data and AI.