What is S3 Intelligent Tiering and How Does it Work?

Automatic movement of data across AWS S3 tiers.

For over a decade, Amazon Simple Storage Service (S3) has offered different levels of data storage classes to efficiently assist users in need of cloud-based storage infrastructure. Several additional storage classes have been added to increase the variety of use cases that S3 can support.

Today, there are seven major storage classes available in AWS S3, which are optimized based on the frequency of access, importance of data, and archiving needs for the storage solution.

Classes of AWS S3 Storage

These are the main storage classes available through S3:

  1. Standard (S3) – Used for frequently accessed data (hot data)
  2. Intelligent Tiering – Used for data with unknown or changing access patterns or uncertain need of access
  3. Standard-Infrequent Access (S3 Standard-IA) – Used for infrequently accessed, long-lived data that needs to be retained but is not being actively used
  4. One Zone Infrequent Access (S3 One Zone-IA) – Used for infrequently accessed data that’s long-lived but not critical enough to be covered by storage redundancies across multiple locations
  5. Glacier – Used to archive infrequently accessed, long-lived data (cold data) Glacier has a latency of a few hours to retrieve data.
  6. Glacier Deep Archive – Used for data that is hardly ever or never accessed and for digital preservation purposes for regulatory compliance
  7. Outposts – Used for data on-premises that has local data residency requirements or requires being close to on-premises applications for high performance reasons

Simple lifecycle policies based on objects’ dates of creation can be implemented to move objects automatically to cheaper S3 storage classes to optimize costs. However, when the pattern of data access is less predictable or data is widely accessed by many applications, a more intelligent tiering model is often more cost-efficient.

How S3 Intelligent Tiering Works

For a fee, Amazon’s S3 Intelligent Tiering automatically moves objects between tiers within the service. When objects have not been accessed for a certain period of time, they are moved into the infrequent access tier; but if they are accessed at a later point in time, they are then moved back into the frequent access tier automatically. Users can further choose to automatically send data to archive tiers that offer asynchronous access.

This type of data management strategy can help organizations save on storage costs mainly in environments where the frequency of data access is uncertain. But it may not always be the best choice of storage class if there is high confidence in access frequency eg via analytics.

Also, S3 Intelligent Tiering is a storage class and you cannot have different treatment for different tiers in the storage class. S3 Intelligent Tiering acts as a black-box – you move objects into it and cannot transparently access different tiers or set different versioning policies for the different tiers. You have to manipulate the whole of S3 Intelligent Tier as a single bucket. For example, if you want to transition an object that has versioning enabled, then you have to transition all the versions. Also, when objects move to the archive tiers, the latency of access is much higher than the access tiers. Not all applications may be able to deal with the high latency.

This may not always be the most cost-effective option – for instance, if you want to have versioning enabled for the access tiers but not for the archive tiers, you cannot accomplish this within S3 Intelligent Tiering. You have to enable versioning for the entire class.

S3 Intelligent Tiering when configured to automatically send data to archive storage classes may require changes to existing workflows if access to archived data is required since it does not automatically restore data tiered to the archive tiers. Additionally, once sufficient time has passed such that the probability of access on archived data is low, data needs be transitioned to Glacier and Glacier Deep Archive storage classes using lifecycle policies to avoid paying recurring S3 Intelligent Tiering monitoring costs.

In contrast, you can intelligently tier data based on accurate access patterns for your custom data sets across all S3 storage classes including Glacier and Glacier Deep Archive with Komprise Intelligent Data Management. You can also set different versioning and other policies for each tier or storage class with Komprise.

S3 Intelligent Tiering Pricing

The cost of Intelligent Tiering is based on how much of each type of storage is being used, how many requests are being made, and how many objects are being monitored. Amazon charges $0.0025 per 1,000 objects monitored.

Advantages of S3 Intelligent Tiering

  • Objects can be assigned a tier upon upload
  • No retrieval fees
  • No tiering fees
  • Objects are moved automatically to cheaper, appropriate tiers based on monitored access patterns
  • No operational overhead
  • No impact on performance
  • Designed for 99.999999999% durability and 99.9% availability over annual average

Disadvantages of S3 Intelligent Tiering

  • If access patterns are predictable, then Lifecycle rules may be more cost-effective than Intelligent Tiering
  • There may be disruption to users and applications if objects in the archive tiers need to be accessed. These will need to be identified and then manually restored to the frequent tier to enable access.
  • It is not straightforward to identify objects that have been in the archive tiers for a long time so that these can be transitioned to Glacier and Glacier Deep Archive storage classes to avoid the S3 Intelligent Tiering fees
  • It is limited only to the S3, infrequent and archive tiers whereas some users may need to move data across EFS, FSX, S3 and Glacier storage classes for maximum efficiency
  • Policies to tier to archive tiers cannot be greater than 2 years
  • Minimum object storage requirement of 30 days
  • Objects smaller than 128KB are never moved from the frequent access tier
  • You cannot configure different policies for different groups or custom data sets, as it is an automated management solution that applies to entire buckets, prefixes or tagged data sets
  • Tiering configurations need to be managed and configured for each bucket level instead of an account or global level for multiple buckets
  • You cannot set different versioning and backup policies for different tiers of S3 Intelligent Tiering, the policy has to apply to the entire bucket

Alternatives to S3 Intelligent Tiering

As an AWS Advanced Tier partner, Komprise offers intelligent data management tools that can provide significant savings on AWS storage costs with strategies built from analytics-driven input.

Komprise enables analytics-driven intelligent tiering across File, S3 and Glacier storage classes in AWS seamlessly so you can maximize price performance across all your data on Amazon.

Komprise also provides analytics and cost visibility across all the storage classes to enable intelligent tiering.

See how much you could save with the right data management platform providing in-depth insight into AWS storage efficiency. Get in touch with an expert at Komprise today for more information.

Getting Started with Komprise:

Contact | Demo