What is Azure Table storage
The Azure Table storage service stores large amounts of structured data. The service is a NoSQL datastore which accepts authenticated calls from inside and outside the Azure cloud. Azure tables are ideal for storing structured, non-relational data. Common uses of the Table service include:
- Storing TBs of structured data capable of serving web scale applications
- Storing datasets that don’t require complex joins, foreign keys, or stored procedures and can be de-normalized for fast access
- Quickly querying data using a clustered index
- Accessing data using the OData protocol and LINQ queries with WCF Data Service .NET Libraries
You can use the Table service to store and query huge sets of structured, non-relational data, and your tables will scale as demand increases.
Table Service Concepts
The Table service contains the following components:
- URL format: Code addresses tables in an account using this address format: http://<storage account>.table.core.windows.net/<table>
You can address Azure tables directly using this address with the OData protocol. For more information, see OData.org
- Storage Account: All access to Azure Storage is done through a storage account. See Azure Storage Scalability and Performance Targets for details about storage account capacity.
- Table: A table is a collection of entities. Tables don’t enforce a schema on entities, which means a single table can contain entities that have different sets of properties. The number of tables that a storage account can contain is limited only by the storage account capacity limit.
- Entity: An entity is a set of properties, similar to a database row. An entity can be up to 1MB in size.
- Properties: A property is a name-value pair. Each entity can include up to 252 properties to store data. Each entity also has 3 system properties that specify a partition key, a row key, and a timestamp. Entities with the same partition key can be queried more quickly, and inserted/updated in atomic operations. An entity’s row key is its unique identifier within a partition.
I recommend watching this short video for a great overview.
Azure Table Storage 101
Real World Example by Troy Hunt
An excellent real world example of how efficient, cheap and damn right awesome Azure Table Storage is would be https://haveibeenpwned.com/
Working with 154 million records on Azure Table Storage – the story of “Have I been pwned?”
Cheap data storage
In other words, if I want 100GB of storage and I want to hit it 10 million times, it’ll cost $8 a month.
Importing data speed.
The max throughput Troy Hunt achieved with this in a single console instance was when all 17 million rows were processed in only 47 minutes – that’s a sustained average of over 6,000 rows per second for three quarters of an hour.
Troy Hunt actually had to add “sleep” time in the service because the service was returning the data too fast giving it the appearance that the service wasn’t working. “That can’t be working, it’s too fast”
154M records being searched and the connection overhead plus some validation of the email address and then splitting it into partition and row keys and it’s all wrapped up in 4ms.