Database Sharding Explained: Techniques, Trade-offs and When to Use It

Database sharding basics: geo, range and hash-based sharding, manual vs automatic sharding, hotspots, and the pros and cons before you split your database.

By Tech Team Strategy · Nov 17, 2026 · 3 min read

Database Sharding Explained: Techniques, Trade-offs and When to Use It
Watch the interview

In a short explainer, Aced (formerly Exponent), the interview-prep company, covers database sharding: what it is, the main techniques, manual versus automatic sharding, and the advantages and disadvantages to weigh before you split a database.

This article is our own summary, with short quotes, from the video's automatic transcript. Watch the full video above for everything that was said.

The short version

  • Why shard. Traditional relational databases hit CPU, memory or disk bottlenecks as data grows, and need ever more expensive hardware. Sharding breaks large tables into horizontal partitions, each holding a subset of rows, and puts each on its own database server. Each partition is a shard.
  • Three common techniques: geo-based, range-based and hash-based sharding, each with its own risk of uneven data.
  • Automatic beats manual. Some databases repartition data on their own when it becomes uneven. Many monolithic databases can't, which pushes sharding into the application, with real costs.
  • The upside: scale out, faster queries on smaller indexes, partial outages instead of total ones, and cheaper commodity hardware.
  • The downside: not all data shards well, cross-shard joins are expensive or impossible, sharding is hard to undo, and every shard is a production server to keep available.
Rate this story
00
Vocabulary14 terms

The words worth stealing from this conversation, what they mean, and who said them.

Elegant words

amenable
Open, responsive, or suited to a particular treatment or process.
Database Sharding: “not all data is amenable to sharding”
backward compatible
A system or design that can successfully work with input or data from older versions.
Database Sharding: “if schema changes are not Backward Compatible the system will need to make sure that all shards have the same schema copy”
bottlenecks
Points of congestion in a system that slow down or halt the entire process.
Database Sharding: “however as data size increases traditional database systems run into bottlenecks on CPU memory or disk usage”
commodity hardware
Standard, relatively inexpensive, and widely available computer hardware components.
Database Sharding: “smaller amounts of data in each Shard mean that the nodes can run on commodity hardware”
runtime
The period of time during which a software program is actively executing or running.
Database Sharding: “the application has to figure out how to rebalance the data partitions at runtime”

AI vocabulary

AI
Computer systems capable of performing tasks that historically required human intelligence, such as learning and decision-making.
Database Sharding: “We help owners turn AI trends into a plan, a product and a launch.”

Tech vocabulary

application layer
The part of an application program where user-facing logic and data processing decisions are made.
Database Sharding: “if you need to continue using these databases but you have increasing data demands then the sharding needs to be done at the application layer”
latency
The delay or time it takes for data to travel from one point to another over a network.
Database Sharding: “this technique allows users to be routed to the node closest to their location thus reducing latency”
monolithic databases
Traditional, single-tier database systems where all data and processing are managed as a single, unified unit.
Database Sharding: “unfortunately many monolithic databases do not support automatic sharding”
normalization
The process of organizing data in a database to reduce redundancy and improve data integrity.
Database Sharding: “a process of normalization would instead store the data in Separate Tables joined together by Foreign Keys”
rdbms
A relational database management system that stores data in tables consisting of rows and columns.
Database Sharding: “traditionally data has been stored in an rdbms or relational database management system”
replication
The practice of copying data across multiple servers to ensure high availability and reliability.
Database Sharding: “you need to ensure High availability via replication or other techniques”
sharding
The process of breaking up large database tables into smaller, horizontal partitions and storing them across multiple servers.
Database Sharding: “this process of breaking up large tables into horizontal data partitions... and putting each partition in a separate database server is called sharding”

Trends and buzzwords

AI trends
The current directions, developments, and popular movements in artificial intelligence technology.
Database Sharding: “We help owners turn AI trends into a plan, a product and a launch.”
Quiz: two questions+10

Get both right to win 10 tokens. One try only.

  1. 1. What should you try before deciding to shard your database?

  2. 2. Which of the following is a listed downside of database sharding?

Free account. Your answers are kept while you sign in.

Sharding techniques

  • Geo-based: partition by user location, such as a continent or "East US" versus "West US," usually fixed at account creation. Users reach the nearest node, which reduces latency, but users may not be spread evenly.
  • Range-based: partition by ranges of a key, for example the first letter of a first name, which gives 26 buckets for English names. Simple to compute, but splits can be uneven.
  • Hash-based: hash the key and use the hash to pick the partition. A good hash spreads data evenly and avoids hotspots, but related rows often land on different shards, so the server can't predict and preload future queries.

Manual versus automatic sharding

With automatic sharding, the database manages partitions and rebalances them when it detects uneven data or queries, which gives better performance and scalability. When the database doesn't support it, sharding moves to the application layer, and the video lists the downsides:

  • Much more development complexity: choosing the technique and the number of shards from projected data trends, then rebalancing at runtime if those assumptions change, and knowing which shard holds each piece of data.
  • Uneven distribution as data trends drift, creating hotspots that can cause performance problems and server crashes.
  • Repartitioning when the original number of shards was too low, which is hard with no downtime.
  • Schema changes that must be applied, and data migrated, correctly on every shard.

Advantages

  1. The system can scale out as data grows, beyond what a single relational database can handle.
  2. Smaller data per shard means smaller indexes and faster queries.
  3. An outage on one shard leaves most of the system available.
  4. Shards can run on commodity hardware instead of expensive high-end servers.

Disadvantages

  1. Not all data is suited to sharding; foreign key relationships only hold within a single shard.
  2. Manual sharding is complex and can create hotspots.
  3. Cross-shard queries such as joins are very expensive or not possible.
  4. Once set up, sharding is very hard (on some systems impossible) to undo, and the shard key is hard to change.
  5. Each shard is a live production server needing high availability through replication, which raises operating costs.

What it means for your business

  1. Don't shard first. Try indexing, caching and read replicas before splitting your database.
  2. Choose the shard key with care. It is hard to change later and decides whether data stays even.
  3. Prefer databases that shard automatically if you expect large growth.
  4. Design so most queries stay inside one shard, for example by customer or region.
  5. Budget for operations. More shards means more servers to monitor, back up and replicate.

Video details

  • Title: The Basics of Database Sharding and Partitioning in System Design
  • Published by: Aced (formerly Exponent) on YouTube
  • Watch: youtube.com/watch?v=be6PLMKKSto
  • Note: this summary is based on the video's automatic transcript. Check the video for exact wording.

Want to put this to work?

We help owners turn AI trends into a plan, a product and a launch. Book a free call or see our services.

Full transcript

Read the whole interview, word for word, with a timestamp on every answer. Free with a sign-in.

Sign in to read it

Originally published by Tech Team Strategy

What did you think of this story?

+10 tokens

Tap to vote ↓

GreatAverageNot great

Sign in to vote

Comments

Sign in to commentJoin the conversation and earn 10 tokens.

  • No comments yet. Be the first.

Get the briefing

The week's essential stories, free in your inbox.

More