Skip to content

Commit cb5ec81

Browse files
authored
DOCSP-47276-reshard-to-same-shard-key (#11328)
* DOCSP-47276-reshard-to-same-shard-key * adds page * adds tutorial * adds monitor resharding operation step * external review * external review; * external review * ex rev * internal review
1 parent 7aa7151 commit cb5ec81

16 files changed

+495
-47
lines changed

source/administration/sharded-cluster-administration.txt

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ Sharded Cluster Administration
2929
Drop Hashed Shard Key Index </tutorial/drop-a-hashed-shard-key-index>
3030
Config Shard </core/config-shard>
3131
Start with a Config Shard </tutorial/start-a-sharded-cluster-with-config-shard>
32+
Reshard to the Same Shard Key </core/reshard-to-same-key>
33+
Reshard a Collection back to the Same Shard Key </tutorial/resharding-back-to-same-key>
34+
Resharding for Adding and Removing Shards </tutorial/resharding-for-adding-and-removing-shards>
3235

3336
:doc:`/tutorial/view-sharded-cluster-configuration`
3437
View status information about the cluster's databases, shards, and
@@ -57,4 +60,13 @@ Sharded Cluster Administration
5760
*and* the usual sharded cluster metadata.
5861

5962
:doc:`/tutorial/start-a-sharded-cluster-with-config-shard`
60-
Start a Sharded Cluster with a Config Shard.
63+
Start a Sharded Cluster with a Config Shard.
64+
65+
:doc:`/core/reshard-to-same-key`
66+
Reshard to the Same Shard Key.
67+
68+
:doc:`/tutorial/resharding-back-to-same-key`
69+
Reshard a Collection back to the Same Shard Key.
70+
71+
:doc:`/tutorial/resharding-for-adding-and-removing-shards`
72+
Resharding for Adding and Removing Shards.

source/core/reshard-to-same-key.txt

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
.. _reshard-to-same-key:
2+
3+
=============================
4+
Reshard to the Same Shard Key
5+
=============================
6+
7+
.. facet::
8+
:name: genre
9+
:values: reference
10+
11+
.. meta::
12+
:description: Reshard to the Same Shard Key to Shard a Collection, Add or Remove Shards Concept
13+
14+
Starting in MongoDB 8.0, you can reshard to the same shard key to move
15+
data with no downtime or impact on workload. This enables you to:
16+
17+
.. include:: /includes/resharding-about.rst
18+
19+
Starting in MongoDB 8.0, resharding reads data using natural order scan.
20+
Resharding first clones all the data and then builds relevant indexes,
21+
resulting in orders of magnitude speed improvement of the resharding
22+
process.
23+
24+
Command Syntax
25+
--------------
26+
27+
You can reshard to the same key using the ``reshardCollection`` command
28+
with ``forceRedistribution`` set to ``true``.
29+
30+
The ``reshardCollection`` command has the following syntax:
31+
32+
.. include:: /includes/reshardCollection-syntax.rst
33+
34+
For details, see :dbcommand:`reshardCollection`.
35+
36+
Use Cases
37+
---------
38+
39+
Resharding is a strategy to move data with no downtime or impact on
40+
workload. Use the :ref:`<reshard-to-shard-def>` technique to shard a
41+
collection and distribute data across all shards.
42+
43+
Use resharding to distribute your collections across all relevant shards
44+
faster than :ref:`chunk migrations <migrate-chunks-sharded-cluster>`.
45+
Resharding writes to all shards in parallel while each shard can only
46+
participate in one chunk migration at a time. Resharding drops the old
47+
collection at the end of the process. There are no orphan documents at
48+
the end of resharding.
49+
50+
.. _reshard-to-shard-def:
51+
52+
Reshard to Shard
53+
~~~~~~~~~~~~~~~~
54+
55+
The **Reshard to Shard** technique lets you use resharding to shard a
56+
collection and distribute the data to all of the shards in a cluster.
57+
58+
Consider using **Reshard to Shard** when you are initially sharding a
59+
collection of any size across any number of shards. If your deployment
60+
meets the :ref:`resource requirements <reshard-to-same-key-behavior>`,
61+
use **Reshard to Shard** no matter how large the collection you want to
62+
shard.
63+
64+
.. _reshard-to-same-key-behavior:
65+
66+
Behavior
67+
--------
68+
69+
Storage
70+
~~~~~~~
71+
72+
.. include:: /includes/reshard-to-same-key/storage.rst
73+
74+
Latency
75+
~~~~~~~
76+
77+
.. include:: /includes/reshard-to-same-key/latency.rst
78+
79+
Additional Resource Requirements
80+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
81+
82+
.. include:: /includes/reshard-to-same-key/addl-resource-reqs.rst
83+
84+
Get Started
85+
-----------
86+
87+
- :ref:`<resharding-a-collection-back-to-same-key>`
88+
- :ref:`<resharding-for-adding-and-removing-shards-tutorial>`
89+
90+
Learn More
91+
----------
92+
93+
- :method:`sh.reshardCollection()`
94+
- :ref:`<sharding-resharding>`

source/includes/remainingOperationTimeEstimatedSecs-details.rst

Lines changed: 7 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -3,26 +3,14 @@ seconds for the current :ref:`resharding operation
33
<sharding-resharding>`. It is returned as ``-1`` when a new resharding
44
operation starts.
55

6-
Starting in:
6+
Starting in MongoDB 7.0, ``remainingOperationTimeEstimatedSecs`` is also
7+
available on the coordinator during a resharding operation.
78

8-
- MongoDB 5.0, but before MongoDB 7.0,
9-
``remainingOperationTimeEstimatedSecs`` is only available on a
10-
:ref:`recipient shard <resharding-process-details>` during a
11-
resharding operation.
12-
- MongoDB 7.0, ``remainingOperationTimeEstimatedSecs`` is also available
13-
on the coordinator during a resharding operation.
9+
``remainingOperationTimeEstimatedSecs``
10+
is set to a pessimistic time estimate:
1411

15-
The resharding operation performs these phases in order:
16-
17-
#. The clone phase duplicates the current collection data.
18-
#. The building indexes phase builds indexes on the resharded collection.
19-
#. The catch-up phase applies any pending write operations to the
20-
resharded collection.
21-
22-
``remainingOperationTimeEstimatedSecs`` is set to a pessimistic time
23-
estimate:
24-
25-
- The catch-up phase time estimate is set to the clone phase time, which
12+
- The catch-up phase time estimate is set to the clone phase time, which
2613
is a relatively long time.
27-
- In practice, if there are only a few pending write operations, the
14+
- In practice, if there are only a few pending write operations, the
2815
actual catch-up phase time is relatively short.
16+
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Your cluster must meet these additional requirements:
2+
3+
- A minimum oplog window of 24 hours.
4+
- I/O capacity below 50%.
5+
- CPU load below 80%.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
You must ensure that your application can tolerate two seconds where
2+
the collection being resharded blocks writes. When writes are blocked,
3+
your application experiences an increase in latency. If your workload
4+
cannot tolerate this requirement, use chunk migrations to balance your
5+
cluster.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
Calculate the required storage space for the resharding operation by
2+
adding your collection size and index size, assuming a minimum oplog
3+
window of 24 hours by using this formula:
4+
5+
.. code-block:: none
6+
7+
Available storage required on each shard = [(collection size + index size) *2 ] / number of shards the collection will be distributed across.
8+
9+
For example, a 2TB collection and 400GB of indexes distributed across
10+
4 shards will need a minimum of 1.2TB of available storage per shard:
11+
12+
.. code-block:: none
13+
14+
[ (2 TB + 400GB) * 2 ] / 4 shards = 1.2 TB / shard
15+
16+
You must confirm that you have the available storage space in your
17+
cluster.
18+
19+
If there is insufficient space or I/O headroom available, you must
20+
increase the storage size. If there is insufficient CPU headroom, you
21+
must scale up the cluster by selecting a higher instance size.
22+
23+
.. tip::
24+
25+
If your MongoDB cluster is hosted on Atlas, you can use the Atlas UI to
26+
review storage, CPU, and I/O headroom metrics.
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
.. code-block:: javascript
2+
3+
db.adminCommand(
4+
{
5+
reshardCollection: "<database>.<collection>",
6+
key: { "<shardkey>" },
7+
zones: [
8+
{
9+
min: { "<document with same shape as shardkey>" },
10+
max: { "<document with same shape as shardkey>" },
11+
zone: <string> | null
12+
},
13+
],
14+
forceRedistribution: <bool>
15+
}
16+
)
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
.. code-block:: javascript
2+
3+
db.adminCommand(
4+
{
5+
reshardCollection: "<database>.<collection>",
6+
key: { "<shardkey>" },
7+
unique: <boolean>,
8+
numInitialChunks: <integer>,
9+
collation: { locale: "simple" },
10+
zones: [
11+
{
12+
min: { "<document with same shape as shardkey>" },
13+
max: { "<document with same shape as shardkey>" },
14+
zone: <string> | null
15+
},
16+
],
17+
forceRedistribution: <bool>
18+
}
19+
)

source/includes/resharding-about.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
- Use the :ref:`Reshard to Shard <reshard-to-shard-def>` technique to shard a collection and distribute its data across all relevant shards
2+
- Add new shards faster
3+
- Remove shards faster
4+
- Rewrite collections to reclaim disk space
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
The resharding operation performs these phases in order:
2+
3+
#. The clone phase duplicates the current collection data.
4+
#. The building indexes phase builds indexes on the resharded collection.
5+
#. The catch-up phase applies any pending write operations to the
6+
resharded collection.
7+
#. The commit phase renames the temporary collection and drops the old
8+
collection to perform a cut-over.

source/reference/command/reshardCollection.txt

Lines changed: 1 addition & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -43,26 +43,7 @@ Syntax
4343

4444
The command has the following syntax:
4545

46-
.. code-block:: javascript
47-
48-
db.adminCommand(
49-
{
50-
reshardCollection: "<database>.<collection>",
51-
key: <shardkey>,
52-
unique: <boolean>,
53-
numInitialChunks: <integer>,
54-
collation: { locale: "simple" },
55-
zones: [
56-
{
57-
min: <document with same shape as shardkey>,
58-
max: <document with same shape as shardkey>,
59-
zone: <string> | null
60-
},
61-
...
62-
],
63-
forceRedistribution: <bool>
64-
}
65-
)
46+
.. include:: /includes/reshardCollection-syntax.rst
6647

6748
Command Fields
6849
--------------

source/reference/command/shardCollection.txt

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -329,9 +329,11 @@ in the ``records`` database and uses the ``zipcode`` field as the
329329
db.adminCommand( { shardCollection: "records.people", key: { zipcode: 1 } } )
330330

331331

332-
.. seealso::
332+
Learn More
333+
----------
333334

334-
- :dbcommand:`refineCollectionShardKey`
335-
- :method:`sh.balancerCollectionStatus()`
336-
- :method:`sh.shardCollection()`
337-
- :doc:`/sharding`
335+
- :dbcommand:`refineCollectionShardKey`
336+
- :method:`sh.balancerCollectionStatus()`
337+
- :method:`sh.shardCollection()`
338+
- :doc:`/sharding`
339+
- :ref:`reshard-to-same-key`

source/reference/method/sh.reshardCollection.txt

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,7 +222,7 @@ Example output:
222222

223223
.. _reshardCollection-to-same-key:
224224

225-
Reshard a Collection to the Same Shard Key
225+
Reshard a Collection to the same Shard Key
226226
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
227227

228228
In order to reshard to the same shard key, set :ref:`forceRedistribution
@@ -255,6 +255,8 @@ Example output:
255255
operationTime: Timestamp({ t: 1733502241, i: 20 })
256256
}
257257

258+
For details, see :ref:`<resharding-a-collection-back-to-same-key>`.
259+
258260
Reshard a Collection with Zones
259261
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
260262

source/tutorial/remove-shards-from-cluster.txt

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,10 @@ Steps
138138
waiting for the balancer to migrate chunks. The cluster ensures
139139
that data is not placed on any draining shards. You can't run
140140
``moveCollection`` and ``reshardCollection`` operations
141-
simultaneously.
141+
simultaneously.
142+
143+
For the full procedure, see
144+
:ref:`resharding-for-adding-and-removing-shards-tutorial`.
142145

143146
.. step:: Move unsharded collections to another shard
144147

0 commit comments

Comments
 (0)