Enable Consistent SHA256 Hashing with reduced Planner Context #3091
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
aporialiao
added a commit
to aporialiao/torchrec
that referenced
this pull request
Jun 13, 2025
…h#3091) Summary: Pull Request resolved: pytorch#3091 Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs. This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances. To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation. Namely: * enumerator.enumerate(...)'s output - which is used as the `search_space` in both LP and OSS planner * We are storing the output of enumerate as an attribute `last_stored_search_space`. **This assumes enumerate will have been called before we hash the planner context inputs**. * StorageResveration's policy (aka whether `HeuristicalStorageReservation` is used or `FixedStorageReservation` * StorageResveration's initialization attributes: * _percentage * _parameter_multiplier for HeuristicalStorageReservation * _dense_tensor_estimate for HeuristicalStorageReservation Created helper functions: * `hash_planner_context_inputs` to be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272) * `hash_sha256_to_int` to be passed in as the default hash function in hash_planner_context_inputs Also created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input. Differential Revision: D76303748
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
aporialiao
force-pushed
the
export-D76303748
branch
from
eb425d8 to
2d29d80
Compare
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
aporialiao
added a commit
to aporialiao/torchrec
that referenced
this pull request
Jun 13, 2025
…h#3091) Summary: Pull Request resolved: pytorch#3091 Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs. This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances. To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation. Namely: * enumerator.enumerate(...)'s output - which is used as the `search_space` in both LP and OSS planner * We are storing the output of enumerate as an attribute `last_stored_search_space`. **This assumes enumerate will have been called before we hash the planner context inputs**. * StorageResveration's policy (aka whether `HeuristicalStorageReservation` is used or `FixedStorageReservation` * StorageResveration's initialization attributes: * _percentage * _parameter_multiplier for HeuristicalStorageReservation * _dense_tensor_estimate for HeuristicalStorageReservation Created helper functions: * `hash_planner_context_inputs` to be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272) * `hash_sha256_to_int` to be passed in as the default hash function in hash_planner_context_inputs Also created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input. Differential Revision: D76303748
aporialiao
force-pushed
the
export-D76303748
branch
from
2d29d80 to
7034b2d
Compare
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
2 similar comments
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
aporialiao
added a commit
to aporialiao/torchrec
that referenced
this pull request
Jun 13, 2025
…h#3091) Summary: Pull Request resolved: pytorch#3091 Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs. This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances. To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation. Namely: * enumerator.enumerate(...)'s output - which is used as the `search_space` in both LP and OSS planner * We are storing the output of enumerate as an attribute `last_stored_search_space`. **This assumes enumerate will have been called before we hash the planner context inputs**. * StorageResveration's policy (aka whether `HeuristicalStorageReservation` is used or `FixedStorageReservation` * StorageResveration's initialization attributes: * _percentage * _parameter_multiplier for HeuristicalStorageReservation * _dense_tensor_estimate for HeuristicalStorageReservation Created helper functions: * `hash_planner_context_inputs` to be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272) * `hash_sha256_to_int` to be passed in as the default hash function in hash_planner_context_inputs Also created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input. Differential Revision: D76303748
aporialiao
force-pushed
the
export-D76303748
branch
from
7034b2d to
10d8108
Compare
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
aporialiao
added a commit
to aporialiao/torchrec
that referenced
this pull request
Jun 13, 2025
…h#3091) Summary: Pull Request resolved: pytorch#3091 Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs. This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances. To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation. Namely: * enumerator.enumerate(...)'s output - which is used as the `search_space` in both LP and OSS planner * We are storing the output of enumerate as an attribute `last_stored_search_space`. **This assumes enumerate will have been called before we hash the planner context inputs**. * StorageResveration's policy (aka whether `HeuristicalStorageReservation` is used or `FixedStorageReservation` * StorageResveration's initialization attributes: * _percentage * _parameter_multiplier for HeuristicalStorageReservation * _dense_tensor_estimate for HeuristicalStorageReservation Created helper functions: * `hash_planner_context_inputs` to be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272) * `hash_sha256_to_int` to be passed in as the default hash function in hash_planner_context_inputs Also created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input. Differential Revision: D76303748
aporialiao
force-pushed
the
export-D76303748
branch
from
10d8108 to
ecba624
Compare
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
aporialiao
force-pushed
the
export-D76303748
branch
from
ecba624 to
d14d02c
Compare
aporialiao
added a commit
to aporialiao/torchrec
that referenced
this pull request
Jun 13, 2025
…h#3091) Summary: Pull Request resolved: pytorch#3091 Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs. This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances. To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation. Namely: * enumerator.enumerate(...)'s output - which is used as the `search_space` in both LP and OSS planner * We are storing the output of enumerate as an attribute `last_stored_search_space`. **This assumes enumerate will have been called before we hash the planner context inputs**. * StorageResveration's policy (aka whether `HeuristicalStorageReservation` is used or `FixedStorageReservation` * StorageResveration's initialization attributes: * _percentage * _parameter_multiplier for HeuristicalStorageReservation * _dense_tensor_estimate for HeuristicalStorageReservation Created helper functions: * `hash_planner_context_inputs` to be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272) * `hash_sha256_to_int` to be passed in as the default hash function in hash_planner_context_inputs Also created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input. Differential Revision: D76303748
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
aporialiao
force-pushed
the
export-D76303748
branch
from
d14d02c to
c94ff70
Compare
aporialiao
added a commit
to aporialiao/torchrec
that referenced
this pull request
Jun 13, 2025
…h#3091) Summary: Pull Request resolved: pytorch#3091 Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs. This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances. To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation. Namely: * enumerator.enumerate(...)'s output - which is used as the `search_space` in both LP and OSS planner * We are storing the output of enumerate as an attribute `last_stored_search_space`. **This assumes enumerate will have been called before we hash the planner context inputs**. * StorageResveration's policy (aka whether `HeuristicalStorageReservation` is used or `FixedStorageReservation` * StorageResveration's initialization attributes: * _percentage * _parameter_multiplier for HeuristicalStorageReservation * _dense_tensor_estimate for HeuristicalStorageReservation Created helper functions: * `hash_planner_context_inputs` to be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272) * `hash_sha256_to_int` to be passed in as the default hash function in hash_planner_context_inputs Also created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input. Differential Revision: D76303748
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
aporialiao
added a commit
to aporialiao/torchrec
that referenced
this pull request
Jun 13, 2025
…h#3091) Summary: Pull Request resolved: pytorch#3091 Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs. This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances. To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation. Namely: * enumerator.enumerate(...)'s output - which is used as the `search_space` in both LP and OSS planner * We are storing the output of enumerate as an attribute `last_stored_search_space`. **This assumes enumerate will have been called before we hash the planner context inputs**. * StorageResveration's policy (aka whether `HeuristicalStorageReservation` is used or `FixedStorageReservation` * StorageResveration's initialization attributes: * _percentage * _parameter_multiplier for HeuristicalStorageReservation * _dense_tensor_estimate for HeuristicalStorageReservation Created helper functions: * `hash_planner_context_inputs` to be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272) * `hash_sha256_to_int` to be passed in as the default hash function in hash_planner_context_inputs Also created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input. Reviewed By: micrain Differential Revision: D76303748
aporialiao
force-pushed
the
export-D76303748
branch
from
c94ff70 to
d2f5fd4
Compare
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
…h#3091) Summary: Pull Request resolved: pytorch#3091 Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs. This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances. To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation. Namely: * enumerator.enumerate(...)'s output - which is used as the `search_space` in both LP and OSS planner * We are storing the output of enumerate as an attribute `last_stored_search_space`. **This assumes enumerate will have been called before we hash the planner context inputs**. * StorageResveration's policy (aka whether `HeuristicalStorageReservation` is used or `FixedStorageReservation` * StorageResveration's initialization attributes: * _percentage * _parameter_multiplier for HeuristicalStorageReservation * _dense_tensor_estimate for HeuristicalStorageReservation Created helper functions: * `hash_planner_context_inputs` to be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272) * `hash_sha256_to_int` to be passed in as the default hash function in hash_planner_context_inputs Also created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input. Reviewed By: micrain Differential Revision: D76303748
aporialiao
force-pushed
the
export-D76303748
branch
from
d2f5fd4 to
76eb8e3
Compare
|
This pull request was exported from Phabricator. Differential Revision: D76303748 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Even though SHA256 hashing is used, we're still not seeing the expected same hash generated from the original planner context inputs.
This problem is due to Enumerator and Storage Reservation objects we were originally trying to hash containing attributes that differ between processes/instances.
To resolve this we reduced the hashing context to only use the specific attributes we need from enumerator and storage reservation.
Namely:
search_spacein both LP and OSS plannerlast_stored_search_space. This assumes enumerate will have been called before we hash the planner context inputs.HeuristicalStorageReservationis used orFixedStorageReservationCreated helper functions:
hash_planner_context_inputsto be called in both planner.hash_planner_context_inputs and manifold loading call site (see D75723272)hash_sha256_to_intto be passed in as the default hash function in hash_planner_context_inputsAlso created a multiprocess unit test to quickly check if consistent hashes are being generated across different processes given the same input.
Differential Revision: D76303748