Skip to content

Hashing calculation doesn't work on large workloads #437

@MiYanni

Description

@MiYanni

Summary

Currently, the mono packs workload doesn't properly calculate the unique hash value that we use when creating a unique URL for the VS drop upload. This causes the (biggest) workload to be uploaded multiple times (per servicing branch count). It will emit this many, many times in the build log:

Exception calling "Write" with "1" argument(s): "Exception of type 'System.OutOfMemoryException' was thrown."
  At D:\a\_work\1\s\eng\create-workload-drops.ps1:35 char:76
  + ... t-Content -Encoding ASCII -Raw | ForEach-Object { $writer.Write($_) }
  +                                                       ~~~~~~~~~~~~~~~~~
      + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
      + FullyQualifiedErrorId : OutOfMemoryException

Basically, the algo is collecting the contents of all of the files in the workload, then creating the hash. This obviously doesn't work if there is too much content in a single workload. The algo would need to be changed so that:

  • Each file is individually hashed
  • Then calculate the hash of hashes

If this change is done, it needs to be done on all branches. Otherwise, different calculations on different branches would invalidate the point of the hash.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions