Periodic updates to the dataset #15
Description
Goals
- The data in S3 should get updated periodically
- Automation should run using github actions, with some schedule that we will set up. (so, we shall rely heavily on metadata to identify what has been downloaded and what needs to be downloaded, we can't copy the whole data)
- Assume that the cases for past dates can also be added to ecourts website at a later point in time, we should have a way to resync the whole data at a different interval (less frequent since this can take long time)