tests, scripts: cleanup tests and dev environment things

[flakey5]

Alternative title: making the tests & local dev environment not annoying and messy

This is an attempt to make the tests and local dev environment a lot nicer to work with.

Some of the main problems with them right now are:

1. No Sentry = no error visibility, nothing is logged to console
2. You can't run the worker in development unless it's in wrangler's remote mode. This requires a Cloudflare account & a R2 bucket to be setup on that account which is annoying
3. Tests are rather messy and haven't really been updated like the rest of the codebase has been. This was good when we were changing a lot so we could ensure we didn't break anything, but, it's time now to clean these up imo

This pr:

1. Rewrites tests to be a lot cleaner & conform with the style of the rest of the codebase
2. Adds an example of the structure of the dist-prod bucket (called dev-bucket)
   • Replaces the bucket data at tests/e2e/test-data/R2_BUCKET for e2e tests
   • Can be used in a dev script to populate a bucket for local testing (see 3)
3. npm run dev - starts a locally running version of the worker with a populated R2 bucket
   • Populated from the dev-bucket directory

What's TODO:

• Finish implementing e2e tests
• Go through code to see if there's anything that should be tested that isn't
• Make the local dev script nicer

Some current annoyances:

• The local dev script uses Miniflare since there isn't actually a way to populate an R2 bucket locally with Wrangler, so any niceness that that has we need to reimplement or go without. Hot reloading might not be possible either since Miniflare works off of the compiled worker.js file. I do have some ideas, but not entirely sure if they're gonna work.
• Ideally we would be able to use Workers' vitest integration, but it's not really possible since we're doing things like mocking an S3 api in our e2e tests. Vitest patches a lot of modules and globals making things like node:http unavailable. Workaround for this was found, so we can use the Vitest integration now

[ovflowd]

@flakey5 -- do you need a review now or is this still WIP?

[flakey5]

Still a wip

[flakey5]

Gonna go ahead and mark this for review now.

This unfortunately doesn't resolve the annoyances with running the worker locally though, there's still some things that I need to play around with to see if I can get it working in an ideal way.

[flakey5]

I forget what the actual number is but there is a limit on the about of bindings & vars a worker can have, so might be beneficial to have these in a FLAGS object or something

[avivkeller]

Can you use node:test?

[ovflowd]

That'd be good yes

[flakey5]

Yes (this is how the tests currently work) but vitest is the preferred way to write tests for workers. With the vitest integration the tests run inside of workerd instead of node + better typescript support (although gap is closing with Node's type stripping I think?). Iirc with vitest we also get better stack traces when things error since we're not testing a built & minified version of the worker.