WIP hf_xet.XetSession

[assafvayner]

fix XET-528

adds XetSession export out of hf_xet.

[hoytak]

I like the idea here; I proposed we do this a while back but at that point we just wanted something reliable working.

I see a main advantage of sessions being that it would enable sharing of lookups and other resources, e.g. for deduplication or caching, across calls to upload_files or download_files within the same session. We don't have to do this at first -- since it can be just for tracking -- but I think this could help some things in the future. Given this direction, I think it makes more sense to put the guts of this into the data/ folder and just put a thin wrapper in hf_xet. Thoughts?

[assafvayner]

Given this direction, I think it makes more sense to put the guts of this into the data/ folder and just put a thin wrapper in hf_xet. Thoughts?

I expect there to need to be a somewhat different implementation of the "session" for different clients, e.g. this session in the data/ crate will not be WASM compatible and likely not export the same exact interface, we will anyways need to re-create it. I'd be more enthusiastic to move it out of hf_xet into data/ when/if we are actively working on a second client that can utilize it.

Another way to look at it is that the hf_xet.XetSession is a thin wrapper over the FileUploadSession in such a way that a python context can hold a handle to hf_xet state object so that the caller in python can control when to create/reuse the session and pass it between function scopes.

[assafvayner]

closing in favor of progress with #403