This is an updated version of !504 (closed) with many of the side-change already integrated in another form.
Here is a small list of things I feel like need work right now (some in this MR, some later, take it a small brain dump)
- The revlog special case could be more standard. Right now we have a special function doing it own opening and returning filename and size. We could reuse some of the "open revlog from StoreEntry code" that we use elsewhere (e.g. upgrade) to reduce the amount of special code around this. It would also seem simpler and safer is that logic returned open file point, or chunk iterator for the data, so skip the need to open the file and the small associated race window. (I also plan to poke a bit at the performance aspect of that step too)
- We should have a test for the race condition that motivate this whole work : having a revlog being split while streaming
- We should look into upgrading the format to make it possible for concurrent reading and writing of the streamed file. A MR like !578 have very promising number so we should make such evolution simple for v3.
Landing the first part (pre-revlog special case) early is probably a good option here.