BFS Partition Resizer: Final Report

Blog post by ahenriksson on Wed, 2012-09-05 09:55

First of all, I apologize for the delay. I have now returned from my vacation, had a few days to settle in and explain to my neighbours that I'm not dead (!). Anyways, on to the interesting stuff.

On the surface, the status of things is mostly the same as in my last report, with a few bugs less. I thought I had dedicated more than enough time for bugfixing, but that turned out to not be the case. This is partly due to the slower development cycle when testing natively (compile, copy driver to image, boot virtual machine, test, repeat), and the bugs only showing up after doing several resizes with other IO going on. All the bugs of this kind that I know about have been eliminated.

To summarize the things I have accomplished during the summer:
  • Resize support in BFS driver, save for vnode mapping and growing a full file system.
  • Getting the resizing "pipeline" from userspace to driver to a working state (still needs some checking to verify that it's robust).

It turned out toward the end of summer that the way I had been handling changed vnode ID's would not work out. The vnode ID is a number that refers to a specific file, and the current BFS code simply uses the block on the disk where the file is located for this purpose. This becomes a problem when a file is moved, as is necessary when resizing. My earlier solution was to "trick" the system to use the new file location as the ID instead, but this doesn't really work out. What needs to be done is to decouple the block number and vnode ID in the BFS module, so a file can be moved without changing the ID. I have started to work a little on this, and it's looking very promising.

The ability to grow a full file system, while not of absolutely critical importance, would be very nice to have. I have written a bit about this in the previous post, so I won't repeat myself. The same goes for resize support in DriveSetup.

My plan for the future is to continue working on this, at the very least until it can be merged. Would be a shame to do all that work for nothing :). It's hard to give a timeline for this, it depends on how the school situation turns out, but I hope it will be done fairly soon.

The code can be found in my github repository, in the 'production' branch. It needs to be merged with the 'fixes' branch (this is not conflict free...), as for some reason I got started on a strange git workflow. I intended to have this sorted out by the end of the coding period, but that didn't happen.

I'd like to give a big thank you to mentors and others for making this happen, all the help and the welcoming attitude. I've learned a great deal (I hear the job market is screaming for people with detailed knowledge of the Be File System :)). Seriously though, I've gained a lot of knowledge of file systems in general, how operating systems work on a lower level, concurrent programming and so on, both from my own work and reading mailing list posts.

Have a nice day
Andreas

Comments

Re: BFS Partition Resizer: Final Report

Resizing a partition to the full size of the media may be more important than you anticipated. The common use of this would be to resize a nightly image to the full size of a usb memory stick.

You have a higher probability of completing your work if you set aside a block of time every week where it is treated as a real job. This is easy to say yet hard to do, especially when you're still in school and there are more fun things to do than fixing file system errors. All I can say is that this work you're doing is super important, but it has to be perfect or it won't get used. You can do it!

Re: BFS Partition Resizer: Final Report

I think you are misunderstanding. With "a full file system", I mean a file system which is almost completely full of data, not growing to the full media size which should be working fine. However, since the nightly images tend to be rather full, this could certainly cause problems in the scenario you describe, which I agree is important.