Skip to content

Even allocation of files among writable branches#89

Open
csabahenk wants to merge 2 commits into
rpodgorny:masterfrom
csabahenk:master
Open

Even allocation of files among writable branches#89
csabahenk wants to merge 2 commits into
rpodgorny:masterfrom
csabahenk:master

Conversation

@csabahenk

Copy link
Copy Markdown

Unionfs can be used as poor man's clustered storage. To make this use case feasible, it's desired that file creations are distributed evenly among the writable branches (using a heuristics that a given new file is randomly placed on branches with likelihoods proportional to available space).

To ease debugging while I was working on this feature, I added timestamps to the debug messages. I include that in the PR as a nicety, but it's not essential for the feature delivered.

@lavdnone

Copy link
Copy Markdown

good feature, seems stable, using it on couple occasions
what I encountered: when in place rename it tries to move file between branches
it should also check and do the move on the same branch

@rpodgorny

Copy link
Copy Markdown
Owner

thanks for the (independent) feedback! ...i'll try to spare some to review the patch.

first impressions from a quick look:

  1. why the heuristic? why not just strictly decide by the free space? i tend to prefer determinism...
  2. if bsd/macos is still in a "todo" phase, i'd rather have this option ifdef-ed for linux only
  3. malloc+free - can we do with static pre-allocation?

@lavdnone

Copy link
Copy Markdown

Quick fix for the move problem was

+448 src/fuse_ops.c
        } else {
                j = i;
        }

@lavdnone

Copy link
Copy Markdown

PS I would not call it "poor man's clustered storage".
It is more of a Single mount JBOD.
With the idea that failure of one drive will not require fixing whole raid0 (for example).
And if you have 100 processes writing something on those drives it will still distribute IO and get you speed off all the drives.
If the drive is gone - it is just files on that drive. With good SMART check reporting one can easy copy real simple files from failing drive to another.

@ajax-lives

Copy link
Copy Markdown

Is this available with fstab at this time? I'm interested in making use of this :D

@ajax-lives

Copy link
Copy Markdown

And will this be merged eventually? Looks extremely neat!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants