Robinhood Sync Acceleration for PanFS#108
Open
elliswilson wants to merge 2 commits intocea-hpc:masterfrom
Open
Robinhood Sync Acceleration for PanFS#108elliswilson wants to merge 2 commits intocea-hpc:masterfrom
elliswilson wants to merge 2 commits intocea-hpc:masterfrom
Conversation
Change-Id: I88c3deb32f977452cbcdb9b0cc3521f1cbc58eaa
Panasas pan_snap_delta update functionality 6 See merge request panasas/robinhood!1
tl-cea
reviewed
Aug 30, 2024
| return rc; | ||
| } | ||
| #ifdef _PANFS | ||
| st = &stn; |
Member
There was a problem hiding this comment.
I'd prefer you include this large piece of code in a subfonction.
tl-cea
reviewed
Aug 30, 2024
| } | ||
|
|
||
| #ifdef _PANFS | ||
| signal_scan_finished(); |
Member
There was a problem hiding this comment.
I don't understand why a scan with PanFS would require a different behavior here.
tl-cea
reviewed
Aug 30, 2024
| if (fstatat(parentfd, name, inode, AT_SYMLINK_NOFOLLOW) == -1) | ||
| return -errno; | ||
| #ifdef _PANFS | ||
| ssize_t vallentmp; |
Member
There was a problem hiding this comment.
I'd prefer this code is included to a subfunction here, to avoid overloading the code.
tl-cea
reviewed
Aug 30, 2024
| return -errno; | ||
|
|
||
| #ifdef _PANFS | ||
| else{ |
tl-cea
reviewed
Aug 30, 2024
| } | ||
|
|
||
| #ifdef _PANFS | ||
| size_t vallentmp; |
tl-cea
reviewed
Aug 30, 2024
| Exit(1); | ||
| } | ||
| #ifdef _PANFS | ||
| ssize_t vallentmp; |
tl-cea
reviewed
Aug 30, 2024
Member
tl-cea
left a comment
tl-cea
left a comment
There was a problem hiding this comment.
Great work!
Overall, I would prefer PanFS specific code to be located in specific functions and just have thinks like:
#ifdef _PANFS
rc = PanFS_xxxx();
#endif
rather than large pieces of code included in the main function
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request contains changes made this summer by employees at Panasas to provide support akin to what already exists for Lustre to accelerate updates to the Robinhood DB for the Panasas filesystem. Unlike in Lustre, which leverages it's changelog to keep the Robinhood DB up-to-date, in this change we use the PanFS snapshot delta utility to compute differences between successive snapshots, track which was the last snapshot we successfully updated from, and update when a newer snapshot is available using just the computed differences.
In the interest of de-risking existing users of Lustre or just the POSIX version of Robinhood that does a full tree walk, we compile out almost all of our code unless --enable-panfs is passed to configure.
Detailed changes include the following:
Please let me know if there are any issues with the code in this request and I'll be glad to make modifications to alleviate such concerns. Thanks!