PLEASE BE AWARE THAT ANY INFORMATION YOU MAY FIND HERE MAY BE INACCURATE, AND COULD INCLUDE TECHNICAL INACCURACIES, TYPOGRAPHICAL ERRORS, AND EVEN SPELLING ERRORS.
From the MANUAL page:
The zdb command is used by support engineers to diagnose
failures and gather statistics. Since the ZFS file system is
always consistent on disk and is self-repairing, zdb should
only be run under the direction by a support engineer.
DO NOT TRY IT IN PRODUCTION. USE AT YOUR OWN RISK!
Hello there… well, after securing the title at #UFCRio, i will do something less agressive. ;-) hehe, you don’t know how many jokes i’m listening about this…
In all these years working with ZFS, just a few times i did need to execute similar procedures as i will describe in this post. And (luck) all the times i could stress the filesystem and face this scenarios at lab, and rely on a stable configuration on production. Hope that do not change!
This time i did face the “where-is-my-pool” trying to exercise the capabilities of ZFS as “Zettabyte” filesystem (Creating and using EB and ZB thin provisioned). In my experience, the sparse and “really big” dataset is a good way to stress ZFS (actually, any FS i think). The demand to space is growing do you know… so, the trigger this time was: destroying the dataset.
BTW, we are talking about the Illumos source code on this blog from now on. Specifically in this time, the build 151.
As from the begininig of this series, here we talk about joys that the ZFS hackers did hide on the source code, and so we mortals do need to seek for them. In the old days the procedure to do what we will do in this post was much more complicated, and this is the first time i need to dig on this, after the new features to address restoring a faulted pool integrated on the ZFS code.
After my tests, i was presented in a situation where the machine was hang deleting a dataset, and after the reset, freezing again. A caotic situation. So, as i did know that the txg rewind code was present on the new ZFS implementation, i did take a look at the zpool command and could not find anything about it. So, my next stop was the source code: zpool_main.c
The idea was to find a comment, so doing a “/rewind” on vi, the first match was this:
* -F Attempt rewind if necessary.
C’mon, why that is not on the zpool manual page?!
Oh, that reminds me the You are not expected to understand this. Ok…
Well, i think that would do the job. Just issuing “-F”. But as that comment section describes two undocumented options (explicitly), i will put this here:
* -V Import even in the presence of faulted vdevs. This is an
* intentionally undocumented option for testing purposes, and
* treats the pool configuration as complete, leaving any bad
* vdevs in the FAULTED state. In other words, it does verbatim
* import.
...
* -T Specify a starting txg to use for import. This option is
* intentionally undocumented option for testing purposes.
If you look at the “zpool -h” switch, you will see a lot of options, but no way to know nothing about them…
The last point was to look at the code that actually handles the options. So, i think these two lines tell a lot:
/* check options */
while ((c = getopt(argc, argv, ":aCc:d:DEfFmnNo:rR:T:VX")) != -1) {
X? ;-)
No way to stop now, we will need to look at the “case” statement… so, here is some excerpts of the “case” code:
case 'F':
do_rewind = B_TRUE;
break;
case 'm':
flags |= ZFS_IMPORT_MISSING_LOG;
break;
case 'n':
dryrun = B_TRUE;
break;
...
case 'T':
errno = 0;
txg = strtoull(optarg, &endptr, 10);
if (errno != 0 || *endptr != '\0') {
(void) fprintf(stderr,
gettext("invalid txg value\n"));
usage(B_FALSE);
}
rewind_policy = ZPOOL_DO_REWIND | ZPOOL_EXTREME_REWIND;
break;
case 'V':
flags |= ZFS_IMPORT_VERBATIM;
break;
case 'X':
xtreme_rewind = B_TRUE;
break;
...
After that reading, i had two bullets: -F and -X (for who was with none, is a lot). ;-)
Let’s use them both from once: zpool import -FX mypool
ps.: The -X will be used just after -F on rewind code…
The pool online and in good shape! The filesystem i did remove was not there, so was after the deletion, and a scrub did run without any issues. But i think it could be easier if the information was public. We could talk about the -T, for sure about the -m, -V options too, but this post is very extense already.
So, we could make some tests and explore that in another blog post. hoping nobody did loose a pool (with real data), without knowing about these options.
The bug? I’m trying to participate and inform that on the illumos mailing list, but seems like the people there are really busy.
peace
