Discussion:
raid10 drive replacement
(too old to reply)
Vincent.
2014-10-18 01:47:05 UTC
Permalink
Raw Message
Hi !

I have a faulty drive in my raid10 and want it to be replaced.
Working drive are xvd[bef] and replacement drive is xvdc.

When I mount my drive in RW:
#mount -odegraded /dev/xvdb /tank
#dmesg -c
[ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
devid 1 transid 551398 /dev/xvdb
[ 6207.327357] btrfs: allowing degraded mounts
[ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
[ 6219.785929] btrfs: open_ctree failed

When I mount my drive in RO:
#mount -odegraded,ro /dev/xvdb /tank
#btrfs filesystem show
Label: none uuid: 728ef4d8-928c-435c-b707-f71c459e1520
Total devices 4 FS bytes used 4.70TiB
devid 1 size 2.73TiB used 2.73TiB path /dev/xvdb
devid 2 size 2.73TiB used 2.73TiB path
devid 3 size 2.73TiB used 2.73TiB path /dev/xvde
devid 4 size 2.73TiB used 2.73TiB path /dev/xvdf

Btrfs v3.12

Of course, because my mount is in RO, i can't add device and do a balance:
#btrfs device add /dev/xvdc /tank
ERROR: error adding the device '/dev/xvdc' - Read-only file system

Neither a "replace without disk":
#btrfs replace start -Br 2 /dev/xvdc /tank
ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
system, no error

FYI I'm on ubuntu 14.04 with btrfs 3.12

Any idea ?
Thanks !
--
fensoft
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Suman Chakravartula
2014-10-18 02:02:49 UTC
Permalink
Raw Message
Post by Vincent.
Hi !
I have a faulty drive in my raid10 and want it to be replaced.
Working drive are xvd[bef] and replacement drive is xvdc.
This is something I ran into the other day. Key difference is that I was
running 3.17.1 kernel and 3.16 btrfs-progs
Post by Vincent.
#mount -odegraded /dev/xvdb /tank
#dmesg -c
[ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
devid 1 transid 551398 /dev/xvdb
[ 6207.327357] btrfs: allowing degraded mounts
[ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
[ 6219.785929] btrfs: open_ctree failed
In my case, I was able to rw mount. May be update btrfs-progs and retry?
Post by Vincent.
#mount -odegraded,ro /dev/xvdb /tank
#btrfs filesystem show
Label: none uuid: 728ef4d8-928c-435c-b707-f71c459e1520
Total devices 4 FS bytes used 4.70TiB
devid 1 size 2.73TiB used 2.73TiB path /dev/xvdb
devid 2 size 2.73TiB used 2.73TiB path
devid 3 size 2.73TiB used 2.73TiB path /dev/xvde
devid 4 size 2.73TiB used 2.73TiB path /dev/xvdf
Btrfs v3.12
#btrfs device add /dev/xvdc /tank
ERROR: error adding the device '/dev/xvdc' - Read-only file system
#btrfs replace start -Br 2 /dev/xvdc /tank
ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
system, no error
Because my mount was rw, replace worked.
Post by Vincent.
FYI I'm on ubuntu 14.04 with btrfs 3.12
Any idea ?
Thanks !
--
fensoft
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Anand Jain
2014-10-18 04:37:53 UTC
Permalink
Raw Message
somethings aren't matching well. the issue is..
----
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not
allowed
----
But per Vincent only xvdc is missing in a raid10 (both data and metadata
are raid10 ?)


Anand
Post by Suman Chakravartula
Post by Vincent.
Hi !
I have a faulty drive in my raid10 and want it to be replaced.
Working drive are xvd[bef] and replacement drive is xvdc.
This is something I ran into the other day. Key difference is that I was
running 3.17.1 kernel and 3.16 btrfs-progs
Post by Vincent.
#mount -odegraded /dev/xvdb /tank
#dmesg -c
[ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
devid 1 transid 551398 /dev/xvdb
[ 6207.327357] btrfs: allowing degraded mounts
[ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
[ 6219.785929] btrfs: open_ctree failed
In my case, I was able to rw mount. May be update btrfs-progs and retry?
Post by Vincent.
#mount -odegraded,ro /dev/xvdb /tank
#btrfs filesystem show
Label: none uuid: 728ef4d8-928c-435c-b707-f71c459e1520
Total devices 4 FS bytes used 4.70TiB
devid 1 size 2.73TiB used 2.73TiB path /dev/xvdb
devid 2 size 2.73TiB used 2.73TiB path
devid 3 size 2.73TiB used 2.73TiB path /dev/xvde
devid 4 size 2.73TiB used 2.73TiB path /dev/xvdf
Btrfs v3.12
#btrfs device add /dev/xvdc /tank
ERROR: error adding the device '/dev/xvdc' - Read-only file system
#btrfs replace start -Br 2 /dev/xvdc /tank
ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
system, no error
Because my mount was rw, replace worked.
Post by Vincent.
FYI I'm on ubuntu 14.04 with btrfs 3.12
Any idea ?
Thanks !
--
fensoft
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Vincent.
2014-10-18 23:12:09 UTC
Permalink
Raw Message
i've upgraded to kernel 3.17.0 to update btrfs.ko and it's still not
mountable in -odegraded (-odegraded,ro works) with this error:
[ 361.828752] BTRFS info (device xvde): allowing degraded mounts
[ 361.984823] BTRFS: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[ 374.253555] BTRFS: too many missing devices, writeable mount is not allowed
[ 374.332474] BTRFS: open_ctree failed

and, yes, data + metadata are raid10
--
Vincent
Post by Anand Jain
somethings aren't matching well. the issue is..
----
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
----
But per Vincent only xvdc is missing in a raid10 (both data and metadata are
raid10 ?)
Anand
Post by Suman Chakravartula
Post by Vincent.
Hi !
I have a faulty drive in my raid10 and want it to be replaced.
Working drive are xvd[bef] and replacement drive is xvdc.
This is something I ran into the other day. Key difference is that I was
running 3.17.1 kernel and 3.16 btrfs-progs
Post by Vincent.
#mount -odegraded /dev/xvdb /tank
#dmesg -c
[ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
devid 1 transid 551398 /dev/xvdb
[ 6207.327357] btrfs: allowing degraded mounts
[ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
[ 6219.785929] btrfs: open_ctree failed
In my case, I was able to rw mount. May be update btrfs-progs and retry?
Post by Vincent.
#mount -odegraded,ro /dev/xvdb /tank
#btrfs filesystem show
Label: none uuid: 728ef4d8-928c-435c-b707-f71c459e1520
Total devices 4 FS bytes used 4.70TiB
devid 1 size 2.73TiB used 2.73TiB path /dev/xvdb
devid 2 size 2.73TiB used 2.73TiB path
devid 3 size 2.73TiB used 2.73TiB path /dev/xvde
devid 4 size 2.73TiB used 2.73TiB path /dev/xvdf
Btrfs v3.12
#btrfs device add /dev/xvdc /tank
ERROR: error adding the device '/dev/xvdc' - Read-only file system
#btrfs replace start -Br 2 /dev/xvdc /tank
ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
system, no error
Because my mount was rw, replace worked.
Post by Vincent.
FYI I'm on ubuntu 14.04 with btrfs 3.12
Any idea ?
Thanks !
--
fensoft
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Anand Jain
2014-10-20 01:33:07 UTC
Permalink
Raw Message
still somethings aren't matching, it could be that
underlying group profile might have changed without
your notice (yes in some circumstance it could).
can you show 'btrfs fi df <mnt>'

Anand
Post by Vincent.
i've upgraded to kernel 3.17.0 to update btrfs.ko and it's still not
[ 361.828752] BTRFS info (device xvde): allowing degraded mounts
[ 361.984823] BTRFS: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[ 374.253555] BTRFS: too many missing devices, writeable mount is not allowed
[ 374.332474] BTRFS: open_ctree failed
and, yes, data + metadata are raid10
--
Vincent
Post by Anand Jain
somethings aren't matching well. the issue is..
----
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
----
But per Vincent only xvdc is missing in a raid10 (both data and metadata are
raid10 ?)
Anand
Post by Suman Chakravartula
Post by Vincent.
Hi !
I have a faulty drive in my raid10 and want it to be replaced.
Working drive are xvd[bef] and replacement drive is xvdc.
This is something I ran into the other day. Key difference is that I was
running 3.17.1 kernel and 3.16 btrfs-progs
Post by Vincent.
#mount -odegraded /dev/xvdb /tank
#dmesg -c
[ 6207.294513] btrfs: device fsid 728ef4d8-928c-435c-b707-f71c459e1520
devid 1 transid 551398 /dev/xvdb
[ 6207.327357] btrfs: allowing degraded mounts
[ 6207.477041] btrfs: bdev (null) errs: wr 15211054, rd 3038899, flush
0, corrupt 0, gen 0
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
[ 6219.785929] btrfs: open_ctree failed
In my case, I was able to rw mount. May be update btrfs-progs and retry?
Post by Vincent.
#mount -odegraded,ro /dev/xvdb /tank
#btrfs filesystem show
Label: none uuid: 728ef4d8-928c-435c-b707-f71c459e1520
Total devices 4 FS bytes used 4.70TiB
devid 1 size 2.73TiB used 2.73TiB path /dev/xvdb
devid 2 size 2.73TiB used 2.73TiB path
devid 3 size 2.73TiB used 2.73TiB path /dev/xvde
devid 4 size 2.73TiB used 2.73TiB path /dev/xvdf
Btrfs v3.12
#btrfs device add /dev/xvdc /tank
ERROR: error adding the device '/dev/xvdc' - Read-only file system
#btrfs replace start -Br 2 /dev/xvdc /tank
ERROR: ioctl(DEV_REPLACE_START) failed on "/tank": Read-only file
system, no error
Because my mount was rw, replace worked.
Post by Vincent.
FYI I'm on ubuntu 14.04 with btrfs 3.12
Any idea ?
Thanks !
--
fensoft
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Duncan
2014-10-18 23:27:52 UTC
Permalink
Raw Message
Suman Chakravartula posted on Fri, 17 Oct 2014 19:02:49 -0700 as
Post by Suman Chakravartula
Hi !
I have a faulty drive in my raid10 and want it to be replaced. Working
drive are xvd[bef] and replacement drive is xvdc.
This is something I ran into the other day. Key difference is that I was
running 3.17.1 kernel and 3.16 btrfs-progs
#mount -odegraded /dev/xvdb /tank
[ 6219.703606] Btrfs: too many missing devices, writeable mount is not allowed
In my case, I was able to rw mount. May be update btrfs-progs and retry?
Because my mount was rw, replace worked.
FYI I'm on ubuntu 14.04 with btrfs 3.12
I'd suggest updating both btrfs-progs (as Suman suggested) and the kernel
(the version of which you don't mention).

There was a period when btrfs was too strict in what it would allow to
mount writable, thus not allowing a device replace to fix the problem.
It would appear ubuntu 14.4's versions fell in that period.

For kernel, given some recent bugs, you want the latest stable 3.16,
definitely > 3.16.2 as that contained a vital btrfs bugfix. Current 3.17
stable has a different bug that didn't affect 3.16, patch pending but not
in 3.17, or in 3.18-rc yet, but 3.16.4 (I believe the latest) should be
good. You can also try the latest long-term-stable 3.14.x as it's pretty
stable btrfs-wise, but 3.14 might have still been in the can't-mount-
writable period and I'm not sure they backported that fix to stable-
series. But 3.16.4+ should be great.

Because btrfs is evolving so fast, in general you'll want to run a
current kernel and keep up with the list to know about any current bugs
(like the one currently affecting 3.17 and the 3.18-rcs, and the earlier
one affecting the entire 3.15 series and 3.16 before 3.16.2). Most
distros run somewhat outdated kernels so sticking to a distro kernel
doesn't always work so well.

btrfs-progs userspace isn't quite so vital to keep current as the kernel,
but you still want to keep at least reasonably current. 3.12 is getting
a bit long in the tooth, now, and I'd recommend at least 3.14.2, with
3.16.1 preferred. (A 3.17 is in rc ATM but not yet released.)

And (you may not need this but we get people all the time for whom it's a
bit late so it's worth repeating) of course, because btrfs /is/ evolving
so fast and isn't yet completely stable, the general rule that if you
value it you have backups for it that applies even to mature and stable
filesystems such as ext3 and ext4, applies even more to btrfs. If
something goes wrong and you don't have a backup, no matter HOW much you /
claim/ to value that data in hindsight, your actions say you simply
didn't value it enough to ensure yourself against such possibilities by
keeping backups, which means you by your actions you considered it throw-
away data if something /did/ go wrong. Also, be aware that a backup
that's not tested can't be considered a backup at all, because you don't
know for sure that you can recover from it. And RAID isn't a substitute
for backups.

So if you value your data, have (tested) backups! Not doing so simply
means you don't value that data, no matter what you /claim/.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Loading...