Read me (install, use)

Requirements

• Three or more x86-64 machines,

• Corosync cluster engine

Install

1. Install x86-64 Linux.

2. Download Sheepdog codes.

   $ git clone git://sheepdog.git.sourceforge.net/gitroot/sheepdog/qemu-kvm
   $ git clone git://sheepdog.git.sourceforge.net/gitroot/sheepdog/sheepdog

3. Build

   $ cd qemu-kvm
   $ ./configure
   $ make install
   $ cd ../sheepdog
   $ ./configure
   $ make
   $ cd ..

Usage

Setup Sheepdog

1. Launch sheepdog on each machines of the cluster.

   $ cd sheepdog
   $ ./script/start-sheepdog -d=/dir/

   Note

   • -d option is a directory to store objects. The directory must be on the Btrfs to get tolerance against the failure. If you want to try Sheepdog and allow data loss, you can use ext3 file system with a user_xattr mount option.

2. Make fs

   $ shepherd mkfs --copies=3

   –copies specifies the number of default data redundancy. In this case, the replicated data is stored on three machines.

3. Check cluster state

   Following list shows that Sheepdog is running on 32 nodes.

   $ shepherd info -t dog
     Idx   Node id (SHA1)                               - Host:Port
   -----------------------------------------------------------------------
     0     2955c401:33c08e24:206c6918:c866a873:5223c484 - 10.68.13.16:9015
     1     b539ca04:8861b35a:62dbdbbe:a38dbcd8:fe048d32 - 10.68.14.6:9021
     2     81b73309:48a0d5fa:0f7e2339:c1ed14ac:ef53b546 - 10.68.13.15:9014
     3     4666cd10:4eda7287:ab8fbc74:15b77641:f2e57b24 - 10.68.13.3:9002
     4     ccdb611a:f842d41c:1b3146b1:8f0ab969:1a750919 - 10.68.13.8:9007
     5     92fba820:97b1402e:bf76909b:12007979:d2155180 - 10.68.14.5:9020
     6     9014ec29:a47a0d40:d8089e7f:23365c81:3a67fdd5 - 10.68.13.9:9008
   * 7     e6512130:43e6dd9e:dda810ad:dca40869:79465710 - 10.68.13.1:9000
     8     8c85c333:7d8e1e46:5514a722:ba3d2794:1eb8aab9 - 10.68.14.11:9026
     9     99d32636:964143e5:e9ba0388:0ebac1af:fee68ced - 10.68.14.14:9029
     10    789e9339:6a329f0a:8a7c7826:658bcb46:4e7c61eb - 10.68.13.5:9004
     11    492fe752:25019280:24d00dd5:b3f75076:72cbbd07 - 10.68.13.13:9012
     12    9ba0da70:19859fa0:d6cbb075:df1bcf25:f5e37965 - 10.68.14.16:9031
     13    c0006b72:73f2c470:f72df52b:31607eaa:0fe66d21 - 10.68.14.10:9025
     14    aae44077:7d952109:263ce633:144170ad:f8b2c010 - 10.68.13.10:9009
     15    69eb0279:17d31876:c445d38e:0b4b9845:fe8cda51 - 10.68.13.14:9013
     16    e752977c:49289195:fe1a9bb0:2196b311:c9c204d0 - 10.68.14.4:9019
     17    1e978e82:3c2aa630:244909af:635a71ab:8d491a7e - 10.68.14.1:9016
     18    5c7a2ca0:5a8153a9:d573f42e:03330aa9:f22079a9 - 10.68.14.15:9030
     19    0448c9b0:f3ededef:5f99acf4:1f30d5e8:f1ce920e - 10.68.14.3:9018
     20    a0d12bb2:5bd2b1f5:317a58a1:307fc1f3:15d6e8e2 - 10.68.13.12:9011
     21    850da9b7:b484dc42:cd59247c:84db4233:3896fd14 - 10.68.14.8:9023
     22    1c116fc1:7aa0a40c:8e365225:8cb396f3:16d177f8 - 10.68.13.4:9003
     23    0757a7c8:7d050f04:80088635:88e0cbf3:03397aea - 10.68.14.2:9017
     24    4671e3c8:096433ee:86b4994d:dd8104d6:b33aa6f8 - 10.68.14.9:9024
     25    fd9e30e1:a1b5013d:2db409ba:7488b01f:41edc5ed - 10.68.14.7:9022
     26    36bf42ec:7f3f299b:950c8d5f:c0d0ca3e:217ce595 - 10.68.14.12:9027
     27    d34228f1:38b07549:6081f6e7:0daf28a0:b26eb7b5 - 10.68.14.13:9028
     28    79aca0f1:d9e4a597:b145be96:1f59ff1f:18397bd4 - 10.68.13.2:9001
     29    48c293fc:7f458710:d17a2d89:b722b4d3:2e758d97 - 10.68.13.6:9005
     30    4b3c14fd:81676f6d:06b6d397:444e6044:13a74761 - 10.68.13.11:9010
     31    29cefcfe:39cfc03a:081355c7:a6a5b654:c0005391 - 10.68.13.7:9006

Create a virtual machine image

1. Create a 256 GB virtual machine image of Alice.

   $ qemu-img create -f sheepdog "Alice's Disk" 256G

2. You can also convert from existing KVM images to Sheepdog ones.

   $ qemu-img convert -f raw -O sheepdog ~/amd64.raw "Bob's Disk"

3. See Sheepdog images by the following command.

   $ shepherd info -t vdi
      40000 : Alice's Disk  256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:17:18, tag:        0, current
      80000 : Bob's Disk    256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:29:20, tag:        0, current

Boot the virtual machine

1. Boot the virtual machine.

   $ qemu-system-x86_64 --drive format=sheepdog,file="Alice's Disk"

2. Following command checks used images.

   $ shepherd info -t vm
   Name            |Vdi size |Allocated| Shared  | Status
   ----------------+---------+---------+---------+------------
   Alice's Disk    |   256 GB|     0 MB|     0 MB| running on xx.xx.xx.xx
   Bob's Disk      |   256 GB|     0 MB|     0 MB| not running

Snapshot

1. Snapshot

   $ qemu-img snapshot -c name sheepdog:"Alice's Disk"

   -c flag is meaningless currently

2. You can boot from the snapshot image by specifying tag id

   $ qemu-system-x86_64 --drive format=sheepdog,file="Alice's Disk":0

3. After getting snapshot, a new virtual machine images are added as a not-current image.

   $ shepherd info -t vdi
      40000 : Alice's Disk  256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:17:18, tag:        0, not current
      80000 : Bob's Disk    256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:29:20, tag:        0, current
      c0000 : Alice's Disk  256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:37:31, tag: 4ad6cc8b, current

Cloning from the snapshot

1. Create a Charlie’s image as a clone of Alice’s image.

   $ qemu-img create -b sheepdog:"Alice's Disk":0 -f sheepdog "Charlie's Disk"

2. Charlie’s image is added to the virtual machine list.

   $ shepherd info -t vdi
      40000 : Alice's Disk   256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:17:18, tag:        0, not current
      80000 : Bob's Disk     256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:29:20, tag:        0, current
      c0000 : Alice's Disk   256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:37:31, tag: 4ad6cc8b, current
     100000 : Charlie's Disk 256 GB (allocated: 0 MB, shared: 0 MB), 2009-10-15 16:47:46, tag:        0, current

3. To check parent-child relationship of the images, use tree format output.

   $ shepherd info -t vdi -f tree
   Alice's Disk---[09-10-16 10:02:31]-+-[09-10-19 09:42:21]---(You Are Here)
                                      `-[09-10-18 20:53:47]---[09-10-18 23:11:42]
   Bob's Disk---[09-10-15 16:29:20]-+-[09-10-18 20:51:13]
                                    `-(You Are Here)
   Charlie's Disk---(You Are Here)

Test Environment

• Debian lenny amd64