forked from MatiasBjorling/kvblade
-
Notifications
You must be signed in to change notification settings - Fork 1
/
README
87 lines (70 loc) · 2.93 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
kvblade: a 4.14 to 4.19+ linux kernel module
* Standardized on the original Linux Kernel aoe.h header file
* Removed all backwards compatibility with older kernels
* Removed code areas that were shown to have problems in the past
---
kvblade: a 4.2 to 4.8+ linux kernel module
---
Kvblade is a kernel module implementing the target
side of the AoE protocol. Users can command the
module through sysfs to export block devices on
specified network interfaces. The loopback device
should be used as an intermediary for exporting
regular files with kvblade.
An exported target has a tuple that uniquely defines
it:
{ major, minor, netif }
Major is the aoemajor / shelf address, minor is the
aoeminor / slot address, and netif is the name of
the interface the target is bound on. It is illegal
to attempt to create more than one of these in kvblade.
Three shell scripts have been created to facilitate interfacing
with kvblade through sysfs: kvstat, kvadd, and kvdel. Kvstat
prints the list of currently exported vblades. Kvadd and
kvdel are used to manage the exported vblades.
This is alpha code. It appears stable, but has limitations
that need to be addressed. See the TODO file for a list of
things that you can help with.
Author:
Sam Hopkins
Coraid, Inc.
John Sharratt
Tokera Ltd.
Address support requests to
Thanks to Lelik P. Korchagin for writing the first
kernel vblade module. It was quite helpful as a
reference implementation.
Thanks to Sam. Hopkins for providing the original version
those many years ago.
---
Forked for the Tokera initiative
Improvements:
* Added support for linux kernel version 4.14.9 (long term kernel)
* Fixed a compile issue caused by the changed field bi_sector in BIO objects
* Removed the 16 static buffers and replaced them with 512 slab objects that are NUMA aligned
* Imported the AOE headers from the latest linux kernel version (4.2.9 - 4.8)
* Fixed incorrect reporting of capacity when attaching to a partition device
* Converted the single threaded worker to use a highly optimized multithreading architecture (pinned thread per core)
* Removed the worker spinlock and replaced with RCU symantics
* Removed a zero memory operation on new packets (which was wasted CPU time)
* Added a new system operation to broadcast AOE announce packets for all hosted devices (echo > /sys/kvblade/announce)
* Impremented per_cpu optimizations to reduce cache line thrashing
* Removed a cache line hot spot on aoedev.busy which was using atomic operations
Testing:
* Tested on kernel 4.8.17
* Number of physical server: 3
* Number of physical CPUs: 2
* Cores per CPU: 12
* RAM: 96GB
* Networking: 10GBit/s
* Disks: SSD (1GB) and HDD (4GB)
* Blocksize: 4096 bytes
* MTU: 8192 bytes
Results
* Command: dd if=/dev/zero of=/dev/etherd/e1.0 bs=1MB count=1000 oflag=direct
* Previous bandwidth: 80Mbit/s
* Current bandwidth: 4792Mbit/s
* Previous IOPS: 10k IOPS
* Current IOPS: 153k IOPS